Configuring haproxy SSL offloading with a Self Signed Certificate on Content Gateway

The following steps are for RHEL/CentOS 7.x specifically.

The following configuration steps are needed to configure HAProxy as an SSL offloader for Content Gateway.

Step-by-step guide

Verify Content Gateway is listening on port 8080 for SCSP, 8090 for S3 and 8091 for Service Proxy

/etc/caringo/cloudgateway/gateway.cfg

[scsp]
enabled = true
bindAddress = 0.0.0.0
bindPort = 8080
externalHTTPPort = 80
externalHTTPSPort = 443

[s3]
enabled = true
bindAddress = 0.0.0.0
bindPort = 8090

[cluster_admin]
enabled = true
bindAddress = 0.0.0.0
bindPort = 8091
externalHTTPSPort = 91

Note: In this example HTTP access to those personalities is provided. The bind address needs to be modified to 127.0.0.1 for all 3 personalities if security hardening is desired and https needs to be forced.

Setup and install HAProxy. This package is part of the EPEL repository.
Use the following configuration for /etc/haproxy/haproxy.cfg

global
    log 127.0.0.1 local2
    chroot /var/lib/haproxy
    stats socket /var/lib/haproxy/stats mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon

    ca-base /etc/pki/tls/certs
    crt-base /etc/pki/tls/private

    ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
    ssl-default-bind-options no-sslv3
    maxconn 2048
    tune.ssl.default-dh-param 2048

defaults
    log     global
    mode    http
    option  forwardfor
    # Do not use "option  http-server-close", it causes S3 PUT incompatibility with some clients including FileFly!
    option  httplog
    option  dontlognull
    timeout connect 5000
    timeout client  50000
    # This timeout should always be larger than gateway.cfg's [storage_cluster] indexerSocketTimeout.
    timeout server  130000

frontend www-http
    bind 0.0.0.0:80
    http-request set-header X-Forwarded-Proto http
    http-request set-header X-Forwarded-Port 80
    default_backend www-backend-scsp
    acl iss3 hdr_sub(Authorization) AWS
    acl iss3 url_reg [?&](AWSAccessKeyId|X-Amz-Credential)=
    use_backend www-backend-s3 if iss3

frontend www-https
    bind 0.0.0.0:443 ssl crt /etc/pki/tls/certs/selfsignedcert.pem
    http-request set-header X-Forwarded-Proto https
    http-request set-header X-Forwarded-Port 443
    default_backend www-backend-scsp
    acl iss3 hdr_sub(Authorization) AWS
    acl iss3 url_reg [?&](AWSAccessKeyId|X-Amz-Credential)=
    use_backend www-backend-s3 if iss3

frontend www-https-svc
    bind 0.0.0.0:91 ssl crt /etc/pki/tls/certs/selfsignedcert.pem
    http-request set-header X-Forwarded-Proto https
    http-request set-header X-Forwarded-Port 91
    default_backend www-backend-svc

backend www-backend-scsp
    #redirect scheme https if !{ ssl_fc }   <--- Uncomment this line if you want to force HTTPS
    server gw1 127.0.0.1:8080 check

backend www-backend-s3
    #redirect scheme https if !{ ssl_fc }    <--- Uncomment this line if you want to force HTTPS
    server gw1 127.0.0.1:8090 check

backend www-backend-svc
    # This rule rewrites CORS header to add the port number used on frontend
    http-request replace-value Access-Control-Allow-Origin (.*) \1:91
    redirect scheme https if !{ ssl_fc }
    server gw1 127.0.0.1:8091 check

Start haproxy
systemctl restart haproxy
Create a Self-Signed SSL Certificate
Execute the following to create a self signed certificate for the domain:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout $DOMAIN.key -out $DOMAIN.crt

Concatenate the DOMAIN.crt (first) and the DOMAIN.key (second) into a DOMAIN.pem file.
Copy the DOMAIN.pem to /etc/pki/tls/certs/DOMAIN.pem
Best practice for S3 is to create a wildcard SSL certificate, so repeat the same steps as above and when prompted for the "Common Name" use "*.DOMAIN" copy the new CRT file to /etc/pki/tls/certs/DOMAIN-wildcard.pem
To add the new certificate to /etc/haproxy/haproxy.cfg change
bind 0.0.0.0:443 ssl crt /etc/pki/tls/certs/selfsignedcer.pem
to
bind 0.0.0.0:443 ssl crt /etc/pki/tls/certs/DOMAIN.pem crt /etc/pki/tls/certs/DOMAIN-wildcard.pem

Perform this for all bind statements with the ssl keyword configured. Restart haproxy to activate: systemctl restart haproxy

If when restarting HA Proxy this error is thrown “Starting frontend www-https-svc: cannot bind socket [0.0.0.0:91]”, either disable SELinux or run this command

setsebool -P haproxy_connect_any=1

NOTE: The following setting must appear and be set properly in the /etc/caringo/cloudgateway/gateway.cfg file if the content gateway is going to be used as the destination for a remote replication feed:

[scsp]
...
...
allowSwarmAdminIP=172.30

In the example above, replicate "172.30" with the IP addresses (or prefix) of clients sending administrative requests to the gateway. The most common example is the IP addresses (or prefix) of the nodes in a cluster using a remote replication feed with the gateway as the destination.

The host <contentgatewayIP>:8091 needs to be used in the login page to connect for Swarm UI in the configuration example above.

Copy both crt files to /etc/pki/ca-trust/source/anchors and run update-ca-trust to test the new certificate from a CentOS 7.x client. curl may then used to test the certificate validation once the command completes.