Configuring haproxy SSL offloading with a Self Signed Certificate on CentOS7/8
CentOS 8 / RHEL 8 / RockyLinux 8 : yum install haproxy
CentOS 7/RHEL 7 follow instructions provided here CentOS Security Update - [CentOS-announce] Announcing release of...
Note: On CentOS7/RHEL7 the active haproxy.cfg will be in '/etc/opt/rh/rh-haproxy18/haproxy'
Haproxy is already pre-installed on the Swarm Cluster Installer provisioned - and VMware bundle provided Gateway VM
The following configuration steps are needed to configure HAProxy as an SSL offloader for Content Gateway.
Step-by-step guide
Verify Content Gateway is listening on port 8080 for SCSP, 8090 for S3 and 8091 for Service Proxy:
The example below are not the default gateway ports, this is done on purpose to avoid port conflicts.
/etc/caringo/cloudgateway/gateway.cfg
[scsp]
enabled = true
bindAddress = 0.0.0.0
bindPort = 8080
externalHTTPPort = 80
externalHTTPSPort = 443
[s3]
enabled = true
bindAddress = 0.0.0.0
bindPort = 8090
[cluster_admin]
enabled = true
bindAddress = 0.0.0.0
bindPort = 8091
externalHTTPSPort = 91
Setup and install HAProxy. This package is part of the EPEL repository.
Use the following example configuration for
/etc/haproxy/haproxy.cfg
or on CentOS7 with haproxy1.8 /etc/opt/rh/rh-haproxy18-haproxy/haproxy.cfg
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
stats socket /var/lib/haproxy/stats mode 660 level admin
user haproxy
group haproxy
daemon
ca-base /etc/pki/tls/certs
crt-base /etc/pki/tls/private
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3
maxconn 2048
tune.ssl.default-dh-param 2048
defaults
log global
mode http
option forwardfor
# Do not use "option http-server-close", it causes S3 PUT incompatibility with some clients including FileFly!
option httplog
option dontlognull
timeout connect 5000
timeout client 50000
# This timeout should always be larger than gateway.cfg's [storage_cluster] indexerSocketTimeout
# and large enough to not timeout an S3 CopyObject or an SCSP COPY in a versioned bucket.
timeout server 130000
frontend www-http
bind 0.0.0.0:80
http-request set-header X-Forwarded-Proto http
http-request set-header X-Forwarded-Port 80
default_backend www-backend-scsp
acl iss3 hdr_sub(Authorization) AWS
acl iss3 url_reg [?&](AWSAccessKeyId|X-Amz-Credential)=
use_backend www-backend-s3 if iss3
frontend www-https
bind 0.0.0.0:443 ssl crt /etc/pki/tls/certs/YOUR_DOMAIN.pem
http-request set-header X-Forwarded-Proto https
http-request set-header X-Forwarded-Port 443
default_backend www-backend-scsp
acl iss3 hdr_sub(Authorization) AWS
acl iss3 url_reg [?&](AWSAccessKeyId|X-Amz-Credential)=
use_backend www-backend-s3 if iss3
frontend www-https-svc
bind 0.0.0.0:91 ssl crt /etc/pki/tls/certs/YOUR_DOMAIN.pem
http-request set-header X-Forwarded-Proto https
http-request set-header X-Forwarded-Port 91
default_backend www-backend-svc
backend www-backend-scsp
balance leastconn
# On HAPROXY < 2.2
option httpchk HEAD /_admin/manage/version HTTP/1.1\r\nHost:\ haproxy-healthcheck
http-check expect status 200
# On HAPROXY > 2.2
#option httpchk
#http-check connect
#http-check send meth HEAD uri /_admin/manage/version ver HTTP/1.1 hdr Host haproxy-healthcheck
#http-check expect status 200
#redirect scheme https if !{ ssl_fc } <--- Uncomment this line if you want to force HTTPS
server gw1 YOUR_GATEWAY1_IP:8080 check inter 10s fall 3 rise 2
server gw2 YOUR_GATEWAY2_IP:8080 check inter 10s fall 3 rise 2
backend www-backend-s3
balance leastconn
# On HAPROXY < 2.2
option httpchk HEAD / HTTP/1.1\r\nHost:\ haproxy-healthcheck
http-check expect status 403
# On HAPROXY > 2.2
#option httpchk
#http-check connect
#http-check send meth HEAD uri / ver HTTP/1.1 hdr Host haproxy-healthcheck
#http-check expect status 403
#redirect scheme https if !{ ssl_fc } <--- Uncomment this line if you want to force HTTPS
server gw1 YOUR_GATEWAY1_IP:8090 check inter 10s fall 3 rise 2
server gw2 YOUR_GATEWAY2_IP:8090 check inter 10s fall 3 rise 2
backend www-backend-svc
balance leastconn
# On HAPROXY < 2.2
option httpchk HEAD / HTTP/1.1\r\nHost:\ haproxy-healthcheck
http-check expect status 401
# On HAPROXY > 2.2
#option httpchk
#http-check connect
#http-check send meth HEAD uri / ver HTTP/1.1 hdr Host haproxy-healthcheck
#http-check expect status 401
# This rule rewrites CORS header to add the port number used on frontend
http-request replace-value Access-Control-Allow-Origin (.*) \1:91
#redirect scheme https if !{ ssl_fc } <--- Uncomment this line if you want to force HTTPS
server gw1 YOUR_GATEWAY1_IP:8091 check inter 10s fall 3 rise 2
server gw2 YOUR_GATEWAY2_IP:8091 check inter 10s fall 3 rise 2
Start HAProxy:
systemctl restart haproxy
Loadbalancing healthcheck
If you have 2 or more gateways, it is recommended to use the http-check directives to verify the service is live.
Each protocol requires different healthcheck configuration.
SCSP Protocol
HAPROXY < 2.2
option httpchk HEAD /_admin/manage/version HTTP/1.1\r\nHost:\ haproxy-healthcheck
http-check expect status 200
HAPROXY > 2.2
S3 Protocol
HAPROXY < 2.2
HAPROXY > 2.2
Admin Protocol ( cluster_admin )
HAPROXY < 2.2
HAPROXY > 2.2
Backend section for all protocols
Example Logging with SCSP health check
In the following example I ran a while loop, to read an object every second… and while doing it shutdown 1 of the 2 gateways waited a few seconds then started it back up.
Create a Self-Signed SSL Certificate
First we need to make an openssl.conf file, in this example YOUR_DOMAIN = swarm.example.com
Generate the self-signed CA private key
Generate the self-signed CA root certificate
Generate wildcard custom domain private key
Generate certificate signing request for your domain
Generate the final domain certificate
Generate the certificate pem file.
Place the YOUR_DOMAIN.pem file where you configured it in haproxy.cfg example here put it in /etc/pki/tls/certs
Copy the selfsignCA.crt to /etc/pki/ca-trust/source/anchors
and run update-ca-trust to tell CentOS to trust your self-signed root certificate.
Restart haproxy to activate the changes.
Now you can copy the selfsignCA.crt file to your clients and follow the browser specific procedures to install and accept it.
The host <contentgatewayIP>:8091 needs to be used in the login page to connect for Swarm UI in the configuration example above.
Replication Feed configuration
The following setting must appear and be set properly in the /etc/caringo/cloudgateway/gateway.cfg
file if the content gateway is going to be used as the destination for a remote replication feed:
In the example above, replicate "172.30" with the IP addresses (or prefix) of clients sending administrative requests to the gateway.
The most common example is the IP addresses (or prefix) of the nodes in a cluster using a remote replication feed with the gateway as the destination.
Troubleshooting tips
If you occasionally see the error SEC_ERROR_UNKNOWN_ISSUER , verify you don't have lingering old haproxy processes , and kill them manually.
To check if your selfsigned CA certificate is trusted run the following command:
for CentOS7 and haproxy 1.8 by default require additional log settings to see the output in a file, add the following line in /etc/rsyslog.conf after the boot.log
then run
you should now see haproxy logging in /var/log/haproxy.log
Do not forget to enable the ports on your firewall