Versions Compared

Old Version 39

changes.mady.by.user Abraham Felsenstein

Saved on

compared with

New Version Current

changes.mady.by.user Jamshid Afshar

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: set timeout server to not timeout a server-side operation

...

Code Block
global
    log 127.0.0.1 local2
    chroot /var/lib/haproxy
    stats socket /var/lib/haproxy/stats mode 660 level admin
    user haproxy
    group haproxy
    daemon

    ca-base /etc/pki/tls/certs
    crt-base /etc/pki/tls/private

    ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
    ssl-default-bind-options no-sslv3
    maxconn 2048
    tune.ssl.default-dh-param 2048

defaults
    log     global
    mode    http
    option  forwardfor
    # Do not use "option  http-server-close", it causes S3 PUT incompatibility with some clients including FileFly!
    option  httplog
    option  dontlognull
    timeout connect 5000
    timeout client  50000
    # This timeout should always be larger than gateway.cfg's [storage_cluster] indexerSocketTimeout
    # and large enough to not timeout an S3 CopyObject or an SCSP COPY in a versioned bucket.
    timeout server  130000

frontend www-http
    bind 0.0.0.0:80
    http-request set-header X-Forwarded-Proto http
    http-request set-header X-Forwarded-Port 80
    default_backend www-backend-scsp
    acl iss3 hdr_sub(Authorization) AWS
    acl iss3 url_reg [?&](AWSAccessKeyId|X-Amz-Credential)=
    use_backend www-backend-s3 if iss3

frontend www-https
    bind 0.0.0.0:443 ssl crt /etc/pki/tls/certs/YOUR_DOMAIN.pem
    http-request set-header X-Forwarded-Proto https
    http-request set-header X-Forwarded-Port 443
    default_backend www-backend-scsp
    acl iss3 hdr_sub(Authorization) AWS
    acl iss3 url_reg [?&](AWSAccessKeyId|X-Amz-Credential)=
    use_backend www-backend-s3 if iss3

frontend www-https-svc
    bind 0.0.0.0:91 ssl crt /etc/pki/tls/certs/YOUR_DOMAIN.pem
    http-request set-header X-Forwarded-Proto https
    http-request set-header X-Forwarded-Port 91
    default_backend www-backend-svc

backend www-backend-scsp
    balance leastconn
    option# httpchkOn HAPROXY < 2.2
 http-check connect     http-check send methoption httpchk HEAD uri /_admin/manage/version HTTP/1.1\r\nHost:\ haproxy-healthcheck
    http-check expect status 200
    # On  #redirect scheme https if !{ ssl_fc }HAPROXY > 2.2
    #option httpchk
    <#http---check Uncommentconnect
this line if you want#http-check tosend forcemeth HTTPSHEAD uri /_admin/manage/version ver HTTP/1.1 serverhdr gw1 YOUR_GATEWAY1_IP:8080 check inter 10s fall 3 rise 2
Host haproxy-healthcheck
    #http-check expect status 200
  
server gw2    #redirect scheme https if !{ ssl_fc }   <--- Uncomment this line if you want to force HTTPS
    server gw1 YOUR_GATEWAY1_IP:8080 check inter 10s fall 3 rise 2
    server gw2 YOUR_GATEWAY2_IP:8080 check inter 10s fall 3 rise 2

backend www-backend-s3
    balance leastconn
    option# httpchkOn HAPROXY < 2.2
 http-check connect     http-check send methoption httpchk HEAD uri/ /_admin/manage/versionHTTP/1.1\r\nHost:\ haproxy-healthcheck
    http-check expect status 200403
    # On HAPROXY > 2.2
#redirect scheme https if !{ ssl_fc }#option httpchk
    <#http---check Uncommentconnect
this line if you want#http-check tosend force HTTPS
    server gw1 YOUR_GATEWAY1_IP:8090 check inter 10s fall 3 rise 2meth HEAD uri / ver HTTP/1.1 hdr Host haproxy-healthcheck
    #http-check expect status 403
    
  server gw2 YOUR_GATEWAY2_IP:8090 check inter 10s fall 3 rise 2

backend www-backend-svc
    balance leastconn
    option httpchk
    http-check connect
    http-check send meth HEAD uri /_admin/manage/version
    http-check expect status 200
    
    # This rule rewrites CORS header to add the port number used on frontend
    http-request replace-value Access-Control-Allow-Origin (.*) \1:91
    #redirect scheme https if !{ ssl_fc } <--- Uncomment this line if you want to force HTTPS
    server gw1 YOUR_GATEWAY1_IP:8091 check inter 10s fall 3 rise 2
    server gw2 YOUR_GATEWAY2_IP:8091 check inter 10s fall 3 rise 2

  • Start HAProxy:
    systemctl restart haproxy

Note

If when restarting HAProxy this error is thrown “Starting frontend www-https-svc: cannot bind socket [0.0.0.0:91]”, either disable SELinux or run this command:

setsebool -P haproxy_connect_any=1

Loadbalancing healtcheck

...

 #redirect scheme https if !{ ssl_fc }    <--- Uncomment this line if you want to force HTTPS
    server gw1 YOUR_GATEWAY1_IP:8090 check inter 10s fall 3 rise 2
    server gw2 YOUR_GATEWAY2_IP:8090 check inter 10s fall 3 rise 2

backend www-backend-svc
    balance leastconn
    # On HAPROXY < 2.2
    option httpchk HEAD / HTTP/1.1\r\nHost:\ haproxy-healthcheck
    http-check expect status 401
    # On HAPROXY > 2.2
    #option httpchk
    #http-check connect
    #http-check send meth HEAD uri / ver HTTP/1.1 hdr Host haproxy-healthcheck
    #http-check expect status 401
    
    # This rule rewrites CORS header to add the port number used on frontend
    http-request replace-value Access-Control-Allow-Origin (.*) \1:91
    #redirect scheme https if !{ ssl_fc } <--- Uncomment this line if you want to force HTTPS
    server gw1 YOUR_GATEWAY1_IP:8091 check inter 10s fall 3 rise 2
    server gw2 YOUR_GATEWAY2_IP:8091 check inter 10s fall 3 rise 2

  • Start HAProxy:
    systemctl restart haproxy

Note

If when restarting HAProxy this error is thrown “Starting frontend www-https-svc: cannot bind socket [0.0.0.0:91]”, either disable SELinux or run this command:

setsebool -P haproxy_connect_any=1

Loadbalancing healthcheck

If you have 2 or more gateways, it is recommended to use the http-check directives to verify the service is live.

Each protocol requires different healthcheck configuration.

SCSP Protocol

HAPROXY < 2.2

Code Block
    option httpchk HEAD /_admin/manage/version HTTP/1.1\r\nHost:\ haproxy-healthcheck
    http-check expect status 200

HAPROXY > 2.2

Code Block
    option httpchk
    http-check connect
    http-check send meth HEAD uri /_admin/manage/version ver HTTP/1.1 hdr Host haproxy-healthcheck
    http-check expect status 200

S3 Protocol

HAPROXY < 2.2

Code Block
    option httpchk HEAD / HTTP/1.1\r\nHost:\ haproxy-healthcheck
    http-check expect status 403

HAPROXY > 2.2

Code Block
    option httpchk
    http-check connect
    http-check send meth HEAD uri / ver HTTP/1.1 hdr Host haproxy-healthcheck
    http-check expect status 403

Admin Protocol ( cluster_admin )

HAPROXY < 2.2

Code Block
    option httpchk HEAD / HTTP/1.1\r\nHost:\ haproxy-healthcheck
    http-check expect status 401

HAPROXY > 2.2

Code Block
    option httpchk
    http-check connect
    http-check send meth HEAD uri /_admin/manage/version ver HTTP/1.1 hdr Host haproxy-healthcheck
    http-check expect status 200
    401

Backend section for all protocols

Code Block
server gw1 YOUR_GATEWAY1_IP:8090 check inter 10s fall 3 rise 2
    server gw2 YOUR_GATEWAY2_IP:8090 check inter 10s fall 3 rise 2

Example Logging with SCSP health check

In the following example I ran a while loop, to read an object every second… and while doing it shutdown 1 of the 2 gateways waited a few seconds then started it back up.

...