How to setup HAProxy for Rancher v2.x

Task

Setup HAProxy as a frontend load balancer for Rancher v2.x.

Overview

Install HAProxy

Ubuntu

apt update
apt install -y haproxy
systemctl enable haproxy
systemctl start haproxy

CentOS / RedHat

yum update
yum install haproxy -y
systemctl enable haproxy
systemctl start haproxy

Example HAProxy Config

Option A - Full SSL

• Follow Rancher install doc https://rancher.com/docs/rancher/v2.x/en/installation/k8s-install/helm-rancher/
• Verify Rancher URL works when connecting directly to a Rancher node. For example:

  curl -k --header "Host: rancher.example.com" https://192.168.1.103/ping

• Copy cert and key into a single file called /etc/haproxy/cert.pem
• Add frontend to /etc/haproxy/haproxy.cfg:

  frontend www-http
  bind *:80
  reqadd X-Forwarded-Proto:\ http
  default_backend rancher-http

  frontend www-https
  bind *:443 ssl crt /etc/haproxy/cert.pem
  reqadd X-Forwarded-Proto:\ https
  default_backend rancher-https

• Add backends to /etc/haproxy/haproxy.cfg:

  backend rancher-http
  mode http
  option httpchk HEAD /healthz HTTP/1.0
  server rancher01 192.168.1.103:80 check weight 1 maxconn 1024
  server rancher02 192.168.1.104:80 check weight 1 maxconn 1024
  server rancher03 192.168.1.105:80 check weight 1 maxconn 1024

  backend rancher-https
  mode http
  option httpchk HEAD /healthz HTTP/1.0
  server rancher01 192.168.1.103:443 check weight 1 maxconn 1024 ssl verify none
  server rancher02 192.168.1.104:443 check weight 1 maxconn 1024 ssl verify none
  server rancher03 192.168.1.105:443 check weight 1 maxconn 1024 ssl verify none

• Test the configuration:

  haproxy -f /etc/haproxy/haproxy.cfg -c

• Reload HAProxy:

  systemctl reload haproxy

Example config

Option B - External TLS Termination

• Follow Rancher install doc https://rancher.com/docs/rancher/v2.x/en/installation/options/chart-options/#external-tls-termination
• Verify Rancher URL works went connecting directly to a Rancher node. For example:

  curl --header "Host: rancher.example.com" http://192.168.1.103/ping

• Copy cert and key into a single file called /etc/haproxy/cert.pem
• Create frontends:

  frontend www-http
  bind *:80
  reqadd X-Forwarded-Proto:\ http
  default_backend rancher-http

  frontend www-https
  bind *:443 ssl crt /etc/haproxy/cert.pem
  reqadd X-Forwarded-Proto:\ https
  default_backend rancher-http

• Create backends:

  backend rancher-http
  mode http
  option httpchk HEAD /healthz HTTP/1.0
  server rancher01 192.168.1.103:80 check weight 1 maxconn 1024
  server rancher02 192.168.1.104:80 check weight 1 maxconn 1024
  server rancher03 192.168.1.105:80 check weight 1 maxconn 1024

• Test the configuration:

  haproxy -f /etc/haproxy/haproxy.cfg -c

• Reload HAProxy:

  systemctl reload haproxy

Example config

Option C - TCP pass-through

• Follow Rancher install doc https://rancher.com/docs/rancher/v2.x/en/installation/k8s-install/helm-rancher/
• Verify Rancher URL works when connecting directly to a Rancher node. For example:

  curl -k --header "Host: rancher.example.com" https://192.168.1.103/ping

• NOTE: The default gateway for all 3 Rancher nodes must be the load balancer. Doc: https://www.haproxy.com/blog/howto-transparent-proxying-and-binding-with-haproxy-and-aloha-load-balancer/
• Create frontends:

  frontend www-http
  bind *:80
  mode tcp
  option tcplog
  tcp-request inspect-delay 5s
  default_backend rancher-http

  frontend www-https
  bind *:443
  mode tcp
  option tcplog
  tcp-request inspect-delay 5s
  default_backend rancher-https

• Create backends:

  backend rancher-http
  mode tcp
  balance roundrobin
  source 0.0.0.0 usesrc client
  server rancher01 192.168.1.103:80
  server rancher02 192.168.1.104:80
  server rancher03 192.168.1.105:80

  backend rancher-https
  mode tcp
  balance roundrobin
  source 0.0.0.0 usesrc client
  server rancher01 192.168.1.103:443
  server rancher02 192.168.1.104:443
  server rancher03 192.168.1.105:443

• Test the configuration:

  haproxy -f /etc/haproxy/haproxy.cfg -c

• Reload HAProxy:

  systemctl reload haproxy

Example config

Troubleshooting

• Add the following to /etc/haproxy/haproxy.cfg before the frontend section.

  listen stats
  bind :9000
  mode http
  stats enable
  stats hide-version
  stats realm Haproxy\ Statistics
  stats uri /
  stats auth admin:admin

• Go to http://load01.example.com:9000/
• Username/Password: admin/admin
• If there are firewall rules blocking port 9000, use ssh tunneling to proxy the connection:

  ssh -f -N -L 9000:127.0.0.1:9000 root@192.168.1.101

• Go to http://localhost:9000/