Help

HAProxy configuration for JSS Load Balancing our cluster

luke_reagor
Contributor II

Posted on ‎03-27-2017 08:23 AM

@ssrussell Here's how we configured our HAProxy server to work with our JSS Cluster. :)

These are the steps we went through when setting up our HAProxy 1.7 server on Ubuntu 14.4.
Our HAProxy server is configured for SSL termination.
Our DB is on its own server separate from the JSS frontends.
We have a JSS master server that is not in the cluster and its only responsibility is to be the brain of the cluster.

These pages were a big help on getting us going in the right direction:
[https://serversforhackers.com/using-ssl-certificates-with-haproxy](link URL)
[https://www.digitalocean.com/community/tutorials/how-to-implement-ssl-termination-with-haproxy-on-ubuntu-14-04](link URL)

These documents will help get User Images working with Apple School Manager when using HAProxy as the SSL Terminator:
This is the main doc:
[https://drive.google.com/open?id=0B1G0Rt4a6S4vTldNQ1JRWkVzaE0](link URL)
This is the haproxy additions/changes:
[https://drive.google.com/open?id=1O_gXyCguULy6Qw_oBA7glhkyQgDOGX4ljaOTqHEzXwA](link URL)

#Installing haproxy 1.7
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:vbernat/haproxy-1.7
sudo apt-get update
sudo apt-get install haproxy

#Enabling haproxy Logging
sudo nano /etc/rsyslog.conf
#Then find the following two lines, and uncomment them to enable UDP syslog reception. It should look like the following when you are done:
$ModLoad imudp
$UDPServerRun 514
$UDPServerAddress 127.0.0.1
#You now should have logging in /var/log/haproxy.log


#We made a cert with no key so we don't have to put in the key every time the service is started.
#To make the keyless cert, we ran the command 'openssl rsa -in server.key.org -out server.key' as suggested in the following page:
#[http://webmasters.stackexchange.com/questions/1247/can-i-skip-the-pem-pass-phrase-question-when-i-restart-the-webserver](link URL)

#Here is our config from /etc/haproxy/haproxy.cfg
##################################################

global
    log /dev/log    local0
    log /dev/log    local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon

    maxconn 8000 # max connections haproxy will handle

    # Default SSL material locations
    ca-base /etc/ssl/certs
    crt-base /etc/ssl/private

    # Default ciphers to use on SSL-enabled listening sockets.
    # For more information, see ciphers(1SSL). This list is from:
    #  [https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/](link URL)
    # An alternative list with additional directives can be obtained from
    #  [https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy](link URL)

    #Default with 1.7.2
    #ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
    #ssl-default-bind-options no-sslv3

    # set default parameters to the intermediate configuration (from mozilla link above)
    tune.ssl.default-dh-param 2048
    ssl-default-bind-ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS
    ssl-default-bind-options no-sslv3 no-tls-tickets
    ssl-default-server-ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS
    ssl-default-server-options no-sslv3 no-tls-tickets

    # set default parameters to the modern configuration (from mozilla link above)
    #ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
    #ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
    #ssl-default-server-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
    #ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets

defaults
    log global
    option  dontlognull
    option log-health-checks
    maxconn 6000 # max connections per listener
    timeout connect 5000
    timeout client  50000
    timeout server  50000
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 408 /etc/haproxy/errors/408.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http

listen stats # goto port 9999 for stats
    mode http
    bind *:9999
    stats enable
    stats uri /

listen smb-distribution #port forwarding for jamf smb distribution
    mode tcp
    bind *:445
    option tcplog
    option tcp-check
    server dbserver 192.168.0.2:445 check

listen https-distribution #port forwarding for jamf https distribution
    #Must be enabled in the JSS GUI for package distribution
    #Using 9991 for images with apple school
    #Cert on port 9991 was created with User Images docs above
    mode http
    bind *:443 ssl crt /path/to/cert/with/no/key.pem
    bind *:9991 ssl crt /path/to/User/Images/cert/webcert_w_key.pem ca-file /path/to/User/Images/cert/photoca.pem verify required
    option httplog
    option forwardfor
    option httpchk
    server dbserver 192.168.0.2:80 check

frontend jssloadbalancer #JSS load balancing
    mode http
    bind *:8080 # JAMF Non-SSL port
    bind *:8443 ssl crt /path/to/cert/with/no/key.pem
    timeout client 10000
    default_backend jsscluster

backend jsscluster #JSS load balancing
    mode http
    option forwardfor
    option http-server-close
    balance leastconn
    option httpchk
    fullconn 1100
    cookie SRVSTICKY insert indirect nocache # use session cookies. keep on same server.
    server jss01 192.168.0.3:8080 check cookie jss01 inter 2000 fall 5
    server jss02 192.168.0.4:8080 check cookie jss02 inter 2000 fall 5
    server jss03 192.168.0.5:8080 check cookie jss03 inter 2000 fall 5
    http-request set-header X-Forwarded-Port %[dst_port]
    http-request add-header X-Forwarded-Proto https if { ssl_fc }

##################################################

Once we had it all set up, we had to do the following to have it startup after the networking came online.
update-rc.d -f haproxy remove
update-rc.d haproxy start 35 2 3 4 5 . stop 20 0 1 6 .

0 Kudos
Reply
1 REPLY 1

russeller
Contributor III

Posted on ‎04-12-2017 06:31 PM

Thanks @luke.reagor I used your config as a template and adjusted it for my environment, it was extremely helpful and it's up and running in my production!

One thing I just noticed today, and I don't know if you've seen the same thing, but we've noticed that we can't enroll 3rd Gen AppleTVs anymore through Apple Configurator 2. It gives us a "We don't trust the servers cert" type of error while enrolling into the JSS via AC2. I tried it against our dev JSS and it worked as intended (not behind a load balancer). DEP and Manual iPad enrollments work, just not AC2 enrollments. Curious if you ever ran into the same issue.

Here are the console logs from the AppleTV:

Apr 12 18:21:53 Apple-TV profiled[140] <Notice>: (Error) MC: Installation failed. Error: NSError:
    Desc   : Profile Installation Failed
    Sugg   : The server certificate for “https://jss.domain.org:8443//otaenroll/” is invalid.
    US Desc: Profile Installation Failed
    US Sugg: The server certificate for “https://jss.domain.org:8443//otaenroll/” is invalid.
    Domain : MCInstallationErrorDomain
    Code   : 4001
    Type   : MCFatalError
    ...Underlying error:
    NSError:
    Desc   : The server certificate for “https://jss.domain.org:8443//otaenroll/” is invalid.
    US Desc: The server certificate for “https://jss.domain.org:8443//otaenroll/” is invalid.
    Domain : MCHTTPTransactionErrorDomain
    Code   : 23002
    Type   : MCFatalError
    Params : (
        "https://jss.domain.org:8443//otaenroll/"
    )
    Extra info:
    {
        isPrimary = 1;
    }

Thanks again for all your help. I'm also going to reach out to JAMF support to see if they have any suggestions. Thanks!

0 Kudos
Reply