Use HAProxy to loadbalance the ruby processes

Installing HAProxy

apt-get install haproxy

We need to enable HAProxy to be started by the init script. 

nano /etc/default/haproxy

Set the ENABLED option to 1 

ENABLED=1

Test whether it's installed and enabled

sudo service haproxy status

Configuring HAProxy

We'll move the default configuration file and create our own one.

mv /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.backup
Create and edit a new configuration file:

nano /etc/haproxy/haproxy.cfg
Let us begin by adding configuration block by block to this file:

global
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        user haproxy
        group haproxy
        daemon

defaults
        option http-server-close # for slowloris like attacks
        mode http
        timeout http-request 5s # for slowloris like attacks, wait only 5sec for                                                                                                                                                              header
        timeout connect 5000ms
        timeout client 50000ms
        timeout server 50000ms

        log     global
        mode    http
        option  httplog
        option  dontlognull

frontend WebServer-In
    bind *:801
    default_backend Web-Servers

backend Web-Servers
    balance url_param user_id
    server web1 127.0.0.1:3000 maxconn 12 check
    server web2 127.0.0.1:3001 maxconn 12 check
    server web3 127.0.0.1:3002 maxconn 12 check

frontend API-In
    bind *:802
    default_backend API-Servers

backend API-Servers
    balance url_param user_id
    server api1 127.0.0.1:4567 maxconn 6 check
    server api2 127.0.0.1:4568 maxconn 6 check
    server api3 127.0.0.1:4569 maxconn 6 check

listen Stats ip_address:port
    mode http
    stats enable
    stats hide-version
    stats realm Haproxy\ Statistics
    stats uri /
    stats auth username:password

#This include both frontend and backend
listen appname 0.0.0.0:80
    mode http
    stats enable
    stats uri /haproxy?stats
    stats realm Strictly\ Private
    stats auth A_Username:YourPassword
    stats auth Another_User:passwd
    balance roundrobin
    option httpclose
    option forwardfor
    server lamp1 10.0.0.1:80 check
    server lamp2 10.0.0.2:80 check

#Use subdomain and re-route the clusters
frontend http-in
        bind *:80

        # Define hosts
        acl host_bacon hdr(host) -i ilovebacon.com
        acl host_milkshakes hdr(host) -i bobsmilkshakes.com

        ## figure out which one to use
        use_backend bacon_cluster if host_bacon
        use_backend milshake_cluster if host_milkshakes

backend baconcluster
        balance leastconn
        option httpclose
        option forwardfor
        cookie JSESSIONID prefix
        server node1 10.0.0.1:8080 cookie A check
        server node1 10.0.0.2:8080 cookie A check
        server node1 10.0.0.3:8080 cookie A check


backend milshake_cluster
        balance leastconn
        option httpclose
        option forwardfor
        cookie JSESSIONID prefix
        server node1 10.0.0.4:8080 cookie A check
        server node1 10.0.0.5:8080 cookie A check
        server node1 10.0.0.6:8080 cookie A check

Save and close the file
sudo service haproxy start

multiple Subdomain setups

To keep performance at a maximum (avoiding a regex every hit) but still cleaning up the config, I'd use an external file for your ACLs here. For example let's say you had a file called /etc/haproxy/sub1urls, which was exactly this:

apple.gamma.com
banana.gamma.com
cherry.gamma.com
Then in your config the ACL could simply be:

acl is_sub1 hdr(host) -i -f /etc/haproxy/sub1urls
Putting the other hosts in a sub2urls file the same way reduces your config down to:

frontend http-in
    bind *:80

    acl alpha     hdr(host) -i alpha.com
    acl beta      hdr(host) -i beta.com
    acl is_sub1   hdr(host) -i -f /etc/haproxy/sub1urls
    acl is_sub2   hdr(host) -i -f /etc/haproxy/sub2urls
    acl gamma     hdr(host) -i gamma.com

    use_backend a if alpha
    use_backend b if beta
    use_backend sub1 if is_sub1
    use_backend sub2 if is_sub2
    use_backend g if gamma

    default_backend default
This makes it very easy to maintain those other files, since they're just lists of hosts. It opens up the list of who can edit them and exposes less risk as well. For example, we have people editing these ACL lists like this in puppet who don't have to know the HAProxy config syntax at all.