Load balancing with nginx

title

Load balancing with nginx

author

Bernard Szlachta (NobleProg Ltd)

nginx as load balancer ⌘

• A common usage pattern - one or more nginx servers, serving static content directly, and sending dynamic requests onto a cluster of Java servers
• Helps provide good uptime and improved performance

Load balancing algorithms ⌘

• Round-robin (default)

• Connection 1 to host 1, connection 2 to host 2, connection 3 to host 1 etc etc

• ip_hash

• Provides consistent mapping between client and upstream server

• least connections

• Sends each connection to the backend with the fewest connections
• Use with care

Simple load balancing example ⌘

upstream app {
 server app1.example.com:8080;
 server app2.example.com:8080;
 server app3.example.com:8080;
}

server {
 location / {
 proxy_pass http://app;
 }
}

Backend health ⌘

• We can disable a backend if we are unable to connect to it:

upstream app {
 server app1.example.com:8080 max_fails=3 fail_timeout=30s;
 server app2.example.com:8080 max_fails=3 fail_timeout=30s;
 server app3.example.com:8080 max_fails=3 fail_timeout=30s;
}

• This stops trying to use a backend after connection attempts have ailed three times in a row
• Will try using it again after 30 seconds

'Sticky' backends ⌘

• Want each client to be served by the same backend?
• Use ip_hash
• Only looks at the first three octets of the IP

• Therefore often unsuitable for internal-facing applications

'Sticky' backends example ⌘

upstream app {
 ip_hash;
 server app1.example.com:8080 max_fails=3 fail_timeout=30s;
 server app2.example.com:8080 max_fails=3 fail_timeout=30s;
 server app3.example.com:8080 max_fails=3 fail_timeout=30s;
}

Least connections ⌘

• Good in theory
• Can lead to 'interesting' problems with servers constantly being added/removed

• Server overloaded, starts to fail connections
• Removed from pool, number of connections falls
• Now healthy again and has no connections, so suddenly flooded with connections
• Server overloaded, starts to fail connections...

Least connections example ⌘

upstream app {
 least_conn;
 server app1.example.com:8080 max_fails=3 fail_timeout=30s;
 server app2.example.com:8080 max_fails=3 fail_timeout=30s;
 server app3.example.com:8080 max_fails=3 fail_timeout=30s;
}

Keepalive connections ⌘

• Keep a connection 'pool' to the backends and recycles them
• Lower overhead for each individual request as no need to create a new TCP connection
• If more connections are required than the pool size, these will be dynamically created as before

Keepalive example ⌘

upstream app {
 ip_hash;
 keepalive 32;
 server app1.example.com:8080 max_fails=3 fail_timeout=30s;
 server app2.example.com:8080 max_fails=3 fail_timeout=30s;
 server app3.example.com:8080 max_fails=3 fail_timeout=30s;
}

Weights ⌘

• Sometimes you may be balancing across backends of varying power
• Use weights to signify which servers should get more or less connections
• Higher weight - more connections

Weight example ⌘

upstream backend {
 ip_hash;
 server   backend1.example.com weight=2;
 server   backend2.example.com;
 server   backend3.example.com;
 server   backend4.example.com;
}

Node maintenance ⌘

• A node can be temporarily disabled in the config, for example when upgrades are to be carried out on it

upstream backend {
 ip_hash;
 server   backend1.example.com down;
 server   backend2.example.com;
 server   backend3.example.com;
 server   backend4.example.com;
}

Exercise ⌘

• Download the HTTP daemons:

File:Httpservers.tar.gz

• Run each of them in separate windows
• Create round-robin load balancer setup with backends at 127.0.0.1:8001 and 127.0.0.1:8002
• Stop the daemon on 8002. What happens?
• Change the config so that connections to the 8002 daemon automatically cease when it has failed