Smart v. Dumb Load Balancing

Dan Iverson

Setting up Load Balancing is common for production systems. Lately, I have seen a few organizations set up their load balancing so they end undoing all their good intentions. At the top of the infrastructure is a smart device, the load balancer, that can look at traffic and the servers underneath it to see where the new connections should be assigned. You want to the load balancer to decide where traffic goes so your web and app servers can focus on processing requests. Often times, the web server is also set up to load balance connections as well.

In the configuration.properties file, the psserver parameter will automatically use a round robin method to assign connections to any server listed. I call this dumb load balancing because there is no logic to assigning traffic. There are many reasons why multiple servers are listed in the psserver parameter, but there are new changes in PeopleTools that let us make smarter decisions with load balancing. First, let’s cover the most common load balancing algorithms to see why they are smart. Next, we’ll talk about the load balancing for the app server layer and the changes we should make. Then, we’ll put it all together and explore different options for optimal load balancing.

Algorithms

Load balancers have many algorithms to choose how traffic is divided, but these three are the most common:

• Round Robin The Round Robin method assigned connections down a circular queue. Round Robin is naive about traffic and there are no decisions on which servers are assigned connections. If the server is next in the queue, the load balancer will assign the connection. Round robin does not look at traffic load or performance, so it can lead to unbalance server loads.
• Least Connection The Least Connection method tracks connections to each pool member and assigns new connections to the server with the fewest connections. This algorithm will keep the connection levels on each server as close as possible. Least connections attempts to balance server load by keeping the connection counts the same. (This article from F5)[https://devcentral.f5.com/articles/back-to-basics-least-connections-is-not-least-loaded] has a great overview on Least Connections.
• Fastest Connection The Fastest Connection method tracks server response times and assigns new connections to the pool members with the fast responses. Fasted Connection prioritizes speed and can be useful for environments where performance is a top priority. The assumption is the fastest server has the capacity to handle more connections. With Fastest Connection, you want to set a connection limit on the pool members. You don’t want to saturate one server with too many connections. More on Fastest Connection.

App Server

“Load Balancing” When people configure the web server, they want to make sure anyone who connects to that web server can reach an application server.

In the psserver parameter, they list multiple application servers. The web server will use a round robin algorithm to “balance” the load. This also gives the web server failover capabilities. If app server 1 is down, the web server will send all the connections to app server 2 and 3.

That diagram looks busy – it must be a good design! 😉

In earlier versions of PeopleTools and WebLogic, this was the only way to offer failover at the web layer (without putting another load balancer between the web and app server). This worked to get failover from the web to app server layers, but it removes any smart load balancing. The other option was to go 1 to 1 between the web and app server. So web server 1 points only to app server 1, web server 2 point only to app server 2. You’re smart load balancing will work much better, but if app server 2 goes down, any user routed to web server 2 will get error messages. That’s not a good user experience.

Smart Load Balancing

Starting with PeopleTools 8.50, you can take advantage of Jolt Failover groups. Jolt Failover groups let you keep the smart decisions at the load balancer but still keep app server failover on the web server layer. To use Jolt Failover, use this syntax in the psserver parameter in the configuration.properties file:

server1:9000{server2:9000;server3:9000}

The web server will only assign connections to server1 while it is up and responsive. When the server1 goes down, the web server will move to server2. If server2 goes down, the web server will send connections to server3. The failover group also supports round robin within the failover group. Instead of a semi-colon, use a comma. E.g,

server1:9000{server2:9000,server3:9000}

You can also you weighted connections for round-robin. This is useful if one server is more powerful than others. E.g,

server1:9000{server2:9000#5,server3:9000#3}

In this case, if server1 is down, server2 will receive 5 connections for every 3 connections assigned to server3.

The main advantage of using failover groups is to keep the smart decisions for assigning traffic at the load balancer. You can read more about Jolt Configuration options in PeopleBooks.