Why Does a Server Occasionally Time Out When a Client Access It Through Different Load Balancers or Different Listeners of a Load Balancer?

Symptom

5-tuple Conflicts

There will be 5-tuple conflicts if the following happens:

• The source port is hard coded as a fixed one, for example, 80, when the client accesses different load balancers.
• The client OS automatically selects the same source port when the client accesses different load balancers (different IP addresses).

Figure 1 5-tuple conflicts

Figure 1 shows how two load balancers (with Transfer Client IP Address enabled) forward client requests to the same backend server over the same source port.

1. The client (IP-Client) uses the same source port (port 100) to send TCP request 1 to EIP1 (12.xx.xx.xx) and TCP request 2 to EIP2 (13.xx.xx.xx) bound to two load balancers.
2. The two TCP listeners forward client requests over the same port (port 80), and the two load balancers receive the 5-tuples as follows:

   Table 2 5-tuples received by the two load balancers

   Name	Protocol	Source IP Address	Source Port	Destination IP Address	Destination Port
   5-tuple 1-ELB	TCP	IP-Client	100	12.xx.xx.xx	80
   5-tuple 2-ELB	TCP	IP-Client	100	13.xx.xx.xx	80

3. Each load balancer forwards the client requests to the same backend server (IP-ECS) over port 90.
4. Transfer Client IP Address has been enabled for the two TCP listeners, so the source IP address and source port are not changed. In this case, IP-ECS receives the 5-tuples from load balancers as follows:

   Table 3 5-tuples received by IP-ECS

   Name	Protocol	Source IP Address	Source Port	Destination IP Address	Destination Port
   5-tuple 1-ECS	TCP	IP-Client	100	IP-ECS	90
   5-tuple 2-ECS	TCP	IP-Client	100	IP-ECS	90

5. IP-ECS receives requests 1 and 2 with the same 5-tuple, causing connection failures.

Solution

You can solve the problem by taking the following measures:

• If the source port is hard coded as a fixed port, change the configuration so that the client can access different load balancers over different fixed ports.
• If the source port is automatically selected by the OS:
  • Allow the client to resend requests using different source ports to load balancers.
  • Add IP addresses as backend servers if your service is sensitive to reset connections. If IP as a Backend is enabled, ELB uses FullNAT to forward requests and Transfer Client IP Address will not take effect. In this case, the source IP address is changed to an IP address in the backend subnet of the load balancer. If you want to transfer the source IP address of the client, Configuring the TOA Plug-in.

• If you use different listeners to forward client requests, configure listeners to route requests to different backend servers.