Using Deregistration Delay of a Dedicated Load Balancer to Ensure Smooth Service Termination

Scenarios

When a backend server is detected as unhealthy or gets removed, connections established with the backend server are not interrupted immediately. Client requests keep going to this server, which can lead to service termination failures or request issues.

To address this issue, you can use the deregistration delay feature of a dedicated load balancer. After deregistration delay is enabled, if a backend server is removed or the health check fails, ELB continues to route in-flight requests to this server until the deregistration delay timeout expires.

Preparations

• Create a dedicated load balancer.
• Create two ECSs in the same VPC as the load balancer.

  This practice uses CentOS ECSs as an example. An ECS (ECS_client) acts as the client to send requests, and it must support persistent connections. The other ECS (ECS_server) serves as the backend server for deploying backend applications.

Step 1: Create a Backend Server Group and Enable Deregistration Delay

In this practice, enable Deregistration Delay and set Deregistration Delay Timeout (s) to 30 seconds.

1. Go to the backend server group list page.
2. Click Create Backend Server Group in the upper right corner.
3. Configure the parameters based on Table 1. Retain the default values for other parameters.

   Table 1 Parameters required for configuring a backend server group

   Parameter	Example Value	Description
   Backend Server Group Name	server_group	Specifies the name of the backend server group.
   Type	Dedicated	Specifies the type of the load balancer that can use the backend server group.
   Load Balancer	Associate existing	Specifies whether to associate a load balancer now. Click Associate existing and select the load balancer you have created.
   Backend Protocol	TCP	Specifies the protocol that backend servers in the backend server group use to receive requests from the listeners. Select TCP.
   Load Balancing Algorithm	Weighted round robin	Select Weighted round robin.
   Deregistration Delay	Enable it.	This parameter is enabled by default if the backend protocol is TCP, UDP, or QUIC. If a backend server is removed or the health check fails, ELB stops routing new requests but continues to route in-flight requests to this server until the deregistration delay timeout expires.
   Deregistration Delay Timeout (s)	30	ELB continues to route in-flight requests to the backend server until the deregistration delay timeout expires. In this practice, set it to 30 seconds.

4. Click Next to add backend servers and configure health check.
5. Click Add Cloud Server, select ECS_server, and retain the default values for other parameters.
6. Enable health check and retain the default values for other health check parameters.
7. Click Next.
8. Confirm the configurations and click Create Now.

Step 2: Add a TCP Listener and Associate It with the Backend Server Group

This practice uses a TCP listener as an example to distribute traffic.

1. Go to the load balancer list page.
2. Locate the target load balancer and click Add Listener in the Operation column.
3. On the Add Listener page, set Protocol to TCP and Port to 80, and retain the default values for other parameters.
4. Click Next: Configure Request Routing Policy and configure the backend server group.

   Click Use Existing and select the backend server group created in Step 1: Create a Backend Server Group and Enable Deregistration Delay.

5. Click Next: Confirm, confirm the configurations, and submit your request.

Step 3: Deploy the Application on the Backend Server

1. Remotely log in to ECS_server.

   Multiple methods are available for logging in to an ECS. For details, see Logging In to an ECS.

2. Create the Test directory on the backend server.

   mkdir Test

3. Create the server.py file in the Test directory and deploy the service for testing deregistration delay in the file to print the connection status between the server and client per second.
   1. Create the server.py file.

      python server.py

   2. Press i to enter editing mode.

      Example Code

      • The following shows the server.py script and the default port is 80.

        import socket
        import threading
        def handle_client(client_socket, addr):
            print(f"[+] Connection request from {addr}")
            try:
                while True:
                    data = client_socket.recv(1024).decode()
                    if not data:
                        break
                    print(f"[{addr}] Received: {data}")
                    client_socket.send(f"ACK: {data}".encode())
            except ConnectionResetError:
                print(f"[-] {addr} Connection interrupted")
            finally:
                client_socket.close()
                print(f"[-] {addr} Connection closed")
        def start_server():
            server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            server.bind(('0.0.0.0', 80))
            server.listen(5)
            print("[*] Server started, waiting to be connected...")
            while True:
                client_sock, addr = server.accept()
                client_thread = threading.Thread(target=handle_client, args=(client_sock, addr))
                client_thread.start()
        if __name__ == "__main__":
            start_server()

4. Press Esc and enter :wq to save the server.py file.
5. Run the server.py file.

   python server.py

6. Wait until the command output shown in Figure 1 is displayed, which indicates the backend service is successfully deployed.
   Figure 1 Backend service started
   

Step 4: Deploy the Application on the Client Server

1. Remotely log in to ECS_client.
2. Create the Test_client directory on the client server.

   mkdir Test_client

3. Create the python_client.py file in the Test_client directory and deploy the application service in the file to print the connection status between the client and server every second.
   1. Create the python_client.py file.

      vi python_client.py

   2. Press i to enter editing mode.

      Example Code

      • The following shows the python_client.py script and the default port is 80.

        import socket
        import time
        import threading
        def client_loop():
            try:
                s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
                s.connect(('<your_elb_ip>', 80)) # <your_elb_ip>: Replace it with the private IP address of your load balancer.
                print("[+] Connected to the server")
                while True:
                    try:
                        s.send("keep alive".encode())
                        response = s.recv(1024).decode()
                        print(f"[Status] {time.ctime()}: Response received -> {response}")
                        time.sleep(1)
                    except (ConnectionResetError, ConnectionAbortedError):
                        print("[-] Connection to server interrupted")
                        break
            except Exception as e:
                print(f"[Error] Connection failed: {e}")
            finally:
                s.close()
        if __name__ == "__main__":
            client_thread = threading.Thread(target=client_loop)
            client_thread.start()
            client_thread.join()

4. Press Esc and enter :wq to save the client.py file.
5. Run the client.py file.

   python client.py

6. Wait until the command output shown in Figure 2 is displayed, which indicates the backend service is deployed successfully.
   Figure 2 Client started
   
7. Check whether ECS_server receives the client request and prints the connection status with the client as shown in Figure 3.
   Figure 3 Client request received
   

Step 5: Remove the Backend Server and Record the Removal Time

In this practice, we remove a backend server to check how deregistration delay works.

1. Go to the backend server group list page.
2. On the backend server group list page, click the name of the target backend server group.
3. Switch to the Backend Servers tab and click Cloud Servers.
4. Select the backend servers you want to remove and click Remove above the backend server list.
5. In the displayed dialog box, click OK.
6. Record the time when the backend server is removed. The client prints the time 09:52:33.

Step 6: Verify the Deregistration Delay Function

1. Observe the connection status printed by the client and server before deregistration delay timeout expires.
   If the connection status as shown in Figure 4 and Figure 5 is displayed, the persistent connection between the client and the server remains active.

   Figure 4 Connection status printed by the client
   

   Figure 5 Connection status printed by the server
   
2. Observe the connection status printed by the client and server after deregistration delay timeout (30s) expires.
   If the connection status as shown in Figure 6 and Figure 7 is displayed, the persistent connection between the client and server is disconnected. According to Figure 6, the connection was disconnected at 09:53:03, 30s later than the time when the backend server was removed (09:52:33).

   Figure 6 Connection status (disconnected) printed by the client
   

   Figure 7 Connection status (disconnected) printed by the server