Help Center/ Elastic Load Balance/ User Guide (ME-Abu Dhabi Region)/ FAQs/ Health Check/ What Are the Precautions of Using UDP for Health Checks?
Updated on 2022-01-25 GMT+08:00
View PDF
Share

What Are the Precautions of Using UDP for Health Checks?

How UDP Health Checks Work

UDP is a connectionless protocol, which does not establish a three-way handshake before sending data. A UDP health check is implemented as follows:

  1. The health check node sends an ICMP request message to the backend server based on the health check configuration.
    • If the health check node receives an ICMP reply message from the backend server, it considers the backend server healthy and continues the health check.
    • If the health check node does not receive an ICMP reply message from the backend server, it considers the backend server unhealthy.
  2. After receiving the ICMP reply message, the health check node sends a UDP probe packet to the backend server.
    • If the health check node receives an ICMP Port Unreachable message from the backend server within the timeout duration, the backend server is considered unhealthy.
    • If the health check node does not receive an ICMP Port Unreachable message from the backend server within the timeout duration, the backend server is healthy.

When you use UDP for health checks, you are advised to retain the default settings of the parameters.

Troubleshooting Procedure

Use either of the following methods to locate the fault:

  1. Check whether the timeout duration is too short.

    A possible cause is that the ICMP Echo Reply or ICMP Port Unreachable message returned by the backend server does not reach the health check node within the timeout duration. As a result, the health check result is inaccurate.

    It is recommended that you change the timeout duration to a larger value.

    UDP health checks are different from other health checks. If the health check timeout duration is too short, the health check result of the backend server changes between Healthy and Unhealthy frequently.

  2. Check whether the backend server restricts the rate at which ICMP messages are generated.

For Linux servers, run the following commands to query the rate limit and rate mask:

sysctl -q net.ipv4.icmp_ratelimit

The default rate limit is 1000.

sysctl -q net.ipv4.icmp_ratemask

The default rate mask is 6168.

If the returned value of the first command is the default value or 0, run the following command to remove the rate limit of Port Unreachable messages:

sysctl -w net.ipv4.icmp_ratemask=6160

For more information, see the Linux Programmer's Manual. On the Linux CLI, run the following command to display the manual:

man 7 icmp

Alternatively, visit http://man7.org/linux/man-pages/man7/icmp.7.html.

Once the rate limit is lifted, the number of ICMP Port Unreachable messages on the backend server will not be limited.

Precautions

Pay attention to the following when you configure UDP health checks:

  • UDP health checks use ping packets to detect the health of the backend server. To ensure smooth transmission of these packets, ensure that ICMP is enabled on the backend server by performing the following:

    Log in to the server and run the following command as user root:

    cat /proc/sys/net/ipv4/icmp_echo_ignore_all

    • If the returned value is 1, ICMP is disabled.
    • If the returned value is 0, ICMP is enabled.
  • The health check result may be different from the actual health of the backend server.

    If the backend server runs a Linux OS, the rate of ICMP packets is limited due to protection against ICMP floods of Linux when there is a large number of concurrent requests. In this case, if a service exception occurs, the load balancer will not receive error message port XX unreachable and will still determine that the health check is successful. As a result, there is an inconsistency between the health check result and the actual server health.

Parent topic: Health Check