Configuring the TOA Module

Scenarios

ELB provides customized strategies for managing service access. Before these strategies can be customized, the clients' IP addresses contained in the requests are required. To obtain the IP addresses, you can install the TCP Option Address (TOA) kernel module on backend servers.

This section provides detailed operations for you to compile the module in the OS if you use TCP to distribute incoming traffic.

The operations for Linux OSs with kernel version of 2.6.32 are different from those for Linux OSs with kernel version of 3.0 or later.

Prerequisites

• The development environment for compiling the module must be the same as that of the current kernel. For example, if the kernel version is kernel-3.10.0-693.11.1.el7, the kernel development package version must be kernel-devel-3.10.0-693.11.1.el7.
• Servers can access OS repositories.
• Users other than root must have sudo permissions.

Procedure

• In the following operations, the Linux kernel version is 3.0 or later.

1. Prepare the compilation environment.
   

   • During the installation, download the required module development package from the Internet if it cannot be found in the source.
   • If the kernel development package (kernel-devel) cannot be obtained, contact the image provider.

   The following are operations for compiling the module in different Linux OSs. Perform appropriate operations.

   • CentOS
     1. Run the following command to install the GCC:

        sudo yum install gcc

     2. Run the following command to install the make tool:

        sudo yum install make

     3. Run the following command to install the module development package (the package header and module library must have the same version as the kernel):

        sudo yum install kernel-devel-`uname -r`

        

        • During the installation, download the required module development package from the following address if it cannot be found in the source:

          https://mirror.netcologne.de/oracle-linux-repos/ol7_latest/getPackage/

          For example, to install 3.10.0-693.11.1.el7.x86_64, run the following command:

          rpm -ivh kernel-devel-3.10.0-693.11.1.el7.x86_64.rpm

        • If the kernel development package (kernel-devel) cannot be obtained, contact the image provider.
   • Ubuntu and Debian
     1. Run the following command to install the GCC:

        sudo apt-get install gcc

     2. Run the following command to install the make tool:

        sudo apt-get install make

     3. Run the following command to install the module development package (the package header and module library must have the same version as the kernel):

        sudo apt-get install linux-headers-`uname -r`

   • SUSE
     1. Run the following command to install the GCC:

        sudo zypper install gcc

     2. Run the following command to install the make tool:

        sudo zypper install make

     3. Run the following command to install the module development package (the package header and module library must have the same version as the kernel):

        sudo zypper install kernel-default-devel

2. Compile the module.
   1. Use the git tool and run the following command to download the module source code:

      git clone https://github.com/Huawei/TCP_option_address.git

      

      If the git tool is not installed, download the module source code from the following link:

      https://github.com/Huawei/TCP_option_address

   2. Run the following commands to enter the source code directory and compile the module:

      cd src

      make

      If no warning or error code is prompted, the compilation was successful. Verify that the toa.ko file was generated in the current directory.

      

      If error message "config_retpoline=y but not supported by the compiler, Compiler update recommended" is displayed, the GCC version is too old. Upgrade the GCC to a later version.

3. Load the module.
   1. Run the following command to load the module:

      sudo insmod toa.ko

   2. Run the following command to check the module loading and to view the kernel output information:

      dmesg | grep TOA

      If TOA: toa loaded is displayed in the command output, the module has been loaded.

      

      After compiling the CoreOS module in the container, copy it to the host system and then load it. The container for compiling the module shares the /lib/modules directory with the host system, so you can copy the module in the container to this directory, allowing the host system to use it.

4. Set the script to enable it to automatically load the module.

   To make the module take effect when the system starts, add the command for loading the module to your startup script.

   You can use either of the following methods to automatically load the module:

   • Add the command for loading the module to a customized startup script as required.
   • Perform the following operations to configure a startup script:
     1. Create the toa.modules file in the /etc/sysconfig/modules/ directory. This file contains the module loading script.

        The following is an example of the content in the toa.modules file.

        #!/bin/sh

        /sbin/modinfo -F filename /root/toa/toa.ko > /dev/null 2>&1

        if [ $? -eq 0 ]; then

        /sbin/insmod /root/toa/toa.ko

        fi

        /root/toa/toa.ko is the path of the module file. You need to replace it with their actual path.

     2. Run the following command to add execution permissions for the toa.modules startup script:

        sudo chmod +x /etc/sysconfig/modules/toa.modules

        

        If the kernel is upgraded, the current module will no longer match. Compile the module again.

5. Install the module on multiple servers.

   To load the module in the same OS, copy the toa.ko file to servers where the module is to be loaded and then perform the operations in 3.

   After the module is successfully loaded, applications can obtain the real IP address contained in the request.

   

   The OS of the server must have the same version as the kernel.

6. Verify the module.

   After the module is successfully installed, the source address can be directly obtained. The following provides an example for verification.

   Run the following command to start a simple HTTP service on the backend server where Python is installed:

   python -m SimpleHTTPServer port

   The value of port must be the same as the port configured for the backend server, and the default value is 80.

   Access the IP address of the load balancer from a client. Access logs on the server are as follows:

   192.168.0.90 - - [06/Aug/2020 14:24:21] "GET / HTTP/1.1" 200 –

   

   192.168.0.90 indicates the client's source IP address that is obtained by the backend server.