Help Center/ Virtual Private Cloud/ Best Practices/ Configuring Policy-based Routes for an ECS with Multiple NICs/ Configuring Policy-based Routes for a Linux ECS with Multiple NICs
Updated on 2024-05-06 GMT+08:00
View PDF

Configuring Policy-based Routes for a Linux ECS with Multiple NICs

Scenarios

This section describes how to configure policy-based routes for a dual-NIC ECS running CentOS 8.0 (64-bit).

For details about the background knowledge and networking of dual-NIC ECSs, see Overview.

Procedure (Linux ECS Using IPv4)

  1. Collect the ECS network information required for configuring policy-based routes.

    For details, see Collecting ECS Network Information.

  2. Log in to an ECS.
  3. Check whether the source ECS can use its primary NIC to communicate with the destination ECS:

    ping -I IP address of the primary NIC on the source ECS IP address of the destination ECS

    In this example, run the following command:

    ping -I 10.0.0.115 10.0.2.12

    If information similar to the following is displayed, the source ECS can use its primary NIC to communicate with the destination ECS. 
    [root@ecs-resource ~]# ping -I 10.0.0.115 10.0.2.12
PING 10.0.2.12 (10.0.2.12) from 10.0.0.115 : 56(84) bytes of data.
64 bytes from 10.0.2.12: icmp_seq=1 ttl=64 time=0.775 ms
64 bytes from 10.0.2.12: icmp_seq=2 ttl=64 time=0.268 ms
64 bytes from 10.0.2.12: icmp_seq=3 ttl=64 time=0.220 ms
64 bytes from 10.0.2.12: icmp_seq=4 ttl=64 time=0.167 ms
^C
--- 10.0.2.12 ping statistics ---

    Before configuring policy-based routes, ensure that the source ECS can use its primary NIC to communicate with the destination ECS.

  4. Query the NIC names of the ECS:

    ifconfig

    Search for the NIC name based on the NIC address.
    • 10.0.0.115 is the IP address of the primary NIC, and the NIC name is eth0.
    • 10.0.1.183 is the IP address of the extension NIC, and the NIC name is eth1.
    [root@ecs-resource ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.0.115  netmask 255.255.255.0  broadcast 10.0.0.255
        inet6 fe80::f816:3eff:fe92:6e0e  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:92:6e:0e  txqueuelen 1000  (Ethernet)
        RX packets 432288  bytes 135762012 (129.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 1655
        TX packets 423744  bytes 106716932 (101.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.1.183  netmask 255.255.255.0  broadcast 10.0.1.255
        inet6 fe80::f816:3eff:febf:5818  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:bf:58:18  txqueuelen 1000  (Ethernet)
        RX packets 9028  bytes 536972 (524.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 1915
        TX packets 6290  bytes 272473 (266.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  5. Configure temporary routes for the ECS.

    Temporary routes take effect immediately after being configured and will be lost after the ECS is restarted. To prevent network interruptions after the ECS is restarted, perform 6 after this step to configure persistent routes.

    1. Configure policy-based routes for both the primary and extension NICs:
      • Primary NIC

        ip route add default via Subnet gateway dev NIC name table Route table name

        ip route add Subnet CIDR block dev NIC name table Route table name

        ip rule add from NIC address table Route table name

      • Extension NIC

        ip route add default via Subnet gateway dev NIC name table Route table name

        ip route add Subnet CIDR block dev NIC name table Route table name

        ip rule add from NIC address table Route table name

      Configure the parameters as follows:
      • NIC name: Enter the name obtained in 4.
      • Route table name: Customize a route table name using a number.
      • Other network information: Enter the IP addresses collected in 1.

      In this example, run the following commands:

      • Primary NIC

        ip route add default via 10.0.0.1 dev eth0 table 10

        ip route add 10.0.0.0/24 dev eth0 table 10

        ip rule add from 10.0.0.115 table 10

      • Extension NIC

        ip route add default via 10.0.1.1 dev eth1 table 20

        ip route add 10.0.1.0/24 dev eth1 table 20

        ip rule add from 10.0.1.183 table 20

      If the ECS has multiple NICs, configure policy-based routes for all NICs one by one.

    2. Check whether the policy-based routes are successfully added.

      ip rule

      ip route show table Route table name of the primary NIC

      ip route show table Route table name of the extension NIC

      The route table name is customized in 5.a.

      In this example, run the following commands:

      ip rule

      ip route show table 10

      ip route show table 20

      If information similar to the following is displayed, the policy-based routes have been added. 
      [root@ecs-resource ~]# ip rule
0:      from all lookup local
32764:  from 10.0.1.183 lookup 20
32765:  from 10.0.0.115 lookup 10
32766:  from all lookup main
32767:  from all lookup default
[root@ecs-resource ~]# ip route show table 10
default via 10.0.0.1 dev eth0 
10.0.0.0/24 dev eth0 scope link 
[root@ecs-resource ~]# ip route show table 20
default via 10.0.1.1 dev eth1 
10.0.1.0/24 dev eth1 scope link
    3. Check whether the source ECS and the destination ECS can communicate with each other.

      ping -I IP address of the primary NIC on the source ECS IP address of the destination ECS

      ping -I IP address of the extension NIC on the source ECS IP address of the destination ECS

      In this example, run the following commands:

      ping -I 10.0.0.115 10.0.2.12

      ping -I 10.0.1.183 10.0.2.12

      If information similar to the following is displayed, both the NICs of the source ECS can communicate with the destination ECS.

      [root@ecs-resource ~]# ping -I 10.0.0.115 10.0.2.12
PING 10.0.2.12 (10.0.2.12) from 10.0.0.115 : 56(84) bytes of data.
64 bytes from 10.0.2.12: icmp_seq=1 ttl=64 time=0.775 ms
64 bytes from 10.0.2.12: icmp_seq=2 ttl=64 time=0.268 ms
64 bytes from 10.0.2.12: icmp_seq=3 ttl=64 time=0.220 ms
64 bytes from 10.0.2.12: icmp_seq=4 ttl=64 time=0.167 ms
^C
--- 10.0.2.12 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 102ms
rtt min/avg/max/mdev = 0.167/0.357/0.775/0.244 ms
[root@ecs-resource ~]# ping -I 10.0.1.183 10.0.2.12
PING 10.0.2.12 (10.0.2.12) from 10.0.1.183 : 56(84) bytes of data.
64 bytes from 10.0.2.12: icmp_seq=1 ttl=64 time=2.84 ms
64 bytes from 10.0.2.12: icmp_seq=2 ttl=64 time=0.258 ms
64 bytes from 10.0.2.12: icmp_seq=3 ttl=64 time=0.234 ms
64 bytes from 10.0.2.12: icmp_seq=4 ttl=64 time=0.153 ms
^C
--- 10.0.2.12 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 92ms
rtt min/avg/max/mdev = 0.153/0.871/2.840/1.137 ms
  6. Configure persistent routes for the ECS.
    1. Run the following command to open the /etc/rc.local file:

      vi /etc/rc.local

    2. Press i to enter the editing mode.
    3. Add the following content to the end of the file: 
      # wait for nics up
sleep 5
# Add v4 routes for eth0
ip route flush table 10
ip route add default via 10.0.0.1 dev eth0 table 10
ip route add 10.0.0.0/24 dev eth0 table 10
ip rule add from 10.0.0.115 table 10
# Add v4 routes for eth1
ip route flush table 20
ip route add default via 10.0.1.1 dev eth1 table 20
ip route add 10.0.1.0/24 dev eth1 table 20
ip rule add from 10.0.1.183 table 20
# Add v4 routes for cloud-init
ip rule add to 169.254.169.254 table main

      Parameters are described as follows:

      • wait for nics up: file startup time. Set the value to be the same as that in the preceding configurations.
      • Add v4 routes for eth0: policy-based routes of the primary NIC. Set the value to be the same as that configured in 5.a.
      • Add v4 routes for eth1: policy-based routes of the extension NIC. Set the value to be the same as that configured in 5.a.
      • Add v4 routes for cloud-init: Configure the Cloud-Init address. Set the value to be the same as that in the preceding configurations.
    4. Press ESC to exit and enter :wq! to save the configuration.
    5. Run the following command to assign execute permissions to the /etc/rc.local file:

      chmod +x /etc/rc.local

      If your operating system is Red Hat or EulerOS, run the following command after you perform 6.e:

      chmod +x /etc/rc.d/rc.local

    6. Run the following command to restart the ECS:

      reboot

      Policy-based routes added to the /etc/rc.local file take effect only after the ECS is restarted. Ensure that workloads on the ECS will not be affected before restarting the ECS.

    7. Repeat 5.b to 5.c to check whether the policy-based routes are added and whether the source ECS and the destination ECS can communicate with each other.