Help Center> Enterprise Router> Best Practices> Setting Up a Hybrid Cloud Network Using Enterprise Router and a Pair of Direct Connect Connections (Virtual Gateway)> Network and Resource Planning
Updated on 2024-06-27 GMT+08:00
View PDF

Network and Resource Planning

To attach both Direct Connect connections to an enterprise router to allow them to work in load balancing mode, you need:
  • Network Planning: Plan CIDR blocks of VPCs and their subnets, Direct Connect connections, and enterprise router, as well as the routes of these resources.
  • Resource Planning: Plan the quantity, names, and other parameters of cloud resources, such as VPC, Direct Connect connection, and enterprise router.

Network Planning

Figure 1 shows the network set up using Direct Connect connections that work in load balancing mode. Table 2 describes the network planning.

Figure 1 Network diagram of two Direct Connect connections

Two Direct Connect connections work in load balancing mode and connect the on-premises data center to the VPCs. Table 1 describes the network traffic flows in detail.

Table 1 Network traffic flows

Path

Description

Request traffic: from VPC-A to the on-premises data center
  1. In the route table of VPC-A, there are routes with the next hop set to the enterprise router to forward traffic from VPC-A to the enterprise router.
  2. In the route table of the enterprise router, there are routes with the next hop set to virtual gateway VGW-A attachment to forward traffic from the enterprise router to virtual gateway VGW-A.
    • There are two routes with the next hop set to VGW-A. The destination of one route is 172.16.1.0/24, which is the on-premises network CIDR block. The destination of the other route is 10.0.0.0/30, which is used to assign gateway addresses to virtual interface VIF-A.
    • The next hops of the routes destined for 172.16.1.0/24 are VGW-A and VGW-B. The two routes are equal-cost routes for load balancing. Traffic is sent over the connection selected based on the hash algorithm. In this example, connection DC-A with global DC gateway VGW-A is selected.
  3. Virtual interface VIF-A is connected to virtual gateway VGW-A. Traffic from the virtual gateway is forwarded to the connection through the remote gateway of the virtual interface.
  4. Traffic is forwarded to the on-premises data center over connection DC-A.

Response traffic: from the on-premises data center to VPC-A
  1. Traffic is forwarded to virtual interface VIF-B over connection DC-B.

    In the on-premises data center, there are also two equal-cost routes that point to the cloud and are used for load balancing. Traffic is returned over a connection selected by the hash algorithm. In this example, DC-B with virtual gateway VGW-B is selected.

  2. Virtual interface VIF-B is associated with the virtual gateway VGW-B. Traffic from the virtual interface is forwarded to the virtual gateway through the local gateway of the virtual interface.
  3. Traffic is forwarded from virtual gateway VGW-B attachment to the enterprise router.
  4. In the route table of the enterprise router, there is a route with the next hop set to the VPC-A attachment to forward traffic from the enterprise router to VPC-A.
Table 2 Network planning using two Direct Connect connections

Cloud Service/Resource

Description

VPC
A VPC is used to run your workloads and needs to be attached to the enterprise router.
  • The CIDR blocks of the VPC and of the on-premises data center cannot overlap.
  • The VPC has a default route table.
  • Table 3 lists the routes in the default VPC route table.
    • Three routes to fixed CIDR blocks: 10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16. If Auto Add Routes is enabled when the VPC is attached to the enterprise router, static routes will be automatically configured in the VPC route table. If more than one VPC is attached to an enterprise router, traffic from one VPC to the other VPCs can be forwarded to the enterprise router over these routes, and is then to the next-hop network instance through the enterprise router.
    • A route to the on-premises network: In addition to the automatically-added routes to the three VPC CIDR blocks, you need to add a route whose destination is the on-premises network CIDR block (172.16.1.0/24 in this example) and next hop is the enterprise router in the VPC route table. Traffic from the VPC is forwarded to the enterprise router and then to the next-hop network instance through the enterprise router.
    NOTICE:

    If an existing route in the VPC route table has a destination to 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16, the route that points to each CIDR block will fail to be added. In this case, do not enable Auto Add Routes. After the attachment is created, manually add the routes.

Direct Connect

Two connections work in load balancing mode.

  • Both connections link your on-premises data center to the cloud.
  • Each connection has a virtual gateway associated, and both virtual gateways are attached to the enterprise router.
  • A virtual interface is required for connecting each virtual gateway to the connection. The two virtual interfaces work in load balancing mode.

Enterprise Router
After Default Route Table Association and Default Route Table Propagation are enabled and an attachment is created, Enterprise Router will automatically:
  • VPC
    • Associate the VPC attachment with the default route table of the enterprise router.
    • Propagate the VPC attachment to the default route table of the enterprise router. The route table automatically learns the VPC CIDR block as the destination of the route. For details, see Table 4.
  • Direct Connect
    • Associate the two virtual gateway attachments with the default route table of the enterprise router.
    • Propagate the virtual gateway attachments to the default route table of the enterprise router. The route table automatically learns the route information of the virtual gateway attachments. For details, see Table 4.

Route policy
  • If the on-premises BGP routes learned by the enterprise router through two virtual gateway attachments are equal-cost routes, load balancing is automatically implemented, and you do not need to create a route policy.

    In this example, the routes with 172.16.1.0/24 as the destination and VGW-A and VGW-B as the next hops are equal-cost routes.

  • If the on-premises BGP routes learned by the enterprise router through two virtual gateway attachments are not equal-cost routes, load balancing cannot be implemented. In this case, you need to associate a route policy with the propagation of the two virtual gateway attachments. After the AS_Path are replaced, the routes from the enterprise router to the on-premises data center through the two virtual gateways will work as equal-cost routes.

    For this to work, you need to add two nodes to the route policy:

    • Node 1 has a higher priority and matches BGP routes. The AS_Path of matched BGP routes is replaced with the BGP ASN of the virtual gateways.
    • Node 2 has a lower priority and matches all routes, ensuring normal communication through non-BGP routes.

    For details, see Route Policies.

    NOTICE:

    Replace the AS_Path of the routes. The same AS_Path may cause network loops. Before configuring a route policy, check your network plan.

ECS

An ECS is deployed in the VPC to verify communications between the cloud and the on-premises data center.

If you have multiple ECSs that are associated with different security groups, you need to add rules to the security groups to allow network access.

On-premises data center

Two equal-cost routes from the on-premises data center to the enterprise router for load balancing.
Table 3 VPC route table

Destination

Next Hop

Route Type

Fixed CIDR block: 10.0.0.0/8

Enterprise router

Static route (custom)

Fixed CIDR block: 172.16.0.0/12

Enterprise router

Static route (custom)

Fixed CIDR block: 192.168.0.0/16

Enterprise router

Static route (custom)

On-premises network CIDR block: 172.16.1.0/24

Enterprise router

Static route (custom)
Table 4 Enterprise router route table

Destination

Next Hop

Route Type

VPC-A CIDR block: 192.168.0.0/16

VPC-A attachment: er-attach-vpc-A

Propagated

VIF-A gateway: 10.0.0.0/30

VGW-A attachment: er-attach-vgw-A

Propagated

VIF-B gateway: 10.1.0.0/30

VGW-B attachment: er-attach-vgw-B

Propagated

On-premises network CIDR block: 172.16.1.0/24
Two equal-cost routes for the two connections to work in load balancing mode:
  • VGW-A attachment: er-attach-vgw-A
  • VGW-B attachment: er-attach-vgw-B

Propagated

Resource Planning

One enterprise router, two Direct Connect connections, one VPC, and one ECS are in the same region but can be in different AZs.

The following resource details are only examples. You can modify them if needed.

Table 5 Details of required resources

Resource

Quantity

Description

VPC

1
A VPC is required to run your workloads and needs to be attached to the enterprise router.
  • VPC name: Set it based on site requirements. In this example, VPC-A is used.
  • VPC IPv4 CIDR block: The CIDR block must be different from that of the on-premises data center. Set it based on site requirements. In this example, 192.168.0.0/16 is used.
  • Subnet name: Set it based on site requirements. In this example, subnet-A01 is used.
  • Subnet IPv4 CIDR block: The CIDR block must be different from the on-premises network CIDR block. Set it based on site requirements. In this example, 192.168.0.0/24 is used.

Enterprise Router

1
  • Name: Set it based on site requirements. In this example, ER-X is used.
  • ASN: Set an ASN that is different from that of the on-premises data center. In this example, the ASN is 64512.
  • Default Route Table Association: Enable
  • Default Route Table Propagation: Enable
  • Auto Accept Shared Attachments: Set it based on site requirements. In this example, enable this option.
  • Three attachments on the enterprise router:
    • VPC-A attachment: er-attach-vpc-A
    • VGW-A attachment: er-attach-vgw-A
    • VGW-B attachment: er-attach-vgw-B

Route policy

1

If the on-premises BGP routes learned by the enterprise router through two virtual gateway attachments are not equal-cost routes, load balancing cannot be implemented. If this happens, you need to configure a route policy to associate it with two virtual gateway attachments.

For this to work, you need to add two nodes to the route policy:
  • Node 1 has a higher priority. The AS_Path of BGP routes is replaced, so the routes learned by the enterprise router through the two virtual gateway attachments can work as equal-cost routes.
    • Node Number: A node with a smaller node number is executed first. The node number of node 1 must be smaller than that of node 2. Set it to 10.
    • Action: Set it to Allow.
    • Match Condition: Select Route type and then BGP.
    • Policy Value 1: Select AS_Path.
    • Action: Select Replace. The value of Replace must be the same as the BGP ASN of the virtual gateways. In this example, the value is 64513.
  • Node 2 has a lower priority and matches all routes, ensuring normal communication through non-BGP routes.
    • Node Number: Set a value greater than that of node 1. In this example, set it to 20.
    • Action: Set it to Allow.

    Leave other parameters blank, indicating that other routes that do not match node 1 can match node 2. This ensures that the route policy allows all routes.

Direct Connect

2

Two connections are required.

In this example, the two connections are DC-A and DC-B.
Two virtual gateways are required.
  • Name: Set it based on site requirements. In this example, VGW-A and VGW-B are used.
  • Associate With: Select Enterprise Router.
  • Enterprise Router: Select your enterprise router. In this example, ER-X is used.
  • BGP ASN: The ASN of the two virtual gateways must be the same and can be the same as or different from that of the enterprise router. In this example, the ASN of the two virtual gateways is 64513.
Two virtual interfaces are required.
  • Name: In this example, the two virtual interfaces are VIF-A and VIF-B.
  • Virtual Interface Priority: Select Preferred for both virtual interfaces, indicating that load balancing will be implemented.
  • Connection: In this example, virtual interface VIF-A is associated with connection DC-A, and virtual interface VIF-B is associated with connection DC-B.
  • Virtual Gateway: In this example, the virtual gateways associated with virtual interface VIF-A is VGW-A, and that associated with VIF-B is associated with VGW-B.
  • Local Gateway: In this example, the local gateway IP address range for virtual interface VIF-A is 10.0.0.1/30, and that for VIF-B is 10.1.0.1/30.
  • Remote Gateway: In this example, the remote gateway IP address range for virtual interface VIF-A is 10.0.0.2/30, and that for VIF-B is 10.1.0.2/30.
  • Remote Subnet: In this example, the on-premises network CIDR block is 172.16.1.0/24.
  • Routing Mode: Select BGP.
  • BGP ASN: ASN of the on-premises data center, which must be different from the ASN of the virtual gateways on the cloud. In this example, 64555 is used.

ECS

1

An ECS is required to verify connectivity.

  • ECS Name: Set it based on site requirements. In this example, ecs-A is used.
  • Image: Select an image based on site requirements. In this example, a public image (CentOS 8.2 64bit) is used.
  • Network
    • VPC: Select the service VPC. In this example, select VPC-A.
    • Subnet: Select the subnet that communicates with the on-premises data center. In this example, the subnet is subnet-A01.
  • Security Group: Select a security group based on site requirements. In this example, the security group sg-demo uses a general-purpose web server template.
  • Private IP address: 192.168.0.137
  • The two Direct Connect connections work in load balancing mode. To prevent network loops and form equal-cost routes, the ASN of the two virtual gateways must be the same. In this example, the ASN is 64513.
  • The ASN of the enterprise router can be the same as or different from that of the virtual gateways. In this example, 64512 is used.
  • The ASN of the on-premises data center must be different from that used on the cloud. Set this ASN of the on-premises data center based on site requirements. In this example, 64555 is used.