Planning CIDR Blocks for a Cluster

• VPC CIDR Block

  A VPC enables you to provision logically isolated, configurable, and manageable virtual networks for cloud servers, cloud containers, and cloud databases. You can configure CIDR blocks, subnets, security groups, and more in a VPC, and assign EIPs and bandwidth to your service systems.

• Subnet CIDR Block
  A subnet manages ECS network planes. It supports IP address management and DNS. ECSs in a subnet are assigned IP addresses from this subnet.

  Figure 1 VPC CIDR block architecture
  

  By default, ECSs in all subnets of the same VPC can communicate with one another, but ECSs in different VPCs cannot.

  You can create a VPC peering connection to enable ECSs in different VPCs to communicate with each other.

• Container (Pod) CIDR Block

  Pod is a Kubernetes concept. Each pod has an IP address.

  When creating a cluster on CCE, you can specify the pod (container) CIDR block. The container CIDR block cannot overlap with the VPC CIDR block or subnet CIDR block. For example, if the subnet CIDR block is 192.168.0.0/16, the container CIDR block of the cluster cannot be 192.168.0.0/18 or 192.168.64.0/18 because these CIDR blocks are covered by 192.168.0.0/16.

• Container Subnet (Only for CCE Turbo Clusters)

  In a CCE Turbo cluster, a container is assigned an IP address from the VPC CIDR block. The container subnet can overlap with the subnet CIDR block. Note that the subnet you select determines the maximum number of pods in the cluster.

• Service CIDR Block

  Service is also a Kubernetes concept. Each Service has an IP address. When creating a cluster on CCE, you can specify the Service CIDR block. Similarly, the Service CIDR block cannot overlap with the VPC CIDR block, subnet CIDR block, or container CIDR block. The Service CIDR block can be used only within a cluster.

Clusters Using the VPC Networks

Pod packets are forwarded through VPC routes. CCE automatically configures a route table on the VPC routes for each container CIDR block. The network scale is limited by the VPC route table. Figure 4 shows the CIDR block planning of such a cluster.

• A VPC CIDR block is where the cluster resides. The size of this CIDR block affects the maximum number of nodes that can be created in the cluster.
• A subnet CIDR block cannot overlap with any container CIDR block.
• If there are multiple clusters that use the VPC networks in a single VPC, the container CIDR blocks of all clusters cannot overlap with each other because these clusters use the same route table. In this case, if the node security group in a cluster allows the container CIDR block from the peer cluster, pods in the cluster can directly access pods in the peer cluster through the pod IP addresses.
• A Service CIDR block can be used only in clusters. Therefore, the Service CIDR blocks of different clusters can overlap with each other, but cannot overlap with the subnet CIDR block or container CIDR block of other clusters.

Figure 4 Network planning for multiple clusters using the VPC networks

Clusters Using the Tunnel Networks

In a container tunnel network, the container network is an overlay network built on top of the VPC network. Although there is a small amount of performance overhead due to tunnel encapsulation, this kind of network offers strong generality, excellent interoperability, and comprehensive support for advanced features such as network policy-based isolation, making it suitable for most application scenarios. Figure 5 shows the CIDR block planning of the cluster.

• A VPC CIDR block is where the cluster resides. The size of this CIDR block affects the maximum number of nodes that can be created in the cluster.
• A subnet CIDR block cannot overlap with any container CIDR block.
• The container CIDR blocks of all clusters can overlap with each other. In this case, pods in different clusters cannot be directly accessed through pod IP addresses. Services are needed for accessing pods in different clusters. The LoadBalancer Services are recommended.
• A Service CIDR block can be used only in clusters. Therefore, the Service CIDR blocks of different clusters can overlap with each other, but cannot overlap with the subnet CIDR block or container CIDR block of other clusters.

Figure 5 Network planning for multiple clusters using the tunnel networks

CCE Turbo Clusters Using the Cloud Native 2.0 Networks

In a Cloud Native 2.0 network, container IP addresses are allocated from the VPC CIDR block. Clusters using this kind of network support multiple VPC-based capabilities, such as ELB passthrough networking to pods and direct association of security groups with containers. They can deliver faster network connection and forwarding.

• A VPC CIDR block is where a cluster resides. In a CCE Turbo cluster, the CIDR block size affects the total number of nodes and containers that can be created in the cluster.
• There is no special restriction on the subnet CIDR blocks in CCE Turbo clusters.
• A container subnet CIDR block is included in the VPC CIDR block. CIDR blocks of container subnets in different clusters can overlap with each other or with the subnet CIDR block. However, you are advised to use different container CIDR blocks for different clusters and ensure that the container subnet CIDR blocks have sufficient IP addresses. In this case, if the network interface security group of a cluster allows the container CIDR block of the peer cluster, pods in these clusters can directly access each other through the IP addresses.
• A Service CIDR block can be used only in clusters. Therefore, the Service CIDR blocks of different clusters can overlap with each other, but cannot overlap with the subnet CIDR block or container subnet CIDR block of the clusters.

Figure 6 Network planning for multiple clusters using the Cloud Native 2.0 networks

Clusters Using Different Networks

For a VPC that contains clusters using different networks, comply with the following rules when creating a cluster:

• All these clusters are in the same VPC CIDR block. Ensure that there are sufficient available IP addresses in the VPC.
• A subnet CIDR block cannot overlap with any container CIDR block. Even in some scenarios, for example, where there are also some CCE Turbo clusters, the subnet CIDR block can overlap with the container (subnet) CIDR block. However, this is not recommended.
• The container CIDR blocks of the clusters that use the VPC networks cannot overlap with each other.
• The Service CIDR blocks of all these clusters can overlap with each other, but cannot overlap with any subnet CIDR block or container CIDR block of the clusters.

Different VPCs cannot communicate with each other. A VPC peering connection can connect VPCs in the same region. For two VPC networks that are connected, you can use route tables to configure the packets to be sent to the peer VPC. For details, see VPC Peering Connection Overview.

Clusters Using the VPC Networks

To allow clusters that use the VPC networks to access each other across VPCs, add routes to both ends of the VPC peering after a VPC peering connection is created.

Figure 7 Network planning for clusters in different VPCs

When creating a VPC peering connection for clusters in different VPCs, pay attention to the following points:

• The VPCs to which the clusters belong must not overlap with each other. In each cluster, the subnet CIDR block cannot overlap with the container CIDR block.
• The container CIDR blocks of the clusters at both ends cannot overlap with each other, but the Service CIDR blocks can.
• If the request end cluster uses the VPC network, check whether the node security group in the peer cluster allows the container CIDR block of the request end cluster. If yes, pods in these clusters can directly access each other through the pod IP addresses. Similarly, if nodes running in the clusters at the both ends of the VPC peering connection need to access each other, the node security group must allow the VPC CIDR block of the peer cluster.
• Both VPC route tables must include routes to the peer CIDR block. For example, the route table of VPC 1 must include a route to the VPC 2 CIDR block, and likewise, the route table of VPC 2 must include a route to the VPC 1 CIDR block.
  • Add the peer cluster's VPC CIDR block:: After adding the route of the VPC CIDR block, pods can access nodes in the other cluster, such as reaching a NodePort Service.
  • Add the peer cluster's container CIDR block:: After adding the route of the container CIDR block, pods can directly access pods in the other cluster using their IP addresses.
  Figure 8 Adding the peer cluster's container CIDR block to the local route
  

Clusters Using the Tunnel Networks

To allow clusters that use the tunnel networks to access each other across VPCs, add routes to both ends of the VPC peering after a VPC peering connection is created.

Figure 9 Network planning for clusters using the tunnel networks

Pay attention to the following:

• The VPC CIDR blocks of the clusters at the two ends must not overlap with each other.
• The container CIDR blocks of the clusters can overlap with each other, so do the Service CIDR blocks.
• If the request end cluster uses the tunnel network, check whether the node security group in the destination cluster allows the VPC CIDR block (including the node subnets) of the request end cluster. If yes, nodes in both clusters can access each other. However, pods in these clusters cannot directly access each other using pod IP addresses. To achieve this, Services are needed. The LoadBalancer Services are recommended.
• Both VPC route tables must include routes to the peer cluster's VPC CIDR block. For example, the route table of VPC 1 must include a route to the VPC 2 CIDR block, and likewise, the route table of VPC 2 must include a route to the VPC 1 CIDR block. After the routes of the VPC CIDR blocks are added, pods can access nodes in the other cluster, such as reaching a NodePort Service.
  Figure 10 Adding the node subnet CIDR block of the peer cluster to the local route
  

CCE Turbo Clusters Using the Cloud Native 2.0 Networks

Clusters Using Different Networks

If clusters using different networks need to communicate with each other across VPCs, every one of them may serve as the request end or destination end. Pay attention to the following:

• The VPC CIDR block of a cluster cannot overlap with the VPC CIDR block of the other cluster.
• The subnet CIDR blocks cannot overlap with any of the container CIDR blocks.
• The container CIDR blocks in these clusters cannot overlap with each other.
• If pods or nodes in these clusters need to access each other, the security groups in the clusters on both ends must allow the corresponding CIDR blocks based on the following rules:
  • If the request end cluster uses a VPC network, the node security group of the destination cluster must allow the VPC CIDR block (including the node subnets and container CIDR block) of the request end cluster.
  • If the request end cluster uses a tunnel network, the node security group of the destination cluster must allow the VPC CIDR block (including the node subnets) of the request end cluster.
  • If the request end cluster uses a Cloud Native 2.0 network, the network interface security group and node security group of the destination cluster must allow the VPC CIDR block (including node subnets and container CIDR block) of the request end cluster.
• Both VPC route tables must include routes to the peer cluster's VPC CIDR block. For example, the route table of VPC 1 must include a route to the VPC 2 CIDR block, and likewise, the route table of VPC 2 must include a route to the VPC 1 CIDR block. After the routes of the VPC CIDR blocks are added, pods can access nodes in the other cluster, such as reaching a NodePort Service.

  If one of the clusters uses a VPC network, both VPC route tables must also include the container CIDR blocks. After the routes of the container CIDR blocks are added, pods can directly access pods in the other cluster using their IP addresses.

CCE clusters can be used with other cloud services, such as RDS and Kafka. To ensure stable communication and avoid network conflicts, a cluster's network must be carefully planned. Specifically, the CIDR blocks used by other services must not overlap with those used by the cluster. The following requirements apply based on the cluster's network:

• For a cluster that uses a VPC network: The CIDR blocks of other cloud services must not overlap with the container CIDR block or Service CIDR block within the cluster.
• For a cluster that uses a tunnel network: The CIDR blocks of other cloud services must not overlap with the container CIDR block or Service CIDR block within the cluster.
• For a CCE Turbo cluster that uses a Cloud Native 2.0 network: The CIDR blocks of other cloud services must not overlap with the Service CIDR block within the cluster.

• In addition to the CIDR block planning, it is essential to verify whether the target cloud service supports external access and whether the relevant security group rules or ACL configurations permit communication. For details, see Accessing Cloud Services from a Pod in the Same VPC or Accessing Cloud Services from a Pod in a Different VPC.
• If the CCE cluster and the cloud services reside in different VPCs, their VPC CIDR blocks must remain non-overlapping. To enable communication between the cluster and external services across VPCs, a VPC peering connection can be established.

Similar to the VPC connection scenario, some CIDR blocks in the VPC may be routed to an IDC, so the pod IP addresses of the CCE cluster must not overlap with those CIDR blocks. If the IDC needs to access pods in the cluster, corresponding routes to the private line must also be configured in the IDC route table.