Container Networks

Kubernetes is not responsible for network communications. It only provides the Container Networking Interface (CNI) for networking through CNI plugins. There are many open source CNI plugins, such as Flannel and Calico. CCE offers various network add-ons for clusters that use different network models, enabling seamless network communications within clusters.

Kubernetes requires that cluster networks meet the following requirements:

Pods in a cluster are accessible to each other through a non-NAT network. The source IP addresses of the data packets received by a pod are the IP addresses of the pods that the data packets were sent from.
Nodes can communicate with each other without NAT.

Pod Communication

Communications Between Pods on the Same Node

A pod communicates with external systems through virtual Ethernet (veth) pairs. A veth pair is like a network cable, with one end inside the pod and the other end outside the pod. The pods on the same node communicate with each other through a Linux bridge.

Figure 1 Communications between pods on the same node

Pods on the same node connect to the bridge through veth devices and obtain IP addresses through the bridge. These IP addresses are from the same CIDR block as the bridge IP address. Additionally, the default routes of all pods on the node point to the bridge, which forwards all traffic from the IP addresses that are not on the local network. The pods on the node can communicate with each other.

Communications Between Pods on Different Nodes

Kubernetes requires the IP address of each pod in a cluster be unique. Each node in the cluster is allocated a subnet to ensure that the IP addresses of the pods are unique within the cluster. Pods running on different nodes communicate with each other through IP addresses. This process is implemented using cluster networking plugins. Pods can communicate with each other using three types of networks: overlay, routing, and underlay:

An overlay network is set up on the node network using tunnel encapsulation. Such a network has its own IP addresses and IP switching and routing. VXLAN is a mainstream overlay network tunneling protocol.
In a routing network, a VPC route table is used with the network for communications between pods and nodes. The performance surpasses that of the overlay tunnel encapsulation.
In an underlay network, drivers expose network interfaces on nodes to pods for high-performance network communications. IP VLANs are commonly used on underlay networks.

Figure 2 Communications between pods on different nodes