Container Networking

Kubernetes is not responsible for network communication, but it provides the Container Networking Interface (CNI) for networking through CNI plug-ins. There are many open-source CNI plug-ins, such as Flannel and Calico. CCE offers various network add-ons for clusters of different network models, enabling seamless network communication within clusters.

According to Kubernetes, cluster networking must meet the following requirements:

Pods in a cluster must be accessible to each other through a non-NAT network. In this way, the source IP address of the data packets received by a pod is the IP address of the pod from which the data packets are sent.
Nodes can communicate with each other without NAT.

Pod Communication

Communication between pods on the same node

A pod communicates with external systems through veth devices created in interconnected pairs. The veth devices are virtual Ethernet devices acting as tunnels between network namespaces. The pods on the same node communicate with each other through a Linux bridge.

Figure 1 Communication between pods on the same node

The pods on the same node connect to the bridge through veth devices. The IP addresses of these pods are dynamically obtained through the bridge and belong to the same CIDR block as the bridge IP address. Additionally, the default routes of all pods on the same node point to the bridge, and the bridge forwards all traffic with the source addresses that are not of the local network. In this way, the pods on the same node can directly communicate with each other.

Communication between pods on different nodes

According to Kubernetes, the address of each pod in a cluster must be unique. Each node in the cluster is allocated with a subnet to ensure that the IP addresses of the pods are unique in the cluster. Pods running on different nodes communicate with each other through IP addresses in overlay, routing, or underlay networking mode based on the underlying dependency. This process is implemented using cluster networking plug-ins.

An overlay network is separately constructed using tunnel encapsulation on the node network. Such a network has its own IP address space and IP switching/routing. VXLAN is a mainstream overlay network tunneling protocol.
In a routing network, a VPC routing table is used with the underlying network for convenient communication between pods and nodes. The performance of routing surpasses that of the overlay tunnel encapsulation.
In an underlay network, drivers expose underlying network interfaces on nodes to pods for high-performance network communication. VLANs are commonly used in underlay networking.

Figure 2 Communication for pods on different nodes