Comparing iptables, nftables, and IPVS
kube-proxy is a key component of a Kubernetes cluster. It is used for load balancing and forwarding data between a Service and its backend pods.
CCE supports the following Service forwarding modes: iptables, nftables (v1.35 and later), and IPVS. The table below summarizes the key characteristics of each option.
| Feature Difference | nftables | iptables | IPVS |
|---|---|---|---|
| Positioning | A high-performance kube-proxy mode that eliminates iptables' performance bottlenecks and scalability limitations at scale. It provides a unified configuration framework for multi-protocol rule management. For details, see nftables. | A mature, stable kube-proxy mode, but its performance is average. It applies to scenarios where the number of Services is small (less than 3,000) or there are a large number of short concurrent connections on the client. For details, see iptables. | A high-performance kube-proxy mode. This mode is suitable for large cluster scales or when there are a large number of Services. For details, see IPVS. |
| Throughput | Relatively high | Relatively low | Relatively high |
| Complexity | O(1) or O(log n). Rule processing overhead typically remains constant regardless of cluster size. | O(n). n increases with the number of Services and backend pods in the cluster. | O(1). In most cases, the connection processing efficiency is irrelevant to the cluster scale. |
| Load balancing algorithm | Random, round-robin | Random, round-robin | Round-robin, shortest expected delay, least connection, and various hashing methods |
| ClusterIP connectivity | ClusterIP addresses cannot be pinged. | ClusterIP addresses cannot be pinged. | ClusterIP addresses can be pinged. NOTE: The IP address in clusters of v1.27 or later cannot be pinged due to security hardening. |
| Additional restrictions | Available in CCE clusters v1.35 and later. | When there are more than 3000 Services in a cluster, network delay may occur. |
|
nftables
nftables is a high-performance packet filtering and management framework built into the Linux kernel, designed to replace iptables. It eliminates performance bottlenecks and scalability limitations while providing more efficient and flexible network traffic control. Built on the Netfilter hook mechanism, nftables uses a unified configuration framework to integrate rule management across multiple protocols (IPv4, IPv6, ARP, and others), eliminating the need for separate per-protocol rule sets. It also optimizes rule matching through hash-based data structures, improving efficiency in high-concurrency scenarios.
In nftables mode, kube-proxy uses nftables instead of iptables to configure packet forwarding and NAT rules. Compared with iptables, nftables offers simpler syntax and more flexible rule organization. It supports incremental rule updates by modifying individual rules without replacing the entire rule set, thereby optimizing rule management performance in large-scale service scenarios and better accommodating Kubernetes cluster growth. For details, see nftables proxy mode.
To enable kube-proxy to use nftables for packet forwarding, nodes must run kernel version 5.13 or later and nftables version 1.0.1 or later.
iptables
iptables is a Linux kernel function for processing and filtering a large number of data packets. It allows flexible sequences of rules to be attached to various hooks in the packet processing pipeline. When iptables is used, kube-proxy implements NAT and load balancing in the NAT pre-routing hook. For each Service, kube-proxy installs an iptables rule which captures the traffic destined for the Service's ClusterIP and ports and redirects the traffic to one of the backend pods. By default, iptables randomly selects a backend pod. For details, see iptables proxy mode.
IPVS
IPVS is constructed on top of Netfilter and balances transport-layer loads as part of the Linux kernel. IPVS can direct requests for TCP- or UDP-based services to the real servers, and make services of the real servers appear as virtual services on a single IP address.
In the IPVS mode, kube-proxy uses IPVS load balancing instead of iptables. IPVS is designed to balance loads for a large number of Services. It has a set of optimized APIs and uses optimized search algorithms instead of simply searching for rules from a list. For details, see IPVS proxy mode.
Helpful Links
- To create a Service, see Service Overview.
- If a node's kernel version is earlier than 5.9 and a CCE cluster accesses an internal service through a Service in IPVS mode, there may be a 1-second latency or access failure after the backend is upgraded. For details, see What Should I Do If There Is a Service Access Failure After a Backend Service Upgrade or a 1-Second Latency When a Service Accesses a CCE Cluster?
- In a cluster running in IPVS mode, packets may be lost after CoreDNS is upgraded on a node, causing a domain name resolution failure. To resolve this issue, see What Should I Do If a DNS Resolution Failure Occurs Due to a Defect in IPVS?
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot