Resource Data Volume of Different Cluster Sizes
As the service scale expands, the maximum number of nodes in a Kubernetes cluster directly affects service scalability and flexibility. When more services or pods are deployed in an existing cluster, cluster resources may become insufficient. As a result, service deployment may fail or performance may deteriorate. The resource data volume of different cluster sizes may be limited. You can select cluster sizes that match the service scale based on Table 1 to meet the requirements of large-scale service deployment and improve resource utilization and service capacity of clusters.
| Resource Type (Theoretical Upper Limit) | 50 Nodes | 200 Nodes | 1,000 Nodes | 2,000 Nodes |
|---|---|---|---|---|
| Total etcd storage capacity | 200 MiB | 500 MiB | 2 GiB | 4 GiB |
| Total size of etcd objects of each resource type | 20 MiB | 50 MiB | 200 MiB | 400 MiB |
| Number of nodes | 50 | 200 | 1,000 | 2,000 |
| Pod (The average pod size cannot exceed 15 KB, and the number of credential-handling containers in a single pod cannot exceed 2.) | 1,000 No more than 2,000 secrets or ConfigMaps can be mounted to pods. | 4,000 No more than 8,000 secrets or ConfigMaps can be mounted to pods. | 20,000 No more than 40,000 secrets or ConfigMaps can be mounted to pods. | 40,000 No more than 80,000 secrets or ConfigMaps can be mounted to pods. |
| Number of pending pods in a cluster | 100 | 400 | 2,000 | 4,000 |
| Number of pods that can be concurrently created or deleted (The average pod size cannot exceed 15 KB, and the number of credential-handling containers in a single pod cannot exceed 2.) | 100 | 400 | 2,000 | 4,000 |
| Total HPA/CronHPA policies | 20 | 40 | 200 | 400 |
| Namespace | 20 | 80 | 400 | 800 |
| ConfigMap (The average resource size cannot exceed 15 KB.) | 250 | 1,000 | 4,000 | 10,000 |
| Secret (The average resource size cannot exceed 15 KB.) | 250 | 1,000 | 4,000 | 10,000 |
| PVC | 500 | 2,000 | 10,000 | 20,000 |
| PV | 500 | 2,000 | 10,000 | 20,000 |
| Service |
|
|
|
|
| Role | 250 | 1,000 | 4,000 | 20,000 |
| RoleBinding | 250 | 1,000 | 4,000 | 20,000 |
| CRD | 40 | 80 | 200 | 400 |
| Total number of CRs of all CRD types (The average CR size cannot exceed 15 KB.) | 2,000 | 4,000 | 20,000 | 40,000 |
| Maximum QPS of all update operations (POST, PUT, PATCH, and DELETE) in a cluster (The average size of to-be-updated objects is estimated to be no more than 15 KB.) | 50 QPS | 100 QPS | 200 QPS | 400 QPS |
| Maximum QPS of all read-only requests (GET, List, and Watch) in a cluster and the total amount of data read per second (Clusters v1.31 and later must support streaming output.) | 100 QPS 50 MiB/s | 200 QPS 100 MiB/s | 400 QPS 200 MiB/s | 800 QPS 300 MiB/s |
| Maximum QPS of GET requests for all custom CRs (15 KB per CR) | 5 QPS | 5 QPS | 20 QPS | 20 QPS |
| Number of persistent watch stream connections for user components to access kube-apiserver | 50 | 200 | 1,000 | 2,000 |
| Number of concurrent List requests to large data volume (Clusters v1.31 or later must support streaming output.) | Average data volume returned by List operations at a time: 50 MiB | Average data volume returned by List operations at a time: 100 MiB | Average data volume returned by List operations at a time: 200 MiB | Average data volume returned by List operations at a time: 300 MiB |
- Except for the number of nodes, other resources are recommended limits. If the cluster usage exceeds the specifications, the cluster may be overloaded. You need to select a cluster of the corresponding size based on your resource usage.
- The resource data volume refers to the size of resource data in JSON format queried through the API, including the managedFields field. (You can run kubectl get {resourceType} {resourceName} -ojson -n {namespace} --show-managed-fields > tmp.json to obtain the size of a single resource. The size of the tmp.json file is the resource size.) For example, if the size of a ConfigMap is 10 KB and there are 10,000 ConfigMaps in a cluster, the total size of ConfigMap data is about 100 MiB.
- The quantity of each resource is calculated based on the average size of 15 KB. If the average size of resources is greater than 15 KB, the supported data volume decreases. For example, a large-size cluster (1,000 nodes) supports 20,000 pods (with average size less than 15 KB) by default. If the average pod size is 30 KB, the maximum number of pods is reduced to 10,000.
The same calculation method applies to the request QPS. For example, if the QPS of update operations on resources with the average size less than 15 KB is 200 for a large-size cluster (1,000 nodes), and the average size of resources to be updated is 30 KB, the actual QPS decreases to 100.
- The resource size in an update request refers to the resource size, not the size of the request body.
What is your overall rating for this page?
Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot