Virtual Node Configurations

Description

A cluster can contain one global virtual node and multiple AZ-level virtual nodes. Pods for the workloads scheduled to the global virtual node are randomly scheduled to any supported AZ for CCI, and pods for the workloads scheduled to an AZ-level virtual node are only scheduled to the corresponding AZ for CCI. If you do not want random scheduling on CCI, you need to manually set the global node (bursting-node) to the unschedulable state.
An AZ-level virtual node has two labels: failure-domain.beta.kubernetes.io/zone and topology.kubernetes.io/zone. The value of each label is the AZ name, based on which pods can be scheduled.
The global virtual node cannot be deleted. AZ-level nodes can be dynamically added or deleted based on the ConfigMap configuration.
vCPU and memory capacities and allocatable vCPUs and memory on the nodes can be configured.
From v1.5.29 of the add-on, CCI sets the corresponding virtual node to be unschedulable when an AZ becomes faulty. You can view the status.conditions information of the virtual node to check the faulty AZ and fault cause. When CCI sets the virtual node to be unschedulable, your existing services may be affected. New services should not be scheduled to the virtual node.

Constraints

If a key does not contain a period (.), the key name is the AZ name (the key cannot be set to global, which indicates a global node). If the key is set to true, the virtual node in the corresponding AZ is enabled.
If the key contains periods (.), the part before the first period is the AZ name, and the part after the first period indicates the node parameters. Currently, cpu and memory are supported. The values are the same as those for Kubernetes resources (in units such as k, Mi, and Gi). cpu and memory are optional. If they are not configured, the default values (cpu: 200000; memory: 1600000Gi) are used.
The key can contain only lowercase letters and must comply with the RFC 1123 subdomain format (same as the node naming rule). The AZ name can contain a maximum of 47 characters.
A maximum of 20 AZ-level virtual nodes are supported.
The default AZ fault detection period is 5 minutes. You can set the startup parameter --az-status-sync-period of resource-syncer to specify the detection period. The minimum period is 1 minute.

Example Configuration

Configure the following ConfigMap in the kube-system namespace:

apiVersion: v1
kind: ConfigMap
metadata:
  name: bursting-node
  namespace: kube-system
data:
  global.cpu: 2k          # global indicates the global virtual node, which is created by default and cannot be deleted. You can configure the available vCPUs and memory of the virtual node. If the available vCPUs and memory are not configured, the default values (cpu: 200000, memory: 1600000Gi) are used.
  global.memory: 4000Gi
  region-a: "true"      # If a ConfigMap is added and its key is the AZ name and its value set to true, virtual nodes in the corresponding AZ are automatically created.
  region-a.cpu: 5k     # cpu and memory are optional. If they are not configured, the default values (cpu: 200000, memory: 1600000Gi) are used.
  region-a.memory: 8000Gi
  region-b: "true"

The cluster is expected to have three nodes: