CCE Advanced Cluster Autoscaler

Introduction

CCE Advanced Cluster Autoscaler (ACA) is a predictive cluster autoscaling add-on that extends the upstream Kubernetes Cluster Autoscaler. It integrates with AASP, a flexible resource forecasting service that uses modeling to anticipate cluster-wide load changes and execute proactive capacity adjustments. ACA is optimized for workloads with predictable periodic patterns.

AASP leverages Transformer-based deep learning models to analyze historical workload metrics and real-time monitoring data, generating forward-looking compute demand forecasts that drive cluster-level elastic scheduling.

AASP must be enabled via customer support before ACA can be deployed.

How It Works

ACA manages both scale-out and scale-in operations.

• Intelligent automatic scale-out: Dual-mode triggering
  • Predictive scale-out: When AASP forecasts that the cluster will require more nodes than are currently available, ACA provisions those additional nodes ahead of demand. This eliminates the minute-level delay typically associated with cloud server initialization.
  • Reactive scale-out: If an unexpected traffic burst causes pods to enter a pending state due to insufficient worker node resources, ACA falls back to upstream Cluster Autoscaler behavior to trigger reactive scaling. This provisions nodes that match the resource profile of the target node pool.
• Smooth auto scale-in

  To determine the appropriate cluster size during scale-in, ACA evaluates both the aggregate resource requirements of all running pods and the node count predicted by AASP. It then scales in gradually to whichever of these two values is larger. This conservative approach buffers the cluster against transient traffic spikes and prevents premature node removal that could destabilize running workloads.

Notes and Constraints

• This add-on is incompatible with CCE Cluster Autoscaler. Do not enable both simultaneously.
• AOM must have at least two weeks of metrics history before you can use this add-on.
• Each node pool supports only one CPU/memory flavor (for example, 4 vCPUs and 8 GiB memory). Only one node pool can be created per cluster.
• Ensure the created nodes can accept pod scheduling.
• This add-on supports only VM nodes. PM and BMS nodes are not supported.
• The default node pool does not support autoscaling. For details, see Description of DefaultPool.
• Node scale-in causes irreversible data loss for PVCs and PVs of the local PV type. These PVCs and PVs cannot be restored or reused. When a node is scaled in, any pod using a local PV is evicted. A replacement pod is created but remains in pending state because the PVC retains a node affinity constraint that prevents scheduling to a different node.
• Advanced Cluster Autoscaler requires CCE clusters v1.33 or later.

Installing the Add-on

1. Create an agency for the add-on.
   1. Log in to the IAM console.
   2. In the navigation pane, choose Agencies. Then, click Create Agency in the upper right corner.
   3. Set Agency Type to Cloud service and choose Cloud Container Engine (CCE).
   4. Click OK. In the dialog box displayed, click Authorize.
   5. Click Create Policy in the upper right corner. The policy content is as follows:

      {
          "Version": "1.1",
          "Statement": [
              {
                  "Action": [
                      "cce:node:list",
                      "cce:node:delete",
                      "cce:nodepool:update",
                      "cce:nodepool:list",
                      "cce:nodepool:get",
                      "cce:cluster:get",
                      "cce:nodepool:scale",
                      "ecs:cloudServerFlavors:get",
                      "ecs:flavors:get",
                      "ecs:flavors:get",
                      "evs:types:get"
                  ],
                  "Effect": "Allow"
              }
          ]
      }

   6. Select the newly created policy for authorization.

2. Log in to the CCE console and click the cluster name to access the cluster console.
3. In the navigation pane, choose Add-ons. On the displayed page, locate Advanced Cluster Autoscaler and click Install.
4. On the add-on installation page, select the number of pods and resource quotas as needed.

   Recommended memory request and limit:

   • Memory request = (Number of pods × Pod YAML file size in KB/10000) × 0.28 GiB + 1 GiB
   • Memory limit = Memory request + 2 GiB

   For example, there are 20,000 pods with a 10 KB YAML file size per pod, the memory request would be 6.6 GiB (2 × 10 × 0.28 GiB + 1 GiB) and the memory limit would be 8.6 GiB (6.6 GiB + 2 GiB). (Calculated values may differ from the recommendations in Table 1. Use either the table or these formulas as your reference.)

   Table 1 Recommended add-on memory sizes in large-scale scenarios

   Pods (10 KB for Each Pod YAML)	Recommended Memory Request	Recommended Memory Limit
   10000	4 GiB	6 GiB
   30000	8 GiB	10 GiB
   50000	16 GiB	18 GiB
   80000	24 GiB	26 GiB
   100000	28 GiB	30 GiB

5. Configure parameters. The following parameters are mandatory:
   • agencyName: name of the target agency. Use the agency created in step 1.
   • clusterId: cluster ID
   • instanceId: AASP service instance ID
   • nodepoolName: node pool name
   • tenantId: project ID
   • stabilizationWindowSeconds: the stabilization window, which is the recommended interval between node count updates. The default value is 300.
   • minPoolSize: the minimum number of nodes in a node pool. The default value is 0.
   • maxPoolSize: the maximum number of nodes in a node pool. The default value is 50.

6. Configure deployment policies for the add-on pods.
   

   • Scheduling policies do not take effect on the DaemonSet pods of the add-on.
   • When configuring multi-AZ deployment or node affinity, ensure that there are nodes meeting the scheduling policy and that resources are sufficient in the cluster. Otherwise, the add-on pods cannot run.

   Table 2 Configurations for add-on scheduling

   Parameter	Description
   Multi-AZ Deployment	Preferred: Deployment pods of the add-on will be preferentially scheduled to nodes in different AZs. If all the nodes in the cluster are deployed in the same AZ, the pods will be scheduled to different nodes in that AZ. Equivalent mode: Deployment pods of the add-on are evenly scheduled to the nodes in the cluster in each AZ. If a new AZ is added, you are advised to increase add-on pods for cross-AZ HA deployment. With the Equivalent multi-AZ deployment, the difference between the number of add-on pods in different AZs will be less than or equal to 1. If resources in one of the AZs are insufficient, pods cannot be scheduled to that AZ. Forcible: Deployment pods of the add-on are forcibly scheduled to nodes in different AZs. There can be at most one pod in each AZ. If nodes in a cluster are not in different AZs, some add-on pods cannot run properly. If a node is faulty, the add-on pods on it may fail to be migrated.
   Node Affinity	Not configured: Node affinity is disabled for the add-on pods. Specify node: Specify the nodes where the add-on pods are deployed. If you do not specify the nodes, the add-on pods will be randomly scheduled based on the default cluster scheduling policy. Specify node pool: Specify the node pool where the add-on pods are deployed. If you do not specify the node pools, the add-on pods will be randomly scheduled based on the default cluster scheduling policy. Customize affinity: Enter the labels of the nodes where the add-on pods are to be deployed for more flexible scheduling policies. If you do not specify node labels, the add-on pods will be randomly scheduled based on the default cluster scheduling policy. If multiple custom affinity policies are configured, ensure that there are nodes that meet all the affinity policies in the cluster. Otherwise, the add-on pods cannot run.
   Toleration	Using both taints and tolerations enables (but does not require) the add-on's Deployment pods to be scheduled on nodes with matching taints, and allows control over pod eviction policies when host nodes are tainted. The add-on applies default toleration policies for the node.kubernetes.io/not-ready and node.kubernetes.io/unreachable taints on pods. The tolerance time window is 60s. For details, see Configuring Tolerance Policies.

7. After the configuration is complete, click Install.