Creating a CustomedHPA Policy
HPA can automatically adjust the number of pods based on the CPU usage and memory usage. However, as application complexity increases and service requirements diversify, enterprises and developers find that this basic policy may not meet needs in some specific scenarios. For example, the loads of some applications may change sharply in a specific period of time, or more refined control (such as the scaling step) is required based on metrics.
Enhanced based on HPA, CustomedHPA provides more flexible, diverse auto scaling capabilities to meet complex, changing service requirements. It scales Deployment pods based on metrics (CPU usage and memory usage) or periodically (daily, weekly, monthly, or yearly at a specific time).
- Scaling based on the percentage of the current number of pods: You can configure the scaling percentage to adjust the number of pods for different loads more flexibly.
- Minimum step for a scaling action: You can configure the minimum step to avoid frequent scaling adjustments, reducing system overhead and resource fluctuations.
- Different scaling actions based on actual metric values: You can define multiple metric thresholds and corresponding scaling policies, so that the system can handle different loads.
Prerequisites
To use CustomedHPA policies, you must install CCE Advanced HPA. If the add-on version is earlier than 1.2.11, prometheus must be installed. If the add-on version is 1.2.11 or later, you must install an add-on that provides the Metrics API. Select one of the following add-ons based on your cluster version and requirements:
- Kubernetes Metrics Server: provides basic resource usage metrics, such as container CPU and memory usage. The default collection period is 60s. It is supported by all cluster versions.
- Cloud Native Cluster Monitoring: To use HPA policies, enable local data storage in this add-on. This add-on is available in clusters v1.17 or later. The default collection period is 15s.
- Auto scaling based on basic resource metrics: Prometheus needs to be registered as a metrics API. For details, see Providing Basic Resource Metrics Through the Metrics API. If Kubernetes Metrics Server has been installed in the cluster, the Metrics API is provided by default. No manual registration is required.
- Auto scaling based on custom metrics: In addition to registering Prometheus as a Metrics API service, you need to aggregate custom metrics to the Kubernetes API server. For details, see Creating an HPA Policy Using Custom Metrics.
- Prometheus (EOM): If Kubernetes Metrics Server is not installed in the cluster, you need to manually create the Metrics API for Prometheus. For details, see Providing Resource Metrics Through the Metrics API. This add-on supports only clusters v1.21 or earlier. This add-on is no longer maintained in clusters v1.21 or later. This add-on is not recommended for CCE clusters. You are advised to use Cloud Native Cluster Monitoring or Kubernetes Metrics Server instead.
Constraints
- CustomedHPA policies apply only to clusters v1.15 or later.
- For clusters earlier than v1.19.10, if a CustomedHPA policy is used to scale out a workload with an EVS volume mounted, existing pods lose access to the volume while it is being unmounted and mounted to the node where a new pod is scheduled.
For clusters v1.19.10 and later, if a CustomedHPA policy is used to scale out a workload with an EVS volume mounted, new pods cannot start because the EVS volume cannot be mounted to multiple nodes simultaneously.
- The specifications of the CCE Advanced HPA add-on are determined based on the total number of containers in the cluster and the number of scaling policies. Configure 500m and 1,000 MiB of memory for every 5,000 containers, and 100m and 500 MiB of memory for every 1,000 scaling policies.
- After a CustomedHPA policy is created, the type of its associated workload cannot be changed.
Procedure
- Log in to the CCE console and click the cluster name to access the cluster console.
- Choose Workloads in the navigation pane. Locate the target workload and choose More > Auto Scaling in the Operation column.
- Set Policy Type to CustomedHPA and configure policy parameters.
Table 1 CustomedHPA policy parameters Parameter
Description
Pod Range
Minimum and maximum numbers of pods.
When a policy is triggered, the workload pods are scaled within this range.
NOTICE:In CCE Turbo clusters, if you use a dedicated load balancer for your workload, the number of pods cannot exceed the backend server group quota of the load balancer, which is 500 by default. If you exceed this limit, you will not be able to add any more pods to the load balancer backend.
Cooldown Period
Enter an interval, in minutes.
This parameter indicates the interval between consecutive scaling operations. The cooldown period ensures that a scaling operation is initiated only when the previous one is completed and the system is running stably.
NOTE:If you are using CCE Advanced HPA of v1.3.10 or later, the cooldown period will only apply to metric policies. Periodic policies will not be affected by the cooldown period.
Rule
Click
. In the dialog box displayed, configure the scaling policy parameters.- Type: You can select Metric-based (Table 2) or Periodic (Table 3). Then, configure trigger conditions and actions.
- Enable: Enable or disable the policy rule.
After configuring the preceding parameters, click OK. Then, the added policy rule is displayed in the rule list.
Table 2 Metric-based rules Parameter
Description
Trigger
Select CPU usage or Memory usage, choose > or <, and enter a percentage.NOTE:Usage = Average resource usage of all pods in a workload/Requested resources
Action
Set an action to be performed when the trigger condition is met. Multiple actions can be added.- Scale To: Adjust the number of pods to the specified value. Both a number and a percentage will do. This action can be used to scale in or out pods. If the current number of pods is less than the target value or the target percentage is greater than 100%, the number of pods will be scaled out to the target value. If the current number of pods is greater than the target value or the target percentage is less than 100%, the number of pods will be scaled in to the target value.
- Add: Configure this parameter when Trigger is set to >. Add the number of pods. You can specify a number or a percentage. This action can only be used to scale out pods.
- Reduce: Configure this parameter when Trigger is set to <. Reduce the number of pods. You can specify a number or a percentage. This action can only be used to scale in pods.
NOTE:You can enter a number or a percentage for the preceding actions.
When entering a percentage, you are required to specify the minimum number of available pods. Final number of pods = Current number of pods x Percentage. The result is rounded up. If the result is smaller than the minimum number of available pods, the preset value is used. Otherwise, the calculation result is used.
As shown below, when the CPU usage exceeds 50%, the number of pods is scaled out to 5. When the CPU usage exceeds 70%, the number of pods is scaled out to 8. When the CPU usage exceeds 90%, the number of pods is scaled out to 18 (adding 10 more pods). These rules also work for scale-in operations.
Figure 1 Setting a trigger condition
Table 3 Periodic-based rules Parameter
Description
Trigger Time
You can select a specific time every day, every week, every month, or every year.
Action
Set an action to be performed at the Triggered Time. As shown below, one pod will be added at 17:00 every day.- Scale To: Adjust the number of pods to the specified value. Both a number and a percentage will do. This action can be used to scale in or out pods. If the current number of pods is less than the target value or the target percentage is greater than 100%, the number of pods will be scaled out to the target value. If the current number of pods is greater than the target value or the target percentage is less than 100%, the number of pods will be scaled in to the target value.
- Add: Add the number of pods. You can specify a number or a percentage. This action can only be used to scale out pods.
- Reduce: Reduce the number of pods. You can specify a number or a percentage. This action can only be used to scale in pods.
NOTE:You can enter a number or a percentage for the preceding actions.
When entering a percentage, you are required to specify the minimum number of available pods. Final number of pods = Current number of pods x Percentage. The result is rounded up. If the result is smaller than the minimum number of available pods, the preset value is used. Otherwise, the calculation result is used.
Figure 2 Periodic triggering (Daily)
- Click Create.
Using kubectl
A CustomedHPA policy is a CustomResourceDefinition (CRD) and can be defined as follows in YAML:
apiVersion: autoscaling.cce.io/v1alpha1
kind: CustomedHorizontalPodAutoscaler
metadata:
name: customhpa-example
namespace: default
spec:
coolDownTime: 3m # Cooldown period
maxReplicas: 10 # Maximum number of pods
minReplicas: 1 # Minimum number of pods
rules:
- actions: # Policy rules
- metricRange: 0,0.1 # Metric range: 0 to 10% (exclusive)
operationType: ScaleDown # Scaling type. ScaleDown indicates downsizing.
operationUnit: Task # Operation unit. Task indicates the number of tasks.
operationValue: 1 # Resource quantity in each scaling
- metricRange: 0.1,0.3 # Metric range: 10% to 30% (exclusive)
operationType: ScaleDown
operationUnit: Task
operationValue: 2
disable: false # Whether to disable this rule
metricTrigger: # Trigger condition
hitThreshold: 1
metricName: CPURatioToRequest # Metric name, where CPURatioToRequest indicates the CPU usage, and MemoryRatioToRequest indicates the memory usage.
metricOperation: < # Metric expression operator, indicating that the rule is triggered when the metric value is less than metricValue.
metricValue: 0.3 # Value on the right of the metric expression
periodSeconds: 60
statistic: instantaneous
ruleName: low # Rule name
ruleType: Metric # Rule type, indicating that the rule is triggered by metric
- actions:
- metricRange: 0.7,0.9
operationType: ScaleUp
operationUnit: Task
operationValue: 1
- metricRange: 0.9,+Infinity
operationType: ScaleUp
operationUnit: Task
operationValue: 2
disable: false
metricTrigger:
hitThreshold: 1
metricName: CPURatioToRequest
metricOperation: '>'
metricValue: 0.7
periodSeconds: 60
statistic: instantaneous
ruleName: high
ruleType: Metric
scaleTargetRef: # Associated workload
apiVersion: apps/v1
kind: Deployment
name: nginx Helpful Links
- For scaling based on CPU and memory usage, see Creating an HPA Policy.
- To use custom metrics for auto scaling, see Creating an HPA Policy with Custom Metrics.
- To scale pods periodically using a Crontab-like schedule, see Creating a Scheduled CronHPA Policy.
- To automatically adjust pod resource limits based on actual usage, see Creating a VPA Policy.
- For periodic resource usage that cannot be defined by fixed rules, use AHPA policies. AHPA automatically identifies the scaling period based on historical metrics and performs scaling. For details, see Creating an AHPA Policy.
What is your overall rating for this page?
Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot