Scaling a Workload Using an HPA Policy

1. Log in to the CCE console. Click the cluster name to access the cluster console.
2. In the navigation pane, choose Workloads. In the upper right corner, click Create Workload.
3. Configure basic workload information. For details about the parameters, see Figure 6 and Table 6.

   In this example, only some mandatory parameters are described. You can keep the default values for other parameters. For details about other parameters, see Creating a Workload. You can select a reference document based on the workload type.

   Figure 6 Basic workload information
   

   Table 6 Basic workload information

   Parameter	Example Value	Description
   Workload Type	Deployment	A workload defines the creation, status, and lifecycle of pods. By creating a workload, you can manage and control the behavior of multiple pods, such as scaling, update, and restoration. Deployment: runs a stateless application. It supports online deployment, rolling upgrade, replica creation, and restoration. StatefulSet: runs a stateful application. Each pod for running the application has an independent state, and the data can be restored when the pod is restarted or migrated, ensuring application reliability and consistency. Job: a one-off job. After the job is complete, the pods are automatically deleted. Cron Job: a time-based job runs a specified job in a specified period. For more information about workloads, see Workload Overview. hpa-example is mainly used for stress tests and does not need to store any persistent data locally. Therefore, hpa-example is deployed as a Deployment in this example.
   Workload Name	hpa-example	Enter a name for the workload. Enter 1 to 63 characters, starting with a lowercase letter and ending with a lowercase letter or digit. Only lowercase letters, digits, and hyphens (-) are allowed.
   Namespace	default	A namespace is a conceptual grouping of resources or objects. Each namespace provides isolation for data from other namespaces. After a cluster is created, the default namespace is created by default. If there is no special requirement, select default.
   Pods	1	Enter the number of pods. The policy for setting the number of pods is as follows: High availability: To ensure high availability of the workload, at least 2 pods are required to prevent individual faults. High performance: Set the number of pods based on the workload traffic and resource requirements to avoid overload or resource waste. In this example, set the number of pods set to 1.

4. Configure container information. For details about the parameters, see Figure 7 and Table 7.

   In this example, only some mandatory parameters are described. You can keep the default values for other parameters. For details about other parameters, see Creating a Workload. You can select a reference document based on the workload type.

   Figure 7 Container settings
   

   Table 7 Container settings

   Parameter	Example Value	Description
   Image Name	hpa-example	Click Select Image. In the displayed dialog box, switch to the My Images tab and select the hpa-example image pushed.
   Image Tag	latest	Select the required image tag.
   CPU Quota	0.25 cores	Specify the CPU limit, which defines the maximum number of CPU cores that can be used by a container. The default value is 0.25 cores.
   Memory Quota	512 MiB	Specify the memory limit, which is the maximum memory available for a container. The default value is 512 MiB. If the memory exceeds 512 MiB, the container will be terminated.

5. Click under Service Settings. On the page that is displayed, configure the Service. For details about the parameters, see Figure 8 and Table 8.

   In this example, only some mandatory parameters are described. You can keep the default values for other parameters. For details about the parameters, see Service . Select a reference document based on the Service type.

   Figure 8 Service settings
   

   Table 8 Service settings

   Parameter	Example Value	Description
   Service Name	hpa-example	Enter a Service name. Enter 1 to 63 characters, starting with a lowercase letter and ending with a lowercase letter or digit. Only lowercase letters, digits, and hyphens (-) are allowed.
   Service Type	LoadBalancer	Select a Service type, which determines how the workload is accessed. ClusterIP: The workload can only be accessed using IP addresses in the cluster. LoadBalancer: The workload can be accessed from the public network through a load balancer. In this example, external access to hpa-example is required. So you need to create a Service of the LoadBalancer type. For more information about the Service types, see Service.
   Load Balancer	Dedicated Network (TCP/UDP) & Application (HTTP/HTTPS) Use existing elb-hpa-example	Select Use existing if there is a load balancer available. NOTE: The load balancer must meet the following requirements: It is in the same VPC as the cluster. It is a dedicated load balancer. It has a private IP address bound. If there is no available load balancer, select Auto create to create one with an EIP bound. For details, see Creating a LoadBalancer Service.
   Port	Protocol: TCP	Protocol: Select a protocol for the load balancer listener.
   	Container Port: 80	Container Port: Enter the port on which the application listens. This port must be the same as the listening port provided by the application for external systems.
   	Service Port: 80	Service Port: Enter a custom port. The load balancer will use this port as the listening port to provide an entry for external traffic.

6. Click Create Workload.

   After the Deployment is created, it is in the Running state in the Deployment list.

   Figure 9 A running workload
   

7. Click the workload name to go to the workload details page and obtain the external access address of hpa-example. On the Access Mode tab, you can view the external access address in the format of {EIP bound to the load balancer}:{Access port}. {EIP bound to the load balancer} indicates the public IP address of the load balancer, and {Access port} is the Service Port in 5.
   Figure 10 Access address
   

1. Install kubectl on the ECS. You can check whether kubectl has been installed by running kubectl version. If kubectl has been installed, you can skip this step.
   1. Run the following command to download kubectl:

      cd /home
      curl -LO https://dl.k8s.io/release/{v1.28.0}/bin/linux/amd64/kubectl

      {v1.28.0} specifies the version. Replace it as needed.

   2. Run the following command to install kubectl:

      chmod +x kubectl
      mv -f kubectl /usr/local/bin

2. Configure a credential for kubectl to access the cluster.

   1. Log in to the CCE console and click the cluster name to access the cluster console. In the navigation pane, choose Overview.
   2. Locate the Connection Information area. Click Configure next to kubectl and view the kubectl connection information.
   3. In the window that slides out from the right, locate the Download the kubeconfig file area, select Intranet access for Current data, and download the configuration file.
   4. Log in to the ECS where kubectl has been installed and copy and paste the downloaded configuration file (for example, kubeconfig.yaml) to the /home directory.
   5. Save the kubectl authentication file to the configuration file in the $HOME/.kube directory.

      cd /home
      mkdir -p $HOME/.kube
      mv -f kubeconfig.yaml $HOME/.kube/config

   6. Run the kubectl command to check whether the cluster can be accessed.

      For example, to view the cluster information, run the following command:

      kubectl cluster-info

      If the following information is displayed, the cluster is connected:

      Kubernetes control plane is running at https://xx.xx.xx.xx:5443
      CoreDNS is running at https://xx.xx.xx.xx:5443/api/v1/namespaces/kube-system/services/coredns:dns/proxy
      To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

3. Create a YAML file for creating a workload. In this example, the file name is hpa-example.yaml. You can change the file name as needed.

   vim hpa-example.yaml

   The file content is as follows:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: hpa-example     # Workload name
   spec:
     replicas: 1     # Number of pods
     selector:
       matchLabels:  # Selector for selecting resources with specific labels
         app: hpa-example  
     template:
       metadata:
         labels:     # Labels
           app: hpa-example
       spec:
         containers:
         - name: container-1
           image: 'swr.cn-east-3.myhuaweicloud.com/test/hpa-example:latest' # Replace it with the address of the image you have pushed to SWR.
           resources:
             limits:  # The value of limits must be the same as that of requests to prevent flapping during scaling.
               cpu: 250m
               memory: 512Mi
             requests:
               cpu: 250m
               memory: 512Mi
         imagePullSecrets:
         - name: default-secret

   

   Replace 'swr.cn-east-3.myhuaweicloud.com/test/hpa-example:latest' with the image path in SWR. The image path is the content following docker push in the step of pushing the image to the image repository.

   Press Esc to exit editing mode and enter :wq to save the file.

4. Run the following command to create the workload:

   kubectl create -f hpa-example.yaml

   If information similar to the following is displayed, the workload is being created:

   deployment.apps/hpa-example created

5. Run the following command to check the workload status:

   kubectl get deployment

   If the value of READY is 1/1, the pod created for the workload is available. This means the workload has been created.

   NAME           READY   UP-TO-DATE   AVAILABLE   AGE
   hpa-example    1/1     1            1           4m59s

   The following table describes the parameters in the command output.

   Table 9 Parameters in the command output

   Parameter	Example Value	Description
   NAME	hpa-example	Workload name.
   READY	1/1	The number of available pods/desired pods for the workload.
   UP-TO-DATE	1	The number of pods that have been updated for the workload.
   AVAILABLE	1	The number of pods available for the workload.
   AGE	4m59s	How long the workload has run.

6. Run the following command to create the nginx-elb-svc.yaml file for configuring the LoadBalancer Service and associating the Service with the created workload. You can change the file name.
   An existing load balancer is used to create the Service. To automatically create a load balancer, see Using kubectl to Create a Load Balancer.

   vim hpa-example-elb-svc.yaml

   The file content is as follows:

   apiVersion: v1 
   kind: Service 
   metadata: 
     name: hpa-example      # Service name
     annotations:
       kubernetes.io/elb.id: <your_elb_id>          # Load balancer ID. Replace it with the actual value.
       kubernetes.io/elb.class: performance         # Load balancer type
   spec:
     selector:             # The value must be the same as the value of matchLabels in the YAML file of the workload in 3.
        app: hpa-example
     ports: 
     - name: service0 
       port: 80     
       protocol: TCP 
       targetPort: 80  
     type: LoadBalancer

   Press Esc to exit editing mode and enter :wq to save the file.

   Table 10 Parameters for using an existing load balancer

   Parameter	Example Value	Description
   kubernetes.io/elb.id	405ef586-0397-45c3-bfc4-xxx	ID of an existing load balancer. NOTE: The load balancer must meet the following requirements: It is in the same VPC as the cluster. It is a dedicated load balancer. It has a private IP address bound. How to Obtain: In the navigation pane of Network Console, choose Elastic Load Balance > Load Balancers. Locate the load balancer. The ID is displayed below the name. Click the name of the load balancer. On the Summary tab, verify that the load balancer meets the preceding requirements.
   kubernetes.io/elb.class	performance	Load balancer type. The value can only be performance, which means that only dedicated load balancers are supported.
   selector	app: hpa-example	Selector, which is used by the Service to send traffic to pods with specific labels.
   ports.port	8080	The port used by the load balancer as an entry for external traffic. You can use any port.
   ports.protocol	TCP	Protocol for the load balancer listener.
   ports.targetPort	80	Port used by a Service to access the target container. This port is closely related to the applications running in the container.

7. Run the following command to create a Service:

   kubectl create -f hpa-example-elb-svc.yaml

   If information similar to the following is displayed, the Service has been created:

   service/hpa-example created

8. Run the following command to check the Service:

   kubectl get svc

   If information similar to the following is displayed, the workload access mode has been configured: You can access the workload using {External access address}:{Service port}. The external access address is the first IP address displayed for EXTERNAL-IP, and the Service port is 8080.

   NAME             TYPE           CLUSTER-IP     EXTERNAL-IP                  PORT(S)          AGE
   kubernetes       ClusterIP      10.247.0.1     <none>                       443/TCP          18h
   hpa-example      LoadBalancer   10.247.56.18   xx.xx.xx.xx,xx.xx.xx.xx      8080:30581/TCP   5m8s

In this example, only some mandatory parameters are described. You can keep the default values for other parameters. For details about the default parameters, see Horizontal Pod Autoscaling.

1. Return to the cluster console. In the navigation pane, choose Policies. In the upper right corner, click Create HPA Policy.
2. Configure the parameters. For details about the parameters, see Table 11.
   Figure 11 Policy basic Information
   

   Table 11 Policy basic Information

   Parameter	Example Value	Description
   Policy Name	hpa-example	Enter a policy name.
   Namespace	default	A namespace is a conceptual grouping of resources or objects. Each namespace provides isolation for data from other namespaces. After a cluster is created, the default namespace is created by default. If there is no special requirement, select default.
   Associated Workload	hpa-example	Select the workload created in Step 4: Create a Workload Using the hpa-example Image. NOTE: An HPA policy matches the associated workload by name. If the application labels of multiple workloads in the same namespace are the same, the policy will fail to meet the expectation.

3. Configure the policy. For details about the parameters, see Table 12.
   Figure 12 Policy configuration
   

   Table 12 Policy configuration

   Parameter	Example Value	Description
   Pod Range	1 to 10	The minimum and maximum numbers of pods that can be created for the workload.
   Scaling Behavior	Default	Auto scaling behaviors. The options are as follows: Default: Default behaviors (such as stabilization window and step) recommended by Kubernetes are used. For more information, see Horizontal Pod Autoscaling. Custom: You can create custom scaling behaviors (such as stabilization window and step) for more flexible configuration. For parameters that are not configured, the default values recommended by Kubernetes are used. For more information, see Horizontal Pod Autoscaling.
   System Policy	Metric: CPU usage	HPA can trigger pod scaling based on pod resource metrics. CPU usage = CPU used by the pod/CPU request Memory usage = Memory used by the pod/Memory request
   	Desired Value: 50%	Desired value of the selected metric, which can be used to calculate the number of pods to be scaled. The formula is as follows: Number of pods to be scaled = (Current metric value/Desired value) × Number of current pods
   	Tolerance Range: 30% to 70%	Scale-in and scale-out thresholds. Scaling will not be triggered when the metric value is within the range. This range prevents frequent adjustments caused by slight fluctuation and ensures system stability. Note that the desired value must be within the tolerance range.

4. Click Create. You can view the created policy in the HPA policy list. Its status is Started.
   Figure 13 HPA policy started
   

1. Create a YAML file for configuring an HPA policy. In this example, the file name is hpa-policy.yaml. You can change it as needed.

   vim hpa-policy.yaml

   The file content is as follows:

   apiVersion: autoscaling/v2
   kind: HorizontalPodAutoscaler
   metadata:
     name: hpa-example   # HPA policy name
     namespace: default  # Namespace
     annotations:
       # Define the condition for triggering scaling: When the CPU usage is between 30% and 70%, scaling is not performed. Otherwise, scaling is performed.
       extendedhpa.metrics: '[{"type":"Resource","name":"cpu","targetType":"Utilization","targetRange":{"low":"30","high":"70"}}]'
   spec:
     scaleTargetRef:     # Associated workload
       kind: Deployment
       name: hpa-example
       apiVersion: apps/v1
     minReplicas: 1     # Minimum number of pods in the workload
     maxReplicas: 10  # Maximum number of pods in the workload
     metrics:
       - type: Resource  # Set the resources to be monitored.
         resource:
           name: cpu      # Set the resource to CPU or memory.
           target:
             type: Utilization   # Set the metric to usage.
             averageUtilization: 50  # The HPA controller keeps the average resource usage of pods at 50%.

   Press Esc to exit editing mode and enter :wq to save the file.

2. Create the HPA policy.

   kubectl create -f hpa-policy.yaml

   If information similar to the following is displayed, the HPA policy has been created:

   horizontalpodautoscaler.autoscaling/hpa-example created

3. View the HPA policy.

   kubectl get hpa

   Information similar to the following is displayed:

   NAME          REFERENCE                TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
   hpa-example   Deployment/hpa-example   <unknown>/50%   1         10        0          10s

1. Log in to the ECS used in Step 3: Create a Compute-intensive Application and Upload Its Image to SWR.
2. Run the following command in a loop to access the workload. In the command, ip:port indicates the access address of the workload, which is the same as {EIP bound to the load balancer}:{Access port} in 7.

   while true;do wget -q -O- http://ip:port; done

   Information similar to the following is displayed:

   OK!
   OK!
   ...

3. Log in to the CCE console and click the cluster name to access the cluster console. In the navigation pane, choose Workloads. On the page displayed, click the workload name. Click on the right of the search bar to observe the automatic scale-out of the workload. You can find that the number of pods created for the hpa-example workload has increased from 1 to 2.
   Figure 14 Automatic workload scale-out
   

4. Return to the ECS, press Ctrl+C to stop accessing the workload, and observe the automatic scale-in of the workload.

   Return to the CCE console and click on the right of the search bar. The stabilization window for workload scale-in is 300 seconds by default. After the scale-in is triggered, the workload is not scaled in immediately. Instead, it will be cooled down for 300 seconds to prevent short-term fluctuations.

   Figure 15 Workload scale-in
   

1. Run the following command in a loop to access the workload. In the command, ip:port indicates the access address of the workload, which is the same as {External access address}:{Service port} in 8.

   while true;do wget -q -O- http://ip:port; done

   Information similar to the following is displayed:

   OK!
   OK!
   ...

2. In the upper left corner of the page, choose Terminal > New Session. In the terminal that is displayed, run the following command to view the automatic workload scale-out:

   kubectl get hpa hpa-policy --watch

   Information similar to the following is displayed:

   NAME          REFERENCE                TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
   hpa-example   Deployment/hpa-example   24%/50%   1         10        1          17m
   hpa-example   Deployment/hpa-example   100%/50%   1         10        1          17m
   hpa-example   Deployment/hpa-example   100%/50%   1         10        2          18m
   hpa-example   Deployment/hpa-example   100%/50%   1         10        2          18m
   hpa-example   Deployment/hpa-example   57%/50%    1         10        2          19m
   hpa-example   Deployment/hpa-example   57%/50%    1         10        2          20m
   ...

   Table 13 Parameters in the command output

   Parameter	Example Value	Description
   NAME	hpa-example	HPA policy name. In this example, the HPA policy name is hpa-example.
   REFERENCE	Deployment/hpa-example	Object for which the HPA policy takes effect. In this example, the object is a Deployment named hpa-example.
   TARGETS	24%/50%	Current and desired values of the metric. In this example, the metric is CPU usage. 24% indicates the current CPU usage, and 50% indicates the desired CPU usage.
   MINPODS	1	Minimum number of pods allowed by the HPA policy. In this example, the minimum number of pods is 1. When the number of pods is 1, the scale-in will not be performed even if the current CPU usage is lower than the tolerance range.
   MAXPODS	10	Maximum number of pods allowed by the HPA policy. In this example, the maximum number of pods is 10. When the number of pods is 10, the scale-out will not be performed even if the current CPU usage is higher than the tolerance range.
   REPLICAS	1	Number of running pods.
   AGE	17m	How long the HPA policy has been used.

   • On line 2, the CPU usage of the workload is 100%, which exceeds 70%. As a result, auto scaling is triggered.
   • On line 3, the number of pods is increased from 1 to 2.
   • On line 5, the CPU usage of the workload decreases to 57%, which is between 30% and 70%. No auto scaling is triggered.

3. In the terminal displayed in 1, press Ctrl+C to stop accessing the workload and observe the automatic scale-in.

   On the terminal in 2, view the command output. Note that the stabilization window for workload scale-in is 300 seconds by default. After the scale-in is triggered, the workload is not scaled in immediately. Instead, it will be cooled down for 300 seconds to prevent short-term fluctuations.

   NAME          REFERENCE                TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
   ...
   hpa-example   Deployment/hpa-example   19%/50%    1         10        2          30m
   hpa-example   Deployment/hpa-example   0%/50%     1         10        2          31m
   hpa-example   Deployment/hpa-example   0%/50%     1         10        2          35m
   hpa-example   Deployment/hpa-example   0%/50%     1         10        1          36m
   ...

   • On line 1, the CPU usage of the workload is 19%, which is lower than 30%. As a result, auto scaling is triggered. However, the workload is not scaled in immediately because it is in the cooldown period.
   • On line 4, the number of pods is decreased from 2 to 1, which is the minimum number of pods. In this case, auto scaling is not triggered.