CPU Burst

Notes and Constraints

• Cluster version: CCE Turbo cluster v1.23.5-r0 or later
• OS version: Huawei Cloud EulerOS 2.0
• The Volcano add-on of v1.9.0 or later must be installed in the cluster, and the hybrid deployment function must be enabled by setting colocation_enable in the advanced settings to true.

Procedure

1. Log in to the CCE console and click the cluster name to access the cluster console.
2. In the navigation pane on the left, choose Nodes. Click the Node Pools tab. When creating or updating a node pool, enable hybrid deployment of online and offline services in Advanced Settings.
   • volcano.sh/oversubscription=true
   • volcano.sh/colocation=true
   Figure 3 Node label settings
   

3. In the navigation pane, choose Add-ons and click Install of the Volcano add-on. On the Install Add-on page, enable hybrid deployment in the Parameters area. For details about the installation, see Volcano Scheduler.

   If the Volcano add-on has been installed, click Edit to view or modify the parameter colocation_enable.

   Figure 4 Enabling hybrid deployment of online and offline services
   
   

   After CPU burst is disabled, this function is still enabled on the existing pods where CPU burst has been enabled. Disabling CPU burst takes effect only on new pods.

4. Verify that CPU burst has been enabled.
   After confirming that the Volcano add-on is working, edit the parameter configmap of volcano-agent-configuration in the namespace kube-system. If enable is set to true, CPU burst is enabled. If enable is set to false, CPU burst is disabled.

   kubectl edit configmap -nkube-system volcano-agent-configuration

   Example:

   ...
   data:
     colocation-config: |
       {
           "globalConfig":{
               "cpuBurstConfig":{
                   "enable":true
               },
   ...

   

   After CPU burst is disabled, this function is still enabled on the existing pods where CPU burst has been enabled. Disabling CPU burst takes effect only on new pods.

5. Deploy a workload in a node pool where hybrid deployment has been enabled. Take Nginx as an example. Set requests to 2 and limits to 4, and create a Service that can be accessed in the cluster for the workload.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: nginx
     namespace: default
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: nginx
     template:
       metadata:
         labels:
           app: nginx
         annotations: 
           volcano.sh/enable-quota-burst: "true"
           volcano.sh/quota-burst-time: "200000"
       spec:
         containers:
         - name: container-1
           image: nginx:latest
           resources:
             limits:
               cpu: "4"
             requests:
               cpu: "2"
         imagePullSecrets:
           - name: default-secret
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: nginx
     namespace: default
     labels:
       app: nginx
   spec:
     selector:
       app: nginx
     ports:
       - name: cce-service-0
         targetPort: 80
         nodePort: 0
         port: 80
         protocol: TCP
     type: ClusterIP

   Annotation	Mandatory	Description
   volcano.sh/enable-quota-burst	Yes	CPU burst is enabled for the workload.
   volcano.sh/quota-burst-time	No	To ensure CPU scheduling stability and reduce contention when multiple containers encounter CPU bursts at the same time, the default CPU Burst value is the same as the CPU Quota value, indicating that a container can use a maximum of twice the CPU Limit value. By default, CPU Burst is set for all service containers in a pod. For example, if the CPU Limit of a container is 4, the default CPU Burst value is 400000 (1 core = 100000), indicating that a maximum of four additional CPU cores can be used after the value of CPU limit is reached.

6. Verify CPU burst.

   You can use the wrk tool to increase load of the workload and observe the service latency, traffic limiting, and CPU limit exceeding when CPU Burst is enabled and disabled, respectively.

   1. Run the following command to increase load of the pod. <service_ip> indicates the service IP address associated with the pod.

      # Download and install the wrk tool on the node.
      # The Gzip compression module is enabled in the Apache configuration to simulate the computing logic for the server to process requests.
      # Run the following command to increase the load. Note that you need to change the IP address of the target application.
      wrk -H "Accept-Encoding: deflate, gzip" -t 4 -c 28 -d 120  --latency --timeout 2s http://<service_ip>

   2. Obtain the pod ID.

      kubectl get pod -n <namespace> <pod_name> -o jsonpath='{.metadata.uid}'

   3. You can run the following commands on the node to view the traffic limiting status and CPU limit exceeding status. In the command, <pod_id> indicates the pod ID.

      cat /sys/fs/cgroup/cpu/kubepods/burstable/pod<pod_id>/cpu.stat

      Information similar to the following is displayed:

      nr_periods 0  # Number of scheduling periods
      nr_throttled 0  # Traffic limiting times
      throttled_time 0  # Traffic limiting duration (ns)
      nr_bursts 0 # CPU Limit exceeding times
      burst_time 0 # Total Limit exceeding duration

      Table 1 Result summary in this example

      CPU Burst	P99 latency	nr_throttled Traffic Limiting Times	throttled_time Traffic Limiting Duration	nr_bursts Limit Exceeding Times	bursts_time Total Limit Exceeding Duration
      Not enabled	2.96 ms	986	14.3s	0	0
      Enabled	456 µs	0	0	469	3.7s