CPU Burst

Notes and Constraints

• Cluster version: CCE Turbo cluster v1.23.5-r0 or later
• OS version: Huawei Cloud EulerOS 2.0
• Volcano: v1.9.0 or later; hybrid deployment: enabled

Procedure

1. Log in to the CCE console and click the cluster name to access the cluster console.
2. In the navigation pane, choose Nodes. On the Node Pools tab page, locate the target node pool and choose More > Mixed configuration.

   Ensure that node pool hybrid deployment and traffic limit for CPU burst are enabled.

   

   After CPU burst is disabled, this function is still enabled on the existing pods where CPU burst has been enabled. Disabling CPU burst takes effect only on new pods.

   

3. Deploy a workload in a node pool where hybrid deployment has been enabled. Take Nginx as an example. Set requests to 2 and limits to 4, and create a Service that can be accessed in the cluster for the workload.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: nginx
     namespace: default
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: nginx
     template:
       metadata:
         labels:
           app: nginx
         annotations: 
           volcano.sh/enable-quota-burst: "true"
           volcano.sh/quota-burst-time: "200000"
       spec:
         containers:
         - name: container-1
           image: nginx:latest
           resources:
             limits:
               cpu: "4"
             requests:
               cpu: "2"
         imagePullSecrets:
           - name: default-secret
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: nginx
     namespace: default
     labels:
       app: nginx
   spec:
     selector:
       app: nginx
     ports:
       - name: cce-service-0
         targetPort: 80
         nodePort: 0
         port: 80
         protocol: TCP
     type: ClusterIP

   Annotation	Mandatory	Description
   volcano.sh/enable-quota-burst	Yes	CPU burst is enabled for the workload.
   volcano.sh/quota-burst-time	No	To ensure CPU scheduling stability and reduce contention when multiple containers encounter CPU bursts at the same time, the default CPU Burst value is the same as the CPU Quota value, indicating that a container can use a maximum of twice the CPU Limit value. By default, CPU Burst is set for all service containers in a pod. For example, if the CPU Limit of a container is 4, the default CPU Burst value is 400000 (1 core = 100000), indicating that a maximum of four additional CPU cores can be used after the value of CPU limit is reached.

4. Verify CPU burst.

   You can use the wrk tool to increase load of the workload and observe the service latency, traffic limiting, and CPU limit exceeding when CPU Burst is enabled and disabled, respectively.

   1. Run the following command to increase load of the pod. <service_ip> indicates the service IP address associated with the pod.

      # Download and install the wrk tool on the node.
      # The Gzip compression module is enabled in the Apache configuration to simulate the computing logic for the server to process requests.
      # Run the following command to increase the load. Note that you need to change the IP address of the target application.
      wrk -H "Accept-Encoding: deflate, gzip" -t 4 -c 28 -d 120  --latency --timeout 2s http://<service_ip>

   2. Obtain the pod ID.

      kubectl get pod -n <namespace> <pod_name> -o jsonpath='{.metadata.uid}'

   3. You can run the following commands on the node to view the traffic limiting status and CPU limit exceeding status. In the command, <pod_id> indicates the pod ID.

      cat /sys/fs/cgroup/cpu/kubepods/burstable/pod<pod_id>/cpu.stat

      Information similar to the following is displayed:

      nr_periods 0  # Number of scheduling periods
      nr_throttled 0  # Traffic limiting times
      throttled_time 0  # Traffic limiting duration (ns)
      nr_bursts 0 # CPU Limit exceeding times
      burst_time 0 # Total Limit exceeding duration

      Table 1 Result summary in this example

      CPU Burst	P99 latency	nr_throttled Traffic Limiting Times	throttled_time Traffic Limiting Duration	nr_bursts Limit Exceeding Times	bursts_time Total Limit Exceeding Duration
      Not enabled	2.96 ms	986	14.3s	0	0
      Enabled	456 µs	0	0	469	3.7s