Configuring a ClusterExtensionProfile or ExtensionProfile

Configuring a ClusterExtensionProfile

1. Generate a YAML file.
   The following is an example YAML file:

   apiVersion: bursting.cci.io/v1
   kind: ClusterExtensionProfile
   metadata:
     name: test-cluster-profile
   spec:
     actions:
       annotations: #Add annotations to the pod.
         annotation1: value1
       labels:      #Add labels to the pod.
         label1: value1
       imageReplacement:    #Add an image address replacement policy to the pod.
       - replaceWith: xxx.com/yyy
         repositoryPrefix: harbor.domain/a/b/c/d
       hostPathReplacement: #Add a hostPath volume replacement policy to the pod.
       - emptyDir:
           sizeLimit: 10Gi
         name: volume1
         policyType: replaceByEmptyDir
       evictions:           #Configure a pod eviction policy.
         - type: "Hard"        
           signal: "DiskPressure"  
       podSidecars:         # Add a sidecar to the pod.
         containers:
         - image: nginx:latest
           name: nginx
           position: head
         volumes:
         - name: volume2
           configMap:
             name: configmap1
     namespaceLabels:
       matchLabels:
         key2: value2
     policy: override

2. Create the object.

   kubectl apply -f <test-cluster-profile>     #YAML file name (Replace it with the actual file name.)

3. (Optional) Verify that the object has been created.

   kubectl get cextp

Configuring an ExtensionProfile

1. Generate a YAML file.
   The following is an example YAML file:

   apiVersion: bursting.cci.io/v1
   kind: ExtensionProfile
   metadata:
     name: test-profile
   spec:
     actions:
       annotations: #Add annotations to the pod.
         annotation1: value1
       labels:      #Add labels to the pod.
         label1: value1
       imageReplacement:    #Add an image address replacement policy to the pod.
       - replaceWith: xxx.com/yyy
         repositoryPrefix: harbor.domain/a/b/c/d
       hostPathReplacement: #Add a hostPath volume replacement policy to the pod.
       - emptyDir:
           sizeLimit: 10Gi
         name: volume1
         policyType: replaceByEmptyDir
       evictions:           #Configure a pod eviction policy.
         - type: "Hard"        
           signal: "DiskPressure"  
       podSidecars:         #Add a sidecar to the pod.
         containers:
         - image: nginx:latest
           name: nginx
           position: head
         volumes:
         - name: volume2
           configMap:
             name: configmap1
     objectLabels:
       matchLabels:
         k1: v1
     policy: addOnly

2. Create the object.

   kubectl apply -f <test-profile>     #YAML file name (Replace it with the actual file name.)

3. (Optional) Verify that the object has been created.

   kubectl get extp

spec Parameters

actions Parameters

    hostPathReplacement: 
    - emptyDir:
        sizeLimit: 10Gi
      name: volume1
      policyType: replaceByEmptyDir

Rules for an Effective Profile

• The configuration only takes effect for new pods when they are created. The existing pods are not modified.
• Only one ClusterExtensionProfile or ExtensionProfile can be applied for a pod. If both types of profiles are matched, the ClusterExtensionProfile takes the precedence. An ExtensionProfile is processed based on the pod for which a ClusterExtensionProfile has taken effect. If the ExtensionProfile is set to addOnly and related actions are configured for both the ExtensionProfile and ClusterExtensionProfile, the following rules apply:
  • For labels and annotations, only these with different key names will take effect.
  • imageReplacement, hostPathReplacement, evictions, and podSidecars do not take effect.
  For example, if a pod matches the ClusterExtensionProfile cextp1 and ExtensionProfile extp1, the hostPath replacement policy configured for extp1 does not take effect because the policy of extp1 is addOnly. In annotations, the k1:v2 configured for extp1 does not take effect because the key k1 has been configured for cextp-1. The pod uses annotations k1:v1, k2:v2, and k3:v3, and hostPathReplacement in cextp1.

  apiVersion: bursting.cci.io/v1
  kind: ClusterExtensionProfile
  metadata:
    name: cextp1
  spec:
    actions:
      annotations:
        k1: v1
        k2: v2
      hostPathReplacement:
      - emptyDir:
          sizeLimit: 10Gi
        name: hostpath-1
        policyType: replaceByEmptyDir
      ...
    policy: addOnly
  ---
  apiVersion: bursting.cci.io/v1
  kind: ExtensionProfile
  metadata:
    name: extp1
    namespace: default
  spec:
    actions:
      annotations:
        k1: v2
        k3: v3
      hostPathReplacement:
      - name: '*'
        policyType: remove
      ...
    policy: addOnly

• If the label of the namespace where the pod is located can be matched by multiple ClusterExtensionProfiles, only the most exactly matched profile takes effect.
• If the label of a pod can be matched by multiple ExtensionProfiles, only the most exactly matched profile takes effect.
• Definition of the most exactly matched profile:
  • The larger the sum of the values of all matchLabels and matchExpressions in objectLabels or namespaceLabels, the more accurate the profile is.
  • If the number of profiles is the same, the profile whose name has the smallest alphabetical order is used.

Pod Eviction After Disk Pressure

When the disk space is insufficient, services may be affected. Based on the Kubernetes eviction and scheduling mechanism, you can configure eviction policies for specified ClusterExtensionProfiles or ExtensionProfiles to automatically evict pods upon disk pressure. If a pod is evicted from a Deployment, a new pod is automatically created for the Deployment.

  actions:    
    evictions:      # Only an eviction policy whose type is Hard and signal is DiskPressure can be configured.
      - type: "Hard"        
        signal: "DiskPressure"

• Method 1: Use a ClusterExtensionProfile to configure an eviction policy for a pod with a specified namespace label.

  apiVersion: bursting.cci.io/v1
  kind: ClusterExtensionProfile
  metadata:  
    name: eviction-cluster-profile
  spec:  
    actions:    
      evictions:      
        - type: "Hard"        
          signal: "DiskPressure"  
    namespaceLabels:    
      matchLabels:      
          key: value

• Method 2: Use an ExtensionProfile to configure an eviction policy for a pod with a specified namespace label.

  apiVersion: bursting.cci.io/v1
  kind: ExtensionProfile
  metadata:  
    name: eviction-cluster-profile
    namespace: default
  spec:  
    actions:    
      evictions:      
        - type: "Hard"        
          signal: "DiskPressure"  
    objectLabels:
      matchExpressions:
      - key: key2
        operator: In
        values:
        - v2
        - v3
      - key: key3
        operator: Exists
      matchLabels:
        key1: value1
    policy: addOnly

Principle of Eviction

If an eviction policy is configured, the CCE Cloud Bursting Engine for CCI add-on evicts pods matched by ClusterExtensionProfile or ExtensionProfile based on the condition value in NodeFsDiskPressure and ImageFsDiskPressure of the pod status synchronized from CCI.

If image snapshots are not used and the disk space of a pod is less than 100 MiB, a condition of NodeFsDiskPressure is reported for the pod. If image snapshots are used and the temporary disk space of a pod is less than 100 MiB or the disk space of the container image is less than 100 MiB, a condition of NodeFsDiskPressure or ImageFsDiskPressure is reported. When the temporary disk space of a pod or the disk space of the container image is greater than 100 MiB, the corresponding condition is removed.