Help Center/ Cloud Container Engine_Autopilot/ User Guide/ Network/ Container Network/ Associating a Subnet and Security Group with a Namespace or Workload Using a Container Network Configuration
Updated on 2025-01-03 GMT+08:00
View PDF
Share

Associating a Subnet and Security Group with a Namespace or Workload Using a Container Network Configuration

Scenario

In a CCE Autopilot cluster, you can configure subnets and security groups for containers by namespace or workload using NetworkAttachmentDefinition, which is a CRD resource. To configure a particular container subnet and security group for a specified namespace or workload, create a container network configuration (NetworkAttachmentDefinition) and associate it with the target namespace or workload. In this way, service subnets can be planned or services can be securely isolated.

Figure 1 Custom container network settings
The following table lists the resources that a container network configuration (NetworkAttachmentDefinition) can be associated with.
Table 1 Associated resources

Dimension

Resources That a Container Network Configuration Can Be Associated With

Namespace

Workload

Subnet and security group configurations

All workloads created in the namespace associated with a container network configuration will use the same subnet and security group configurations.

The workloads using the same container network configuration will use the same subnet and security group configurations.

Supported CCE Autopilot cluster version

v1.27.8-r0, v1.28.6-r0, and later

v1.27.8-r0, v1.28.6-r0, and later

Constraint

A namespace can only be associated with one container network configuration.

Only custom container network configurations that are not associated with any namespace can be specified.
  • The priorities (in descending order) of container network configurations used by a pod are as follows: Container network configuration directly associated with the pod > Container network configuration associated with the pod namespace > Default container network configuration of the cluster (default-network)
  • If default-network is available in a cluster, it will be applied on all pods when no custom container network configuration has been configured. The default container subnet in the network settings on the Overview page is the container subnet in default-network. default-network cannot be deleted.
  • If there is only one container network configuration in a cluster, it is the default container network configuration. If there are multiple container network configurations in a cluster, the one with annotation yangtse.io/default-network: true is the default container network configuration. If there is no default container network configuration in a cluster, the pod that is not associated with any container network configuration fails to start after being created because no network interface can be allocated for the pod.

Constraints

Before deleting a custom container network configuration, delete the pods (with the cni.yangtse.io/network-status annotation) created using the configuration in the target namespace. For details, see Deleting a Container Network Configuration.

Creating a Container Network Configuration for a Namespace

After a container network configuration is created and associated with a namespace, all workloads created in the namespace will use the same subnet and security group configurations.

  1. Log in to the CCE console.
  2. Click the cluster name to access the cluster console. Choose Settings in the navigation pane on the left and click the Network tab.
  3. In the Container Network area, click Add. In the window that slides out from the right, configure parameters such as the pod subnet and security group.

    • Name: Enter a name that contains a maximum of 63 characters. Do not use default-network, default, mgnt0, or mgnt1. They are reserved for system use.
    • Associated Resource Type: Select a resource type that will be associated with the custom container network configuration. For details, see Table 1. To create a container network configuration for a namespace, select Namespace.
    • Namespace: Select the namespace to be associated. The namespaces associated with different container network configurations must be unique. If no namespace is available, click Create Namespace to create one.
    • Pod Subnet: Select one or more subnets. A maximum of 100 subnets can be selected. If no subnet is available, create one first. After the subnet is created, click the refresh button.
    • Associate Security Group: The default value is the security group associated with the container network interface. You can also create one. After the security group is created, click the refresh button. A maximum of five security groups can be selected.
    Figure 2 Creating a container network configuration for a namespace

  4. Click OK. You will be redirected to the custom container network configuration list, where you can view new container network configuration.

    Figure 3 Container network configuration list

After a container network configuration is created and associated with a namespace, all workloads created in the namespace will use the same subnet and security group configurations.

  1. Use kubectl to connect to the cluster. For details, see Connecting to a Cluster Using kubectl.
  2. Create a YAML file for creating a custom container network configuration and associating the configuration with a namespace. You can use any file name. In this example, the file name is networkattachment-test.yaml. 

    vim networkattachment-test.yaml

    Example file content:

    apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  annotations:
    yangtse.io/project-id: 05e38**       
  name: example                   
  namespace: kube-system                 
spec:
  config: |
   {
    "type":"eni-neutron",                
    "args":{
      "securityGroups":"41891**",        
      "subnets":[
        {
          "subnetID":"27d95**"           
        }
      ]
    },
    "selector":{
      "namespaceSelector":{             
        "matchLabels":{
          "kubernetes.io/metadata.name":"default"
        }
      }
    }
   }
    Table 2 Key parameters

    Parameter

    Mandatory

    Type

    Description

    apiVersion

    Yes

    String

    API version. The value is fixed at k8s.cni.cncf.io/v1.

    kind

    Yes

    String

    Type of the object to be created. The value is fixed at NetworkAttachmentDefinition.

    yangtse.io/project-id

    Yes

    String

    Project ID in the current region. For details, see Obtaining a Project ID.

    name

    Yes

    String

    Configuration name, which can be customized. Enter up to 63 characters. Start and end with a lowercase letter or digit. Only lowercase letters, digits, hyphens (-), and periods (.) are allowed. Do not use default-network, default, mgnt0, or mgnt1. They are reserved for system use.

    namespace

    Yes

    String

    Namespace of the configuration. The value is fixed at kube-system.

    config

    Yes

    Object

    Configuration content, which is a string in JSON format.

    For details about the data structure of the config field, see Table 3.
    Table 3 Data structure of the config field

    Parameter

    Mandatory

    Type

    Description

    type

    Yes

    String

    Network type. The value is fixed at eni-neutron.

    args

    Yes

    Object

    Security group and subnet configurations.

    For details about the data structure of the args field, see Table 4.

    selector

    No

    Object

    Namespace where the configuration will take effect.

    For details about the data structure of the selector field, see Table 5.
    Table 4 Data structure of the args field

    Parameter

    Mandatory

    Type

    Description

    securityGroups

    No

    String

    ID of the security group to be configured. A maximum of five security groups can be associated. If no security group is planned, ensure that the security group is the same as that in default-network.

    How to obtain:

    Log in to the VPC console. In the navigation pane on the left, choose Access Control > Security Groups. Click the target security group name and copy the ID on the Summary tab.

    subnets

    Yes

    Array of subnetID objects

    Container subnet ID list. Enter 1 to 100 subnet IDs. The format is as follows:

    [{"subnetID":"27d95**"},{"subnetID":"827bb**"},{"subnetID":"bdd6b**"}]

    The container subnet must be in the same VPC as the cluster.

    How to obtain:

    Log in to the VPC console. In the navigation pane on the left, choose Virtual Private Cloud > Subnets. Click the target subnet name and copy the Subnet ID on the Summary tab.
    Table 5 Data structure of the selector field

    Parameter

    Mandatory

    Type

    Description

    namespaceSelector

    No

    matchLabels object

    A Kubernetes standard selector. Enter the namespace label in the following format:

    "matchLabels":{
          "kubernetes.io/metadata.name":"default"
        }

    The namespaces for different configurations must be unique.

  3. Create the container network configuration. 

    kubectl create -f networkattachment-test.yaml

    If information similar to the following is displayed, the configuration has been created:

    networkattachmentdefinition.k8s.cni.cncf.io/example created

Creating a Container Network Configuration for a Workload

After such a container network configuration is created, all workloads associated with it will use the same subnet and security group configurations.

  1. Log in to the CCE console.
  2. Click the cluster name to access the cluster console. Choose Settings in the navigation pane on the left and click the Network tab.
  3. On the displayed page, locate the Container Network area and click Add. In the window that slides out from the right, configure parameters such as the pod subnet and security group.

    • Name: Enter a name that contains a maximum of 63 characters. Do not use default-network, default, mgnt0, or mgnt1. They are reserved for system use.
    • Associated Resource Type: Select a resource type that will be associated with the custom container network configuration. For details, see Table 1. To create a container network configuration for a workload, select Workload.
    • Pod Subnet: Select one or more subnets. A maximum of 100 subnets can be selected. If no subnet is available, create one first. After the subnet is created, click the refresh button.
    • Associate Security Group: The default value is the security group associated with the container network interface. You can also create one. After the security group is created, click the refresh button. A maximum of five security groups can be selected.
    Figure 4 Creating a container network configuration for a workload

  4. Click OK. You will be redirected to the custom container network configuration list, where you can view new container network configuration.

    Figure 5 Container network configuration list

  5. When creating a workload, select a custom container network configuration.

    1. In the navigation pane on the left, choose Workloads. Then click the Deployments tab.
    2. Click Create Workload in the upper right corner of the page. In the Advanced Settings area, choose Network Configuration and determine whether to enable a specified container network configuration.
    3. Select an existing container network configuration. If no configuration is available, click Add to create one.
      Figure 6 Selecting a container network configuration

    4. Configure other parameters and click Create Workload.

      Return to the Settings page. In the container network configuration list, the name of the resource associated with the created container network configuration is displayed.

      Figure 7 Resources associated with a container network configuration

After a container network configuration is created, all workloads associated with it will use the same subnet and security group configurations.

  1. Connect to a cluster using kubectl.
  2. Create a YAML file for creating a custom container network configuration. You can use any file name. In this example, the file name is networkattachment-test.yaml. 

    vim networkattachment-test.yaml

    Example file content:

    apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  annotations:
    yangtse.io/project-id: 80d5a**    
  name: example                       
  namespace: kube-system                     
spec:
  config: |
   {
    "type":"eni-neutron",                    
    "args":{
      "securityGroups":"f4983**",   
      "subnets":[
        {
          "subnetID":"5594b**"       
        }
      ]
    }
   }
    Table 6 Key parameters

    Parameter

    Mandatory

    Type

    Description

    apiVersion

    Yes

    String

    API version. The value is fixed at k8s.cni.cncf.io/v1.

    kind

    Yes

    String

    Type of the object to be created. The value is fixed at NetworkAttachmentDefinition.

    yangtse.io/project-id

    Yes

    String

    Project ID in the current region. For details, see Obtaining a Project ID.

    name

    Yes

    String

    Configuration name, which can be customized. Enter up to 63 characters. Start and end with a lowercase letter or digit. Only lowercase letters, digits, hyphens (-), and periods (.) are allowed. Do not use default-network, default, mgnt0, or mgnt1. They are reserved for system use.

    namespace

    Yes

    String

    Namespace of the configuration. The value is fixed at kube-system.

    config

    Yes

    Object

    Configuration content, which is a string in JSON format.

    For details about the data structure of the config field, see Table 7.
    Table 7 Data structure of the config field

    Parameter

    Mandatory

    Type

    Description

    type

    Yes

    String

    Network type. The value is fixed at eni-neutron.

    args

    Yes

    Object

    Security group and subnet configurations.

    For details about the data structure of the args field, see Table 8.
    Table 8 Data structure of the args field

    Parameter

    Mandatory

    Type

    Description

    securityGroups

    No

    String

    ID of the security group to be configured. A maximum of five security groups can be associated. If no security group is planned, ensure that the security group is the same as that in default-network.

    How to obtain:

    Log in to the VPC console. In the navigation pane on the left, choose Access Control > Security Groups. Click the target security group name and copy the ID on the Summary tab.

    subnets

    Yes

    Array of subnetID objects

    Container subnet ID list. Enter 1 to 100 subnet IDs. The format is as follows:

    [{"subnetID":"27d95**"},{"subnetID":"827bb**"},{"subnetID":"bdd6b**"}]

    The container subnet must be in the same VPC as the cluster.

    How to obtain:

    Log in to the VPC console. In the navigation pane on the left, choose Virtual Private Cloud > Subnets. Click the target subnet name and copy the Subnet ID on the Summary tab.

  3. Create the container network configuration. 

    kubectl create -f networkattachment-test.yaml

    If information similar to the following is displayed, the configuration has been created:

    networkattachmentdefinition.k8s.cni.cncf.io/example created

  4. Create a YAML file for creating a workload (for example, a Deployment) and associating it with the new container network configuration. You can use any file name. In this example, the file name is deploy.yaml. 

    vim deploy.yaml

    Example file content:

    apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
        yangtse.io/network: "example"  # Name of the custom container network configuration, which can be used to obtain all pods associated with the container network configuration by label
      annotations:
        yangtse.io/network: "example"  # Name of the custom container network configuration
    spec:
      containers:
        - name: container-0
          image: nginx:alpine
          resources:
            limits:
              cpu: 100m
              memory: 200Mi
            requests:
              cpu: 100m
              memory: 200Mi
      imagePullSecrets:
        - name: default-secret
    • yangtse.io/network: name of the specified custom container network configuration. Only a container network configuration that is not associated with any namespace can be specified. Add this parameter to the label so that you can use the label to obtain all pods associated with this container network configuration.

      The name of the custom container network configuration specified by yangtse.io/network is the same as the value of name in networkattachment-test.yaml.

  5. Create a Deployment. 

    kubectl create -f deploy.yaml

    Information similar to the following is displayed:

    deployment.apps/nginx created

Verify That a Container Network Configuration Is Associated with a Namespace or Workload

The following are steps to verify that a subnet and security group specified in the container network configuration have been associated with a workload. The steps for verifying that a container network configuration has been associated with a namespace are the same as those provided here.

  1. Verify that a subnet is associated.

    1. Check the IP address of a pod for a workload. 
      kubectl get pod -o wide

      Information similar to the following is displayed:

      NAME                     READY   STATUS    RESTARTS   AGE     IP              NODE                                   NOMINATED NODE   READINESS GATES
nginx-85dbdb8c5d-ng5bb   1/1     Running   0          5d18h   192.168.60.5   ca50a5ae-e1ef-41c6-b3fc-6ebcd10a1e07   <none>           <none>
    2. Log in to Network Console. In the navigation pane on the left, choose Virtual Private Cloud > Subnets. On the displayed page, click the subnet name.
    3. Click IP Addresses. If the IP address of the pod is displayed, the subnet is associated.
      Figure 8 Viewing the subnet associated with a pod

  2. Verify that a security group is associated.

    1. Log in to Network Console. In the navigation pane on the left, choose Access Control > Security Groups. On the displayed page, click the security group name.
    2. On the Associate Instances tab, click Supplementary Network Interfaces.
    3. View the private IP address of the pod in the Private IP Address column. If the private IP address of the pod is displayed, the security group is associated.
      Figure 9 Viewing the IP address associated with a security group

Deleting a Container Network Configuration

You can delete a new container network configuration or view its YAML file.

Before deleting a container network configuration, delete all pods using the configuration, or the deletion will fail.

  1. Run the following command to filter out the pods that use the configuration (example is used as an example here) in the cluster: 
    kubectl get pod -A -o=jsonpath="{.items[?(@.metadata.annotations.cni\.yangtse\.io/network-status=='[{\"name\":\"example\"}]')]['metadata.namespace', 'metadata.name']}"

    The name of each pod and the namespace associated with the configuration will be displayed.

  2. Delete the workload that each pod is running. The workload can be a Deployment, StatefulSet, Job, or CronJob. 
    kubectl delete deployment nginx

    Information similar to the following is displayed:

    deployment.apps "nginx" deleted
  3. Delete the container network configuration. 
    kubectl delete -f networkattachment-test.yaml -n kube-system

    Information similar to the following is displayed:

    networkattachmentdefinition.k8s.cni.cncf.io "example" deleted
Parent Topic: Container Network