Configuration Suggestions on CCE Workload Identity Security

Notes and Constraints

The cluster version must be 1.19.16 or later.

Procedure

1. Obtain the public key of the cluster serviceAccountToken from CCE. For details, see Step 1: Obtain the Public Key for Signature of the CCE Cluster.
2. Create an identity provider on IAM. For details, see Step 2: Configure an Identity Provider.
3. Obtain an IAM token from the workload to simulate an IAM user to access a cloud service. For details, see Step 3: Use a Workload Identity.
   The procedure is as follows:

   1. Deploy the application pod and obtain the OpenID Connect ID token file by mounting the identity provider.
   2. Use the mounted OpenID Connect ID token file in programs in the pod to access IAM and obtain a temporary IAM token.
   3. Access the cloud service using the IAM token in programs in the pod.

   Figure 1 Workflow
   

Step 1: Obtain the Public Key for Signature of the CCE Cluster

1. Use kubectl to access the target cluster.
2. Obtain the public key:

   kubectl get --raw /openid/v1/jwks

   # kubectl get --raw /openid/v1/jwks
   {"keys":[{"use":"sig","kty":"RSA","kid":"*****","alg":"RS256","n":"*****","e":"AQAB"}]}

   The returned field is the public key of the cluster.

Step 2: Configure an Identity Provider

1. Log in to the IAM console, choose Identity Providers in the navigation pane, and click Create Identity Provider in the upper right corner. On the displayed page, set Protocol to OpenID Connect and SSO Type to Virtual user and click OK.

   

2. In the identity provider list, locate the row containing the new identity provider and click Modify in the Operation column to modify the identity provider information.

   Access Type: Select Programmatic access.

   Configuration Information

   • Identity Provider URL: Enter https://kubernetes.default.svc.cluster.local.
   • Client ID: Enter an ID, which will be used when you create a container.
   • Signing Key: Enter the JWKS of the CCE cluster obtained in Step 1: Obtain the Public Key for Signature of the CCE Cluster.

   

   Identity Conversion Rules

   An identity conversion rule maps the ServiceAccount of a workload to IAM user.

   For example, create a ServiceAccount named oidc-token in namespace default of the cluster and map it to user group demo. If you use the identity provider ID to access cloud services, you have the permissions of the demo user group. The attribute must be sub. The value format is system:serviceaccount:Namespace:ServiceAccountName.

   

   Rules are in the JSON format as follows:

   [
       {
           "local": [
               {
                   "user": {
                       "name": "test"
                   }
               },
               {
                   "group": {
                       "name": "demo"
                   }
               }
           ],
           "remote": [
               {
                   "type": "sub",
                   "any_one_of": [
                       "system:serviceaccount:default:oidc-token"
                   ]
               }
           ]
       }
   ]

3. Click OK.

Step 3: Use a Workload Identity

1. Create a ServiceAccount, whose name must be the value of ServiceAccountName set in Step 2: Configure an Identity Provider.

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: oidc-token

2. Mount the identity provider to the workload and obtain the OpenID Connect ID token file.
   An example is as follows:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: nginx
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: nginx
         version: v1
     template:
       metadata:
         labels:
           app: nginx
           version: v1
       spec:
         containers:
         - name: container-1
           image: nginx:latest
           volumeMounts:
           - mountPath: "/var/run/secrets/tokens"     # Mount the serviceAccountToken generated by Kubernetes to the /var/run/secrets/tokens/oidc-token file.
             name: oidc-token
         imagePullSecrets:
         - name: default-secret
         serviceAccountName: oidc-token      # Name of the created ServiceAccount
         volumes:
         - name: oidc-token
           projected:
             defaultMode: 420
             sources:
             - serviceAccountToken:
                 audience: client_id   # Must be the client ID of the identity provider.
                 expirationSeconds: 7200       # Expiry period
                 path: oidc-token              # Path name, which can be customized

3. After the creation is complete, log in to the container. The content of the /var/run/secrets/tokens/oidc-token file is the serviceAccountToken generated by Kubernetes.
   

   If the serviceAccountToken is used for more than 24 hours or 80% of its expiry period, kubelet will automatically rotate the serviceAccountToken.

4. Use the OpenID Connect ID token to call the API for Obtaining a Token with an OpenID Connect ID Token. The X-Subject-Token field in the response header is the IAM token. Then, you can use this token to access cloud services.

   The following shows an example:

   curl -i --location --request POST 'https://{{iam endpoint}}/v3.0/OS-AUTH/id-token/tokens' \
    --header 'X-Idp-Id: workload_identity' \
    --header 'Content-Type: application/json' \
    --data @token_body.json

   Specifically:

   • {{iam endpoint}} indicates the endpoint of IAM. For details, see Regions and Endpoints.
   • workload_identity is the identity provider name, which is the same as that configured in Step 2: Configure an Identity Provider.
   • token_body.json is a local file and its content is as follows:

      { 
        "auth" : { 
          "id_token" : { 
            "id" : "eyJhbGciOiJSU..."
          }, 
          "scope": { 
            "project" : { 
              "id" : "46419baef4324...",
              "name" : ******
            } 
          } 
        } 
      }

     • $.auth.id_token.id: The value is the content of the /var/run/secrets/tokens/oidc-token file in the container.
     • $.auth.scope.project.id: indicates the project ID. For details about how to obtain the project ID, see Obtaining a Project ID.
     • $.auth.scope.project.name: indicates the project name, for example, cn-north-4.