Configuration Suggestions on CCE Workload Identity Security

Notes and Constraints

The cluster version must be 1.19.16 or later.

Procedure

1. Obtain the JSON Web Key Set (JWKS) of the cluster (the signature public key of the service account token) from CCE.
2. Create an identity provider on the IAM console.
3. Deploy the application and bind it with the identity provider.
   1. Use the OIDC token to access IAM and obtain the IAM token (implemented by you).
   2. Use the IAM token to access cloud services (implemented by you).

Obtaining JWKS of a CCE Cluster

1. Use kubectl to connect to the cluster.
2. Run the following command to obtain the public key:

   kubectl get --raw /openid/v1/jwks

   # kubectl get --raw /openid/v1/jwks
   {"keys":[{"use":"sig","kty":"RSA","kid":"*****","alg":"RS256","n":"*****","e":"AQAB"}]}

   The returned field is the public key of the cluster.

Configuring an Identity Provider

1. Log in to the IAM console, create an identity provider, and select OpenID Connect for Protocol.

   

2. Click OK. After the creation, modify the identity provider information.

   Access Type: Select Programmatic access.

   Configuration Information

   • Identity Provider URL: Enter https://kubernetes.default.svc.cluster.local.
   • Client ID: Enter an ID, which will be used when you create a container.
   • Signing Key: Enter the JWKS of the CCE cluster obtained in Obtaining JWKS of a CCE Cluster.

   

   Identity Conversion Rules

   An identity conversion rule maps the ServiceAccount of a workload to IAM user.

   For example, create a ServiceAccount named oidc-token in namespace default of the cluster and map it to user group demo. If you use the identity provider ID to access cloud services, you have the permissions of the demo user group. The attribute must be sub. The value format is system:serviceaccount:Namespace:ServiceAccountName.

   

   Rules are in the JSON format as follows:

   [
       {
           "local": [
               {
                   "user": {
                       "name": "test"
                   }
               },
               {
                   "group": {
                       "name": "demo"
                   }
               }
           ],
           "remote": [
               {
                   "type": "sub",
                   "any_one_of": [
                       "system:serviceaccount:default:oidc-token"
                   ]
               }
           ]
       }
   ]

3. Click OK.

Using a Workload Identity

Create a ServiceAccount, whose name must be the value of ServiceAccountName set in Configuring an Identity Provider.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: oidc-token

Example configuration for the workload:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v1
  template:
    metadata:
      labels:
        app: nginx
        version: v1
    spec:
      containers:
      - name: container-1
        image: nginx:latest
        volumeMounts:
        - mountPath: "/var/run/secrets/tokens"     # Mount the serviceAccountToken generated by Kubernetes to the /var/run/secrets/tokens/oidc-token file.
          name: oidc-token
      imagePullSecrets:
      - name: default-secret
      serviceAccountName: oidc-token      # Name of the created ServiceAccount
      volumes:
      - name: oidc-token
        projected:
          defaultMode: 420
          sources:
          - serviceAccountToken:
              audience: client_id   # Must be the client ID of the identity provider.
              expirationSeconds: 7200       # Expiry period
              path: oidc-token              # Path name, which can be customized.

After the creation, log in to the container. The content of the /var/run/secrets/tokens/oidc-token file is the serviceAccountToken generated by Kubernetes. You can obtain the IAM token by calling the API for obtaining a token with an OpenID Connect ID token. The X-Subject-Token field in the response header is the IAM token. In this way, you can access cloud services using the IAM token.

If the serviceAccountToken is used for over 24 hours or 80% of the expiry period, kubelet automatically rotates the serviceAccountToken.

An example is as follows:

curl -i --location --request POST 'https://{{iam endpoint}}/v3.0/OS-AUTH/id-token/tokens' \
 --header 'X-Idp-Id: workload_identity' \
 --header 'Content-Type: application/json' \
 --data @token_body.json

Specifically:

• {{iam endpoint}} indicates the endpoint of IAM. For details, see Regions and Endpoints.
• workload_identity is the identity provider name, which is the same as that configured in Configuring an Identity Provider.
• token_body.json is a local file and its content is as follows:

   { 
     "auth" : { 
       "id_token" : { 
         "id" : "eyJhbGciOiJSU..."
       }, 
       "scope": { 
         "project" : { 
           "id" : "46419baef4324...",
           "name" : "cn-north-4"
         } 
       } 
     } 
   }

  • $.auth.id_token.id: The value is the content of the /var/run/secrets/tokens/oidc-token file in the container.
  • $.auth.scope.project.id: indicates the project ID. For details about how to obtain the project ID, see Obtaining a Project ID.
  • $.auth.scope.project.name: indicates the project name, for example, cn-north-4.