Help Center/ Cloud Container Engine/ Best Practices/ DevOps/ Installing and Deploying Jenkins on CCE/ Procedure/ Interconnecting Jenkins with RBAC of Kubernetes Clusters (Example)
Updated on 2025-07-11 GMT+08:00
View PDF
Share

Interconnecting Jenkins with RBAC of Kubernetes Clusters (Example)

Prerequisites

RBAC must be enabled for the cluster.

Scenario 1: Namespace-based Permissions Control

Create a service account and a Role, define a RoleBinding, and bind the service account to the Role.

# kubectl create ns dev
# kubectl -n dev create sa dev

# cat <<EOF > dev-user-role.yml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: dev
  name: dev-user-pod
rules:
- apiGroups: ["*"]
  resources: ["deployments", "pods", "pods/log"]
  verbs: ["get", "watch", "list", "update", "create", "delete"]
EOF
# kubectl create -f dev-user-role.yml

# kubectl create rolebinding dev-view-pod \
    --role=dev-user-pod \
    --serviceaccount=dev:dev \
    --namespace=dev

Generate the kubeconfig file of a specified service account.

  • In clusters earlier than v1.21, tokens are obtained by mounting the secret of a service account to a pod. Such tokens are permanent. However, this approach is not recommended in clusters of v1.21 or later. Starting from v1.25, Kubernetes no longer automatically creates secrets for service accounts as part of its community iteration policy.

    Instead, in clusters of v1.21 and later, the recommended approach is to use the TokenRequest API to obtain tokens and mount them via a projected volume to pods. These tokens remain valid only for a fixed period and become invalid once the pods are deleted. For details, see Service Account Token Security Improvement.

  • If you need a token that never expires, you can manually manage secrets for service accounts. Although a permanent service account token can be created manually, you are advised to use a short-lived token by calling the TokenRequest API for better security.

Because the cluster version used in this case is v1.25, the ServiceAccount will not have a secret created automatically. This example shows how to manually create a secret named dev-secret and associate it with the ServiceAccount named dev.

The manually created secret must be in the same namespace as the ServiceAccount to be associated with. Otherwise, the creation may fail.

# kubectl apply -f - <<EOF
apiVersion: v1 
kind: Secret 
metadata:
  namespace: dev  # Namespace
  name: dev-secret   
  annotations:     
    kubernetes.io/service-account.name: dev 
type: kubernetes.io/service-account-token 
EOF

Check whether dev-secret has been created. If dev-secret is present in secrets of the dev namespace, then it has been created.

# kubectl get secrets -n dev
NAME               TYPE                                  DATA        AGE
default-secret     kubernetes.io/dockerconfigjson           1      2d22h
dev-secret       kubernetes.io/service-account-token      3      4h14m
paas.elb           cfe/secure-opaque                        1      2d22h

Generate a kubeconfig file using dev-secret.

# API_SERVER="https://172.22.132.51:6443"
# CA_CERT=$(kubectl -n dev get secret dev-secret -o yaml | awk '/ca.crt:/{print $2}')
# cat <<EOF > dev.conf
apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority-data: $CA_CERT
    server: $API_SERVER
  name: cluster
EOF

# TOKEN=$(kubectl -n dev get secret dev-secret -o go-template='{{.data.token}}')
# kubectl config set-credentials dev-user \
    --token=`echo ${TOKEN} | base64 -d` \
    --kubeconfig=dev.conf

# kubectl config set-context default \
    --cluster=cluster \
    --user=dev-user \
    --kubeconfig=dev.conf

# kubectl config use-context default \
    --kubeconfig=dev.conf

Verify the configuration.

# kubectl --kubeconfig=dev.conf get po
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:dev:dev" cannot list pods in the namespace "default"

# kubectl -n dev --kubeconfig=dev.conf run nginx --image nginx --port 80 --restart=Never
# kubectl -n dev --kubeconfig=dev.conf get po
NAME      READY     STATUS    RESTARTS   AGE
nginx     1/1       Running   0          39s

Verify whether the permissions meet the expectation in Jenkins.

  1. On the Jenkins dashboard page, click Manage Jenkins, choose Security > Manage Credentials, and click the default global credential storage domain of Jenkins. You can also create a domain.

  2. Click Add Credentials. Set Kind to Secret file, Scope to Global (Jenkins, nodes, items, all child items, etc), and File to the downloaded kubeconfig configuration file, retain the default values for other parameters, and click Create.
  3. Start the Jenkins job. In this example, Jenkins fails to be deployed in namespace default but is successfully deployed in namespace dev.

Scenario 2: Resource-based Permissions Control

  1. Generate the service account, role, and binding. 

    # kubectl -n dev create sa sa-test0304

# cat <<EOF > test0304-role.yml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: dev
  name: role-test0304
rules:
- apiGroups: ["*"]
  resources: ["deployments"]
  resourceNames: ["tomcat03", "tomcat04"]
  verbs: ["get", "update", "patch"]
EOF
# kubectl create -f test0304-role.yml

# kubectl create rolebinding test0304-bind \
    --role=role-test0304 \
    --serviceaccount=dev:sa-test0304\
    --namespace=dev

  2. Generate a kubeconfig file.

    • In clusters earlier than v1.21, tokens are obtained by mounting the secret of a service account to a pod. Such tokens are permanent. However, this approach is not recommended in clusters of v1.21 or later. Starting from v1.25, Kubernetes no longer automatically creates secrets for service accounts as part of its community iteration policy.

      Instead, in clusters of v1.21 and later, the recommended approach is to use the TokenRequest API to obtain tokens and mount them via a projected volume to pods. These tokens remain valid only for a fixed period and become invalid once the pods are deleted. For details, see Service Account Token Security Improvement.

    • If you need a token that never expires, you can manually manage secrets for service accounts. Although a permanent service account token can be created manually, you are advised to use a short-lived token by calling the TokenRequest API for better security.

    Because the cluster version used in this case is v1.25, the ServiceAccount will not have a secret created automatically. This example shows how to manually create a secret named test-secret and associate it with the ServiceAccount named sa-test0304.

    # kubectl apply -f - <<EOF
apiVersion: v1 
kind: Secret 
metadata:
  namespace: dev   
  name: test-secret   
  annotations:     
    kubernetes.io/service-account.name: sa-test0304 
type: kubernetes.io/service-account-token 
EOF

    Check whether test-secret has been created. If test-secret is present in secrets of the dev namespace, then it has been created.

    # kubectl get secrets -n dev
NAME               TYPE                                  DATA        AGE
default-secret     kubernetes.io/dockerconfigjson           1      2d22h
dev-secret         kubernetes.io/service-account-token      3      4h14m
paas.elb           cfe/secure-opaque                        1      2d22h
test-secret       kubernetes.io/service-account-token      3      25m

    Generate a kubeconfig file using test-secret.

    # API_SERVER=" https://192.168.0.153:5443"
# CA_CERT=$(kubectl -n dev get secret test-secret -o yaml | awk '/ca.crt:/{print $2}')
# cat <<EOF > test0304.conf
apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority-data: $CA_CERT
    server: $API_SERVER
  name: cluster
EOF

# TOKEN=$(kubectl -n dev get secret test-secret -o go-template='{{.data.token}}')
# kubectl config set-credentials test0304-user \
    --token=`echo ${TOKEN} | base64 -d` \
    --kubeconfig=test0304.conf

# kubectl config set-context default \
    --cluster=cluster \
    --user=test0304-user \
    --kubeconfig=test0304.conf

# kubectl config use-context default \
    --kubeconfig=test0304.conf

  3. Verify that Jenkins is running as expected.

    In the pipeline script, update the Deployments of tomcat03, tomcat04, and tomcat05 in sequence.

           try { 
         kubernetesDeploy(
             kubeconfigId: "test0304",
             configs: "test03.yaml")
         println "hooray, success"
        } catch (e) {
            println "oh no! Deployment failed! "
            println e
        }
        echo "test04"
        try { 
         kubernetesDeploy(
             kubeconfigId: "test0304",
             configs: "test04.yaml")
         println "hooray, success"
        } catch (e) {
            println "oh no! Deployment failed! "
            println e
        }
        echo "test05"
        try { 
         kubernetesDeploy(
             kubeconfigId: "test0304",
             configs: "test05.yaml")
         println "hooray, success"
        } catch (e) {
            println "oh no! Deployment failed! "
            println e
        }

    The following shows examples of the running results.

    Figure 1 test03
    Figure 2 test04

Parent Topic: Procedure