Rotating Cluster Certificates
A Kubernetes cluster relies on multiple certificate chains and credentials. Proper lifecycle management, including timely rotation, is critical to cluster security and workload integrity. When a cluster is created, CCE automatically generates certificates to secure communication among control plane components (kube-apiserver, etcd, kube-controller-manager, kube-scheduler) and cluster nodes. You may also provide custom certificates for agent authentication and kube-apiserver aggregation layer configuration. If certificates expire without rotation, cluster functionality will fail, affecting running workloads.
CCE cluster certificates have a default validity period of five years. To prevent service interruptions, rotate certificates before expiry. CCE supports manual certificate rotation via the console or API. CCE sends a reminder before certificate expiry, displays an expiration alert on the console, and prompts you to take timely action to ensure continuous and secure cluster operations.
Precautions
- Certificate rotation is only available for clusters v1.19 or later. If your cluster is earlier than v1.19, upgrade the cluster first.
- kube-apiserver access is briefly interrupted during each rotation. This may affect the workloads that depend on kube-apiserver. You can schedule the rotations during off-peak hours to reduce service disruptions.
- Kubernetes components on cluster nodes restart during certificate rotation. If CCE Node Problem Detector is installed, you may receive transient exception alerts. These do not indicate actual faults.
- Do not modify the cluster during certificate rotation.
- Rotation duration depends on cluster size. Typically, it takes 10 to 15 minutes. Upon successful rotation, certificates are valid for five years.
- If a cluster certificate has expired, contact technical support.
How to Rotate CCE Cluster Certificates
CCE cluster certificates consist of system-generated certificates and self-managed certificates. You can rotate them directly or with a cluster upgrade.
Before a certificate expires, the Update Certificate button is displayed in the cluster list or on the cluster overview page of the CCE console. You can also configure notifications through internal messages, emails, and SMS messages. For details, see Configuring Message Notification.
- Log in to the CCE console.
- View the cluster certificate expiration alert on the console.
- If a cluster certificate is about to expire in 30 days, a certificate expiration alert is displayed when you access the CCE console, and the alert is also displayed in the cluster list. If you click the cluster name to view the cluster details, the cluster certificate expiration alert is also displayed on the cluster overview page.
- If a cluster certificate is about to expire in 180 days, a certificate expiration alert is displayed on the cluster overview page when you click the cluster name to view the cluster details.
- Click Update Certificate as prompted on the CCE console. Carefully review the risk warnings and complete the certificate rotation as prompted.
Enabling OpenID Connect (OIDC) for a cluster will invalidate the existing cluster signing public key. You must obtain a new key after the certificate rotation completes, or OIDC authentication will fail.
If you provide a self-managed certificate during cluster creation, Update Certificate appears in the cluster list or on the CCE cluster overview page before expiry. Alert behavior is identical to that for system-generated certificates. To complete rotation, upload the CA root certificate, client certificate, and client private key as prompted.
- Log in to the CCE console.
- Click the cluster name to access the cluster console. In the navigation pane, choose Cluster Upgrade.
- Specify a cluster upgrade path and back up data. CCE then performs a pre-upgrade check.
- During upgrade configuration, specify cluster certificate settings and choose whether to rotate certificates synchronously with the upgrade.
- Rotate CCE Cluster Certificate: You can choose whether to rotate a CCE cluster certificate during a cluster upgrade based on the remaining validity period of the certificate. Upon successful rotation, certificates are valid for five years.
Enabling OpenID Connect (OIDC) for a cluster will invalidate the existing cluster signing public key. You must obtain a new key after the cluster upgrade, or OIDC authentication will fail.
- Rotate Self-Managed Certificate: If you provided a self-managed certificate during cluster creation, you can choose whether to rotate it during upgrade. To complete rotation, upload the CA root certificate, client certificate, and client private key as prompted.
- Rotate CCE Cluster Certificate: You can choose whether to rotate a CCE cluster certificate during a cluster upgrade based on the remaining validity period of the certificate. Upon successful rotation, certificates are valid for five years.
- Click Upgrade.
Certificates to Be Rotated
The following table lists the certificates to be rotated on worker nodes.
| Certificate | Certificate Path | Certificate Validity Period |
|---|---|---|
| kube-node-tls.crt kube-node-tls.key | /var/paas/srv/kubernetes | The initial certificate is valid for five years. After the certificate is rotated, the validity period is extended by five years. |
| kube-proxy-tls.crt kube-proxy-tls.key | /var/paas/srv/kubernetes | The initial certificate is valid for five years. After the certificate is rotated, the validity period is extended by five years. |
| kubelet-client-current.pem kubelet-server-current.pem | /opt/cloud/cce/kubernetes/kubelet/pki/ | The certificate is automatically rotated before expiration. The validity period is 1 year. |
| kubecfg.crt kubecfg_crypto.key | /var/paas/srv/kubernetes | The initial certificate is valid for five years. After the certificate is rotated, the validity period is extended by five years. |
Checking the Certificate Expiration Date
You can query the certificate expiration date of a cluster through an API. For details, see Reading a Specified Cluster and Listing Clusters in a Specified Project. For details about how to call the API, see Making an API Request.
The API for reading a specified cluster is used as an example to describe how to check the certificate expiration date. The URI of the API is as follows:
GET /api/v3/projects/{project_id}/clusters/{cluster_id}
| Parameter | Description |
|---|---|
| project_id | Project ID. For details about how to obtain a project ID, see How to Obtain Parameters in the API URI. |
| cluster_id | Cluster ID. For details about how to obtain a cluster ID, see How to Obtain Parameters in the API URI. |
The following is an example of the request body for reading a specified cluster:
Configuring Message Notification
You can add recipients for cluster certificate expiration notifications by email, SMS, or internal message.
- Click
to go to Message Center, choose Message Receiving Management > Recipient Management in the navigation pane, and click Add Recipient to add the recipients and their contact information. 
- Choose SMS & Email Settings in the navigation pane, select O&M > Upgrade reminder, and click Modify Recipient.
- Select the recipients who need to receive the message and click OK.
Common Issues
- OIDC-based workload identity authentication becomes invalid after the cluster certificate is rotated.
After the certificate is rotated, using the old certificate results in a signature verification failure. In this case, you need to update the cluster public key.
Solution
Obtain the signature public key of the CCE cluster again by referring to Step 1 in Using OIDC to Authenticate Workloads in a CCE Cluster, and update the signature public key in the identity provider information in IAM by referring to Step 2.
What is your overall rating for this page?
Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot
