Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Situation Awareness
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive

Before You Start

Updated on 2024-12-18 GMT+08:00

Before the upgrade, you can check whether your cluster can be upgraded and which versions are available on the CCE console. For details, see Upgrade Overview.

Precautions

Before upgrading a cluster, pay attention to the following points:

  • Perform an upgrade during off-peak hours to minimize the impact on your services.
  • Before upgrading a cluster, learn about the features and differences of each cluster version in Kubernetes Release Notes to prevent exceptions due to the use of an incompatible cluster version. For example, check whether any APIs deprecated in the target version are used in the cluster. Otherwise, calling the APIs may fail after the upgrade. For details, see Deprecated APIs.

During a cluster upgrade, pay attention to the following points that may affect your services:

  • During a cluster upgrade, do not perform any operation on the cluster. Do not stop, restart, or delete nodes during cluster upgrade. Otherwise, the upgrade will fail.
  • Before upgrading a cluster, ensure no high-risk operations are performed in the cluster. Otherwise, the cluster upgrade may fail or the configuration may be lost after the upgrade. Common high-risk operations include modifying cluster node configurations locally and modifying the configurations of the listeners managed by CCE on the ELB console. Instead, modify configurations on the CCE console so that the modifications can be automatically inherited during the upgrade.
  • During a cluster upgrade, the running workloads will not be interrupted, but access to the API server will be temporarily interrupted.
  • By default, application scheduling is not restricted during a cluster upgrade. During an upgrade of the following early cluster versions, the node.kubernetes.io/upgrade taint (equivalent to NoSchedule) will be added to the nodes in the cluster. The taint will be removed after the cluster is upgraded:
    • All v1.15 clusters
    • All v1.17 clusters
    • v1.19 clusters with patch versions earlier than or equal to v1.19.16-r4
    • v1.21 clusters with patch versions earlier than or equal to v1.21.7-r0
    • v1.23 clusters with patch versions earlier than or equal to v1.23.5-r0

Notes and Constraints

  • If an error occurred during a cluster upgrade, the cluster can be rolled back using the backup data. If you perform other operations (for example, modifying cluster specifications) after a successful cluster upgrade, the cluster cannot be rolled back using the backup data.
  • When clusters using the tunnel network model are upgraded to v1.19.16-r4, v1.21.7-r0, v1.23.5-r0, v1.25.1-r0, or later, the SNAT rule whose destination address is the container CIDR block but the source address is not the container CIDR block will be removed. If you have configured VPC routes to directly access all pods outside the cluster, only the pods on the corresponding nodes can be directly accessed after the upgrade.
  • For more details, see Version Differences.

Version Differences

Upgrade Path

Version Difference

Self-Check

v1.23 or v1.25

Upgraded to v1.27

Docker is no longer recommended. Use containerd instead. For details, see Container Engine.

This item has been included in the pre-upgrade check.

Table 1 QoS class changes before and after the upgrade

Init Container (Calculated Based on spec.initContainers)

Service Container (Calculated Based on spec.containers)

Pod (Calculated Based on spec.containers and spec.initContainers)

Impacted or Not

Guaranteed

Besteffort

Burstable

Yes

Guaranteed

Burstable

Burstable

No

Guaranteed

Guaranteed

Guaranteed

No

Besteffort

Besteffort

Besteffort

No

Besteffort

Burstable

Burstable

No

Besteffort

Guaranteed

Burstable

Yes

Burstable

Besteffort

Burstable

Yes

Burstable

Burstable

Burstable

No

Burstable

Guaranteed

Burstable

Yes

Deprecated APIs

With the evolution of Kubernetes APIs, APIs are periodically reorganized or upgraded, and old APIs are deprecated and finally deleted. The following tables list the deprecated APIs in each Kubernetes community version. For details about more deprecated APIs, see Deprecated API Migration Guide.

NOTE:

When an API is deprecated, the existing resources are not affected. However, when you create or edit the resources, the API version will be intercepted.

Table 2 APIs deprecated in Kubernetes v1.27

Resource Name

Deprecated API Version

Substitute API Version

Change Description

CSIStorageCapacity

storage.k8s.io/v1beta1

storage.k8s.io/v1

(This API is available since v1.24.)

None

FlowSchema and PriorityLevelConfiguration

flowcontrol.apiserver.k8s.io/v1beta1

flowcontrol.apiserver.k8s.io/v1beta3

(This API is available since v1.26.)

None

HorizontalPodAutoscaler

autoscaling/v2beta2

autoscaling/v2

(This API is available since v1.23.)

None

Table 3 APIs deprecated in Kubernetes v1.25

Resource Name

Deprecated API Version

Substitute API Version

Change Description

CronJob

batch/v1beta1

batch/v1

(This API is available since v1.21.)

None

EndpointSlice

discovery.k8s.io/v1beta1

discovery.k8s.io/v1

(This API is available since v1.21.)

Pay attention to the following changes:

  • In each endpoint, the topology["kubernetes.io/hostname"] field has been deprecated. Replace it with the nodeName field.
  • In each endpoint, the topology["kubernetes.io/zone"] field has been deprecated. Replace it with the zone field.
  • The topology field is replaced with deprecatedTopology and cannot be written in v1.

Event

events.k8s.io/v1beta1

events.k8s.io/v1

(This API is available since v1.19.)

Pay attention to the following changes:

  • The type field can only be set to Normal or Warning.
  • The involvedObject field is renamed regarding.
  • The action, reason, reportingController, and reportingInstance fields are mandatory for creating a new events.k8s.io/v1 event.
  • Use eventTime instead of the deprecated firstTimestamp field (this field has been renamed deprecatedFirstTimestamp and is not allowed to appear in the new events.k8s.io/v1 event object).
  • Use series.lastObservedTime instead of the deprecated lastTimestamp field (this field has been renamed deprecatedLastTimestamp and is not allowed to appear in the new events.k8s.io/v1 event object).
  • Use series.count instead of the deprecated count field (this field has been renamed deprecatedCount and is not allowed to appear in the new events.k8s.io/v1 event object).
  • Use reportingController instead of the deprecated source.component field (this field has been renamed deprecatedSource.component and is not allowed to appear in the new events.k8s.io/v1 event object).
  • Use reportingInstance instead of the deprecated source.host field (this field has been renamed deprecatedSource.host and is not allowed to appear in the new events.k8s.io/v1 event object).

HorizontalPodAutoscaler

autoscaling/v2beta1

autoscaling/v2

(This API is available since v1.23.)

None

PodDisruptionBudget

policy/v1beta1

policy/v1

(This API is available since v1.21.)

If spec.selector is set to null ({}) in PodDisruptionBudget of policy/v1, all pods in the namespace are selected. (In policy/v1beta1, an empty spec.selector means that no pod will be selected.) If spec.selector is not specified, pod will be selected in neither API version.

PodSecurityPolicy

policy/v1beta1

None

Since v1.25, the PodSecurityPolicy resource no longer provides APIs of the policy/v1beta1 version, and the PodSecurityPolicy access controller is deleted.

Use Pod Security Admission instead.

RuntimeClass

node.k8s.io/v1beta1

node.k8s.io/v1 (This API is available since v1.20.)

None

Table 4 APIs deprecated in Kubernetes v1.22

Resource Name

Deprecated API Version

Substitute API Version

Change Description

MutatingWebhookConfiguration

ValidatingWebhookConfiguration

admissionregistration.k8s.io/v1beta1

admissionregistration.k8s.io/v1

(This API is available since v1.16.)

  • The default value of webhooks[*].failurePolicy is changed from Ignore to Fail in v1.
  • The default value of webhooks[*].matchPolicy is changed from Exact to Equivalent in v1.
  • The default value of webhooks[*].timeoutSeconds is changed from 30s to 10s in v1.
  • The default value of webhooks[*].sideEffects is deleted, and this field must be specified. In v1, the value can only be None or NoneOnDryRun.
  • The default value of webhooks[*].admissionReviewVersions is deleted. In v1, this field must be specified. (AdmissionReview v1 and v1beta1 are supported.)
  • webhooks[*].name must be unique in the list of objects created through admissionregistration.k8s.io/v1.

CustomResourceDefinition

apiextensions.k8s.io/v1beta1

apiextensions/v1

(This API is available since v1.16.)

  • The default value of spec.scope is no longer Namespaced. This field must be explicitly specified.
  • spec.version is deleted from v1. Use spec.versions instead.
  • spec.validation is deleted from v1. Use spec.versions[*].schema instead.
  • spec.subresources is deleted from v1. Use spec.versions[*].subresources instead.
  • spec.additionalPrinterColumns is deleted from v1. Use spec.versions[*].additionalPrinterColumns instead.
  • spec.conversion.webhookClientConfig is moved to spec.conversion.webhook.clientConfig in v1.
  • spec.conversion.conversionReviewVersions is moved to spec.conversion.webhook.conversionReviewVersions in v1.
  • spec.versions[*].schema.openAPIV3Schema becomes a mandatory field when the CustomResourceDefinition object of the v1 version is created, and its value must be a structural schema.
  • spec.preserveUnknownFields: true cannot be specified when the CustomResourceDefinition object of the v1 version is created. This configuration must be specified using x-kubernetes-preserve-unknown-fields: true in the schema definition.
  • In v1, the JSONPath field in the additionalPrinterColumns entry is renamed jsonPath (patch #66531).

APIService

apiregistration/v1beta1

apiregistration.k8s.io/v1

(This API is available since v1.10.)

None

TokenReview

authentication.k8s.io/v1beta1

authentication.k8s.io/v1

(This API is available since v1.6.)

None

LocalSubjectAccessReview

SelfSubjectAccessReview

SubjectAccessReview

SelfSubjectRulesReview

authorization.k8s.io/v1beta1

authorization.k8s.io/v1

(This API is available since v1.16.)

spec.group was renamed spec.groups in v1 (patch #32709).

CertificateSigningRequest

certificates.k8s.io/v1beta1

certificates.k8s.io/v1

(This API is available since v1.19.)

Pay attention to the following changes in certificates.k8s.io/v1:
  • For an API client that requests a certificate:
    • spec.signerName becomes a mandatory field (see Known Kubernetes Signers). In addition, the certificates.k8s.io/v1 API cannot be used to create requests whose signer is kubernetes.io/legacy-unknown.
    • spec.usages now becomes a mandatory field, which cannot contain duplicate string values and can contain only known usage strings.
  • For an API client that needs to approve or sign a certificate:
    • status.conditions cannot contain duplicate types.
    • The status.conditions[*].status field is now mandatory.
    • The status.certificate must be PEM-encoded and can contain only the CERTIFICATE data block.

Lease

coordination.k8s.io/v1beta1

coordination.k8s.io/v1

(This API is available since v1.14.)

None

Ingress

networking.k8s.io/v1beta1

extensions/v1beta1

networking.k8s.io/v1

(This API is available since v1.19.)

  • The spec.backend field is renamed spec.defaultBackend.
  • The serviceName field of the backend is renamed service.name.
  • The backend servicePort field represented by a number is renamed service.port.number.
  • The backend servicePort field represented by a string is renamed service.port.name.
  • The pathType field is mandatory for all paths to be specified. The options are Prefix, Exact, and ImplementationSpecific. To match the behavior of not defining the path type in v1beta1, use ImplementationSpecific.

IngressClass

networking.k8s.io/v1beta1

networking.k8s.io/v1

(This API is available since v1.19.)

None

ClusterRole

ClusterRoleBinding

Role

RoleBinding

rbac.authorization.k8s.io/v1beta1

rbac.authorization.k8s.io/v1

(This API is available since v1.8.)

None

PriorityClass

scheduling.k8s.io/v1beta1

scheduling.k8s.io/v1

(This API is available since v1.14.)

None

CSIDriver

CSINode

StorageClass

VolumeAttachment

storage.k8s.io/v1beta1

storage.k8s.io/v1

  • CSIDriver is available in storage.k8s.io/v1 since v1.19.
  • CSINode is available in storage.k8s.io/v1 since v1.17.
  • StorageClass is available in storage.k8s.io/v1 since v1.6.
  • VolumeAttachment is available in storage.k8s.io/v1 since v1.13.
Table 5 APIs deprecated in Kubernetes v1.16

Resource Name

Deprecated API Version

Substitute API Version

Change Description

NetworkPolicy

extensions/v1beta1

networking.k8s.io/v1

(This API is available since v1.8.)

None

DaemonSet

extensions/v1beta1

apps/v1beta2

apps/v1

(This API is available since v1.9.)

  • The spec.templateGeneration field is deleted.
  • spec.selector is now a mandatory field and cannot be changed after the object is created. The label of an existing template can be used as a selector for seamless migration.
  • The default value of spec.updateStrategy.type is changed to RollingUpdate (the default value in the extensions/v1beta1 API version is OnDelete).

Deployment

extensions/v1beta1

apps/v1beta1

apps/v1beta2

apps/v1

(This API is available since v1.9.)

  • The spec.rollbackTo field is deleted.
  • spec.selector is now a mandatory field and cannot be changed after the Deployment is created. The label of an existing template can be used as a selector for seamless migration.
  • The default value of spec.progressDeadlineSeconds is changed to 600 seconds (the default value in extensions/v1beta1 is unlimited).
  • The default value of spec.revisionHistoryLimit is changed to 10. (In the apps/v1beta1 API version, the default value of this field is 2. In the extensions/v1beta1 API version, all historical records are retained by default.)
  • The default values of maxSurge and maxUnavailable are changed to 25%. (In the extensions/v1beta1 API version, these fields default to 1.)

StatefulSet

apps/v1beta1

apps/v1beta2

apps/v1

(This API is available since v1.9.)

  • spec.selector is now a mandatory field and cannot be changed after the StatefulSet is created. The label of an existing template can be used as a selector for seamless migration.
  • The default value of spec.updateStrategy.type is changed to RollingUpdate (the default value in the apps/v1beta1 API version is OnDelete).

ReplicaSet

extensions/v1beta1

apps/v1beta1

apps/v1beta2

apps/v1

(This API is available since v1.9.)

spec.selector is now a mandatory field and cannot be changed after the object is created. The label of an existing template can be used as a selector for seamless migration.

PodSecurityPolicy

extensions/v1beta1

policy/v1beta1

(This API is available since v1.10.)

PodSecurityPolicy for the policy/v1beta1 API version will be removed in v1.25.

Upgrade Backup

How to back up a node:

Backup Type

Backup Object

Backup Mode

Backup Time

Rollback Time

Description

etcd data backup

etcd data

Automatic backup during an upgrade

1-5 minutes

2 hours

Mandatory. The backup is automatically performed during the upgrade.

We use cookies to improve our site and your experience. By continuing to browse our site you accept our cookie policy. Find out more

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback