Updated on 2024-04-23 GMT+08:00

Pre-upgrade Check

The system automatically checks a cluster before its upgrade. If the cluster does not meet the pre-upgrade check conditions, the upgrade cannot continue. To avoid risks, you can perform pre-upgrade check according to the check items and solutions described in this section.

Table 1 Check items

No.

Check Item

Description

1

Node Restrictions

  • Check whether the node is available.
  • Check whether the node OS supports the upgrade.
  • Check whether the node is marked with unexpected node pool labels.
  • Check whether the Kubernetes node name is the same as the ECS name.

2

Upgrade Management

Check whether the target cluster is under upgrade management.

3

Add-ons

  • Check whether the add-on status is normal.
  • Check whether the add-on support the target version.

4

Helm Charts

Check whether the current HelmRelease record contains discarded Kubernetes APIs that are not supported by the target cluster version. If yes, the Helm chart may be unavailable after the upgrade.

5

SSH Connectivity of Master Nodes

Check whether CCE can connect to your master nodes.

6

Security Groups

Check whether the Protocol & Port of the worker node security groups is set to ICMP: All and whether the security group with the source IP address set to the master node security group is deleted.

7

Arm Node Restrictions

  • Check whether the cluster contains Arm nodes.

8

To-Be-Migrated Nodes

Check whether the node needs to be migrated.

9

Discarded Kubernetes Resources

Check whether there are discarded resources in the clusters.

10

Compatibility Risks

Read the version compatibility differences and ensure that they are not affected. The patch upgrade does not involve version compatibility differences.

11

CCE Agent Versions

Check whether cce-agent on the current node is of the latest version.

12

Node CPU Usage

Check whether the CPU usage of the node exceeds 90%.

13

CRDs

  • Check whether the key CRD packageversions.version.cce.io of the cluster is deleted.
  • Check whether the cluster key CRD network-attachment-definitions.k8s.cni.cncf.io is deleted.

14

Node Disks

  • Check whether the key data disks on the node meet the upgrade requirements.
  • Check whether the /tmp directory has 500 MB available space.

15

Node DNS

  • Check whether the DNS configuration of the current node can resolve the OBS address.
  • Check whether the current node can access the OBS address of the storage upgrade component package.

16

Node Key Directory File Permissions

Check whether the owner and owner group of the files in the /var/paas directory used by the CCE are both paas.

17

Kubelet

Check whether the kubelet on the node is running properly.

18

Node Memory

Check whether the memory usage of the node exceeds 90%.

19

Node Clock Synchronization Server

Check whether the clock synchronization server ntpd or chronyd of the node is running properly.

20

Node OS

Check whether the OS kernel version of the node is supported by CCE.

21

Node CPUs

Check whether the number of CPUs on the master node is greater than 2.

22

Node Python Commands

Check whether the Python commands are available on a node.

23

ASM Version

  • Check whether ASM is used by the cluster.
  • Check whether the current ASM version supports the target cluster version.

24

Node Readiness

Check whether the nodes in the cluster are ready.

25

Node journald

Check whether journald of a node is normal.

26

containerd.sock

Check whether the containerd.sock file exists on the node. This file affects the startup of container runtime in the Euler OS.

27

Internal Errors

Before the upgrade, check whether an internal error occurs.

28

Node Mount Points

Check whether inaccessible mount points exist on the node.

29

Kubernetes Node Taints

Check whether the taint needed for cluster upgrade exists on the node.

30

Everest Restrictions

Check whether there are any compatibility restrictions on the current Everest add-on.

31

cce-hpa-controller Restrictions

Check whether the current cce-controller-hpa add-on has compatibility restrictions.

32

Enhanced CPU Policies

Check whether the current cluster version and the target version support enhanced CPU policy.

33

Health of Worker Node Components

Check whether the container runtime and network components on the worker nodes are healthy.

34

Health of Master Node Components

Check whether the Kubernetes, container runtime, and network components of the master nodes are healthy.

35

Memory Resource Limit of Kubernetes Components

Check whether the resources of Kubernetes components, such as etcd and kube-controller-manager, exceed the upper limit.

36

Discarded Kubernetes APIs

The system scans the audit logs of the past day to check whether the user calls the deprecated APIs of the target Kubernetes version.
NOTE:

Due to the limited time range of audit logs, this check item is only an auxiliary method. APIs to be deprecated may have been used in the cluster, but their usage is not included in the audit logs of the past day. Check the API usage carefully.

37

IPv6 Capabilities of a CCE Turbo Cluster

If IPv6 is enabled for a CCE Turbo cluster, check whether the target cluster version supports IPv6.

38

Node NetworkManager

Check whether NetworkManager of a node is normal.

39

Node ID File

Check the ID file format.

40

Node Configuration Consistency

When you upgrade a cluster to v1.19 or later, the system checks whether the following configuration files have been modified on the backend:

41

Node Configuration File

Check whether the configuration files of key components exist on the node.

42

CoreDNS Configuration Consistency

Check whether the current CoreDNS key configuration Corefile is different from the Helm release record. The difference may be overwritten during the add-on upgrade, affecting domain name resolution in the cluster.