Resetting a Node

Scenario

You can reset a node to modify the node configuration, such as the node OS and login mode.

Resetting a node will reinstall the node OS and the Kubernetes software on the node. If a node is unavailable because you modify the node configuration, you can reset the node to rectify the fault.

Notes and Constraints

• Node resetting in CCE standard clusters or CCE Turbo clusters is supported for cluster versions v1.13 and later.
• For Kunpeng clusters, the version must be v1.15 or later to support node resetting.

Precautions

• Only worker nodes can be reset. If the node is still unavailable after the resetting, delete the node and create a new one.
• After a node is reset, its OS will be reinstalled. Before resetting a node, drain it to gracefully evict the pods running on the node to other available nodes. Perform this operation during off-peak hours. If pods remain when the node is reset, CCE will evict them. If a Pod Disruption Budget (PDB) policy is configured for your pods, they may fail to be evicted. In this case, the node will be forcibly reset after a period of time.
• After a node is reset, its system disk and data disks will be cleared. Back up important data before resetting a node.
• If an ECS node has a raw data disk attached (not using LVM), detach it before resetting the node. After resetting, the original attachment information will be cleared. Re-attach the disk to the ECS node to retain the data.
• The IP addresses of the workload pods on the node will change, but the container network access is not affected.
• When a node is reset, the backend will make it unschedulable.
• Resetting a node will clear the Kubernetes labels and taints you added (those added by editing a node pool will not be lost). As a result, node-specific resources (such as local storage and workloads scheduled to this node) may be unavailable. For nodes in the default node pool, the advanced settings on the reset nodes' configuration page automatically detect and populate any custom labels and taints you've previously applied. Review these auto-filled values carefully. If they do not match your requirements, modify them manually before proceeding.
• Resetting a node will cause PVC/PV data loss for the local PV associated with the node. These PVCs and PVs cannot be restored or used again. In this scenario, the pod that uses the local PV is evicted from the node. A new pod is created and stays in the pending state. This is because the PVC used by the pod has a node label, due to which the pod cannot be scheduled. After the node is reset, the pod may be scheduled to the reset node. In this case, the pod remains in the creating state because the underlying logical volume corresponding to the PVC does not exist.

Resetting Nodes in the Default Pool

1. Log in to the CCE console and click the cluster name to access the cluster console.
2. In the navigation pane, choose Nodes. On the displayed page, click the Nodes tab.
3. In the node list of the default pool, select one or more nodes to be reset and choose More > Reset Node in the Operation column.
4. Specify node parameters.
   Compute Settings

   Table 1 Configuration parameters

   Parameter	Description
   Specifications	Specifications cannot be modified when you reset a node.
   Container Engine	CCE supports Docker or containerd as the container runtime. Available runtimes vary by cluster type, version, and OS. Select the runtime shown on the CCE console. For details, see Mapping Between Node OSs and Container Runtimes.
   OS	Select an OS type. Different types of nodes support different OSs. Public image: Select a public image for the node. Private image: Select a private image for the node. For details about how to create a private image, see Creating a Custom CCE Node Image. Shared image: You can select an image shared by other users. This option is supported by BMS and PM nodes. NOTE: Service container runtimes share the kernel and underlying calls of nodes. To ensure compatibility, select a Linux distribution version that is the same as or close to that of the final service container image for the node OS.
   Login Mode	Password The default username is root. Enter the password for logging in to the node and confirm the password. Be sure to remember the password as you will need it when you log in to the node. Key Pair Select the key pair used to log in to the node. Private key pairs are accessible only to you. Account key pairs are shared with all users in the account. A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click Create Key Pair. For details about how to create a key pair, see Creating a Key Pair. Image password (supported for ECSs or PMs whose OSs are private images) Retain the password of the selected image. To use this option, ensure that you have set a password for the selected image.

   Storage Settings

   Configure storage resources on a node for the containers running on it.

   Table 2 Configuration parameters

   Parameter	Description
   System Disk	Directly use the system disk of the cloud server.
   Default Data Disk	Select a data disk for the container runtime and kubelet.
   System Component Storage	Select a disk for storing system components. Data Disk: A default data disk must be added for the container runtime and kubelet. This data disk cannot be detached. Otherwise, the node will be unavailable. System Disk: stores CCE resources such as downloaded images, ephemeral storage for containers, and container stdout logs. If the system disk is fully occupied, it will negatively affect the stability of the node. NOTE: In clusters v1.23.18-r0, v1.25.13-r0, v1.27.10-r0, v1.28.8-r0, v1.29.4-r0, or later, you can select a disk for storing system components. If CCE Node Problem Detector is used, ensure its version is 1.19.2 or later.
   Data Disk	A default data disk must be available for storing container runtime and kubelet components if System Component Storage is set to Data Disk. This data disk cannot be detached. Otherwise, the node will be unavailable. This function is available for clusters of a version earlier than v1.23.18-r0, v1.25.13-r0, v1.27.10-r0, v1.28.8-r0, or v1.29.4-r0. If System Component Storage is set to System Disk, you do not need to add a default data disk. All data disks are general-purpose ones. This function is available for clusters v1.23.18-r0, v1.25.13-r0, v1.27.10-r0, v1.28.8-r0, v1.29.4-r0, or later versions. Click Expand to configure Data Disk Space Allocation. This reserves space for the container runtime, images, and ephemeral storage to ensure normal operation. For details about how to allocate data disk space, see Space Allocation of a Data Disk. For other data disks, a raw disk is created without any processing by default. You can also click Expand and select Mount Disk to mount the data disk to a specified directory. Data disks can also be used as local PVs or local EVs.

   Advanced Settings

   Table 3 Advanced configuration parameters

   Parameter	Description
   Resource Tag	You can add resource tags to classify resources. A maximum of eight resource tags can be added. You can create predefined tags on the TMS console. These tags are available to all resources that support tags. You can use these tags to improve the tag creation and resource migration efficiency. For details, see Creating Predefined Tags. CCE will automatically create the CCE-Dynamic-Provisioning-Node=Node ID tag.
   Kubernetes Label	Click Add Label to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 20 labels can be added. Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see Labels and Selectors.
   Kubernetes Taint	This parameter is left blank by default. You can add taints to configure anti-affinity for the node. A maximum of 20 taints are allowed for each node. Each taint contains the following parameters: Taint Key: A key must contain 1 to 63 characters, starting and ending with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as a key prefix. Taint Value: A value must contain 1 to 63 characters, starting and ending with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. Effect: Available options are NoSchedule, PreferNoSchedule, and NoExecute. NOTE: If taints are used, you must configure pod tolerations. Otherwise, scale-out operations may fail, or pods may fail to schedule onto the added nodes.
   Max. Pods	Maximum number of pods that can run on the node, including the default system pods. Value range: 16 to 256 This limit prevents the node from being overloaded with pods.
   Pre-installation Command	Installation script command, in which CJK characters are not allowed. The script command will be Base64-transcoded. The characters of both the pre-installation and post-installation scripts are centrally calculated, and the total number of characters after transcoding cannot exceed 10,240. The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed.
   Post-installation Command	Installation script command, in which CJK characters are not allowed. The script command will be Base64-transcoded. The characters of both the pre-installation and post-installation scripts are centrally calculated, and the total number of characters after transcoding cannot exceed 10,240. The script will be executed after Kubernetes software is installed, which does not affect the installation. During post-installation script execution, pods can be scheduled normally. However, if the script times out, node installation fails. To prevent pod scheduling on nodes with incomplete script execution, enable the option to schedule pods only after post-installation script completion. CAUTION: Do not use the reboot command in the post-installation script to restart the system immediately. Instead, use the shutdown -r 1 command to restart the system with a one-minute delay.

5. Click Next: Confirm.
6. Read the instructions, select the check box for confirmation, and click Submit.

Resetting Nodes in a Node Pool

• When resetting a node in a node pool, you can only change its storage configuration. All other configurations will follow the settings of the node pool.
• Resetting a node will execute the pre- and post-installation scripts in the current node pool and update the security group configurations to those of the node pool.

1. Log in to the CCE console and click the cluster name to access the cluster console.
2. In the navigation pane, choose Nodes. On the displayed page, click the Nodes tab.
3. In the node list of the target node pool, select a node to be reset and choose More > Reset Node in the Operation column.
4. Modify the node storage parameters.

   Table 4 Configuration parameters

   Parameter	Description
   System Disk	Directly use the system disk of the cloud server.
   Default Data Disk	Select a data disk for container runtime and kubelet.
   System Component Storage	Select a disk for storing system components. Data Disk: A default data disk must be added for the container runtime and kubelet. This data disk cannot be detached. Otherwise, the node will be unavailable. System Disk: stores CCE resources such as downloaded images, ephemeral storage for containers, and container stdout logs. If the system disk is fully occupied, it will negatively affect the stability of the node. NOTE: In clusters v1.23.18-r0, v1.25.13-r0, v1.27.10-r0, v1.28.8-r0, v1.29.4-r0, or later, you can select a disk for storing system components. If CCE Node Problem Detector is used, ensure its version is 1.19.2 or later.
   Data Disk	Configure advanced settings for each data disk. For the default data disk, click Expand to configure Data Disk Space Allocation. This reserves space for the container runtime, images, and ephemeral storage to ensure they run normally. For details, see Space Allocation of a Data Disk. For a non-default data disk, click Expand and configure mount options. Default: The data disk is attached as a raw disk without any settings. Mount Disk: The data disk is attached to the service directory path. This parameter cannot be left blank or set to a key OS path such as the root directory. Use as PV: The data disk is used as persistent storage volumes for PVCs. For details, see Local PVs. Use as ephemeral volume: The data disk is used as ephemeral storage volumes for PVCs. For details, see Local EV.

5. Read the instructions, select the check box for confirmation, and click OK.

Resetting Nodes in a Batch

Resetting nodes in a batch varies depending on application scenarios.

Common Issues