Updated on 2024-12-04 GMT+08:00

Draining a Node

Scenario

After you enable nodal drainage on the console, CCE configures the node to be non-schedulable and securely evicts all pods that comply with Nodal Drainage Rules on the node. Subsequent new pods will not be scheduled to this node.

When a node becomes faulty, nodal drainage quickly isolates the faulty node. The pods evicted from the faulty node will be scheduled by the workload controller to other nodes that are running properly.

Constraints

  • Only clusters of the following versions support the nodal drainage function:
    • v1.21: v1.21.10-r0 or later
    • v1.23: v1.23.8-r0 or later
    • v1.25: v1.25.3-r0 or later
    • v1.25 or later
  • To use the nodal drainage function, an IAM user must have at least one of the following permissions. For details, see Namespace Permissions (Kubernetes RBAC-based).
    • cluster-admin (administrator): read and write permissions on all resources in all namespaces.
    • drainage-editor: drain a node.
    • drainage-viewer: view the nodal drainage status but cannot drain a node.
  • If a disruption budget is not specify for the workload, the workload function may be unavailable during pod rescheduling.

Nodal Drainage Rules

The nodal drainage function securely evicts pods on a node. However, for pods that meet the following filtering criteria, the system performs the corresponding operations:

Filter Criterion

Forced Drainage Enabled

Forced Drainage Disabled

The status.phase field of the pod is Succeeded or Failed.

Deletion

Deletion

The pod is not managed by the workload controller.

Deletion

Drainage cancellation

The pod is managed by DaemonSet.

None

Drainage cancellation

A volume of the emptyDir type is mounted to the pod.

Eviction

Drainage cancellation

The pod is a static pod directly managed by kubelet

None

None

The following operations may be performed on pods during nodal drainage:

  • Deletion: The pod is deleted from the current node and will not be scheduled to other nodes.
  • Eviction: The pod is deleted from the current node and rescheduled to another node.
  • None: The pod will not be evicted or deleted.
  • Drainage cancellation: If a pod on a node cancels drainage, the drainage process of the node is terminated and no pod is evicted or deleted.

Procedure

  1. Log in to the CCE console and click the cluster name to access the cluster console.
  2. In the navigation pane, choose Nodes. On the displayed page, click the Nodes tab.
  3. Locate the target node and choose More > Nodal Drainage in the Operation column.
  4. In the Nodal Drainage window displayed, set parameters.

    • Timeout (s): The drainage task automatically fails after the preset timeout period. The value 0 indicates that the timeout period is not set.
    • Forced Drainage: If this function is enabled, pods managed by DaemonSet will be ignored, and pods with emptyDir volumes and pods not managed by controllers will be deleted. For details, see Nodal Drainage Rules.

  5. Click OK and wait until the nodal drainage is complete.