Node Mounting

Check Items

This section describes how to diagnose and fix node-mount failures caused by CCE storage misconfigurations.

Solution

• Check item 1: Mount point conflicts

  Check Content

  Verify that all required paths are mounted as expected. CCE needs to use the subdirectories of these paths. If the paths have been mounted, they are inaccessible to CCE nodes. In this case, an exception may occur when the node is restarted or a node component is restarted during an upgrade.

  If there is a vgpaas-share volume group (shared disk), check only the /mnt path.

  If there is no vgpaas-share volume group (not a shared disk), check the /mnt, /mnt/paas, /mnt/paas/kubernetes, /var, and /var/lib paths.

  Solution

  If the error message "the file path xxx is mounted, but the subpath is used by the CCE" is displayed, a path is mounted but CCE still needs to access its subpath. Check the node's pre-installation script for any mount commands that target the affected path. Remove the command or remount to an unused path. Then, drain and reset the node.

• Check item 2: symlinks and bind mounts

  Check Content

  Verify that the following key directories are correctly configured:

  1. General check: Check whether /var/lib/kubelet is linked to /mnt/paas/kubernetes/kubelet.
  2. Conditional check: when a vgpaas-share volume group (shared disk) is used
     • If Docker is used, check whether /var/lib/docker is linked to /mnt/paas/runtime.
     • If containerd is used, check whether /var/lib/containerd is linked to /mnt/paas/runtime.

  Solution

  If the error message "the target path of the path link xxx" is displayed, the symlink is missing or points to a wrong location. If ls on the symlink target returns content, drain and reset the node. Use the fixes below only if the symlink or mount point is empty. The following uses /var/lib/kubelet as an example:

  • Case 1: A symlink is missing.

    Create a symbolic link.

    sudo ln -sf /mnt/paas/kubernetes/kubelet /var/lib/kubelet

  • Case 2: A symlink points to a wrong location.
    1. Remove the incorrect symlink.

       sudo rm /var/lib/kubelet

    2. Create a correct symlink.

       sudo ln -sf /mnt/paas/kubernetes/kubelet /var/lib/kubelet

Common Issues

How Do I Check Whether the vgpaas-share Volume Group (Shared Disk) Exists?

1. Log in to the target node based on the check information.
2. Run the lsblk command to check whether vgpaas-share is mounted to /mnt/paas. If yes, a shared disk is used.
   Figure 1 Checking whether a shared disk is used