Help Center/ Cloud Container Engine/ User Guide/ Nodes/ Node O&M/ Differences in kubelet and Runtime Component Configurations Between CCE and the Native Community
Updated on 2024-11-11 GMT+08:00

Differences in kubelet and Runtime Component Configurations Between CCE and the Native Community

To maintain the stability of nodes, CCE stores Kubernetes and container runtime components on separate data disks. Kubernetes uses the /mnt/paas/kubernetes directory, and the container runtime uses the /mnt/paas/runtime directory. The paths to these directories are the same as the default paths used in the community through soft links.

Differences in Default Paths to kubelet and Runtime

Path Name

Kubernetes Native Path

Path on CCE Nodes

kubelet root-dir

/var/lib/kubelet

/mnt/paas/kubernetes/kubelet

kubelet

/var/lib/kubelet

/mnt/paas/kubernetes/kubelet

The soft link from /var/lib/kubelet to /mnt/paas/kubernetes/kubelet is also available.

Container runtime (Docker)

/var/lib/docker

Container runtime (containerd)

/var/lib/containerd

containerd logs

/var/log/pods

/var/lib/containerd/container_logs

The soft link from /var/log/pods to /var/lib/containerd/container_logs is also available.

The differences in default paths to kubelet and runtime between CCE and the native community may have the following impacts:

  • If a soft link file is mounted into a container, the real path that the soft link file points to cannot be accessed.

    For example, if /var/log is mounted to /mnt/log in a container through a hostPath, /mnt/log/pods is an abnormal soft link file in the container, and the actual data in /var/log/pods cannot be accessed.

    Mount the actual file path to the container to prevent a file read failure caused by soft links.

  • The kubelet root-dir path (/mnt/paas/kubernetes/kubelet) is inconsistent with the community path (/var/lib/kubelet). If the container mount path of a third-party CSI add-on is the community path, the file mounting does not take effect.

    For example, when Vault (an open-source third-party add-on) uses secrets-store-csi-driver to mount a secret, if the root-dir path to the add-on is inconsistent with the CCE configuration path (the default value of the add-on is the same as the community path /var/lib/kubelet), the Vault secret cannot be obtained in the container.

    This is because CSI uses mountPropagation to mount volumes to target containers. kubelet includes the mount path (/mnt/paas/kubernetes/kubelet/pods/{podID}/volume/...) related to kubelet root-dir into the CSI request for mounting a volume. When the CSI container mounts a volume to the mount path requested by kubelet CSI, if the mount path is irrelevant to the hostPath, the mount propagation will be invalid. For example, the CSI container mounts host /var/lib/kubelet to container /var/lib/kubelet, and /mnt/paas/kubernetes/kubelet/pods/{podID}/volume/... in the CSI container is irrelevant to the hostPath.

    Update the root-dir path in CSI to match the kubelet root-dir path on the current CCE node.