Help Center/ Cloud Container Engine/ FAQs/ Node/ OSs/ Why Are Pods Evicted by kubelet Due to Abnormal cgroup Statistics?
Updated on 2024-07-04 GMT+08:00
View PDF
Share

Why Are Pods Evicted by kubelet Due to Abnormal cgroup Statistics?

Symptom

On an Arm node, pods are evicted by kubelet due to the abnormal cgroup statistics. As a result, the node runs abnormally.

kubelet keeps evicting pods. After all containers are killed, kubelet still considers that the memory is insufficient.

The resource usage is normal.

The value of usage_in_bytes of cgroup in the /sys/fs/cgroup/memory directory is abnormal.

# cd /sys/fs/cgroup/memory
# cat memory.usage_in_bytes
17618837504

Possible Causes

On an Arm node, the kernel of EulerOS 2.8 and 2.9 has a bug, which causes kubelet to evict pods and results in service unavailability.

This issue has been resolved in the following versions:

  • EulerOS 2.8: kernel-4.19.36-vhulk1907.1.0.h1252.eulerosv2r8.aarch64
  • EulerOS 2.9: kernel-4.19.90-vhulk2103.1.0.h819.eulerosv2r9.aarch64

Solution

  • If your cluster version is 1.19.16-r0, 1.21.7-r0, 1.23.5-r0, 1.25.1-r0, or later, reset the OS of the node to the latest version.
  • If your cluster version does not meet the requirements, upgrade the cluster to the specified version and then reset the node OS to the latest version.
Parent Topic: OSs