Updated on 2024-09-30 GMT+08:00

Container Engines

Introduction to Container Engines

Container engines, one of the most important components of Kubernetes, manage the lifecycle of images and containers. The kubelet interacts with a container runtime through the Container Runtime Interface (CRI).

CCE supports containerd and Docker. containerd is recommended for its shorter traces, fewer components, higher stability, and less consumption of node resources.

Kubernetes has removed dockershim from v1.24 and does not support Docker by default. For details, see Kubernetes is Moving on From Dockershim: Commitments and Next Steps. CCE is going to stop the support for Docker. Change the node container engine from Docker to containerd. For details, see Migrating Nodes from Docker to containerd.

Table 1 Comparison between container engines

Item

containerd

Docker

Tracing

kubelet --> CRI plugin (in the containerd process) --> containerd

  • Docker (Kubernetes v1.23 and earlier versions):

    kubelet --> dockershim (in the kubelet process) --> docker --> containerd

  • Docker (community solution for Kubernetes v1.24 or later):

    kubelet --> cri-dockerd (kubelet uses CRI to connect to cri-dockerd) --> docker--> containerd

Command

crictl/ctr

docker

Kubernetes CRI

Native support

Support through dockershim or cri-dockerd

Pod delayed startup

Minor

High

kubelet CPU/memory usage

Minor

High

Runtime's CPU/memory usage

Minor

High

Mapping Between Node OSs and Container Engines

VPC network clusters of v1.23 or later versions support containerd. Tunnel network clusters of v1.23.2-r0 or later versions support containerd.

Table 2 Node OSs and container engines in CCE clusters

OS

Kernel Version

Container Engine

Container Storage Rootfs

Container Runtime

CentOS 7.6

3.x

Docker

Clusters of v1.23 and later support containerd.

Clusters of v1.19.16 and earlier use Device Mapper.

Clusters of v1.19.16 and later use OverlayFS.

runC

EulerOS 2.3

3.x

Docker

Device Mapper

runC

EulerOS 2.5

3.x

Docker

Device Mapper

runC

EulerOS 2.9

4.x

Docker

Clusters of v1.23 and later support containerd.

OverlayFS

runC

Ubuntu 18.04

4.x

Docker

Clusters of v1.23 and later support containerd.

OverlayFS

runC

Ubuntu 22.04

4.x

Docker

Clusters of v1.23 and later support containerd.

OverlayFS

runC

Huawei Cloud EulerOS 1.1

3.x

Docker

containerd

OverlayFS

runC

Huawei Cloud EulerOS 2.0

5.x

Docker

containerd

OverlayFS

runC

Table 3 Node OSs and container engines in CCE Turbo clusters

Node Type

OS

Kernel Version

Container Engine

Container Storage Rootfs

Container Runtime

ECS (VM)

CentOS 7.6

3.x

Docker

containerd

OverlayFS

runC

Ubuntu 18.04

4.x

EulerOS 2.9

4.x

Huawei Cloud EulerOS 1.1

3.x

Huawei Cloud EulerOS 2.0

5.x

ECS (PM)

EulerOS 2.9

4.x

containerd

Device Mapper

Kata

Table 4 Kunpeng Node OSs and container engines in CCE

OS

Kernel Version

Container Engine

Container Storage Rootfs

Container Runtime

Huawei Cloud EulerOS 2.0

5.x

Docker

containerd

OverlayFS

runC

EulerOS 2.9

4.x

Docker

containerd

OverlayFS

runC

EulerOS 2.8

4.x

Docker

OverlayFS

runC

Common Commands of containerd and Docker

containerd does not support Docker APIs and Docker CLI, but you can run crictl commands to implement similar functions.

Table 5 Image-related commands

Operation

Docker Command

containerd Command

docker

crictl

ctr

List local images.

docker images

crictl images

ctr -n k8s.io i ls

Pull images.

docker pull

crictl pull

ctr -n k8s.io i pull

Push images.

docker push

None

ctr -n k8s.io i push

Delete a local image.

docker rmi

crictl rmi

ctr -n k8s.io i rm

Check images.

docker inspect

crictl inspecti

None

Table 6 Container-related commands

Operation

Docker Command

containerd Command

docker

crictl

ctr

List containers.

docker ps

crictl ps

ctr -n k8s.io c ls

Create a container.

docker create

crictl create

ctr -n k8s.io c create

Start a container.

docker start

crictl start

ctr -n k8s.io run

Stop a container.

docker stop

crictl stop

None

Delete a container.

docker rm

crictl rm

ctr -n k8s.io c del

Connect to a container.

docker attach

crictl attach

None

Access the container.

docker exec

crictl exec

None

Query container details.

docker inspect

crictl inspect

ctr -n k8s.io c info

View container logs.

docker logs

crictl logs

None

Check the resource usage of the container.

docker stats

crictl stats

None

Update container resource limits.

docker update

crictl update

None

Table 7 Pod-related commands

Operation

Docker Command

containerd Command

docker

crictl

ctr

List pods.

None

crictl pods

None

View pod details.

None

crictl inspectp

None

Start a pod.

None

crictl start

None

Run a pod.

None

crictl runp

None

Stop a pod.

None

crictl stopp

None

Delete a pod.

None

crictl rmp

None

Containers created and started by containerd are immediately deleted by kubelet. containerd does not support suspending, resuming, restarting, renaming, and waiting for containers, nor Docker image build, import, export, comparison, push, search, and labeling. containerd does not support file copy. You can log in to the image repository by modifying the configuration file of containerd.

Differences in Tracing

  • Docker (Kubernetes v1.23 and earlier versions):

    kubelet --> dockershim (in the kubelet process) --> docker --> containerd

  • Docker (community solution for Kubernetes v1.24 or later):

    kubelet --> cri-dockerd (kubelet uses CRI to connect to cri-dockerd) --> docker--> containerd

  • containerd:

    kubelet --> CRI plugin (in the containerd process) --> containerd

Although Docker has added functions such as swarm cluster, docker build, and Docker APIs, it also introduces bugs. Compared with containerd, Docker has one more layer of calling. Therefore, containerd is more resource-saving and secure.

Container Engine Versions

  • Docker
    • EulerOS/CentOS: docker-engine 18.9.0, a Docker version customized for CCE. Security vulnerabilities will be fixed promptly.
    • Ubuntu: docker-ce: 24.0.9 (community version). You are advised to use the containerd engine for Ubuntu nodes.

      The open source docker-ce of Ubuntu may trigger bugs when concurrent exec operations are performed (for example, multiple exec probes are configured). You are advised to use HTTP/TCP probes.

  • containerd: 1.6.14

    If the cluster version is v1.28.8-r0, v1.29.4-r0, v1.30.1-r0, or later, upgrade containerd to 1.7.16.