Using Dual-Architecture Images (x86 and Arm) in CCE

Background

CCE allows you to create x86 and Arm nodes in the same cluster. Due to different underlying architectures, Arm images (applications) cannot run on x86 nodes, and vice versa. As a result, workloads may fail to be deployed in the clusters containing x86 and Arm nodes.

Solution

To address this issue, use either of the following methods:

• Set the service affinity when you create a workload so that the pod can be scheduled to an Arm node when the Arm-based image is used or to an x86 node when the x86-based image is used.

  

• Build a dual-architecture image that supports both x86 and Arm architectures. When a pod is scheduled to an Arm node, the Arm variant in the image is pulled. When a pod is scheduled to an x86 node, the x86 variant in the image is pulled. A dual-architecture image has two variants but has one unified access path. When deploying a workload, you only need to specify one image path without configuring the service affinity. In this case, the workload description file is simpler and easier to maintain.

  

Affinity Configuration Description

When creating a node, CCE automatically adds the kubernetes.io/arch label to the node to indicate the node architecture.

kubernetes.io/arch=amd64

The value amd64 indicates the x86 architecture, and arm64 indicates the Arm architecture.

When creating a workload, you can configure the node affinity to schedule pods to nodes using the corresponding architecture.

You can use nodeSelector in the YAML file to configure the architecture.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
spec:
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
    spec:
      nodeSelector:
        kubernetes.io/arch: amd64
      containers:
      - name: container0
        image: swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend-linux-amd64:1.5
        resources:
          limits:
            cpu: 250m
            memory: 512Mi
          requests:
            cpu: 250m
            memory: 512Mi
      imagePullSecrets:
      - name: default-secret

Building a Dual-Architecture Image

To create a dual-architecture image, ensure that the Docker client version is later than 18.03.

The essence of building a dual-architecture image is to build images based on the x86 and Arm architectures separately and then build the dual-architecture image manifest.

For example, the defaultbackend-linux-amd64:1.5 and defaultbackend-linux-arm64:1.5 are images based on the x86 and Arm architectures, respectively.

Upload the two images to SWR. For details about how to upload an image, see Uploading an Image Through a Container Engine Client.

# Add a tag to the original amd64 image defaultbackend-linux-amd64:1.5.
docker tag defaultbackend-linux-amd64:1.5  swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend-linux-amd64:1.5
# Add a tag to the original arm64 image defaultbackend-linux-arm64:1.5.
docker tag defaultbackend-linux-arm64:1.5  swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend-linux-arm64:1.5
# Push the amd64 image to the image repository.
docker push swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend-linux-amd64:1.5
# Push the arm64 image to the image repository.
docker push swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend-linux-arm64:1.5

Create a dual-architecture manifest file and upload it.

# Enable DOCKER_CLI_EXPERIMENTAL.
export DOCKER_CLI_EXPERIMENTAL=enabled
# Create the manifest image file.
docker manifest create --amend --insecure swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend:1.5 swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend-linux-arm64:1.5 swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend-linux-amd64:1.5
# Add arch information to the manifest image file.
docker manifest annotate swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend:1.5 swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend-linux-amd64:1.5 --arch amd64
docker manifest annotate swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend:1.5 swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend-linux-arm64:1.5 --arch arm64
# Push the manifest image file to the image repository.
docker manifest push -p --insecure swr.ap-southeast-1.myhuaweicloud.com/test-namespace/defaultbackend:1.5