Using AppArmor to Confine Container Access to Resources

Constraints

• This feature is supported only in clusters of v1.31.6-r0 or later, and the nodes must run Ubuntu. Ensure that your cluster version and node OS meet the requirements.
• The version of the Everest add-on in the cluster must be 2.4.158 or later.
• After AppArmor is enabled on a node, Docker or containerd applies the default AppArmor profile to pods scheduled to the node even if the appArmorProfile parameters are not configured for the pods. The AppArmor profile is not enforced when the appArmorProfile.type parameter is explicitly set to Unconfined for the pods. For details, see Using AppArmor Profiles to Secure Pods.
• Operations that trigger node reset, such as resetting a node or synchronizing a node pool, will disable AppArmor. If you need to disable AppArmor on a node, reset the node. Currently, AppArmor cannot be manually disabled.

Enabling AppArmor on a Node

1. Log in to the Ubuntu node and install AppArmor.

   sudo DEBIAN_FRONTEND=noninteractive apt-get -y install apparmor

   

   If a private Ubuntu image is used, pre-install AppArmor in the image.

2. Check the AppArmor status.

   sudo aa-status

   If the following information is displayed, AppArmor has been loaded:

   apparmor module is loaded.

3. Start AppArmor and set it to be automatically started upon system boot. When the node is restarted, the AppArmor profile is automatically reloaded.

   sudo systemctl enable apparmor
   sudo systemctl start apparmor

4. Check whether the system is started. If Y is returned, the system is started.

   sudo cat /sys/module/apparmor/parameters/enabled

   The parameters in the command output are read by kubelet to check whether AppArmor is enabled on the node. If AppArmor is not enabled on the node, pods with AppArmor settings will be intercepted when they are scheduled to this node.

   Restart kubelet to read the parameters in the command output. (AppArmor is enabled by default in kubelet 1.31 GA.)

   sudo systemctl restart kubelet 

Enabling AppArmor for the Container Engine

If the container engine is containerd, take the following steps:

1. Modify the containerd configuration.

   vim /etc/containerd/config.toml

2. Modify the following content and save the modification:

   [plugins."io.containerd.grpc.v1.cri"]
     disable_apparmor = false 

3. Restart containerd.

   sudo systemctl restart containerd

If the container engine is Docker, take the following steps:

1. Modify the Docker configuration.

   vim /etc/default/docker

2. Use # to comment out container='disable apparmor'.

   # container='disable apparmor'

3. Restart Docker.

   sudo systemctl restart docker

Using AppArmor Profiles to Secure Pods

AppArmor profiles can be specified at the pod or container level. If a pod-level profile is set, it will be used as the default profile for all containers (including init, sidecar, and ephemeral containers) in the pod. If both pod- and container-level profiles are set, the container-level profile is preferentially used. The following shows an example configuration:

...
securityContext:
  appArmorProfile:
    type: <profile_type>
...

The options of <profile_type> are as follows:

• Localhost: The profile loaded on the node is used. For details, see Example: Using Custom AppArmor Profile to Confine a Pod.
• RuntimeDefault: The default profile is used.
• Unconfined: The AppArmor profile is not enforced.

For details about the AppArmor profile API, see Specifying AppArmor Confinement.

When privileged containers are enabled for a pod (privileged: true), configure rules for the AppArmor configuration to take effect. For details, see Rules for AppArmor Configurations to Take Effect After AppArmor Is Enabled for Privileged Containers.

To verify that a container's root process is running with the correct profile, you can check its process attributes.

kubectl exec <pod_name> -- cat /proc/1/attr/current 

An example output is as follows:

cri-containerd.apparmor.d (enforce)

Example: Using Custom AppArmor Profile to Confine a Pod

1. Load the custom AppArmor profile onto the node. You can define the profile name.

   vim /etc/apparmor.d/k8s-apparmor-example-deny-write

   The following is an example. The profile denies all file write operations. For details about the profile writing syntax, see the AppArmor Profile Quick Guide.

   #include <tunables/global>
   profile k8s-apparmor-example-deny-write flags=(attach_disconnected) {
     #include <abstractions/base>
     file,
     # Deny all file writes.
     deny /** w,
   }

   Load the profile.

   apparmor_parser -r /etc/apparmor.d/k8s-apparmor-example-deny-write

   

   The AppArmor profile needs to be loaded onto each node that needs to use AppArmor.

2. Create a simple pod as follows and add the AppArmor rules for the pod to reject file writing as set in the previous step.
   Create the hello-apparmor.yaml file and add the following content to the file:

   apiVersion: v1
   kind: Pod
   metadata:
     name: hello-apparmor
   spec:
     securityContext:
       appArmorProfile:
         type: Localhost  # Use the AppArmor profile loaded on the node.
         localhostProfile: k8s-apparmor-example-deny-write  # Define the name of the AppArmor profile loaded on the node.
     containers:
     - name: hello
       image: busybox:1.28
       command: [ "sh", "-c", "echo 'Hello AppArmor!' && sleep 1h" ]

   

   If the cluster contains nodes that do not support AppArmor, use affinity scheduling to schedule the pod to a node that supports AppArmor. For details, see Configuring Node Affinity Scheduling (nodeAffinity).

3. Create a sample pod.

   kubectl create -f hello-apparmor.yaml

4. When the pod is running, check the /proc/1/attr/current file of the container to verify that the container is running with the AppArmor profile.

   kubectl exec hello-apparmor -- cat /proc/1/attr/current

   If the following information is displayed, the AppArmor profile has been applied to the container:

   k8s-apparmor-example-deny-write (enforce)

5. Write files to violate the AppArmor profile and check whether the profile takes effect.

   kubectl exec hello-apparmor -- touch /tmp/test

   If the following information is displayed, the write operation is rejected, and the profile takes effect:

   touch: /tmp/test: Permission denied
   command terminated with exit code 1

Rules for AppArmor Configurations to Take Effect After AppArmor Is Enabled for Privileged Containers

If privileged containers are enabled for a pod (privileged: true), the rules for AppArmor configurations to take effect are as follows.

Helpful Links

• AppArmor official website
• Using AppArmor in Ubuntu
• Using AppArmor to Confine Container Access to Resources
• AppArmor Profile Quick Guide