How Do I Enable the API Server Audit for an On-Premises Kubernetes Container?

Scenario

On-premises Kubernetes containers are used.

Prerequisites

• Container protection has been enabled. For details, see Enabling Container Node Protection.
• API server audit is disabled. Perform the following steps to check its status:
  1. Log in to the node where kube-apiserver is located.
  2. Check the kube-apiserver.yaml file or the started kube-apiserver process.
     • Go to the /etc/kubernetes/manifest directory and check whether --audit-log-path and --audit-policy-file exist in kube-apiserver.yaml. If they do not exist, API server audit is disabled.
     • Run the ps command to check whether --audit-log-path and --audit-policy-file exist in the command lines of the kube-apiserver process. If they do not exist, the audit function of the kube-apiserver process is disabled.

Enabling API Server Audit

1. Copy the following YAML content, save it to the YAML file, and name the file audit-policy.yaml.
   This YAML file is the configuration file of the Kubernetes audit function. You can directly use the file or compile it as needed.

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103

   apiVersion: audit.k8s.io/v1 # This is required. kind: Policy # Don't generate audit events for all requests in RequestReceived stage. omitStages: - "RequestReceived" rules: # The following requests were manually identified as high-volume and low-risk, # so drop them. # Kube-Proxy running on each node will watch services and endpoint objects in real time - level: None users: ["system:kube-proxy"] verbs: ["watch"] resources: - group: "" # core resources: ["endpoints", "services"] # Some health checks - level: None users: ["kubelet"] # legacy kubelet identity verbs: ["get"] resources: - group: "" # core resources: ["nodes"] - level: None userGroups: ["system:nodes"] verbs: ["get"] resources: - group: "" # core resources: ["nodes"] - level: None users: ["system:apiserver"] verbs: ["get"] resources: - group: "" # core resources: ["namespaces"] # Some system component certificates reuse the master user, which cannot be accurately distinguished from user behavior, # considering that subsequent new functions may continue to add system operations under kube-system, the cost of targeted configuration is relatively high, # in terms of the overall strategy, it is not recommended (allowed) for users to operate under the kube-system, # so overall drop has no direct impact on user experience - level: None verbs: ["get", "update"] namespaces: ["kube-system"] # Don't log these read-only URLs. - level: None nonResourceURLs: - /healthz* - /version - /swagger* # Don't log events requests. - level: None resources: - group: "" # core resources: ["events"] # Don't log leases requests - level: None verbs: [ "get", "update" ] resources: - group: "coordination.k8s.io" resources: ["leases"] # Secrets, ConfigMaps, and TokenReviews can contain sensitive & binary data, # so only log at the Metadata level. - level: Metadata resources: - group: "" # core resources: ["secrets", "configmaps"] - group: authentication.k8s.io resources: ["tokenreviews"] # Get responses can be large; skip them. - level: Request verbs: ["get", "list", "watch"] resources: - group: "" # core - group: "admissionregistration.k8s.io" - group: "apps" - group: "authentication.k8s.io" - group: "authorization.k8s.io" - group: "autoscaling" - group: "batch" - group: "certificates.k8s.io" - group: "extensions" - group: "networking.k8s.io" - group: "policy" - group: "rbac.authorization.k8s.io" - group: "settings.k8s.io" - group: "storage.k8s.io" # Default level for known APIs - level: RequestResponse resources: - group: "" # core - group: "admissionregistration.k8s.io" - group: "apps" - group: "authentication.k8s.io" - group: "authorization.k8s.io" - group: "autoscaling" - group: "batch" - group: "certificates.k8s.io" - group: "extensions" - group: "networking.k8s.io" - group: "policy" - group: "rbac.authorization.k8s.io" - group: "settings.k8s.io" - group: "storage.k8s.io" # Default level for all other requests. - level: Metadata

2. Upload the audit-policy.yaml file to the /etc/kubernetes/ directory.
3. Go to the /etc/kubernetes/manifests directory and add the following content to the kube-apiserver.yaml file to enable API server audit:

   1 2 3 4 5

   --audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-log-path=/var/log/kubernetes/audit/audit.log --audit-log-maxsize=100 --audit-log-maxage=1 --audit-log-maxbackup=10

   

   • --audit-policy-file: configuration file used by the audit function.
   • --audit-log-path: path of the log file where audit events are written. If this flag is not specified, the logging backend will be disabled.
   • --audit-log-maxsize: maximum size (in MB) of an audit log file before rotation.
   • --audit-log-maxage: maximum number of days for storing old audit log files.
   • --audit-log-maxbackup: maximum number of retained audit log files.
   • Add the preceding parameters to the kube-apiserver.yaml file, ensure that the format of the parameters is the same as that in the kube-apiserver.yaml file and cannot contain tab characters.

4. (Optional) If your kube-apiserver runs as a pod, perform the following steps to persist logs on the server:
   1. Locate the volumeMounts field in kube-apiserver.yaml and configure volume mounting as follows:

      1 2 3 4 5 6 7

      volumeMounts: - mountPath: /etc/kubernetes/audit-policy.yaml name: audit readOnly: true - mountPath: /var/log/kubernetes/audit/ name: audit-log readOnly: false

   2. Locate the volumes field in kube-apiserver.yaml and configure it as follows:

      1 2 3 4 5 6 7 8 9

      volumes: - name: audit hostPath: path: /etc/kubernetes/audit-policy.yaml type: File - name: audit-log hostPath: path: /var/log/kubernetes/audit/ type: DirectoryOrCreate

5. Restart kube-apiserver to apply the configuration.

   The method of restarting kube-apiserver varies depending on the environment.

   If kube-apiserver is managed by systemd, run the following command to restart the service:

   systemctl restart kube-apiserver.service