更新时间:2024-08-14 GMT+08:00

自建k8s容器如何开启apiserver审计功能?

适用场景

用户自建k8s容器。

前提条件

  • 已开启容器防护。
  • 已确认apiserver审计功能未开启,确认步骤如下:
    1. 登录到kube-apiserver所在的节点。
    2. 查看kube-apiserver.yaml文件或者已经启动的kube-apiserver进程。
      • 进入/etc/kubernetes/manifest目录,查看kube-apiserver.yaml中是否存在--audit-log-path和--audit-policy-file,不存在即表示apiserver审计功能未正常开启。
      • 执行ps命令,查看kube-apiserver的进程命令行中是否存在--audit-log-path和--audit-policy-file,不存在即表示apiserver审计功能未正常开启。

开启apiserver审计功能

  1. 将以下yaml内容复制并保存至yaml文件,并将yaml文件命名为“audit-policy.yaml”

    该yaml内容为k8s审计功能的配置文件,您可以直接使用或者根据实际业务情况编写。
    apiVersion: audit.k8s.io/v1 # This is required.
    kind: Policy
    # Don't generate audit events for all requests in RequestReceived stage.
    omitStages:
      - "RequestReceived"
    rules:
      # The following requests were manually identified as high-volume and low-risk,
      # so drop them.
      # Kube-Proxy running on each node will watch services and endpoint objects in real time
      - level: None
        users: ["system:kube-proxy"]
        verbs: ["watch"]
        resources:
          - group: "" # core
            resources: ["endpoints", "services"]
      # Some health checks
      - level: None
        users: ["kubelet"] # legacy kubelet identity
        verbs: ["get"]
        resources:
          - group: "" # core
            resources: ["nodes"]
      - level: None
        userGroups: ["system:nodes"]
        verbs: ["get"]
        resources:
          - group: "" # core
            resources: ["nodes"]
      - level: None
        users: ["system:apiserver"]
        verbs: ["get"]
        resources:
          - group: "" # core
            resources: ["namespaces"]
      # Some system component certificates reuse the master user, which cannot be accurately distinguished from user behavior,
      # considering that subsequent new functions may continue to add system operations under kube-system, the cost of targeted configuration is relatively high,
      # in terms of the overall strategy, it is not recommended (allowed) for users to operate under the kube-system,
      # so overall drop has no direct impact on user experience
      - level: None
        verbs: ["get", "update"]
        namespaces: ["kube-system"]
      # Don't log these read-only URLs.
      - level: None
        nonResourceURLs:
          - /healthz*
          - /version
          - /swagger*
      # Don't log events requests.
      - level: None
        resources:
          - group: "" # core
            resources: ["events"]
      # Don't log leases requests
      - level: None
        verbs: [ "get", "update" ]
        resources:
          - group: "coordination.k8s.io"
            resources: ["leases"]
      # Secrets, ConfigMaps, and TokenReviews can contain sensitive & binary data,
      # so only log at the Metadata level.
      - level: Metadata
        resources:
          - group: "" # core
            resources: ["secrets", "configmaps"]
          - group: authentication.k8s.io
            resources: ["tokenreviews"]
      # Get responses can be large; skip them.
      - level: Request
        verbs: ["get", "list", "watch"]
        resources:
          - group: "" # core
          - group: "admissionregistration.k8s.io"
          - group: "apps"
          - group: "authentication.k8s.io"
          - group: "authorization.k8s.io"
          - group: "autoscaling"
          - group: "batch"
          - group: "certificates.k8s.io"
          - group: "extensions"
          - group: "networking.k8s.io"
          - group: "policy"
          - group: "rbac.authorization.k8s.io"
          - group: "settings.k8s.io"
          - group: "storage.k8s.io"
      # Default level for known APIs
      - level: RequestResponse
        resources:
          - group: "" # core
          - group: "admissionregistration.k8s.io"
          - group: "apps"
          - group: "authentication.k8s.io"
          - group: "authorization.k8s.io"
          - group: "autoscaling"
          - group: "batch"
          - group: "certificates.k8s.io"
          - group: "extensions"
          - group: "networking.k8s.io"
          - group: "policy"
          - group: "rbac.authorization.k8s.io"
          - group: "settings.k8s.io"
          - group: "storage.k8s.io"
      # Default level for all other requests.
      - level: Metadata

  1. 将audit-policy.yaml文件上传至/etc/kubernetes/路径下。
  2. 进入/etc/kubernetes/manifests目录,将以下内容填写至配置文件kube-apiserver.yaml中,开启apiserver审计功能。

    --audit-policy-file=/etc/kubernetes/audit-policy.yaml
    --audit-log-path=/var/log/kubernetes/audit/audit.log
    --audit-log-maxsize=100
    --audit-log-maxage=1
    --audit-log-maxbackup=10
    • --audit-policy-file:指定审计功能所使用的配置文件。
    • --audit-log-path:指定用来写入审计事件的日志文件路径。不指定此标志会禁用日志后端。
    • --audit-log-maxsize:定义审计日志文件轮转之前的最大大小(兆字节)。
    • --audit-log-maxage:定义保留旧审计日志文件的最大天数。
    • --audit-log-maxbackup:定义要保留的审计日志文件的最大数量。
    • 将上述启动参数填写至配置文件kube-apiserver.yaml时,注意与kube-apiserver.yaml中的启动参数格式保持一致,且不能存在制表符(tab)。

  1. (可选)如果您的kube-apiserver是以Pod形式存在,请按如下步骤将审计日志持久化到主机上。

    1. 在kube-apiserver.yaml中找到volumeMounts字段,按如下配置挂载数据卷。
      volumeMounts:
        - mountPath: /etc/kubernetes/audit-policy.yaml
          name: audit
          readOnly: true
        - mountPath: /var/log/kubernetes/audit/
          name: audit-log
          readOnly: false
    2. 在kube-apiserver.yaml中找到volumes字段,按如下配置挂载。
      volumes:
      - name: audit
        hostPath:
          path: /etc/kubernetes/audit-policy.yaml
          type: File
      - name: audit-log
        hostPath:
          path: /var/log/kubernetes/audit/
          type: DirectoryOrCreate