Help Center/ Cloud Container Engine_Autopilot/ User Guide/ Add-ons/ Cloud Native Logging
Updated on 2024-12-18 GMT+08:00
View PDF
Share

Cloud Native Logging

Introduction

The Cloud Native Logging add-on (log-agent) is developed based on Fluent Bit and OpenTelemetry for collecting logs and Kubernetes events. After the add-on is installed, the fluent-bit container component is automatically created for log collection. This add-on supports CRD-based log collection policies. It collects and forwards standard output logs, container file logs, and Kubernetes event logs in a cluster based on configured policies. It also reports all abnormal Kubernetes events and some normal Kubernetes events to AOM. For details about how to collect logs, see Collecting Logs.

The fluent-bit container component is provided by Huawei Cloud for free.

Constraints

The constraints on using the log-agent add-on are as follows:
  • A maximum of 50 log collection rules can be configured for each cluster.
  • log-agent cannot collect .gz, .tar, or .zip log files.
  • If the container runtime is containerd, stdout logs cannot be multi-line.
  • In each cluster, up to 10,000 single-line logs can be collected per second, and up to 2,000 multiple-line logs can be collected per second.

Permissions

The fluent-bit component reads and collects the standard output logs and container file logs based on the collection configuration.

The following permissions are required for running the fluent-bit component:

  • CAP_DAC_OVERRIDE: ignores the discretionary access control (DAC) restrictions on files.
  • CAP_FOWNER: ignores the restrictions that the file owner ID must match the process user ID.
  • DAC_READ_SEARCH: ignores the DAC restrictions on file reading and catalog research.
  • SYS_PTRACE: allows all processes to be traced.

Installing the Add-on

  1. Log in to the CCE console and click the cluster name to access the cluster console. In the navigation pane on the left, choose Add-ons. Locate Cloud Native Logging on the right and click Install.
  2. On the Install Add-on page, configure the specifications.

    Table 1 Cloud Native Logging specifications

    Parameter

    Description

    Instances

    Number of pods that will be created to match the selected add-on specifications.

    If you select Custom, you can adjust the number of pods as required.

    Containers

    The add-on contains the following container components, whose specifications can be adjusted as required:

    • log-operator: parses and updates log rules.
    • otel-collector: forwards logs collected by fluent-bit to LTS.

  3. Click Install.

Components

Table 2 log-agent components

Component

Description

Resource Type

fluent-bit

A lightweight log collector and forwarder for collecting logs.

Pod

log-operator

Used to generate internal configuration files

Deployment

otel-collector

Used to collect logs from applications and services and report the logs to LTS

Deployment
Parent Topic: Add-ons