Updated on 2024-06-17 GMT+08:00

log-agent

log-agent is a cloud native log collection add-on built based on open source fluent-bit and OpenTelemetry. It supports CRD-based log collection policies, collects and forwards standard output logs, container file logs, node logs, and Kubernetes events of containers in a cluster.

The core components of log-agent include fluent-bit, cop-logs, log-operator, and otel-collector.

  • fluent-bit: indicates the log collector, which is installed on each node as a DaemonSet.
  • cop-logs: generates and updates configuration files on the collection side.
  • log-operator: parses and updates log rules.
  • otel-collector: forwards logs collected by fluent-bit to LTS in a centralized manner.

Resource Quota Requirements of Different Specifications

Ensure that the cluster has sufficient CPU and memory resources for scheduling when installing log-agent. Table 1 describes the resource quota requirements of different log-agent specifications.

Table 1 Resource quota requirements of different specifications

Add-on Specification

Container

CPU Quota

Memory Quota

Small specifications

(1 pod)

fluent-bit

Request: 100 m

Limit: 500 m

Request: 200 MiB

Limit: 500 MiB

cop-logs

Request: 100 m

Limit: 1

Request: 100 MiB

Limit: 500 MiB

log-operator

Request: 100 m

Limit: 500 m

Request: 100 MiB

Limit: 500 MiB

otel-collector

Request: 200 m

Limit: 1

Request: 1 GiB

Limit: 2 GiB

Large specifications

(2 pods)

fluent-bit

Request: 100 m

Limit: 500 m

Request: 200 MiB

Limit: 500 MiB

cop-logs

Request: 100 m

Limit: 1

Request: 100 MiB

Limit: 500 MiB

log-operator

Request: 100 m

Limit: 500 m

Request: 100 MiB

Limit: 500 MiB

otel-collector

Request: 200 m

Limit: 1

Request: 1 GiB

Limit: 2 GiB