Cloud Native Logging

Introduction

log-agent is built based on Fluent Bit and OpenTelemetry. It collects logs and Kubernetes events. This add-on supports CRD-based log collection policies. It collects and forwards standard output logs, container file logs, node logs, and Kubernetes event logs in a cluster based on configured policies. It also reports Kubernetes events to AOM for configuring event alarms. By default, all abnormal events and some normal events are reported.

In 1.3.2 and later versions, Cloud Native Logging reports all warning events and some normal events to AOM by default. The reported events can be used to configure alarms. If the cluster version is 1.19.16, 1.21.11, 1.23.9, 1.25.4, or later, after Cloud Native Logging is installed, events are reported to AOM by this add-on instead of the control plane component. After Cloud Native Logging is uninstalled, events will not be reported to AOM.

Notes and Constraints

This add-on is available only in clusters v1.17 or later.

Add-on Performance

Permissions

The fluent-bit component of the log-agent add-on reads and collects the stdout logs on each node, file logs in pods, and node logs based on the collection configuration.

The following permissions are required for running the fluent-bit component:

• CAP_DAC_OVERRIDE: ignores the discretionary access control (DAC) restrictions on files.
• CAP_FOWNER: ignores the restrictions that the file owner ID must match the process user ID.
• DAC_READ_SEARCH: ignores the DAC restrictions on file reading and catalog research.
• SYS_PTRACE: allows all processes to be traced.

Installing the Add-on

1. Log in to the CCE console and click the cluster name to access the cluster console. In the navigation pane, choose Add-ons, locate Cloud Native Logging on the right, and click Install.
2. On the Install Add-on page, configure the specifications as needed.
   • If you selected Preset, you can choose between Low or High depending on the number of logs on nodes. The system will automatically set the number of add-on pods and resource quotas according to the preset specifications. You can see the configurations on the console.

     You can select the Low option for clusters where the logs of a single node are less than 5000/s or 5 MB/s, and select the High option for clusters where the logs on a single node are less than 10000/s or 10 MB/s.

   • If you selected Custom, you can adjust the number of pods and resource quotas as needed. High availability is not possible with a single pod. If an error occurs on the node where the add-on instance runs, the add-on will fail.

3. Configure add-on parameters, which is supported by the add-on of v1.6.2 or later. For details, see Tail.

   Parameter	Description	Default Value
   		Large Specification	Other Specification
   Initial Buffer Size (Buffer_Chunk_Size)	The size of the initial buffer used to read files.	256k	128k
   Maximum Buffer Size (Buffer_Max_Size)	Limit on the buffer size for each monitored file. When a buffer is added, the number of memory buffers that can be added is restricted by this value. If the limit is exceeded, the file will be removed from the monitored file list.	1024k	512k
   Memory Buffer Limit (Mem_Buf_Limit)	Memory limit when data is appended to the engine. When the limit is reached, the add-on will temporarily stop reading log file data. The reading will resume after the data is refreshed.	300mb	40mb

   

   The unit can be case-insensitive k, kb, m, mb, g or gb. If no unit is specified, the default unit of byte will be used.

4. Configure deployment policies for the add-on pods.
   

   Scheduling policies do not take effect on add-on instances of the DaemonSet type.

   Table 1 Configurations for add-on scheduling

   Parameter

   Description

   Multi AZ

   Preferred: Deployment pods of the add-on will be preferentially scheduled to nodes in different AZs. If all the nodes in the cluster are deployed in the same AZ, the pods will be scheduled to different nodes in that AZ. Required: Deployment pods of the add-on are forcibly scheduled to nodes in different AZs. There can be at most one pod in each AZ. If nodes in a cluster are not in different AZs, some add-on pods cannot run properly. If a node is faulty, add-on pods on it may fail to be migrated.

5. Click Install.

Components

Add-on Usage

This add-on can collect container standard output logs, container file logs, node logs, and Kubernetes events. You can use LTS or AOM to store the collected logs. These services support different types of logs. For details, see Table 3.

Change History