Updated on 2024-04-12 GMT+08:00

ICAgent

This section describes the restrictions on the log collector ICAgent.

Table 1 ICAgent file collection restrictions

Item

Description

Remarks

File encoding

UTF 8 is supported. Other encoding formats may cause garbled characters.

You can configure whether to collect log files containing binary content. Binary characters may be displayed as garbled characters.

N/A

Host type

Only logs of Linux hosts can be collected.

N/A

Log file size

Unlimited

N/A

Log file rotation

ICAgent supports configuration of fixed log file names or fuzzy match of log file names. You need to rotate log files manually.

N/A

Log collection path

Linux

  • Collection paths support recursion. You can use double asterisks (**) to collect logs from up to five directory levels. Example: /var/logs/**/a.log
  • Collection paths support fuzzy match. You can use an asterisk (*) to represent one or more characters of a directory or file name. Example: /var/logs/*/a.log or /var/logs/service/a*.log
  • If the collection path is set to a directory, for example, /var/logs/, only .log, .trace, and .out files in the directory are collected. If the collection path is set to name of a text file, that file is directly collected.
  • Each collection path must be unique. That is, the same path of the same host cannot be configured for different log groups and log streams.

N/A

Symbolic link

Symbolic links are not supported.

N/A

Single log size

Configure whether log splitting is supported. A log cannot exceed 500 KB. If a log exceeds 500 KB, the extra part will be truncated and discarded.

If log splitting is enabled, a log exceeding 500 KB will be split into multiple logs for collection. For example, a 600 KB log will be split into a 500 KB log and a 100 KB log. Only Linux hosts and single-line logs are supported.

N/A

Regular expression

Perl regular expressions are supported.

N/A

File collection configuration

A file can be reported to only one log group and stream. If a file is configured for multiple log streams, only one configuration takes effect.

N/A

File opening

Files are opened when being read, and closed after being read.

N/A

First log collection

All logs are collected.

N/A

Table 2 ICAgent performance specifications

Item

Description

Remarks

Log collection rate

Raw logs of a single node are collected at a rate up to 50 MB/s.

Service quality cannot be ensured if this limit is exceeded.

Monitored directories

Up to five levels of directories are supported, with up to 1000 files.

N/A

Monitored files

Container scenarios

  • The ICAgent can collect a maximum of 20 log files from a volume mounting directory.
  • The ICAgent can collect a maximum of 1000 standard container output log files. These files must be in JSON format.

VM scenarios

  • A maximum of 1000 files are supported.

N/A

Default resource restrictions

CPU: Max. two CPU cores

Memory: Max. min{4 GB, Physical memory/2}. A restart is triggered if this memory limit is exceeded. "min{4 GB, Physical memory/2}" means that the smaller value between half of the physical memory and 4 GB is used.

N/A

Resource limit reached

A forcible restart is triggered. Logs may be lost or duplicate if rotated during the restart.

N/A

Agent installation, upgrade, or uninstallation

Unlimited

N/A

Table 3 Other restrictions on ICAgent

Item

Description

Remarks

Configuration update

Configuration updates take effect in 1 to 3 minutes.

N/A

Dynamic configuration loading

Console configurations can be dynamically delivered. The update of one configuration does not affect other configurations.

N/A

Configurations

Unlimited

N/A

Tenant isolation

Tenants are isolated from each other by default.

N/A

Log collection delay

Normally, the delay from writing logs to the disk to collecting the logs is less than 2s (congestion not considered).

N/A

Log upload

File changes are read and uploaded immediately once detected. One or more logs can be uploaded a time.

N/A

Network error handling

Network exceptions trigger retries at an interval of 5s.

N/A

Resource quota used up

If the resources allocated to the ICAgent are insufficient due to massive amounts of logs, the ICAgent continues and retries upon a failure. Logs will be stacked if resources are still insufficient.

N/A

Max. retry timeout

Retry attempts are periodically made.

N/A

Status check

The collector status is monitored through heartbeat detection.

N/A

Checkpoint timeout

Checkpoints are automatically deleted if no updates are made within 12 hours.

N/A

Checkpoint saving

Checkpoints are updated if logs are reported successfully.

N/A

Checkpoint saving path

The default save path:

/var/share/oss/manager/ICProbeAgent/internal/TRACE

N/A

Log loss

Duplicate logs

ICAgent uses multiple mechanisms to ensure log collection reliability and prevent data loss. However, logs may be lost in the following scenarios:

  • Log files are rotated at a high speed, for example, once per second.
  • Logs cannot be forwarded due to improper system security settings or syslog itself.
  • The container running time, for example, shorter than 30s, is extremely short.
  • A single node generates logs at a high speed, exceeding the allowed transmit bandwidth or log collection speed. It is recommended that the log generation speed of a single node is lower than 50 MB/s.

When the ICAgent is restarted, identical data may be collected around the restart time.

N/A