Managing ICAgent

You can use ICAgent to collect logs to LTS. ICAgent is a log collection tool for LTS. It runs on hosts where logs need to be collected. This section describes how to install ICAgent on a host, upgrade ICAgent, uninstall ICAgent, check the ICAgent version description, and check the ICAgent status.

You can select one of the following solutions based on your server type. The usage process and configuration requirements vary in different scenarios.

Host solution: applicable to traditional computing environments, such as physical servers and ECSs. Hosts are classified into intra-region and extra-region hosts based on their locations.
Container solution: applicable to services deployed on Kubernetes container platforms, such as those deployed in CCE clusters, self-built Kubernetes clusters, and ServiceStage.

Installing ICAgent

In host and container scenarios, you can install ICAgent on target servers.

Host Scenario

Installing ICAgent
For details, see Installing ICAgent (Intra-Region Hosts) and Installing ICAgent (Extra-Region Hosts).
Upgrading ICAgent
LTS regularly updates ICAgent to improve log collection. When LTS prompts you that a new ICAgent version is available, perform the following operations to upgrade ICAgent in your Linux environment.

For Windows hosts, you must uninstall ICAgent before installing a new version.
1. Log in to the management console and choose Management & Deployment > Log Tank Service.
2. Choose Host Management > Hosts in the navigation pane.
3. Click the Intra-Region Hosts or Extra-Region Hosts tab. When the system prompts you that a new ICAgent version is available, select one or more hosts where ICAgent is to be upgraded, and click Upgrade ICAgent.
4. In the displayed dialog box, click OK.
  The ICAgent upgrade takes about 1 minute. When the ICAgent status changes from Updating to Running, the ICAgent upgrade has completed.
  
  If ICAgent is abnormal after the upgrade or if the upgrade fails, log in to the host and run the installation command to reinstall ICAgent. Uninstalling the original ICAgent is not required. For details, see Installing ICAgent (Intra-Region Hosts) and Installing ICAgent (Extra-Region Hosts).

Container Scenario

Installing ICAgent in a CCE cluster
1. Choose Host Management > Hosts in the navigation pane.
2. Click the CCE Cluster tab. Select ICAgent Status in the search box to locate clusters whose ICAgent is in Uninstalled state, select one of the clusters, and click Upgrade ICAgent.
  Figure 1 Upgrading ICAgent in a CCE cluster
  - If no CCE cluster is displayed on the Hosts page of LTS, you need to authorize LTS to access AOM 2.0. Log in to the AOM 2.0 console and grant AOM 2.0 the permissions to access cloud services, such as LTS and CCE, by referring to Authorizing AOM to Access Other Cloud Services.
  - If there are no CCE clusters available, Output to AOM is unavailable.
  - To disable the function of collecting container standard output to AOM 1.0, ICAgent 5.12.133 or later is required.
  - When a CCE cluster is created, ICAgent is installed on its hosts by default, with logs reported to LTS. The Output to AOM feature is disabled by default. If you enabled Output to AOM after cluster creation but want to revert to reporting logs to LTS, ensure Output to AOM is disabled before upgrading ICAgent. For Ingesting CCE Application Logs to LTS, you are advised to collect container standard output to LTS instead of AOM.
  - CCE cluster ID: Each cluster has a fixed ID.
  - When ICAgent is upgraded, LTS automatically creates a log group and host group for your CCE cluster, named k8s-log-{Cluster ID}. You can ingest logs of this CCE cluster to the log group when creating an ingestion configuration. For details, see Ingesting CCE Application Logs to LTS.
  - If the cluster's hosts do not have ICAgent installed or have an outdated version, clicking Upgrade ICAgent will install or upgrade ICAgent on all hosts in the cluster.
3. In the displayed dialog box, click OK.
  The ICAgent upgrade takes about 1 minute. When the ICAgent status changes from Updating to Running, the ICAgent upgrade has completed.
  
  Figure 2 Upgrading ICAgent
For details about how to install ICAgent in a self-built Kubernetes cluster, see Install the Log Collection Component.
Upgrading ICAgent in a CCE cluster follows the same process as installing it in a CCE cluster.
Uninstalling ICAgent:
For details, see Managing UniAgents and ICAgents in CCE Clusters.

Upgrading ICAgent

In host and container scenarios, you can upgrade ICAgent on target servers.

LTS automatically checks ICAgent versions. If LTS detects that your host is using an ICAgent version that is no longer maintained, it will display a message once you log in to the LTS console. You can click Authorize Auto Upgrade to allow LTS to automatically upgrade ICAgent during off-peak hours, or click Manually Upgrade to upgrade it on the host management page. If you do not need to upgrade ICAgent, select Do not show again.

Host scenario: For detailed steps, see Upgrading ICAgent.
Container scenario: Upgrading ICAgent in a CCE cluster follows the same process as installing it in a CCE cluster.

Uninstalling ICAgent

If ICAgent is uninstalled from a host, log collection will be affected. Exercise caution when performing this operation.

Uninstalling ICAgent does not delete the installation files. You need to delete them manually if necessary.

Only ICAgent installed on Linux hosts can be uninstalled from the Host Management page of the LTS console. To uninstall ICAgent from a Windows host, go to \ICProbeAgent\bin\manual\win in the directory where the ICAgent installation package was decompressed, and double-click the script named uninstall.bat. If message "ICAgent uninstall success" is displayed, the uninstallation is successful.

Uninstall ICAgent from a Linux environment using any of the following methods:

Uninstalling ICAgent on the console: applicable to uninstall ICAgent that has been successfully installed.
1. Choose Host Management > Hosts in the navigation pane.
2. Select one or more hosts where ICAgent is to be uninstalled and click Uninstall ICAgent.
3. In the displayed dialog box, click OK.
  The uninstallation begins. This process takes about a minute.
  
  After the uninstallation is complete, the ICAgent status of the host will be displayed as Uninstalled in the host list.
  
  To reinstall ICAgent, wait for 5 minutes after it is uninstalled. Otherwise, the ICAgent may be automatically uninstalled again.
Logging in to the host to uninstall ICAgent: applies to the scenario where ICAgent fails to be installed.
1. Log in to a host where ICAgent is to be uninstalled as user root.
2. Run the following command:
```
bash /opt/oss/servicemgr/ICAgent/bin/manual/uninstall.sh
```
  If the message "ICAgent uninstall success" is displayed, the uninstallation has completed.
Remotely uninstalling ICAgent: applicable to remotely uninstall ICAgent that has been successfully installed.
You can uninstall ICAgent on one host remotely from another host.
1. Run the following command on the host where ICAgent has been installed. x.x.x.x indicates the IP address of the host you want to uninstall ICAgent from.
```
bash /opt/oss/servicemgr/ICAgent/bin/remoteUninstall/remote_uninstall.sh -ip x.x.x.x
```
2. Enter the password for user root of the host when prompted.
  - If the Expect tool is installed on the host that has ICAgent installed, the ICAgent uninstallation should be able to complete without prompting you for a password. Otherwise, enter the password as prompted.
  - Ensure that user root can run SSH or SCP commands on the host where ICAgent has been installed to communicate with the remote host.
  - If the message "ICAgent uninstall success" is displayed, the uninstallation has completed.
Batch uninstalling ICAgent: applicable to uninstall ICAgent that has been successfully installed from a batch of hosts.
If ICAgent has been installed on a host and the ICAgent installation package ICProbeAgent.tar.gz is in the /opt/ICAgent/ directory of the host, you can use this method to uninstall ICAgent from multiple hosts at once.

The hosts must all belong to the same VPC and be on the same subnet.

Prerequisites

The IP addresses and passwords of all hosts to uninstall ICAgent have been collected, sorted in the iplist.cfg file, and uploaded to the /opt/ICAgent/ directory on the host that has ICAgent installed. Each IP address and password in the iplist.cfg file must be separated by a space. Examples:

192.168.0.109 Password (Replace the IP address and password with the actual ones)

192.168.0.39 Password (Replace the IP address and password with the actual ones)
- The iplist.cfg file contains sensitive information. You are advised to clear it after using it.
- If all hosts share a password, list only IP addresses in the iplist.cfg file and enter the password manually during execution. If one of the hosts uses a different password, type the password behind its IP address.
1. Run the following command on the host that has ICAgent installed:
```
bash /opt/oss/servicemgr/ICAgent/bin/remoteUninstall/remote_uninstall.sh -batchModeConfig /opt/ICAgent/iplist.cfg
```
  Enter the default password for user root of the hosts to uninstall ICAgent. If the passwords of all hosts have been configured in the iplist.cfg file, press Enter to skip this step.
```
batch uninstall begin
Please input default passwd:
send cmd to 192.168.0.109
send cmd to 192.168.0.39
2 tasks running, please wait...
End of uninstall agent: 192.168.0.109
End of uninstall agent: 192.168.0.39
All hosts uninstall icagent finish.
```
  If message "All hosts uninstall icagent finish." is displayed, the batch uninstallation has completed.
2. Choose Host Management > Hosts in the navigation pane of the LTS console to check the ICAgent status.

Checking the ICAgent Status

To check the ICAgent status of a target host, choose Host Management > Hosts in the navigation pane. The following table lists the ICAgent statuses.

**Table 1** ICAgent statuses
Status	Description
Running	ICAgent is running properly.
Uninstalled	ICAgent is not installed.
Installing	ICAgent is being installed. This process takes about 1 minute.
Installation failed	ICAgent installation failed.
Upgrading	ICAgent is being upgraded. This process takes about 1 minute.
Upgrade failed	ICAgent upgrade failed.
Offline	ICAgent is abnormal because the AK/SK is incorrect. Obtain the correct AK/SK and install ICAgent again.
Faulty	ICAgent is faulty. Contact technical support.
Uninstalling	ICAgent is being uninstalled. This process takes about 1 minute.
Authentication error	Authentication fails because parameters were incorrectly configured during ICAgent installation.

Checking the ICAgent Version

To check the ICAgent version of a target host, choose Host Management > Hosts in the navigation pane. The following table lists the ICAgent versions.

**Table 2** ICAgent version description
Version	Description	Released
7.3.3	Optimized the regular expression verification logic for container log collection. Fixed the issue where container logs failed to collect when the collection path is set to the mount directory's parent.	2025-06-26
7.3.2	Resolved the issue where ICAgent-reported system logs cannot be queried by cluster name on the AOM 1.0 console in self-built Kubernetes scenarios.	2025-06-09
7.3.1	Introduced an option to select logical relationships for whitelist log filtering in ICAgent structuring. Added support for Base64 encoding and decoding for ICAgent log collection and reporting. Added support for ECS OpenStack API V2 for ICAgent. Enhanced container log and standard output collection in self-built Kubernetes to function even if port 10255 is not enabled.	2025-05-26
7.2.6	Improved the AgentId file clearance logic for ICAgent re-installation and uninstallation on ECSs, resolving the issue of AgentId duplication after ICAgent is pre-installed in private images.	2025-05-14
7.2.4	Adjusted the cursor directory structure. Cursor file names no longer include log group or log stream paths, and cursor files now stored directly in their corresponding directories.	2025-04-25
7.2.2	Resolved intermittent standard output collection failures caused by occasional blocking of the collection coroutine during high log volume. Solved the issue where the reason field is not reported when Kubernetes events are reported.	2025-03-13
7.2.1	Added line number processing to the structured single-line regular expression and multi-line regular expression. Optimized the logic for obtaining labels during log sending. Supported the configuration of multiple NTP servers for NTP metrics. Added the function of collecting container NPU metrics by using pod IP addresses.	2025-02-28
7.1.22	Optimized the code to reduce the memory consumed for obtaining pod information.	2025-01-23
7.1.19	Optimized the logic for generating agent IDs. Changed the expiration time of wrapped files to 2 hours. Optimized the log printing function.	2025-01-13
7.1.17	Optimized the log collection process to adjust the size of transmitted blocks.	2025-01-03
7.1.14	Optimized the log collection process. Updated to persistently save signature information during log collection.	2024-12-25
7.1.12	Solved the issue where, in some scenarios, CPU usage increased due to timer leakage caused by the Go version.	2024-12-15
7.1.6	Enabled logs to report image names. Optimized the log collection process. Upgraded ICAgent's base image version.	2024-11-21
7.1.5	Solved the issue that logs continued to be collected after all configurations in a project are deleted or disabled.	2024-11-21
7.1.3	Optimized the httpclient component's performance in log sending.	2024-11-02
7.1.1	Optimized the ICAgent architecture and unified the Agents of AOM 1.0 and AOM 2.0.	2024-10-26
5.12.233	Optimized the structuring performance for container logs. Solved the issue that error diagnosis occasionally failed to be reported in CCE scenarios. Solved the issue that inactive files failed to be collected in VM scenarios.	2024-10-17
5.12.232	Replaced the JSON library used for parsing standard output logs with the sonic library to reduce the CPU usage. Solved the issue of log transmission timeouts when using LTS.	2024-10-14
5.12.231	Solved the issue that ICAgent restarted under the following conditions: Standard output was collected by LTS; there were no log ingestion rules defined in LTS; and a container was created in CCE and standard output logs were printed. Solved the issue where hisfile in the cursor file was incorrectly recognized as a directory. Solved the issue where the incremental collection switch was ineffective.	2024-10-11
5.12.230	Solved the issue where periodic cursor refresh was not functioning. Solved the issue where the cursor could not be reset after the base file was wrapped since the wrapping file could not be found. Solved the issue of high CPU usage caused by file signature.	2024-10-09
5.12.224	Cleared all export HISTSIZE=0 entries (if any) from the .bashrc file in the ECS upgrade scenario.	2024-09-27
5.12.218	Supported GBK encoding in the logs reported by ICAgent. Enabled ICAgent to collect the same log files repeatedly.	2024-09-26
5.12.185	Fixed the invalid blacklist path in the VM log configuration. Optimized the collection of containerd standard output logs.	2024-05-20
5.12.184	Solved the issue of being unable to exclude wrapped files when container logs were collected. Changed the number of concurrent collection coroutines of node log collection to 32.	2024-05-16
5.12.183	Optimized the wrapping files for the container standard output of containerd nodes.	2024-05-11
5.12.182	Solved the syslog switch issue.	2024-04-28
5.12.181	Solved the ICAgent authentication failure on self-built Kubernetes clusters. Solved the log truncation issue. Solved the issue that some files were not collected because wrapped files could not be found when the log rate was high.	2024-04-25
5.12.177	Solved the infinite loop issue of wrapping.	2024-03-28
5.12.176	Optimized .zip streaming parsing: Check whether transferred files are complete. Specified the maximum number of connections to each PodLB host.	2024-03-18
5.12.175	Solved the collection performance bottleneck issue of structured logs.	2024-03-13
5.12.172	Optimized the supported wrapping mode.	2024-02-28
5.12.171	Solved the JSON parsing issue (not deleting escape characters) of standard output logs of Docker nodes.	2024-01-31
5.12.170	Supported incremental collection of host, container, and standard output logs. Resolved the host GPU metric disconnection issue.	2024-01-29
5.12.166	Solved the high memory usage issue of the standard output log collection plug-in. Solved the issue that the VM log collection plug-in repeatedly collected wrapped files. Added the log group and log stream information to the cursor files.	2023-12-27
5.12.165	Added the function of obtaining the initial agent ID from the configuration file. If the agent ID does not meet the verification requirements, the randomly generated UUID is used.	2023-12-21
5.12.163	Supported ICAgent installation using the UniAgent plug-in.	2023-12-13
5.12.159	Solved the leakage issue of the standard output log collection coroutine. Solved the issue that standard output wrapping logs could not be collected after standard output logs were collected to AOM.	2023-11-27
5.12.158	Solved the issue that memory leakage of container metrics caused ICAgent restart when the metric switch was turned off.	2023-11-08
5.12.157	Container log ingestion to LTS: supported Docker driver Device Mapper. Solved the issue that ICAgent restarted due to too many VM logs (fast transferring) and sharp increase of memory usage.	2023-11-06
5.12.156	Solved the issue of obtaining the installation package from OBS and changed the protocol from HTTP to HTTPS.	2023-11-01
5.12.154	Supported log structuring.	2023-10-31
5.12.150	Resolved the cluster name and ID not-set issues. Supported CCE clusters of v1.27.	2023-10-17
5.12.149	Supported mounting wrapping.	2023-10-12
5.12.148	Solved the high CPU usage issue in the multi-GPU scenario.	2023-08-30
5.12.147	Solved the issue that log transfer function could not be restarted and the host GPU metric adaptation issue.	2023-08-17
5.12.142	Supported the collection of container GPU metrics of CCE clusters of v1.25 or later.	2023-06-13
5.12.139	Solved the issue of a large number of TIME_WAIT TCP connections when LTS logs are reported.	2023-04-25
5.12.135	Solved the 0% CPU usage issue. Solved the missing container network metrics of containerd nodes in CCE v1.23 clusters. Supported collection of disk partition metrics of EulerOS 2.5.	2023-02-08
5.12.133	Supported multi-line collection of container standard output logs.	2022-12-17
5.12.130	Supported direct ingestion of CCE logs to LTS.	2022-11-04
5.12.120	Added a metric: maximum number of process handles. Supported changing of the LTS PodLB domain name.	2022-08-28
5.12.111	Added thread metrics and fixed the issue that the LVS disk partition metrics fail to be obtained.	2022-06-09
5.12.100	Added two metrics: used memory working set and memory working set usage. Supported container log tagging with stderr.log or stdout.log during collection. Added the Pod_ip tag for container log reporting. Supported double asterisks (**) for matching files in the current directory.	2022-01-15
5.12.98	Supported LTS log collection blacklists and changed the source of container metrics to working_set.	2021-09-29
5.12.96	Supported discovery of more types of cloud resources.	2021-09-22
5.12.90	Updated the GPU metric source.	2021-07-15
5.12.87	Supported more types of disks.	2021-03-30
5.12.75	Adapted to secure containers.	2021-03-09