Ingesting ServiceStage Cloud Host Logs to LTS

Prerequisites

• ICAgent has been installed and added to the host group.
• A ServiceStage application has been created. For details, see Creating an Application.
• A ServiceStage environment has been created. For details, see Creating an Environment.
• A ServiceStage component has been created. For details, see Creating and Deploying a Component.

Procedure for Creating a Single Ingestion Configuration

1. Log in to the LTS console.
2. In the left navigation pane, choose Log Ingestion. Click ServiceStage - Cloud Host Logs on the Access Wizard tab page. Or, click Ingest Log on the Ingestion Rule tab page and then choose ServiceStage - Cloud Host Logs.
3. Alternatively, choose Log Management in the left navigation pane. Click the name of the target log stream to go to the log details page. Click in the upper right corner. On the displayed page, click the Collection Configuration tab and click Create. In the displayed dialog box, click ServiceStage - Cloud Host Logs.
4. In the Select Log Stream step, set the following parameters:
   1. Select a ServiceStage application and ServiceStage environment.
   2. Select a log group from the Log Group drop-down list. If there are no desired log groups, click Create Log Group to create one.
   3. Select a log stream from the Log Stream drop-down list. If there are no desired log streams, click Create Log Stream to create one.
   4. Click Next: (Optional) Select Host Group.

5. (Optional) Select a host group.
   1. Select one or more host groups from which you want to collect logs. If there are no desired host groups, click Create above the host group list to create one.
      

      You can skip this step and configure host groups after the ingestion configuration is complete. There are two ways to do this:

      • Choose Host Management in the navigation pane, click the Host Groups tab, and associate host groups with ingestion configurations.
      • On the LTS console, choose Log Ingestion in the navigation pane and click an ingestion configuration. On the displayed page, add one or more host groups for association.
   2. Click Next: Configurations.

6. Configure the collection.

   Specify collection rules. For details, see Configuring the Collection.

7. Configure cloud structuring. For details, see Cloud Structuring Parsing.
   

   If the selected log stream has been structured, exercise caution when deleting it.

   • If you have enabled ICAgent structuring parsing configuration, you do not need to configure cloud structuring parsing. For details, see Configuring ICAgent Collection.
   • ICAgent structuring parsing configuration is available only to whitelisted users. To use this function, submit a service ticket.

8. Configure indexes. For details, see Index Settings.
9. Click Submit. An ingestion configuration will be displayed on the Ingestion Rule tab page. You can:
   • Click the name of the ingestion rule to view its details.
   • Click Edit in the Operation column to modify the ingestion rule.
   • Click Configure Tag in the Operation column to add a tag.
   • Click Copy in the Operation column to copy the ingestion rule.
   • Click Delete in the Operation column to delete the ingestion rule.

Configuring the Collection

When you configure ServiceStage log ingestion, the collection configuration details are as follows.

1. Collection Configuration Name: Enter up to 64 characters. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. The name cannot start with a period or underscore, or end with a period.
2. Collection Paths: Add one or more host paths. LTS will collect logs from these paths.
   • Logs can be collected recursively. A double asterisk (**) can represent up to 5 directory levels in a path.

     For example, /var/logs/**/a.log matches the following logs:

     /var/logs/1/a.log 
     /var/logs/1/2/a.log
     /var/logs/1/2/3/a.log
     /var/logs/1/2/3/4/a.log
     /var/logs/1/2/3/4/5/a.log

     

     • /1/2/3/4/5/ indicates the 5 levels of directories under the /var/logs directory. All the a.log files found in all these levels of directories will be collected.
     • Only one double asterisk (**) can be contained in a collection path. For example, /var/logs/**/a.log is acceptable but /opt/test/**/log/** is not.
     • A collection path cannot begin with a double asterisk (**), such as /**/test to avoid collecting system files.
   • You can use an asterisk (*) as a wildcard for fuzzy match. The wildcard (*) can represent one or more characters of a directory or file name.
     

     If a log collection path is similar to C:\windows\system32 but logs cannot be collected, enable the Web Application Firewall (WAF) and configure the path again.

     • Example 1: /var/logs/*/a.log will match all a.log files found in all directories under the /var/logs/ directory:

       /var/logs/1/a.log

       /var/logs/2/a.log

     • Example 2: /var/logs/service-*/a.log will match files as follows:

       /var/logs/service-1/a.log

       /var/logs/service-2/a.log

     • Example 3: /var/logs/service/a*.log will match files as follows:

       /var/logs/service/a1.log

       /var/logs/service/a2.log

   • If the collection path is set to a directory (such as /var/logs/), only .log, .trace, and .out files in the directory are collected.

     If the collection path is set to a file name, the corresponding file is collected. Only text files can be collected. To query the file format, run file -i File name.

   • Add Custom Wrapping Rule: ICAgent determines whether a file is wrapped based on the file name rule. If your wrapping rule does not comply with the built-in rules, you can add a custom wrap rule to prevent log loss during repeated collection and wrapping.

     The built-in rules are {basename}{connector}{wrapping identifier}.{suffix} and {basename}.{suffix}{connector}{wrapping identifier}. The connector is -._, the wrapping identifier is a non-letter symbol, and the suffix is a letter.

     A custom wrapping rule consists of {basename} and the feature regular expression of the wrapped file. Example: If your log file name is /opt/test.out.log, and the wrapped file names are test.2024-01-01.0.out.log and test.2024-01-01.1.out.log, the collection path is /opt/*.log and the wrapping rule is {basename}\.[-0-9\.].out.log.

   

   • Ensure that sensitive information is not collected.
   • If you want to collect system logs from a Windows host, enable the collection of Windows event logs when configuring the collection.
   • LTS cannot collect logs of PostgreSQL (database) instances. It only collects logs of ECS (host) instances.
   • A collection path can be configured only once. It means that a path of a host cannot be added for different log streams. Otherwise, log collection may be abnormal.
   • If a collection path of a host has been configured in AOM, do not configure the path in LTS. If a path is configured in both AOM and LTS, only the path that is configured later takes effect.
   • If log files were last modified more than 12 hours earlier than the time when the path is added, the files are not collected.
3. Set Collection Filters: Blacklisted directories or files will not be collected. If you specify a directory, all files in the directory are filtered out.

   Blacklist filters can be exact matches or wildcard pattern matches. For details, see Collection Paths.

   

   If you blacklist a file or directory that has been set as a collection path in the previous step, the blacklist settings will be used and the file or files in the directory will be filtered out.

4. Collect Windows Event Logs: To collect logs from Windows hosts, enable this option and set the following parameters.

   Table 1 Parameters for collecting windows event logs

   Parameter	Description
   Log Type	Log types include System, Application, Security, and Startup.
   First Collection Time Offset	Example: Set this parameter to 7 to collect logs generated within the 7 days before the collection start time. This offset takes effect only for the first collection to ensure that the logs are not repeatedly collected. Max: 7 days.
   Event Level	You can filter and collect Windows events based on their severity (information, warning, error, critical, and verbose). This function is available only to Windows Vista or later.

5. Select the corresponding component under ServiceStage matching rule.
6. Enable Structuring Parsing. For details, see Configuring ICAgent Collection.
   

   This function is available only to whitelisted users. To use it, submit a service ticket.

7. Perform other configurations.

   Table 2 Other configurations

   Parameter	Description
   Max Directory Depth	The maximum directory depth is 5 levels. ICAgent does not collect log files with directory levels beyond this value. Set this parameter to the appropriate level for a target collection path with fuzzy matching strings to avoid wasted ICAgent resources.
   Split Logs	LTS supports log splitting. If this option is enabled, a single-line log larger than 500 KB will be split into multiple lines for collection. For example, a 600 KB single-line log will be split into a line of 500 KB and a line of 100 KB. If this option is disabled, a log larger than 500 KB will be truncated.
   Collect Binary Files	LTS supports binary file collection. Run the file -i File_name command to view the file type. charset=binary indicates that a log file is a binary file. If this option is enabled, binary log files will be collected, but only UTF-8 strings are supported. Other strings will be garbled on the LTS console. If this option is disabled, binary log files will not be collected.
   Log File Code	The encoding format of log files is UTF-8.
   Collection Policy	Select Incremental or All. Incremental: When collecting a new file, ICAgent reads the file from the end of the file. All: When collecting a new file, ICAgent reads the file from the beginning of the file.

8. Configure the log format and log time.

   Table 3 Log collection settings

   Parameter	Description
   Log Format	Single-line: Each log line is displayed as a single log event. Multi-line: Multiple lines of exception log events can be displayed as a single log event. This is helpful when you check logs to locate problems.
   Log Time	System time: log collection time by default. It is displayed at the beginning of each log event. NOTE: Log collection time is the time when logs are collected and sent by ICAgent to LTS. Log printing time is the time when logs are printed. ICAgent collects and sends logs to LTS with an interval of 1 second. Restriction on log collection time: Logs are collected within 24 hours before and after the system time.
   	Time wildcard: You can set a time wildcard so that ICAgent will look for the log printing time as the beginning of a log event. If the time format in a log event is 2019-01-01 23:59:59.011, the time wildcard should be set to YYYY-MM-DD hh:mm:ss.SSS. If the time format in a log event is 19-1-1 23:59:59.011, the time wildcard should be set to YY-M-D hh:mm:ss.SSS. NOTE: If a log event does not contain year information, ICAgent regards it as printed in the current year. Example: YY - year (19) YYYY - year (2019) M - month (1) MM - month (01) D - day (1) DD - day (01) hh - hours (23) mm - minutes (59) ss - seconds (59) SSS - millisecond (999) hpm - hours (03PM) h:mmpm - hours:minutes (03:04PM) h:mm:sspm - hours:minutes:seconds (03:04:05PM) hh:mm:ss ZZZZ (16:05:06 +0100) hh:mm:ss ZZZ (16:05:06 CET) hh:mm:ss ZZ (16:05:06 +01:00)
   Log Segmentation	This parameter needs to be specified if the Log Format is set to Multi-line. By generation time indicates that a time wildcard is used to detect log boundaries, whereas By regular expression indicates that a regular expression is used.
   By regular expression	You can set a regular expression to look for a specific pattern to indicate the beginning of a log event. This parameter needs to be specified when you select Multi-line for Log Format and By regular expression for Log Segmentation.

Creating Multiple Ingestion Configurations

You can create ingestion tasks in batches on the Ingestion Rule tab page.

1. Click Batch Ingestion to go to the configuration details page. For details, see Table 4.
   

   Structuring parsing configuration is available only to whitelisted users. For details, see Configuring ICAgent Collection. To use this function, submit a service ticket.

   Table 4 Adding configurations in batches

   Type	Operation	Description
   Basic Settings	Ingestion Type	Select ServiceStage - Cloud Host Logs.
   	Configurations to Add	Enter the number of ingestion configurations in the text box and click Add. A maximum of 100 ingestion configurations can be added at a time, including the one already exists under Ingestion Settings by default.
   Ingestion Settings	Configuration List	The ingestion configurations are displayed on the left. You can add up to 99 more configurations. The ingestion configuration details are displayed on the right. For details, see Creating Multiple Ingestion Configurations. After an ingestion configuration is complete, you can click Apply to Other Configurations to copy the configuration to other configurations.

2. Click Check Parameters. After the check is successful, click Submit.
3. The added ingestion configurations will be displayed in the lower part of the Ingestion Rule tab page after the batch creation is successful.
4. (Optional) Perform the following operations on an ingestion configuration:
   • Select multiple existing ingestion configurations and click Modify. On the displayed page, select an ingestion type to modify the corresponding ingestion configurations.
   • Select multiple existing ingestion configurations and click Open or Close. If you toggle off the switch in the Status column of an ingestion configuration, logs will not be collected for this configuration.
   • Select multiple existing ingestion configurations and click Delete.