What's New

1

ICAgent structuring parsing

ICAgent supports parsing using regular expressions, delimiters, JSON, and combined plug-ins.

Commercial use

Setting ICAgent Structuring Parsing Rules

2

Function enhancement

You can select the And or Or relationship when configuring Kubernetes label whitelists and Kubernetes label blacklists for log ingestion.

Commercial use

Ingesting CCE Application Logs to LTS

3

Minute-level traffic monitoring

You can check the resource usage details and top log groups or streams by resource usage on the resource statistics page.

Commercial use

Viewing Log Management

4

Quick analysis

You can specify the number of samples for quick analysis. The value ranges from 100,000 (default) to 10 million. Quick analysis provides a fast overview by sampling field value statistics rather than analyzing all log data. The more logs sampled, the slower the analysis.

Commercial use

Setting Indexes

5

Log download

Up to 20 million logs can be downloaded at a time.

Commercial use

Accessing the Log Search Page

6

Log file code

Log file encoding supports Chinese Internal Code Extension Specification (GBK), which is a Chinese character encoding standard that extends both the ASCII and GB2312 encoding systems.

Commercial use

Ingesting CCE Application Logs to LTS

7

Log alarms

You can search for alarm rules by name.

Commercial use

Viewing Alarms in LTS

8

Query creation

You can create a query to search for log keywords.

Commercial use

Accessing the Log Search Page

9

Style optimization

The operation buttons on the log alarm page are optimized to facilitate user identification.

Commercial use

Configuring Log Alarm Rules

10

Repeated log collection

Logs from Cloud Container Engine (CCE), Elastic Cloud Server (ECS), and self-built Kubernetes can be repeatedly collected to LTS. This function is not available to Windows.

Commercial use

Ingesting CCE Application Logs to LTS

1

Log ingestion

You can customize key-value pairs, the maximum size of a single log (up to 1 MB), and log file path depth (up to 20 levels) when configuring ingestion.

Commercial use

Ingesting CCE Application Logs to LTS

2

Log alarms

You can edit, export, and import alarm rules in batches.

Commercial use

Configuring Log Alarm Rules

3

ICAgent collection

ICAgent for Windows supports host groups with identifiers, cross-cloud collection, and log collection error diagnosis.

Commercial use

Installing ICAgent (Extra-Region Hosts)

4

ICAgent collection diagnosis

Collection diagnosis dashboard templates are provided for monitoring exceptions, overall status, and collection status of ICAgent.

Commercial use

Setting ICAgent Collection

5

Log transfer

You can add variable parameters when setting a custom path for transferring logs to OBS.

Commercial use

Transferring Logs to OBS

6

Index settings

You can configure full-text indexes and field indexes for each log stream separately and customize delimiter configurations.

Commercial use

Setting Indexes

1

Structuring templates

A system template for GeminiDB Mongo audit logs is added.

Commercial use

Selecting a Cloud Structuring Template

2

Log ingestion

The standard output and standard error output can be collected to different files (stdout.log and stderr.log) or the same file (stdout.log).

Commercial use

Ingesting CCE Application Logs to LTS

3

Default delimiters

The backslash (\) and question mark (?) are added to default delimiters.

Commercial use

Configuring Log Content Delimiters

1

Metric generation

You can create log metric rules to extract metrics from log data reported to LTS for central management.

Open beta testing

Generating Metrics from Logs

2

Dashboards

Enterprise projects can be selected for dashboards.

Commercial use

Creating a Dashboard

3

Log groups

Enterprise projects can be selected for log groups.

Commercial use

Managing Log Groups

4

Batch connection

Log ingestion rules can be created in batches for services such as Bare Metal Server (BMS), Cloud Container Engine (CCE), and Elastic Cloud Server (ECS).

Commercial use

Using ICAgent to Collect Logs

5

Custom identifiers

You can set custom identifiers when creating host groups of the Windows type.

Commercial use

Managing Host Groups

1

Simple noise reduction for alarms

The notification frequency can be set for log alarms to prevent alarm storms.

Commercial use

Alarm Rules

2

Alarm action rules

You can create an alarm action rule and bind it with a message template.

Commercial use

Creating an Alarm Action Rule

3

ServiceStage log collection

Containerized application logs of ServiceStage can be ingested to LTS.

Commercial use

Collecting Logs from ServiceStage - Containerized Applications

1

Layout settings can be customized.

Layout settings can be customized on the page of log search.

Commercial use

Log Search

2

CCE event logs

CCE K8s event logs can be collected.

Commercial use

Collecting Logs from CCE

3

Logs of mounted container files can be collected.

CCE log collection allows ICAgent to collect logs of mounted container files.

Commercial use

Collecting Logs from CCE

4

Retention duration

The log retention duration can be set to 10 years. This function is available only for whitelisted users in internal tests.

Open beta testing

Log Groups

1

The simple view is added to viewing context.

On the page of viewing context, you can set the simple view to quickly browse logs by omitting information such as the log time.

Commercial use

Log Search

2

The names of log groups and log streams can be changed.

After a log group and a log stream are created, you can change their names. The original names cannot be changed.

Commercial use

Log Groups

3

Collection configurations can be verified.

When logs are collected from CCE, regular expressions can be verified on the collection configuration page.

Commercial use

Collecting Logs from CCE

1

Optimized charts

Configuration items of table charts, line charts, pie charts, bar charts, and number charts are optimized.

Commercial use

Statistical Charts

2

Interactive search

Interactive search is added for you to quickly add query statements to lower the search threshold.

Commercial use

Log Search

3

Optimized real-time log display

Real-time log display adopts real-time scrolling to improve the display effect.

Commercial use

Viewing Real-Time Logs

4

Quick search

The quota of statements that can be quickly searched for in a single log stream increases from 10 to 50.

Commercial use

Quick Search

5

Log stream tags

After a tag is added to a log stream, the log stream tag information is displayed in the reported charging data record (CDR). This helps you analyze fees by resource tag (log stream tag) in the billing center.

Commercial use

Best Practices

1

Alarm rules

• Alarms can be enabled, disabled, and temporarily disabled in batches, improving editing efficiency.

• Alarm rules can be renamed.

• Alarms can be copied.

Commercial use

Alarm Rules

2

Log ingestion

LTS can collect logs from self-built K8s clusters.

Open beta testing

Self-Built Kubernetes

3

Custom delimiters

You can customize delimiters for full-text indexes and field indexes.

Open beta testing

Structuring Modes

4

Index settings

You can configure full-text indexes and field indexes for each log stream separately to customize delimiter configuration.

Commercial use

Index Settings

5

Delimiter containing multiple characters

A delimiter can contain multiple characters.

Commercial use

Structuring Modes

1

Dashboard templates

Provided ER dashboard templates.

Open beta testing

Dashboard Templates

2

Optimized custom log time

When logs are ingested to LTS, you can enable Custom Log Time to set the time field in the logs as the ingested configuration time.

Commercial use

Customizing the Log Time

3

Log structuring templates

Added the templates of structuring GaussDB(for Cassandra) slow logs, MySQL audit logs, GaussDB(for Mongo) error logs, and GaussDB(for Mongo) slow logs.

Open beta testing

Structuring Templates

1

Dashboard templates

Provided VPC dashboard templates.

Open beta testing

Dashboard Templates

1

Dashboard templates

Provided CDN dashboard templates.

Open beta testing

Dashboard Templates

2

Filters

The dynamic value source in the filter can be fuzzy match.

Open beta testing

Filters

1

Log structuring templates

Added the templates for structuring SQL Server error logs and GaussDB(for Redis) slow logs.

Open beta testing

Log Structuring

2

Custom identifier host group

You can create a host group with a custom identifier that is added automatically.

Open beta testing

Custom Identifier

3

Adding a log stream to favorites

Favorite log streams can be displayed in two modes: My Favorites and My Favorites (Local Cache).

Open beta testing

LTS Console

1

Index settings

You can query and analyze logs after setting indexes.

Open beta testing

Index Settings

2

Creating alarms in batches

You can select multiple log streams to create keyword alarms.

Commercial use

Log Alarms

3

Function jobs

You can create functions from templates or scratch to normalize, transfer, anonymize, and filter logs.

Open beta testing

Function Processing

1

Dashboard templates

Provided APIG, Nginx, and CFW dashboard templates.

Open beta testing

Dashboard Templates

2

Log search syntax

Search syntaxes such as *, in, (), and key:#"abc def" are supported.

Commercial use

Log Search

3

Log ingestion

The console provides guided ingestion of ELB, APIG, and CTS logs.

Commercial use

Collecting Logs from Cloud Services

1

Dashboard templates

Provided DCS, DDS, and ELB dashboard templates.

Open beta testing

Dashboard Templates

2

Usage alerts

If the log resource usage exceeds a specified limit, the system gives an alarm.

Commercial use

Configuring Usage Alerts

3

Log structuring templates

Added the structuring configuration templates for Tomcat, DDS, and DCS.

Commercial use

Log Structuring

1

Custom log time

By default, the log time is the time configured in log ingestion. You can enable User-defined log time to customize the log time.

Commercial use

Custom log time

2

Search tips

During log search, you can press Tab to automatically add the first keyword displayed in the drop-down list.

Commercial use

Log Search

3

Collecting logs from CCE

Logs from CCE can be collected.

Commercial use

Collecting Logs from CCE

1

Optimized log context display

When viewing the context, you can query the number of rows, highlight log content, and filter logs.

Commercial use

Quick Search

1

Cross-account ingestion

If you use the Cross-Account Ingestion - Log Stream Mapping, logs of the delegator account are ingested to LTS.

Commercial use

Cross-Account Ingestion

1

New log search page

The new page provides log sharing, simplified field details, and layout setting.

Commercial use

Log Search

1

Resource statistics

Statistics of log index traffic, read/write traffic, and storage usage are displayed.

Commercial use

Resource Statistics

1

Tag transfer

Tags can be transferred along with logs.

Commercial use

Transferring Logs to OBS

1

Tag fields

Logs can be filtered by tag fields during transfer.

Commercial use

Transferring Logs to OBS

1

Time zone setting

Time zone can be configured for log transfer.

Commercial use

Transferring Logs to OBS

1

Transfer to encrypted OBS buckets

You can transfer logs to an encrypted OBS bucket.

Commercial use

Transferring Logs to OBS

2

Log transfer delegation

You can transfer logs for another account as long as you have been delegated.

Commercial use

Transferring Logs to OBS

1

Log ingestion wizard

Step-by-step wizard for ingesting logs from hosts.

Commercial use

Log Ingestion

2

Host management

Host groups and log ingestion configurations can be associated with each other. This allows you to modify the log collection paths of multiple hosts at a go.

Commercial use

Managing Host Groups

1

Log ingestion using APIs

You can use APIs to ingest logs.

Commercial use

Collecting Logs Using APIs

2

Delimiters

You can configure delimiters to split log content into words, so you can search for logs by these words.

Commercial use

Configuring Delimiters

3

Cloud service log ingestion

You can ingest logs from different cloud services.

Commercial use

Collecting Logs from Cloud Services

1

Custom structuring templates

You can create structuring templates for quick reuse.

Commercial use

Structuring Logs

1

Nginx log structuring

You can parse logs in standard Nginx format.

Commercial use

Structuring Modes

1

JSON, delimiters, and system templates

You can parse logs in standard JSON format, and parse logs using delimiters (such as commas and spaces) or ELB, VPC, and CTS structuring templates.

Commercial use

Structuring Logs

1

Log search

You can filter logs by fields.

Commercial use

Log Search

1

Open APIs

APIs for managing log groups and log streams are available.

Commercial use

Creating a Log Group

1

Log structuring and SQL query

You can structure logs, and use SQL statements to query and analyze structured logs.

Commercial use

Structuring Logs

1

Transferring Logs to DIS

You can transfer logs to Data Ingestion Service (DIS) and further transmit the logs through DIS to EI. DIS supports both log storage and big data analysis. To be specific, DIS can transfer a large number of log files to the cloud for backup, offline analysis, query, and machine learning. In addition, You can use DIS for data recovery and fault analysis after data loss or exceptions. During transfer, a large number of small text files can be merged into a large file to accelerate data processing.

Commercial use

Transferring Logs to DIS