Help Center/ Log Tank Service/ User Guide/ Log Search and Analysis (Pipe Character)/ Overview
Updated on 2025-08-14 GMT+08:00
View PDF

Overview

Log search and analysis are indispensable to O&M. After configuring log ingestion, you can search and analyze the collected log data on LTS. Its efficient and professional log collection, search, and analysis help you monitor and manage your systems and applications.

The search function using the pipe character (|) is available only to whitelisted. To use it, create a service ticket.

Log Structuring

Before searching and analyzing reported logs, you need to configure structuring and indexing for them. Structured data has a unified length and format, which can significantly improve search and analysis efficiency and accuracy.

Log data can be structured or unstructured.

  • Structured data is organized using data models such as tables and relational databases. It has a strict length and format, facilitating storage and analysis.
  • Unstructured data has no pre-defined data models and cannot be fit into two-dimensional tables of databases. It is difficult to analyze unstructured data.

Log structuring extracts logs with fixed formats or high similarity from log streams and filters out irrelevant logs.

Log structuring parsing converts unstructured or semi-structured log data into a structured format for better storage, query, and analysis, improving log data readability, searchability, and query efficiency.

LTS offers both ICAgent-based and cloud-based structuring parsing modes. However, a log stream can only use one parsing mode. For example, if you have configured ICAgent structuring parsing for a log stream and want to configure cloud structuring parsing for it, delete the existing ICAgent structuring parsing configuration first.

If you have configured ICAgent structuring parsing during log ingestion, you do not need to configure cloud structuring parsing.

  • ICAgent structuring parsing is performed on the collection side and supports combined plug-ins for parsing. You can set multiple collection configurations with different structuring parsing rules for a single log stream. This parsing mode is recommended. For details, see Configuring ICAgent Structuring Parsing.
  • Leveraging the computing power of LTS, cloud structuring parsing structures logs in log streams using various log extraction methods. In the future, it will incur log processing traffic fees based on the log volume.

Log Search and Analysis

After the structuring is complete, use the search syntax and SQL functions provided by LTS to set search criteria. Using the pipe character (|) to combine search statements with SQL analysis statements enhances the log data extraction and analysis efficiency, making it easy to visualize the results. For details, see Searching and Analyzing Logs.