Creating a Quick Analysis Task

Constraints

1. The following describes constraints on fields displayed under Quick Analysis.

   

   • The first 100,000 logs can be analyzed.

     The purpose of quick analysis is to quickly return the distribution and change trend of field values. It does not analyze all data, but only samples.

   • Logs can be filtered by query time and criteria for analysis.

     Quick analysis is to analyze the logs queried by query statements. When the number of queried logs is 0, no result is displayed for quick analysis.

   • Quick analysis can be used to generate query statements.

     You can click an analysis result to automatically generate a query statement, query logs, and generate a new quick analysis.

   • The maximum length of a field for quick analysis is 2,000 bytes.
   • The distribution statistics in quick analysis field area displays the first 100 records.
2. In the More Metric Info pane for analyzing a single field:

   The field analysis is performed on all logs that meet the specified time range and search criteria, that is, the search results displayed on the Log Search tab page. If the number of logs in the search results is less than 100 million, the analysis is performed on all of them. If the number of logs exceeds 100 million, a sampling analysis is performed, with a maximum of 100 million logs sampled. To avoid sampling, narrow the time range or add search criteria to reduce the number of logs searched.

Prerequisites

Quick analysis is conducted on fields extracted from structured logs. Ensure that logs have been reported, structuring has been complete, and indexing has been configured before you create a quick analysis task. For details, see Using ICAgent to Collect Host Logs and Configuring Log Indexing.

If you have not configured ICAgent structuring parsing when configuring log ingestion to LTS, you can configure ICAgent or cloud configuring parsing for the target log stream separately. ICAgent structuring parsing is recommended. For details, see Configuring ICAgent Structuring Parsing.

Creating a Quick Analysis Task

Quick analysis is performed on a per-log-stream basis. You can create a quick analysis task as follows:

1. Log in to the LTS console and choose Log Management in the navigation pane.
2. Click the target log group or stream to access the log details page.
3. On the Log Search tab page, click next to Quick Analysis to go to the Index Settings tab page. Under Index Fields, click Auto Configure to generate index fields automatically. Enable quick analysis for these fields.
4. Click OK. After the quick analysis task is created, you can perform statistical analysis on these fields. For example, you can check the basic distribution of fields, various statistical metrics, and the top 5 value sequence chart. These tools provide in-depth insights and visualizations to help you understand and explore the data.
   • abc displayed in front of a field indicates that the field is of the string type.
   • 1.2 displayed in front of a field indicates that the field is of the float type.
   • 123 displayed in front of a field indicates that the field is of the long type.
   • {...} displayed in front of a field indicates that the field is of the JSON type.

5. If the number of statistics records for a displayed field under Quick Analysis is 100 or more, click More to check them on the Charts tab page. The visual charts display data intuitively, enhancing analysis and understanding efficiency.
6. Click next to the target field to access the More Metric Info pane. This function is available only in region AP-Singapore.
   • For a field of the numeric type, the pane displays its basic distribution (such as total number of log lines and total number of lines in the current column), statistics metrics (such as maximum, minimum, and average values), and value distribution histogram.

     Table 1 Description of a numeric field

     Category	Parameter	Description
     Basic Distribution	Total Log Lines	Total number of log lines that meet the time range and search criteria you specify on the Log Search tab page.
     	Lines in Current Column	Total number of log lines containing the field in the current search results.
     	Lines with Missing Values	Total Log Lines minus Lines in Current Column.
     	Total Distinct Values	Number of distinct field values in the column, calculated using the approx_distinct function.
     Statistics Metrics	Max. Value	Maximum value of the field.
     	Min. Value	Minimum value of the field.
     	Avg. Value	Average value of the field.
     	Median	Value in the middle position after the data is sorted in ascending order.
     	First Quartile (Q1)	Value at the 25% position after the data is sorted in ascending order.
     	Third Quartile (Q3)	Value at the 75% position after the data is sorted in ascending order.
     	Sample Standard Deviation	Sample standard deviation of the field.
     	Population Standard Deviation	Population standard deviation of the field.
     	Kurtosis	A statistical concept that indicates the concentration degree of data distribution.
     	Skewness	A statistical concept that indicates the skewness of data.
     Value Distribution Histogram	Statistical histogram created by dividing the value distribution into 10 intervals.

   • You can click Number distribution in the upper right corner of the More Metric Info pane to access the Charts tab page. The visual charts display data intuitively, enhancing analysis and understanding efficiency.
   • For a field of the string type, the pane displays its basic distribution (such as total number of log lines and total number of lines in the current column), statistics metrics (such as maximum, minimum, and average lengths), and a sequence diagram of the top 5 values.

     Table 2 Description of a string field

     Category	Parameter	Description
     Basic Distribution	Total Log Lines	Total number of log lines that meet the time range and search criteria you specify on the Log Search tab page.
     	Lines in Current Column	Total number of log lines containing the field in the current search results.
     	Lines with Missing Values	Total Log Lines minus Lines in Current Column.
     	Missing Value Ratio	Lines with Missing Values divided by Total Log Lines.
     	Total Distinct Values	Number of distinct field values in the column, calculated using the approx_distinct function.
     	Distinct Value Ratio	Total Distinct Values divided by Total Log Lines.
     Statistics Metrics	Max. Characters	Maximum number of characters in the field value.
     	Min. Characters	Minimum number of characters in the field value.
     	Avg. Characters	Average number of characters in the field value.
     Sequence Diagram of Top 5 Values	Trend chart showing how the top 5 values from the specified time range change over time.

   • You can click Field distribution in the upper right corner of the More Metric Info pane to access the Charts tab page. The visual charts display data intuitively, enhancing analysis and understanding efficiency.

7. On the More Metric Info pane, click a blue parameter value to access the Charts tab page and generate the corresponding SQL query statement. In this way, you can check the distribution and change trend of field values more intuitively. For details, see SQL Functions and Visualizing Logs in Statistical Charts.