Using LTS to Implement Top N Statistics Ranking and Time-based Sorting
In cloud service O&M scenarios, log data is the core basis for troubleshooting, performance analysis, and service monitoring. However, O&M personnel face the challenge of how to efficiently extract useful information from a large amount of log data. LTS provides one-stop capabilities, including log collection, storage, query, analysis, and visualization. In actual O&M, top N statistics ranking and time-based sorting are two common log query methods.
- Top N statistics ranking: Rank top N data by various dimensions (such as the traffic of source IP addresses, destination IP addresses, and destination ports) among massive amounts of logs. This helps you focus on core problems and quickly locate root causes.
- Time-based sorting: Sort log data by time and display the metric trend, such as the trend of requests in an hour and the error rate in one minute. This helps you detect performance patterns and time windows with abnormal fluctuations.
This practice describes how to compile SQL statements for top N statistics ranking and time-based sorting, and how to configure charts such as line charts, helping you learn the entire process from log collection to visualized analysis.
Prerequisites
- If the IAM users under your account want to use LTS, you need to grant them the permissions required. For details, see Using IAM to Grant Access to LTS.
- You have created a log group and a log stream. For details, see Creating a Log Group and Creating a Log Stream.
- Logs have been ingested to LTS. For details, see Log Ingestion.
- You have configured log structuring. For details, see Setting Cloud Structuring Parsing.
Top N Statistics Ranking and Visualization
Top N statistics ranking enables you to quickly locate the top N data from massive amounts of logs. This function is suitable for scenarios such as hotspot analysis, exception locating, and resource ranking.
- Log in to the LTS console. The Log Management page is displayed by default.
- Click the target log group or log stream to access the details page.
- Click the Charts tab. On the tab page displayed, select a time range, enter SQL statements for log search and analysis in the SQL search box, and click Search to search for logs. The following uses the cloud-based structuring template of an enterprise router as an example. For details about the enterprise router flow log parameters, see Viewing Details About a Flow Log.
- Example 1: Query the top 3 source addresses ranked by traffic.
* | select srcaddr as "source address" ,sum(bytes)/1024.0/1024.000 as "traffic/MB" from log group by "source address" order by "traffic/MB" desc limit 3
Figure 1 Top 3 source addresses ranked by traffic
- Example 2: Query the top 3 destination addresses and ports ranked by traffic.
* | select dstaddr as "destination address", dstport as "destination port" ,sum(bytes)/1024.0/1024.000 as "traffic/MB" from log group by "destination address","destination port" order by "traffic/MB" desc limit 3
Figure 2 Top 3 destination addresses and ports ranked by traffic
- Example 1: Query the top 3 source addresses ranked by traffic.
Time-based Sorting and Visualization
For time-based sorting, log data is sorted by time and the metric trend is displayed. This function is suitable for scenarios such as traffic monitoring, error rate trend analysis, and performance trend analysis.
- Log in to the LTS console. The Log Management page is displayed by default.
- Click the target log group or log stream to access the details page.
- Click the Charts tab. On the tab page displayed, select a time range, enter SQL statements for log search and analysis in the SQL search box, and click Search to search for logs.
Click
on the right, select the X-axis and dimension. The chart with data sorted based on time can then be displayed.Example: Query the traffic trend of the top 5 source addresses ranked by total traffic per minute.
* | select time_format(from_unixtime(__time - __time%60000), 'yyyy-MM-dd HH:mm:ss') as t, srcaddr as "source address", sum(bytes)/1024.0/1024.000 as "traffic/MB" from log where srcaddr in (select srcaddr from log group by srcaddr order by sum(bytes) desc limit 5) group by t,"source address" order by t limit 1000
Figure 3 Traffic trend of the top 5 source addresses ranked by total traffic per minute

What is your overall rating for this page?
Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot