Displaying Query and Analysis Results on Pages

If there are too many query and analysis results, the display speed and query experience are reduced. LTS can display query and analysis results on multiple pages. You can control the number of logs returned each time. This practice describes how to display query and analysis results on multiple pages.

Pagination Modes

The log search and statistical chart functions of LTS allows you to use query and analysis statements to query keywords and perform SQL analysis on the query results.

Log search: Search raw logs by keyword. You can use the line_num, search_type, limit, and is_desc parameters in the log query API to achieve pagination effects. For more information, see Querying Logs.
Statistical charts: Use SQL statements to analyze query results and obtain statistics. You can use the SQL LIMIT syntax to implement pagination. For more information, see SQL Analysis Syntax.

Pagination in Log Queries

The line_num, search_type, limit, and is_desc parameters in the log query API are described as follows:

line_num: sequence number of the final log event in the last query result. This parameter is not required for the first query, but is required for subsequent pagination queries. The value can be obtained from the response of the last query. The value of line_num should be between the values of start_time and end_time. If the custom time function is enabled, you need to use both this field and the __time__ field for pagination queries.
search_type: The value is init (default value) for the first query, or forwards or backwards for a pagination query. This parameter is used together with is_desc for pagination queries.
limit: number of log events to be queried each time. The default value is 50 when this parameter is not set. You are advised to set this parameter to 100.
is_desc: whether the search order is descending or ascending. The default value is false (ascending order).

For pagination reading, obtain the value of line_num from the last query result, set search_type to determine the direction of pagination (backward or forward), set limit to the number of log events queried each time, and set the time sequence of logs with is_desc. If the returned value of isQueryComplete is true, all data has been read.

The following uses Java code as an example. For more information, see LTS Java SDK.

      
       
         
         
           package com.huaweicloud.sdk.test;
import com.huaweicloud.sdk.core.auth.ICredential;
import com.huaweicloud.sdk.core.auth.BasicCredentials;
import com.huaweicloud.sdk.core.exception.ConnectionException;
import com.huaweicloud.sdk.core.exception.RequestTimeoutException;
import com.huaweicloud.sdk.core.exception.ServiceResponseException;
import com.huaweicloud.sdk.lts.v2.region.LtsRegion;
import com.huaweicloud.sdk.lts.v2.*;
import com.huaweicloud.sdk.lts.v2.model.*;
public class ListLogsSolution  {
    public static void main(String[] args) {
        // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
        // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
        String ak = System.getenv("CLOUD_SDK_AK");
        String sk = System.getenv("CLOUD_SDK_SK");
        ICredential auth = new BasicCredentials()
            .withAk(ak)
            .withSk(sk);
        LtsClient client = LtsClient.newBuilder()
            .withCredential(auth)
            .withRegion(LtsRegion.valueOf("region"))
            .build();
        ListLogsRequest request = new ListLogsRequest();
        request.withLogGroupId("group_id");
        request.withLogStreamId("stream_id");
        QueryLtsLogParams body = new QueryLtsLogParams();
        // Number of raw log events returned each time
        body.withLimit(3);
        // Turn page backwards or forwards. This parameter can be left blank or set to the default value (init) for the first query.
        body.withSearchType(QueryLtsLogParams.SearchTypeEnum.fromValue("init"));
        // Whether to query log events in descending order by time.
        body.withIsDesc(false);
        // Query end time.
        body.withEndTime("1727429160000");
        // Query start time.
        body.withStartTime("1727428260000");
        // Whether to return the number of results.
        body.withIsCount(true);
        request.withBody(body);
        while (true) {
            try {
                ListLogsResponse response = client.listLogs(request);
                System.out.println(response.toString());
                // If the number of log events is equal to the value of limit, some log events have not been queried. Continue to query logs.
                if (response.getCount() < 3) {
                    return;
                }
                // Obtain the sequence number of the final log event in the last query result. Use it as the start sequence number of this query.
                body.withLineNum(response.getLogs().get(2).getLineNum());
                // Turn page backwards or forwards.
                body.withSearchType(QueryLtsLogParams.SearchTypeEnum.fromValue("forwards"));
            } catch (ConnectionException e) {
                e.printStackTrace();
            } catch (RequestTimeoutException e) {
                e.printStackTrace();
            } catch (ServiceResponseException e) {
                e.printStackTrace();
                System.out.println(e.getHttpStatusCode());
                System.out.println(e.getRequestId());
                System.out.println(e.getErrorCode());
                System.out.println(e.getErrorMsg());
            }
        }
    }
}

          

        

      
     

Pagination in Statistical Charts

You can use the Limit syntax in SQL to analyze and display analysis results.

    
         limit num offset num2

Example:

    
         * | select hostid,podname  limit 0
* | select hostid,podname  limit 500 offset 500

The parameters are described as follows:

offset: the line from which the analysis result is read.
limit: the number of lines to be read. The maximum value is 100,000. Reading too many lines at a time will increase network latency and slow down client processing.

Assume that you use the * | select hostid,podname limit 20 offset 10 statement to query and analyze logs and specify that 100 lines of logs are to be returned. You can set the query to read 20 lines at a time and complete the task in five reads as follows.

The following uses Java code as an example. For more information, see .

package com.huaweicloud.sdk.test;
import com.huaweicloud.sdk.core.auth.ICredential;
import com.huaweicloud.sdk.core.auth.BasicCredentials;
import com.huaweicloud.sdk.core.exception.ConnectionException;
import com.huaweicloud.sdk.core.exception.RequestTimeoutException;
import com.huaweicloud.sdk.core.exception.ServiceResponseException;
import com.huaweicloud.sdk.lts.v2.region.LtsRegion;
import com.huaweicloud.sdk.lts.v2.*;
import com.huaweicloud.sdk.lts.v2.model.*;
public class ListLogsSolution {
    public static void main(String[] args) {
        // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
        // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
        String ak = System.getenv("CLOUD_SDK_AK");
        String sk = System.getenv("CLOUD_SDK_SK");
        ICredential auth = new BasicCredentials()
            .withAk(ak)
            .withSk(sk);
        LtsClient client = LtsClient.newBuilder()
            .withCredential(auth)
            .withRegion(LtsRegion.valueOf("region"))
            .build();
        ListLogsRequest request = new ListLogsRequest();
        request.withLogGroupId("group_id");
        request.withLogStreamId("stream_id");
        QueryLtsLogParams body = new QueryLtsLogParams();
        body.withIsAnalysisQuery(true);
        // offset indicates the query offset. Set it to 0 for the first query. limit indicates the number of returned records, with a maximum value of 10,000.
        long offset = 0;
        long limit = 3;
        // Modify the offset in the SQL statement.
        body.withQuery(String.format("* | select hostid,podname limit %s offset %s",limit,offset));
        body.withEndTime("1727429160000");
        body.withStartTime("1727428260000");
        request.withBody(body);
        while (true) {
            try {
                ListLogsResponse response = client.listLogs(request);
                System.out.println(response.toString());
                if (response.getAnalysisLogs().size() < 3) {
                    return;// Pagination query. If an empty result is returned, the query is complete and the loop exits.
                }
                offset = offset + limit;
                body.withQuery(String.format("* | select hostid,podname limit %s offset %s",limit,offset));
            } catch (ConnectionException e) {
                e.printStackTrace();
            } catch (RequestTimeoutException e) {
                e.printStackTrace();
            } catch (ServiceResponseException e) {
                e.printStackTrace();
                System.out.println(e.getHttpStatusCode());
                System.out.println(e.getRequestId());
                System.out.println(e.getErrorCode());
                System.out.println(e.getErrorMsg());
            }
        }
    }
}

Parent topic: Log Search and Analysis

Previous topic: Embedding the LTS Log Query Page into a User-built System

Next topic: Optimizing Regular Expressions for Faster Log Parsing