Help Center/ Log Tank Service/ Best Practices/ Log Search and Analysis/ Querying and Analyzing Website Logs
Updated on 2026-08-28 GMT+08:00
View PDF
Share

Querying and Analyzing Website Logs

This practice will show you how to use Log Tank Service (LTS) to query and analyze website logs.

Prerequisites

Website access logs have been ingested to LTS. For details, see Ingesting ECS Text Logs to LTS.

Step 1: Creating a Log Index

An index is a storage structure used to query log data. Configuring indexing makes log searches and analysis faster and easier.

  1. Log in to the LTS console. The Log Management page is displayed by default.
  2. Click the target log group or log stream to access the log stream details page.
  3. On the Log Search tab page, click next to Quick Analysis to go to the Index Settings tab page.
  4. Enable Index Whole Text by default.
  5. Configure the field index and click OK.

    You can manually add field indexes one by one or click Auto Configure to generate index fields based on the first log event in the last 15 minutes or common system reserved fields (such as hostIP, hostName, and pathFile).
    • Either full-text indexing or field indexing must be configured.
    • By default, LTS has created field indexes for some built-in reserved fields. For details, see System Reserved Fields.
    • After the index configuration is modified, the modification takes effect only for newly written log data.

Step 2: Querying and Analyzing Logs

LTS enables you to search and analyze logs simultaneously using a statement that includes a pipe character (|). The syntax consists of three parts: a statement for searching unstructured and semi-structured data, a pipe character (|), and a statement for analyzing structured data, for example, * and msg:"hello world" | SELECT avg(value).

For details about how to query and analyze logs, see Searching and Analyzing Logs.

Querying statements

  • Query logs that contain Chrome. 
    Chrome
  • Query logs whose request time is longer than 60s. 
    request_time > 60
  • Query logs whose request time is between 60–120s. 
    request_time in [60 120]
  • Query logs about successful GET requests (status codes 200 to 299). 
    request_method : GET and status in [200 299]
  • Query logs whose request_uri is /request/path-2/file-2. 
    request_uri:/request/path-2/file-2

Analyzing logs

  • Collect website visits' page views (PVs).

    Use the count function to collect website visits' PVs.

    * | SELECT count(*) as PV

    The query result shows that the PV value is 8,764.

  • Collect website visits' PVs by minute.

    Use the date_trunc function to truncate the time to minute, group the data by time, and then use the count function to calculate the PVs per minute. Then, sort the data by time.

    * | SELECT count(*) as PV, date_trunc('minute', __time) as time GROUP BY time ORDER BY time
  • Collect the number of requests for each request method every 5 minutes.

    Use __time - __time%300000 to truncate the time to 5-minute intervals and group the data by time. Then, use the count function to calculate the number of requests every 5 minutes and sort the data by time.

    * | SELECT request_method, count(*) as count, __time - __time %300000 as time GROUP BY time, request_method ORDER BY time
  • Collect the week-on-week website visits' PVs.

    Use the count function to calculate the total number of PVs, and then use the ts_compare function to calculate the week-on-week PVs.

    * | SELECT diff[1] as this_week, diff[2] as last_week, time FROM (SELECT ts_compare(pv, 604800) as diff, time FROM (SELECT COUNT(*) as pv, date_trunc('week', __time) as time FROM log GROUP BY time ORDER BY time) GROUP BY time)
  • Collect client address distribution.

    Use the ip_to_province function to obtain the province corresponding to an IP address, group the IP addresses by province, use the count function to calculate the number of occurrences of each IP address, and sort the IP addresses by the number of occurrences.

    * | SELECT count(*) as count, ip_to_province(client_ip) as address GROUP BY address ORDER BY count DESC
  • Collect the top 5 requested paths.

    Group the data by request path, use the count function to calculate the number of visits to each path, and sort the paths by the number of visits.

    * | SELECT count(*) as PV, url_extract_path(request_uri) as PATH GROUP BY PATH ORDER BY PV DESC LIMIT 5

    After the query statement is executed, the returned example result displays the top 5 requested paths (PATH) and the number of visits to each path (PV).

  • Query the logs whose request_uri ends with body.json. 
    * | select * from log where request_uri like '%body.json'

Reference: Website Log Example

hostIP:203.0.113.15
collectTime:1719124688
hostName:website_access_log
body_bytes_sent:8736
client_ip:110.249.135.66
host:demo-business.cn
http_host:demo-business.cn
http_user_agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36
http_x_forwarded_for:110.249.135.66
instance_id:i-089d7f2c
instance_name:business-web-03
network_type:vpc
owner_id:op-789xyz-05
referer:https://search.demo-business.cn
remote_addr:10.0.120.45
remote_user:admin_ops
request_length:6892
request_method:GET
request_time:126
request_uri:/api/user/info/list?page=1&size=20
scheme:https
server_protocol:HTTP/1.1
status:200
time_local:14/Jun/2026:15:38:08
upstream_addr:10.0.56.18
upstream_response_time:87
upstream_status:200
user_agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Geckocko) Chrome/124.0.0.0 Safari/537.36vip_addr:203.0.113.18
vpc_id:6cf942a3****15ce74629b41
Parent topic: Log Search and Analysis