Configuring Cloud Structuring Parsing for DNS

Scenarios

Log search and analysis are indispensable to O&M. After configuring log ingestion, you can search and analyze the collected log data on LTS. Its efficient and professional log collection, search, and analysis help you monitor and manage your systems and applications.

Before searching and analyzing reported logs, you need to configure structuring and indexing for them. Structured data has a unified length and format. This can significantly improve search and analysis efficiency and accuracy.

LTS provides five log structuring modes: regular expressions, JSON, delimiters, Nginx, and structuring templates. You can make your choice flexibly.

This practice describes how to configure log structuring for DNS Resolver and introduces SQL statements commonly used by DNS.

Constraints

DNS Resolver has been enabled in your region.

Process

Procedure

1. Create a log group.

   For details, see Creating a Log Group.

   Configure DNS access logging. For details, see Configuring Access Logging.

2. Create a log stream.

   For details, see Creating a Log Stream.

3. Configure cloud structuring parsing.
   1. After creating the log stream, click Log Settings in the upper right corner.

      

   2. On the displayed page, click the Cloud Structuring Parsing tab and set parameters as instructed.
      

      • If Retain Raw Logs is enabled, raw logs will be stored in LTS as the value of the content field. In this practice, keep this function enabled.
      • If Upload Parsing Failure Log is enabled, raw logs will be uploaded to LTS as the value of the _content_parse_fail_ field. In this practice, keep this function disabled.
      • You can choose one of the methods below to structure logs. In this practice, select Structuring Template.
      • Select a template. In this practice, select DNS resolution request logs.

        The following is an example log. Table 1 describes the key fields extracted from the log.

        2025-04-16 10:23:56.377 ws-api-hu.xfyun.cn. A NOERROR UDP cnnorth4a CNAME_hd-kaicp.xf-gtm.com.|A_120.92.238.203|A_120.92.238.23|A_120.92.238.24|A_120.92.238.30 192.168.0.8 d3b08f84-12ef-411f-8eae-5c2726f42c14

        Table 1 Key fields in DNS resolution request logs

        Parameter	Description	Example Value
        Time	Time	2025-04-16 10:23:56.377
        QueryName	Domain name to be resolved	ws-api-hu.xfyun.cn.
        QueryType	Record set type	A
        Rcode	Response code	NOERROR
        Protocol	Protocol	UDP
        DNS-Region	Region	cnnorth4a
        Answer	Response	CNAME_hd-kaicp.xf-gtm.com.|A_120.92.238.203|A_120.92.238.23|A_120.92.238.24|A_120.92.238.30
        Client	Client	192.168.0.8
        VpcId	VPC ID	d3b08f84-12ef-411f-8eae-5c2726f42c14

   3. Click Save.
4. Configure log indexing.
   1. Click Log Settings in the upper right corner of the log stream page.

      

   2. On the Index Settings tab, click Auto Configure under Index Fields, and retain the default values for other parameters.

      The key fields extracted in step 3.b are automatically configured as indexes.

      

   3. Click OK. The fields for configuring indexes are displayed on the Log Search tab.
5. Create a SQL scheduled job.
   1. Click Log Settings in the upper right corner of the log stream page.

      

   2. On the SQL Scheduled Jobs tab, click Create SQL Scheduled Job.

      

   3. Configure parameters as instructed.
      For details about common DNS SQL statements, see Common DNS SQL Statements.

      Figure 1 Configure Computation
      

      Figure 2 Configure Scheduling
      
   4. Click OK.

      SQL scheduled jobs allows you to save the queried data to another log stream or upload the data to an AOM Prometheus instance for further analysis.

6. Create an alarm rule.
   1. Click Log Settings in the upper right corner of the log stream page.

      

   2. On the Alarm Rules tab, click Create.

      

   3. Configure parameters as instructed and click OK.

      For details, see Configuring Log Alarm Rules.

      

7. Configure access logging for DNS.
   1. Go to the Resolvers page.
   2. Click the Access Logs tab.

   3. Click Configure Access Logging.

   4. Configure the parameters, such as Log Group, Log Stream, and VPC, as instructed.

   5. Click OK.

8. View access logs.
   1. In the access log list, locate the target access log and click View Log Details.

      On the displayed page, view the information about the log group and log stream.

   2. Click the name of the log stream and view its details.

      The following is an example log. For details about the fields in the log, see Table 2. The log format cannot be modified.

      {
          "content": "2024-07-02 09:28:00.304 baidu.com. A NOERROR TCP cnsouthwest2d _ 192.168.0.138 c1e159ce-ac25-4908-8e31-8ff73ad2f57d",
          "_resource_id": "c1e159ce-ac25-4908-8e31-8ff73ad2f57d",
          "_resource_name": "c1e159ce-ac25-4908-8e31-8ff73ad2f57d",
          "_service_type": "DNS",
          "category": "LTS",
          "collectTime": 1719883683977
      }

      Table 2 Fields in a DNS Resolver access log

      Parameter	Description	Value Range	Example Value
      content	DNS Resolver access logs	String	2024-07-02 09:28:00.304 baidu.com. A NOERROR TCP cnsouthwest2d _ 192.168.0.138 c1e159ce-ac25-4908-8e31-8ff73ad2f57d
      _resource_id	Resource ID	uuid	95c2b814-99dc-939a-e811-ae84c61ea9ee
      _resource_name	Resource name	Name of the resource specified by the resource ID	95c2b814-99dc-939a-e811-ae84c61ea9ee
      _service_type	Service for which access logs are collected	Fixed value: DNS	DNS
      category	Log category	Fixed value: LTS	LTS
      collectTime	LTS log collection time	Integer	1704158708902

Common DNS SQL Statements

This section provides some common SQL statements of DNS. If you have other statistical requirements, submit a service ticket to contact us.

• Analysis on the number of requests
  • Obtain the number of requests by time:

    select time_series(__time, 'PT1M', 'yyyy-MM-dd HH:mm:ss', '0', '+08:00') as t_time, count(*) as num group by t_time order by t_time

    

  • Obtain the number of requests by domain name:

    select time_series(__time, 'PT1M', 'yyyy-MM-dd HH:mm:ss', '0', '+08:00') as t_time, count(*) as num where QueryName='Domain name.' group by t_time order by t_time

    

• Request source analysis
  Rank IP addresses by request frequency:

  SELECT Client, COUNT(*) AS request_count GROUP BY Client ORDER BY request_count LIMIT 10 

  

• Response feature analysis
  • Query percentage distribution of DNS response statuses:

    SELECT Rcode, count(*) as rcode_count group by Rcode order by rcode_count

    

  • Rank NXDOMAIN requests by domain name.

    SELECT QueryName, count(*) as rcode_count where Rcode='NXDOMAIN' group by QueryName order by rcode_count

    

• Analysis of response details
  Filter DNS log data where a specific field matches an expected value.

  SELECT * where Field to be queried='Expected value'