Help Center/ Log Tank Service/ More Documents/ User Guide(Paris Regions)/ Log Search and View/ Cloud Structuring Parsing/ Log Structuring
Updated on 2023-11-14 GMT+08:00
View PDF
Share

Log Structuring

Log data can be structured or unstructured. Structured data is quantitative data or can be defined by unified data models. It has a fixed length and format. Unstructured data has no pre-defined data models and cannot be fit into two-dimensional tables of databases.

During log structuring, logs with fixed or similar formats are extracted from a log stream based on your defined structuring method and irrelevant logs are filtered out.

Precautions

  • You have created a log stream.
  • Log structuring is recommended when most logs in a log stream share a similar pattern.

Creating a Structuring Rule

Add structuring rules to a log stream and LTS will extract logs based on the rules.

To structure logs:

  1. Log in to the LTS console and choose Log Management in the navigation pane on the left.
  2. Select a log group and a log stream.
  3. On the log stream details page, click in the upper right corner. On the page displayed, select Log Structuring to structure logs.

    You can then use SQL statements to query structured logs in the same way as you query data in two-dimensional database tables.

    • If a structured field exceeds 20 KB, only the first 20 KB is retained.
    • The following system fields cannot be extracted during log structuring: groupName, logStream, lineNum, content, logContent, logContentSize, collectTime, category, clusterId, clusterName, containerName, hostIP, hostId, hostName, nameSpace, pathFile, and podName.

  4. Click Save.

Modifying a Structuring Rule

To modify a structuring rule, perform the following steps:

  1. On the Log Structuring page, click to modify a structuring rule.

    You can modify the structuring rules, including the structuring mode, log extraction field, and tag field.

  2. Click Save.

Deleting a Structuring Rule

If a log structuring rule is no longer used, perform the following steps to delete it:

  1. On the Log Structuring page, click to delete a structuring rule.
  2. In the displayed dialog box, click OK.

    Deleted structuring rules cannot be restored. Exercise caution when performing this operation.