Help Center/ Log Tank Service/ More Documents/ User Guide (ME-Abu Dhabi Region)/ Log Search and View/ Cloud Structuring Parsing/ Log Structuring
Updated on 2023-11-29 GMT+08:00
View PDF
Share

Log Structuring

Log data can be structured or unstructured. Structured data is quantitative data or can be defined by unified data models. It has a fixed length and format. Unstructured data has no pre-defined data models and cannot be fit into two-dimensional tables of databases.

During log structuring, logs with fixed or similar formats are extracted from a log stream based on your defined structuring method and irrelevant logs are filtered out.

Precautions

  • You have created a log stream.
  • Log structuring is recommended when most logs in a log stream share a similar pattern.

Creating a Structuring Rule

Add structuring rules to a log stream and LTS will extract logs based on the rules.

To structure logs:

  1. Log in to the LTS console and choose Log Management in the navigation pane on the left.
  2. Select a log group and a log stream.
  3. On the log stream details page, click in the upper right corner. On the page displayed, select Log StructuringCloud Structuring Parsing to structure logs.

    Figure 1 Log structuring

    You can then use SQL statements to query structured logs in the same way as you query data in two-dimensional database tables.

    • If a structured field exceeds 20 KB, only the first 20 KB is retained.
    • The following system fields cannot be extracted during log structuring: groupName, logStream, lineNum, content, logContent, logContentSize, collectTime, category, clusterId, clusterName, containerName, hostIP, hostId, hostName, nameSpace, pathFile, and podName.

  4. Click Save.

Modifying a Structuring Rule

To modify a structuring rule, perform the following steps:

  1. On the Log Structuring page, click to modify a structuring rule.

    You can modify the structuring rules, including the structuring mode, log extraction field, and tag field.

  2. Click Save.

Deleting a Structuring Rule

If a log structuring rule is no longer used, perform the following steps to delete it:

  1. On the Log Structuring page, click to delete a structuring rule.
  2. In the displayed dialog box, click OK.

    Deleted structuring rules cannot be restored. Exercise caution when performing this operation.