Help Center/ Log Tank Service/ User Guide/ Log Search and Analysis/ Cloud Structuring Parsing/ Log Structuring
Updated on 2024-02-28 GMT+08:00
View PDF

Log Structuring

Log data can be structured or unstructured. Structured data is quantitative data or can be defined by unified data models. It has a fixed length and format. Unstructured data has no pre-defined data models and cannot be fit into two-dimensional tables of databases.

During log structuring, logs with fixed or similar formats are extracted from a log stream based on your defined structuring method and irrelevant logs are filtered out.

Precautions

  • You have created a log stream.
  • Log structuring is recommended when most logs in a log stream share a similar pattern.
  • After the structuring configuration is modified, the modification takes effect only for newly written log data.

Creating a Structuring Rule

Add structuring rules to a log stream and LTS will extract logs based on the rules.

To structure logs:

  1. Log in to the LTS console and choose Log Management in the navigation pane on the left.
  2. Select a log group and a log stream.
  3. On the log stream details page, click in the upper right corner. On the page displayed, select Log Structuring to structure logs.

    • If a structured field exceeds 20 KB, only the first 20 KB is retained.
    • The following system fields cannot be extracted during log structuring: groupName, logStream, lineNum, content, logContent, logContentSize, collectTime, category, clusterId, clusterName, containerName, hostIP, hostId, hostName, nameSpace, pathFile, and podName.

  4. Click Save.

Modifying a Structuring Rule

To modify a structuring rule, perform the following steps:

  1. On the Log Structuring page, click to modify a structuring rule.

    • You can modify the structuring rules, including the structuring mode, log extraction field, and tag field.
    • System templates cannot be modified.

  2. Click Save.

Deleting a Structuring Rule

If a log structuring rule is no longer used, perform the following steps to delete it:

  1. On the Log Structuring page, click to delete a structuring rule.
  2. In the displayed dialog box, click OK.

    Deleted structuring rules cannot be restored. Exercise caution when performing this operation.