Searching for Logs

Precaution

• To use log streams, enable this function in Menu Settings. For details, see Menu Settings.
• The Log Stream option is not available under Log Analysis (New).

Setting a Filter

1. Log in to the AOM 2.0 console.
2. In the navigation pane, choose Log Analysis > Log Stream.
3. In the filter area of the Log Stream page, filter logs by setting different perspectives (such as cloud log) and parameters. Set log search criteria as prompted.
4. Click Search.

Searching for Raw Logs

1. Log in to the AOM 2.0 console.
2. In the navigation pane, choose Log Analysis > Log Stream.
3. Set filters by referring to Setting a Filter.
4. In the upper right corner of the Raw Logs tab page, select a time range.
5. Search for raw logs in the following ways:
   • In the search area, enter a keyword or select a keyword from the drop-down list, and click Search.
     

     • After you set log structuring, the drop-down list displays both the built-in fields and fields configured for structuring.
     • Built-in fields include appName, category, clusterId, clusterName, collectTime, containerName, hostIP, hostIPv6, hostId, hostName, nameSpace, pathFile, podName and serviceID. By default, the fields are displayed in simplified mode, and hostIP, hostName, and pathFile are displayed at the beginning.

     • The structured fields are displayed in key:value format.
   • Click a field in blue in the log content and the field will be used as a filter. All logs that meet the filtering criteria are displayed.
   • On the Raw Logs page, click a field in blue in the log content and the field will be used as a filter. All logs that meet the filtering criteria are displayed.
   • Click a field for which quick analysis has been created to add it to the search box.
     

     If the field you click already exists in the search box, it will be replaced by this newly added one. If the field is added the first time, fields in the search box are searched using the AND operator.

   • In the search area, press the up and down arrows on the keyboard to select a keyword or search syntax from the drop-down list, press Tab or Enter to select a keyword or syntax, and click Search.

Visualized Log Analysis

You can query and analyze structured log fields using SQL statements. After log structuring, wait about 1–2 minutes for SQL query and analysis.

Before visualized analysis, structure raw logs first.

1. Log in to the AOM 2.0 console.
2. In the navigation pane, choose Log Analysis > Log Stream.
3. Set filters by referring to Setting a Filter.
4. Click the Visualization tab, select a time range, enter an SQL statement, and click Search.
   

   • SQL query constraints:
     • A maximum of 100,000 records can be returned for each query.
     • When the number of aggregation results exceeds 100,000, the aggregation results may be inaccurate.
   • There are some restrictions when you use a string in a WHERE clause.
     • The value should be enclosed by single quotation marks (') for exact match, and by single or double quotation marks (") for fuzzy search. If the key has the same name with one of the SQL reserved fields, enclose the key with double quotation marks (").
     • Recommended formats: WHERE "Key"='Value' and WHERE "Key" like ' %Value%'
   • There are no restrictions on float and long types in WHERE clauses. You are advised to use the formats described above to avoid query exceptions caused by keyword conflicts.

   If the number of logs generated within the specified time range exceeds 1 billion, iterative query is triggered so you can view all logs in multiple queries. The message Query status: Results are accurate is displayed.

5. Select a graph to display the query result. For details about icon types and configurations, see Log Graphs (Table/Bar/Line/Pie/Number/Digital Line/Map Graphs).
6. Perform the following operations on the query result:
   • Click Create. In the displayed dialog box, set Chart Name and SQL Statement, select a chart type, and click OK.
   • Click Save. In the displayed dialog box, set Chart Name, and click OK to save the visual chart. You can also select a chart, click Save, and modify it as required.
   • Click Save As. In the displayed dialog box, set Chart Name, and click OK to copy the existing visual chart.
     

     You must save a chart before saving it as a visual chart.

   • Click Download to download the visual data of the current SQL query result. The file is in .csv format.
   • Click Show Chart to expand the charts of the current log stream.
   • Click Hide Chart to collapse the expanded charts of the current log stream.

Analyzing Real-Time Logs

1. Log in to the AOM 2.0 console.
2. In the navigation pane, choose Log Analysis > Log Stream.
3. Set filters by referring to Setting a Filter.
4. Click the Real-Time Logs tab to view the corresponding real-time logs.

   Logs are refreshed every 5s. You may wait for up to 1 minute before the logs are displayed.

   You can also customize log display by clicking Clear or Pause in the upper right corner.

   • Clear: Displayed logs will be cleared from the real-time view.
   • Pause: Loading of new logs to the real-time view will be paused.

     After you click Pause, the button changes to Continue. You can click Continue to resume the log loading to the real-time view.

   

   Stay on the Real-Time Logs tab to keep updating them in real time. If you leave the Real-Time Logs tab, logs will not be loaded in real time. The next time you access the tab, the logs that were shown before you left the tab will not be displayed.

Common Log Search Operations

These operations include adding alarms, selecting a time range to display logs, and refreshing logs. For details, see Table 1.

Table 1 Common operations

Operation	Description
Configuring quick search	Click and configure quick search.
Refreshing logs	Click to refresh logs. There are two refresh modes: manual and automatic. Manual refresh: Click Refresh Now to refresh logs. Automatic refresh: Select an interval from the drop-down list to automatically refresh logs. The interval can be 15 seconds, 30 seconds, 1 minute, or 5 minutes.
Copying logs	Click to copy log content.
Viewing the context	Click to view the log context.
Simplifying field details	Click to view the simplified field details.
Unfolding	Click to unfold log content. They will be displayed in different lines. NOTE: By default, log content is unfolded and two lines are displayed.
Downloading logs	Click . On the page that is displayed, download logs to the local host. Direct Download: Download log files to the local PC. Up to 5000 logs can be downloaded at a time. Select .csv or .txt from the drop-down list and click Download to export logs to the local PC. NOTE: If you select .csv, logs are exported as a table. If you select .txt, logs are exported as a .txt file.
JSON	Move the cursor over , click JSON, and set JSON formatting. NOTE: Formatting is enabled by default. The default number of expanded levels is 2. Formatting enabled: Set the default number of expanded levels. Maximum value: 10. Formatting disabled: JSON logs will not be formatted for display.
Collapse configuration	Move the cursor over , click Log Collapse, and set the maximum characters to display in a log. If the number of characters in a log exceeds the maximum, the extra characters will be hidden. Click Expand to view all. NOTE: Logs are collapsed by default, with a default character limit of 400.
Log time display	Move the cursor over and click Log time display. On the page that is displayed, set whether to display milliseconds and whether to display the time zone. NOTE: By default, the function of displaying milliseconds is enabled.

Syntax and Examples of Searching by Keyword

Search syntax:

Operators (such as &&, ||, AND, OR, NOT, *, ?, :, >, <, =, >=, and <=) contained in raw logs cannot be used to search for logs.

Search rules:

• Fuzzy search is supported.

  For example, if you enter error*, all logs containing error will be displayed and those start with error will be highlighted.

• You can use a combination of multiple search criteria in the key and value format: key1:value1 AND key2:value2 or key1:value1 OR key2:value2. After entering or selecting key1:value1, you need to add AND or OR before entering or selecting key2:value2 in the search box.
• Click a keyword and select one of the three operations from the displayed drop-down list: Copy, Add To Search, and Exclude from Search.
  • Copy: Copy the field.
  • Add To Search: Add AND field: value to the search statement.
  • Exclude from Search: Add NOT field: value to the query statement.

Search examples:

• Search for logs containing start: Enter start.
• Search for logs containing start to refresh: Enter start to refresh.
• Search for the logs containing both keyword start and unexpected: Enter start && unexpected.
• Search for logs containing both start and unexpected: Enter start AND unexpected or start and unexpected.
• Search for the logs containing keyword start or unexpected: Enter start || unexpected.

• Search for logs containing start or unexpected: Enter start OR unexpected or start or unexpected.
• Logs that do not contain query1: NOT content: query1 or not content: query1.
• error*: logs that contain error.
• er?or: logs that start with er, is followed by any single character, and end with or.
• If your keyword contains a colon (:), use the content: Keyword format. Example: content: "120.46.138.115:80" or content: 120.46.138.115:80.
• query1 AND query2 AND NOT content: query3: logs that contain both query1 and query2 but not query3.

• When you enter a keyword to query logs, the keyword is case-sensitive. Both the log contents you queried and the highlighted log contents are case-sensitive.
• The asterisk (*) and question mark (?) do not match special characters such as hyphens (-) and spaces.
• For fuzzy match, a keyword cannot start with a question mark (?) or an asterisk (*). For example, you can enter ER?OR or ER*R.