Help Center/ Cloud Search Service/ User Guide/ OpenSearch/ Managing Logs/ Querying Logs
Updated on 2026-04-30 GMT+08:00
View PDF
Share

Querying Logs

During routine O&M, when an OpenSearch cluster becomes sluggish, returns an error for a query request, or shows a yellow status, the O&M team must be able to obtain relevant information immediately. CSS provides a log query feature that aggregates underlying server logs to the console, where you can query them by node and log level in seconds. By analyzing run logs, slow query logs, and deprecation logs, you can quickly locate the root causes of faults and issues. Additionally, you can dynamically adjust log levels and enable trace logging for debugging, allowing you to obtain the most detailed log records without restarting your cluster.

Querying Recent Logs

Query recently generated logs that are not yet archived.

  1. Log in to the CSS management console.
  2. In the navigation pane on the left, choose Clusters > OpenSearch.
  3. In the cluster list, click the name of the target cluster. The cluster information page is displayed.
  4. Choose Logs > Log Search. The Log Search page is displayed.

    You can search records by log type, node, log level, or keyword. For a detailed description of each type of logs, see Log Types.

    When a log file reaches 128 MB or when the time reaches 00:00 UTC, the system automatically compresses and archives it. Only unarchived logs appear on the log search page, while archived logs remain accessible through the log backup function. For more information, see Backing Up Logs to OBS.

Adjusting Log Levels Dynamically

Log4j2 is used as the log component in OpenSearch clusters. Multiple log levels (ERROR, WARN, INFO, DEBUG, and TRACE) are supported. The default log level is INFO. To facilitate troubleshooting and debugging, you can dynamically adjust the log levels.

  • INFO is the default log level. The levels, in order of increasing detail, are: ERROR, WARN, INFO, DEBUG, and TRACE. When INFO is set, you will see logs for itself and all higher-severity levels (ERROR and WARN), while more verbose levels (DEBUG and TRACE) are excluded.
  • You can change the log level of a specified module in real time via an OpenSearch API.
  1. Log in to OpenSearch Dashboards and go to the command execution page. OpenSearch clusters support multiple access methods. This topic uses OpenSearch Dashboards as an example to describe the operation procedures.
    1. Log in to the CSS management console.
    2. In the navigation pane on the left, choose Clusters > OpenSearch.
    3. In the cluster list, find the target cluster, and click Dashboards in the Operation column to log in to OpenSearch Dashboards.
    4. In the left navigation pane, choose Dev Tools.

      The left part of the console is the command input box, and the triangle icon in its upper-right corner is the execution button. The right part shows the execution result.

  2. Run the following command to change the log level, for example, to DEBUG: 
    PUT _cluster/settings
{
  "persistent": {
    "logger": {
      "org.opensearch.action": "DEBUG"
    }
  }
}
  3. Refresh the log query page and check whether DEBUG logs have been generated. You should always restore the log level to the default setting once you finish the task at hand.
  4. Run the following command to restore the default log level INFO: 
    PUT _cluster/settings
{
  "persistent": {
    "logger": {
      "org.opensearch.action": null
    }
  }
}

Enabling Trace Logging

To analyze communication details on the HTTP or transport layer, you can enable trace logging for the HTTP or Transport module to obtain detailed log records.

  1. Log in to OpenSearch Dashboards and go to the command execution page. OpenSearch clusters support multiple access methods. This topic uses OpenSearch Dashboards as an example to describe the operation procedures.
    1. Log in to the CSS management console.
    2. In the navigation pane on the left, choose Clusters > OpenSearch.
    3. In the cluster list, find the target cluster, and click Dashboards in the Operation column to log in to OpenSearch Dashboards.
    4. In the left navigation pane, choose Dev Tools.

      The left part of the console is the command input box, and the triangle icon in its upper-right corner is the execution button. The right part shows the execution result.

  2. Run the following command to enable trace logging: 
    PUT _cluster/settings
{
  "transient": {
    "logger.org.opensearch.transport.TransportService.tracer": "trace",
    "transport.tracer.include": "",
    "http.tracer.include": "",
    "logger.org.opensearch.http.HttpTracer": "trace"
  }
}

    Enabling trace logging is a non-persistent configuration and will be disabled upon a cluster restart. It is typically used for emergency packet capture and analysis during troubleshooting.

  3. Go to the log details page to view trace logs.
    1. In the cluster list, click the name of the target cluster. The cluster information page is displayed.
    2. Choose Logs > Log Search. The Log Search page is displayed.
    3. Select all log levels (mandatory) and view trace logs.
      Figure 1 Viewing trace logs

Log Types

Table 1 Introducing different log types

Log Type

Description

Purpose

Run logs

Run logs, or main logs, record the cluster status and key information about write and query operations. For example, write logs record operations such as index creation, index mapping update, and write queue exhaustion; and query logs record query queue status and query exceptions.

Check the status and write and query operations of each cluster node, including inter-node connectivity, full GC, index creation or deletion, and cluster-level query errors.

Slow indexing logs

Slow indexing logs record indexing operations (such as bulk, index, update, and delete) that took a long time to complete, helping you identify performance bottlenecks.

In the case of slow write performance, you can query slow indexing logs to locate the cause.

Slow query logs

Slow query logs record search requests that took a long time to complete. They help you monitor and analyze time-consuming search requests, so you can identify performance bottlenecks, optimize SQL queries, and improve overall system performance.

In the case of slow query performance, you can query slow query logs to locate the cause.

Deprecation logs

Deprecation logs record deprecation warnings. Deprecation warnings are written to this log when you use APIs, configurations, or functions that are marked for removal in future versions.

Check for APIs or features that are about to expire in future versions.

Access logs

Access logs record cluster access requests, such as the request path and source address.

You cannot check access logs on the console. To check them, you need to back them up to an OBS bucket or transfer them to a target cluster first.

If there is a surge in service requests, you can analyze the request sources and paths by checking the access logs.
  • Run log description

    Run logs record the cluster status and key information about write and query operations. For example, the log record below indicates that an index named test was created and afterwards the cluster status changed from YELLOW to GREEN.

    Figure 2 Run log sample
    Log content:
    • 1. Log generation time
    • 2. Log level, which can be DEBUG, INFO, WARN, or ERROR
    • 3. Log-generating module
    • 4. Name of the log-generating node
    • 5. Log content
  • Slow indexing log description

    Slow indexing logs record indexing operations that took a long time to complete. For example, the log record below shows an indexing request that lasted longer than the preset threshold. The log contains the index name, duration, and request content.

    Figure 3 Slow indexing log sample
    Log content:

    • 1. Log generation time

    • 2. Log level, which can be DEBUG, INFO, WARN, or ERROR
    • 3. Log-generating module
    • 4. Name of the log-generating node
    • 5. Index name and ID
    • 6. Log content. In this example, the log recorded the request execution duration, index type, and index request body.
  • Slow query log description

    Slow query logs record search requests that took a long time to complete. For example, the log record below shows a search request that lasted longer than the preset threshold. The log contains the index name, duration, and request content.

    Figure 4 Slow query log sample
    Log content:
    • 1. Log generation time
    • 2. Log level, which can be DEBUG, INFO, WARN, or ERROR
    • 3. Log-generating module
    • 4. Name of the log-generating node
    • 5. Index name and shard ID
    • 6. Log content. In this example, the log recorded the query duration, number of hits, and query request body.
  • Deprecation log description

    Deprecation logs record deprecation warnings. For example, the log record below indicates that GET /_cat/master has been deprecated and should be replaced with GET /_cat/cluster_manager.

    Figure 5 Deprecation log sample
    Log content:
    • 1. Log generation time
    • 2. Log level, which can only be DEPRECATION.
    • 3. Log-generating module
    • 4. Name of the log-generating node
    • 5. Log content
  • Access log description

    Access logs record cluster access requests and source addresses. For example, the log record below has recorded source information for the /_snapshot/my_backup/my_snapshot/_restore?pretty=true operation.

    Figure 6 Access log sample
    Log content:
    • 1. Log generation time
    • 2. Name of the log-generating node
    • 3. Name of the log-generating thread
    • 4. Log level, which can be DEBUG, INFO, WARN, or ERROR
    • 5. Log request method
    • 6. Request path
    • 7. Source and destination addresses of the request
Parent topic: Managing Logs