Help Center/ Cloud Search Service/ User Guide/ Logstash/ Managing Logs/ Querying Logs
Updated on 2026-04-30 GMT+08:00
View PDF
Share

Querying Logs

When using Logstash for large-scale data migration or real-time log ingestion, errors such as incorrect pipeline configurations, broken connections between the source and destination, or erroneous data processing logic may interrupt data flows. CSS provides a log query feature that aggregates underlying server logs to the console, where you can query them by node and keyword in seconds. By analyzing recent logs, you can gain real-time insight into each phase of the pipelines and accurately locate the root causes of exceptions such as connection timeouts or configuration syntax errors.

Querying Recent Logs

Query recently generated logs that are not yet archived.

  1. Log in to the CSS management console.
  2. In the navigation pane on the left, choose Clusters > Logstash.
  3. In the cluster list, click the name of the target cluster. The cluster information page is displayed.
  4. Choose Logs > Log Search. The Log Search page is displayed.

    You can search log records by node or keyword. For a detailed description of each type of logs, see Log Types.

    When a log file reaches 128 MB or when the time reaches 00:00 UTC, the system automatically compresses and archives it. Only unarchived logs appear on the log search page, while archived logs remain accessible through the log backup function. For more information, see Backing Up Logs to OBS.

Log Types

Table 1 Introducing different log types

Log Type

Description

Purpose

Run logs

Run logs record a cluster's node and pipeline status, such as source-destination connectivity, pipeline creation or modification, and pipeline running errors.

Check run logs to troubleshoot pipeline errors.

Deprecation logs

Deprecation logs record deprecation warnings. Deprecation warnings are written to this log when you use APIs, configurations, or functions that are marked for removal in future versions.

You cannot check deprecation logs on the console. To check them, you need to back them up to an OBS bucket first.

Check for APIs or features that are about to expire in future versions.
  • Run log description

    Run logs record a cluster's node and pipeline status. For example, the log record below indicates that the destination cluster could not be reached. You need to check whether the cluster address is correct and whether the cluster status is normal.

    Figure 1 A sample of run logs
    Log content:
    • 1. Log generation time
    • 2. Log level, which can be DEBUG, INFO, WARN, or ERROR
    • 3. Log-generating module
    • 4. Name of the log-generating node
    • 5. Log content
  • Deprecation log description

    Deprecation logs record deprecation warnings.

    Figure 2 A sample of deprecation logs
    Log content:
    • 1. Log generation time
    • 2. Log level, which can only be DEPRECATION.
    • 3. Log-generating module
    • 4. Log content. The log record shown in the figure above indicates that the ECS compatibility mode was not explicitly declared for a plugin when Logstash was started.
Parent topic: Managing Logs