Help Center/ Cloud Search Service/ User Guide/ Elasticsearch/ Managing Logs/ Ingesting Logs into a Cluster
Updated on 2026-04-30 GMT+08:00
View PDF
Share

Ingesting Logs into a Cluster

For large clusters that generate high volumes of logs, traditional search methods may struggle to extract comprehensive insights from the log data. To enable more advanced log analysis, you can leverage CSS's log ingestion feature to ingest logs generated by your cluster in real-time into a designated index in a specified cluster. Tools like Kibana can then be used to visualize and analyze the logs. By converting unstructured log data into searchable indexes, you can perform centralized log analysis across multiple clusters, greatly enhancing the efficiency of troubleshooting complex issues.

Constraints

Only Elasticsearch clusters whose image version is 7.10.2_24.2.0_xxx or later supports log ingestion.

Enabling Real-Time Log Ingestion

Log ingestion converts unstructured log data into structured documents stored in searchable Elasticsearch indexes.

  1. Log in to the CSS management console.
  2. In the navigation pane on the left, choose Clusters > Elasticsearch.
  3. In the cluster list, click the name of the target cluster. The cluster information page is displayed.
  4. Choose Logs > Log Ingestion. The Log Ingestion page is displayed.

    If the Log Ingestion tab is not displayed, the cluster does not support log ingestion.

  5. Enable log ingestion. If it is already enabled, skip this step.
    1. Click Enable Log Ingestion. In the displayed dialog box, configure necessary settings.
      Table 1 Log ingestion settings

      Parameter

      Description

      Index Prefix Name

      If you set a prefix for the log file indexes, the log index names will use the format index prefix name + log ingestion date. The unit of log ingestion is days.

      An index prefix name is a string of 1 to 128 characters. It can only contain digits, lowercase letters, underscores (_), and hyphens (-).

      Retention Period

      Set the log retention period. Ingested logs will be deleted upon expiration of this period.

      Value range: 1 to 3650

      Unit: days

      Log Storage Cluster

      Select a cluster to store ingested logs. Options include Current cluster and Other clusters.

      • Current cluster: Store ingested logs in the current cluster.
      • Other clusters: Store ingested logs in another cluster.

        Select a cluster from the Cluster drop-down list to use as the log storage destination. Make sure this cluster is in the same VPC and have the same version as the current cluster.

        When you select another cluster, click Check to verify network connectivity between the current cluster and the selected other cluster. Log ingestion can start only when "The current cluster is available" is returned.
    2. Click OK to enable log ingestion.

      The configuration information will be displayed on the Log Ingestion tab.

  6. On the Log Ingestion tab, if the task status changes to Running, log ingestion has started.

Viewing Ingested Logs

On the Log Ingestion tab, click Access Kibana. The Kibana login page for the log storage cluster is displayed. Kibana's powerful visualization features simplify log search and analytics.

Figure 1 Viewing ingested logs

To view details about the log storage cluster, click the cluster name in cluster list to go to the cluster overview page.

Disabling Log Ingestion

If you no longer need the log ingestion feature, you can disable it to reclaim storage resources.

On the Log Ingestion page, click Disable Log Ingestion. In the displayed dialog box, click OK.

Disabling log ingestion does not automatically remove ingested log data. Rather, it is deleted by the system upon expiration of its retention period. You can also manually delete it from the log storage cluster before the retention period expires.

Parent topic: Managing Logs