Updated on 2025-08-21 GMT+08:00

Viewing Kafka Topic Logs

Topic logs record the details about leader election in topic partitions, covering the leader election time, topic partitions, and number of leader elections. This section describes how to view topic logs on the console.

Topic logs are stored and can be queried in Log Tank Service (LTS).

Notes and Constraints

  • Unavailable for old instances. See the console.
  • By default, topic logs are retained for 7 days. To store the logs for longer, modify the log group retention period on the LTS console.
  • Enabling logging will create a log group, log stream, and dashboard in LTS. Fees are generated based on the log volume. For details, see LTS pricing details.
  • Frequently generating topic logs may affect instance performance.
  • When Kafka instances store topic logs in one log group and log stream, the log group and stream of each instance contain the topic logs of all the instances.

Prerequisites

  • Ensure that you have permissions to create log groups and log streams in LTS.
  • Topic logging can be enabled or disabled only when the Kafka instance is in the Running state.

Enabling Topic Logging

  1. Log in to the Kafka console.
  2. Click in the upper left corner to select the region where your instance is located.
  3. Click the desired instance to go to the instance details page.
  4. In the navigation pane, choose Analysis & Diagnosis > Topic Logs
  5. Click Enable Logging. The Enable Logging dialog box is displayed.
  6. Click OK. The Configure Logs dialog box is displayed.
  7. Determine whether to enable log configuration as required and click OK. The Background Tasks page is displayed. If the logging enabling task is in the Successful state, topic logging is enabled successfully.

    • Disable: LTS automatically creates a log group and log stream.
    • Enable: Select the log group and log stream for storing the log file topic.log. You can click View Log Group on the right to go to the LTS console to view or create log groups and log streams.

Viewing Topic Logs

  1. Log in to the Kafka console.
  2. Click in the upper left corner to select the region where your instance is located.
  3. Click the desired instance to go to the instance details page.
  4. In the navigation pane, choose Analysis & Diagnosis > Topic Logs
  5. On the Logs tab page, view topic logs.

    For details about how to search for logs, see Accessing the Log Search Page.

    The following is an example of topic logs:

    {
        "level": "INFO",
        "timestamp": "2024-12-27 17:26:13,361",
        "message": {
            "topicPartition": "topic-0",
            "targetState": "OnlinePartition",
            "leaderAndIsr": "LeaderAndIsr(leader=1, leaderEpoch=3, isr=List(1, 0), leaderRecoveryState=RECOVERED, partitionEpoch=3)",
            "partitionState": "OnlinePartition",
            "topic": "topic",
            "type": "ELECT_LEADER"
        }
    }
    
    {
        "level": "INFO",
        "timestamp": "2024-12-27 17:26:13,491",
        "message": {
            "leader": "1",
            "startOffset": "0",
            "topic": "topic",
            "type": "MAKE_LEADER",
            "topicPartition": "topic-0",
            "epoch": "3"
        }
    }

    Table 1 describes the parameters.

    Table 1 Topic log parameters

    Parameter

    Description

    level

    Level of the topic logs. The only value is INFO.

    timestamp

    Time when a leader is elected or determined in a topic partition.

    topicPartition

    Topic partition.

    targetState

    Target state. The options are as follows:

    • NewPartition: The partition is creating.
    • OnlinePartition: The partition is working properly.
    • OfflinePartition: The partition is discontinued.
    • NonExistentPartition: The partition does not exist or has been deleted.

    leaderAndIsr

    Information of the leaderAndIsr request.

    partitionState

    Partition state. The options are as follows:

    • NewPartition: The partition is creating.
    • OnlinePartition: The partition is working properly.
    • OfflinePartition: The partition is discontinued.
    • NonExistentPartition: The partition does not exist or has been deleted.

    topic

    Topic name.

    type

    Phase of a leader. The options are as follows:

    • ELECT_LEADER: Electing
    • MAKE_LEADER: Elected

    leader

    Partition where a leader is located.

    startOffset

    Offset of the first message written by a leader in an epoch.

    Each epoch corresponds to a startOffset.

    epoch

    Number of leader elections. The initial value is 0. The epoch value increases by 1 each time a leader is elected.

Disabling Topic Logging

  1. Log in to the Kafka console.
  2. Click in the upper left corner to select the region where your instance is located.
  3. Click the desired instance to go to the instance details page.
  4. In the navigation pane, choose Analysis & Diagnosis > Topic Logs
  5. Click Disable Logging in the upper right corner. The Disable Logging dialog box is displayed.
  6. Click OK. The Background Tasks page is displayed. The topic log function is disabled when the Disable logging task is in the Successful state.

    This only disables the topic logging function. The logs groups and log streams on LTS are retained and still generate fees. If you no longer need the logs, delete the log groups and log streams.