Viewing Kafka Topic Logs
Topic logs record the details about leader election in topic partitions, covering the leader election time, topic partitions, and number of leader elections. This section describes how to view topic logs on the console.
Topic logs are stored and can be queried in Log Tank Service (LTS).
Notes and Constraints
- Unavailable for old instances. See the console.
- By default, topic logs are retained for 7 days. To store the logs for longer, modify the log group retention period on the LTS console.
- Enabling logging will create a log group, log stream, and dashboard in LTS. Fees are generated based on the log volume. For details, see LTS pricing details.
- Frequently generating topic logs may affect instance performance.
- When Kafka instances store topic logs in one log group and log stream, the log group and stream of each instance contain the topic logs of all the instances.
Prerequisites
- Ensure that you have permissions to create log groups and log streams in LTS.
- Topic logging can be enabled or disabled only when the Kafka instance is in the Running state.
Enabling Topic Logging
- Log in to the Kafka console.
- Click
in the upper left corner to select the region where your instance is located.
- Click the desired instance to go to the instance details page.
- In the navigation pane, choose Analysis & Diagnosis > Topic Logs
- Click Enable Logging. The Enable Logging dialog box is displayed.
- Click OK. The Configure Logs dialog box is displayed.
- Determine whether to enable log configuration as required and click OK. The Background Tasks page is displayed. If the logging enabling task is in the Successful state, topic logging is enabled successfully.
- Disable: LTS automatically creates a log group and log stream.
- Enable: Select the log group and log stream for storing the log file topic.log. You can click View Log Group on the right to go to the LTS console to view or create log groups and log streams.
Viewing Topic Logs
- Log in to the Kafka console.
- Click
in the upper left corner to select the region where your instance is located.
- Click the desired instance to go to the instance details page.
- In the navigation pane, choose Analysis & Diagnosis > Topic Logs
- On the Logs tab page, view topic logs.
For details about how to search for logs, see Accessing the Log Search Page.
The following is an example of topic logs:
{ "level": "INFO", "timestamp": "2024-12-27 17:26:13,361", "message": { "topicPartition": "topic-0", "targetState": "OnlinePartition", "leaderAndIsr": "LeaderAndIsr(leader=1, leaderEpoch=3, isr=List(1, 0), leaderRecoveryState=RECOVERED, partitionEpoch=3)", "partitionState": "OnlinePartition", "topic": "topic", "type": "ELECT_LEADER" } } { "level": "INFO", "timestamp": "2024-12-27 17:26:13,491", "message": { "leader": "1", "startOffset": "0", "topic": "topic", "type": "MAKE_LEADER", "topicPartition": "topic-0", "epoch": "3" } }
Table 1 describes the parameters.
Table 1 Topic log parameters Parameter
Description
level
Level of the topic logs. The only value is INFO.
timestamp
Time when a leader is elected or determined in a topic partition.
topicPartition
Topic partition.
targetState
Target state. The options are as follows:
- NewPartition: The partition is creating.
- OnlinePartition: The partition is working properly.
- OfflinePartition: The partition is discontinued.
- NonExistentPartition: The partition does not exist or has been deleted.
leaderAndIsr
Information of the leaderAndIsr request.
partitionState
Partition state. The options are as follows:
- NewPartition: The partition is creating.
- OnlinePartition: The partition is working properly.
- OfflinePartition: The partition is discontinued.
- NonExistentPartition: The partition does not exist or has been deleted.
topic
Topic name.
type
Phase of a leader. The options are as follows:
- ELECT_LEADER: Electing
- MAKE_LEADER: Elected
leader
Partition where a leader is located.
startOffset
Offset of the first message written by a leader in an epoch.
Each epoch corresponds to a startOffset.
epoch
Number of leader elections. The initial value is 0. The epoch value increases by 1 each time a leader is elected.
Disabling Topic Logging
- Log in to the Kafka console.
- Click
in the upper left corner to select the region where your instance is located.
- Click the desired instance to go to the instance details page.
- In the navigation pane, choose Analysis & Diagnosis > Topic Logs
- Click Disable Logging in the upper right corner. The Disable Logging dialog box is displayed.
- Click OK. The Background Tasks page is displayed. The topic log function is disabled when the Disable logging task is in the Successful state.
This only disables the topic logging function. The logs groups and log streams on LTS are retained and still generate fees. If you no longer need the logs, delete the log groups and log streams.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot