Obtaining Kafka Consumer Offset Information
Symptom
How do I obtain Kafka Consumer offset information when using Kafka Consumer to consume data?
Kafka APIs
- New Producer API
Indicates the API defined in org.apache.kafka.clients.producer.KafkaProducer. When kafka-console-producer.sh is used, the API is used by default.
- Old Producer API
Indicates the API defined in kafka.producer.Producer. When kafka-console-producer.sh is used, the API is invoked to add --old-producer.
- New Consumer API
Indicates the API defined in org.apache.kafka.clients.consumer.KafkaConsumer. When kafka-console-consumer.sh is used, the API is invoked to add --new-consumer.
- Old Consumer API
Indicates the API defined in kafka.consumer.ConsumerConnector. When kafka-console-consumer.sh is used, the API is used by default.
![](https://support.huaweicloud.com/intl/en-us/my-kualalumpur-1-usermanual-mrs/public_sys-resources/note_3.0-en-us.png)
New Producer API and new Consumer API are called new API in general in the document.
Procedure
Old Consumer API
- Prerequisites
- The system administrator has understood service requirements and prepared a Kafka administrator (belonging to the kafkaadmin group).
- The Kafka client has been installed.
- Procedure
- Log in to the node where the Kafka client is installed as the client installation user.
- Switch to the Kafka client installation directory, for example, /opt/kafkaclient.
cd /opt/kafkaclient
- Run the following command to configure environment variables:
- Run the following command to perform user authentication (skip this step for a cluster in common mode):
- Run the following command to switch to the Kafka client installation directory:
- Run the following command to obtain Consumer offset metric information:
bin/kafka-consumer-groups.sh --zookeeper <zookeeper_host:port>/kafka --list
bin/kafka-consumer-groups.sh --zookeeper <zookeeper_host:port>/kafka --describe --group test-consumer-group
Example:
kafka-consumer-groups.sh --zookeeper 192.168.100.100:2181/kafka --list kafka-consumer-groups.sh --zookeeper 192.168.100.100:2181/kafka --describe --group test-consumer-group
New Consumer API
- Prerequisites
- The system administrator has understood service requirements and prepared a Kafka administrator (belonging to the kafkaadmin group).
- The Kafka client has been installed.
- Procedure
- Log in to the node where the Kafka client is installed as the client installation user.
- Switch to the Kafka client installation directory, for example, /opt/client.
cd /opt/client
- Run the following command to configure environment variables:
- Run the following command to perform user authentication (skip this step for a cluster in common mode):
- Run the following command to switch to the Kafka client installation directory:
- Run the following command to obtain Consumer offset metric information:
kafka-consumer-groups.sh --bootstrap-server <broker_host:port> --describe --group my-group
Example:
kafka-consumer-groups.sh --bootstrap-server 192.168.100.100:9092 --describe --group my-group
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot