Kafka Shell Commands

Prerequisites

The Linux client of Kafka has been installed. For details, see Installing a Client.

Shell Command Guide

Methods of running shell commands:

1. Go to any directory of the Kafka client.
2. Run the following command to initialize environment variables:

   source /opt/client/bigdata_env

3. If Kerberos authentication has been enabled for the current cluster, run the following command to authenticate the current user (the user must be added to the kafkaadmin user group and have the Kafka administrator permission): If the Kerberos authentication is disabled for the current cluster, skip this step.

   kinit MRS cluster user

   For example, kinit admin

4. Go to Client installation directory/Kafka/kafka/bin and run the Kafka shell command.

The common commands are as follows:

• Query the list of topics in the current cluster.

  sh kafka-topics.sh --list --zookeeper <ZooKeeper cluster IP address:2181/kafka>

• Query the details of a topic.

  sh kafka-topics.sh --describe --zookeeper <ZooKeeper cluster IP address:2181/kafka> --topic <Topic name>

• Delete a topic (performed by an administrator).

  sh kafka-topics.sh --delete --zookeeper <ZooKeeper cluster IP address:2181/kafka> --topic <Topic name>

• Create a topic (performed by an administrator).

  sh kafka-topics.sh --create --zookeeper <ZooKeeper cluster IP address:2181/kafka> --partitions 6 --replication-factor 2 --topic <Topic name>

• Produce data using the old Producer APIs.

  sh kafka-console-producer.sh --broker-list <Kafka cluster IP address:9092> --topic <Topic name> --old-producer -sync

• Consume data using the old Consumer APIs.

  sh kafka-console-consumer.sh --zookeeper <ZooKeeper cluster IP address:2181/kafka> --topic <Topic name> --from-beginning

• Enable the Producer APIs to produce messages (this operation requires the producer permission of the desired topic).

  sh kafka-console-producer.sh --broker-list <Kafka cluster IP address:21007> --topic <Topic name> --producer.config config/producer.properties

• Enable the Consumer APIs to consume data (this operation requires the consumer permission of the desired topic).

  sh kafka-console-consumer.sh --topic <Topic name> --bootstrap-server <Kafka cluster IP address:21007> --new-consumer --consumer.config config/consumer.properties

• The default port of the Kafka cluster with Kerberos authentication disabled is 9092, and the default port of the Kafka cluster with Kerberos authentication enabled is 21007.
• Log in to MRS Manager, choose Services > ZooKeeper > Instance, and get the OM IP Address of the quorumpeer instance.
• Log in to MRS Manager, choose Services > Kafka > Instance, and obtain the OM IP Address of the Kafka broker instance.