Help Center/ MapReduce Service/ User Guide (ME-Abu Dhabi Region)/ Using an MRS Client/ Using the Client of Each Component/ Using a Kafka Client
Updated on 2022-02-22 GMT+08:00
View PDF
Share

Using a Kafka Client

Scenario

You can create, query, and delete topics on a cluster client.

Prerequisites

The client has been installed. For example, the client is installed in the /opt/hadoopclient directory. The client directory in the following operations is only an example. Change it to the actual installation directory.

Using the Kafka Client (Versions Earlier Than MRS 3.x)

  1. View the IP addresses of the ZooKeeper role instance.

    Record any IP address of the ZooKeeper instance.

  2. Log in to the node where the client is installed.
  3. Run the following command to switch to the client directory, for example, /opt/hadoopclient/Kafka/kafka/bin.

    cd /opt/hadoopclient/Kafka/kafka/bin

  4. Run the following command to configure environment variables:

    source /opt/hadoopclient/bigdata_env

  5. If Kerberos authentication is enabled for the current cluster, run the following command to authenticate the user. If Kerberos authentication is disabled, skip this step.

    kinit Kafka user

  6. Create a topic.

    sh kafka-topics.sh --create --topic Topic name --partitions Number of partitions occupied by the topic --replication-factor Number of replicas of the topic --zookeeper IP address of the node where the ZooKeeper instance resides:clientPort/kafka

  7. Run the following command to view the topic information in the cluster:

    sh kafka-topics.sh --list --zookeeper IP address of the node where the ZooKeeper instance resides:clientPort/kafka

  8. Delete the topic created in 6.

    sh kafka-topics.sh --delete --topic Topic name --zookeeper IP address of the node where the ZooKeeper instance resides:clientPort/kafka

    Type y and press Enter.

Using the Kafka Client (MRS 3.x or Later)

  1. View the IP addresses of the ZooKeeper role instance.

    Record any IP address of the ZooKeeper instance.

  2. Log in to the node where the client is installed.
  3. Run the following command to switch to the client directory, for example, /opt/hadoopclient/Kafka/kafka/bin.

    cd /opt/hadoopclient/Kafka/kafka/bin

  4. Run the following command to configure environment variables:

    source /opt/hadoopclient/bigdata_env

  5. If Kerberos authentication is enabled for the current cluster, run the following command to authenticate the user. If Kerberos authentication is disabled, skip this step.

    kinit Kafka user

  6. Create a topic.

    sh kafka-topics.sh --create --topic Topic name --partitions Number of partitions occupied by the topic --replication-factor Number of replicas of the topic --zookeeper IP address of the node where the ZooKeeper instance resides:clientPort/kafka

  7. Run the following command to view the topic information in the cluster:

    sh kafka-topics.sh --list --zookeeper IP address of the node where the ZooKeeper instance resides:clientPort/kafka

  8. Delete the topic created in 6.

    sh kafka-topics.sh --delete --topic Topic name --zookeeper IP address of the node where the ZooKeeper instance resides:clientPort/kafka

    Type y and press Enter.