Viewing Kafka Consumer Information

Notes and Constraints

Due to cache reasons, the consumer connection addresses displayed on Kafka Manager may have expired. In this case, restart Kafka Manager.

Prerequisites

The consumer list and connection address can be viewed only when consumers in a consumer group are connected to the Kafka instance (that is, the consumer group is in the STABLE state).

Viewing the Consumer List (Console)

1. Log in to the console.
2. Click in the upper left corner to select the region where your instance is located.
3. Click and choose Application > Distributed Message Service (for Kafka) to open the console of DMS for Kafka.
4. Click the desired instance to go to the instance details page.
5. In the navigation pane, choose Consumer Groups.
6. Click the name of the desired consumer group.
7. On the Consumers tab page, view the consumer list.

   In the consumer list, you can view the consumer ID, consumer address, and client ID.

8. (Optional) To query a specific consumer, enter the consumer ID in the search box and press Enter.

Viewing the Consumer List (Kafka CLI)

• For a Kafka instance with ciphertext access disabled, run the following command in the /bin directory of the Kafka client:

  ./kafka-consumer-groups.sh --bootstrap-server ${connection-address} --group ${group-name} --members --describe

  Parameter description:

  • connection-address: can be obtained from the Connection area on the Basic Information page on the Kafka console.
  • group-name: consumer group name.

  Example:

  [root@ecs-kafka bin]# ./kafka-consumer-groups.sh --bootstrap-server 192.168.xx.xx:9092,192.168.xx.xx:9092,192.168.xx.xx:9092 --group test --members --describe
  
  GROUP           CONSUMER-ID                                           HOST            CLIENT-ID        #PARTITIONS     
  test            console-consumer-571a64fe-b0c4-47ce-833d-9e0da5a88d14 /192.168.0.215  console-consumer 3               
  [root@ecs-kafka bin]#

• For a Kafka instance with ciphertext access enabled, do as follows:
  1. (Optional) Modify the client configuration file.
     View Security Protocol in the Connection area on the Basic Information page on the Kafka console. The configuration settings vary depending on the protocol.

     • SASL_PLAINTEXT: Skip this step and go to 2 if the username and password are already set. Otherwise, create the ssl-user-config.properties file in the /config directory on the Kafka client and add the following content to the file:

       security.protocol=SASL_PLAINTEXT
       # If the SASL mechanism is SCRAM-SHA-512, configure as follows:
       sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
       username="**********" \
       password="**********";        
       sasl.mechanism=SCRAM-SHA-512
       # If the SASL mechanism is PLAIN, configure as follows:
       sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
       username="**********" \
       password="**********";        
       sasl.mechanism=PLAIN

       Parameter description: username and password are the ones you set when enabling ciphertext access for the first time or when creating a user.

     • SASL_SSL: Skip this step and go to 2 if the username, password, and SSL certificate are already set. Otherwise, create the ssl-user-config.properties file in the /config directory on the Kafka client and add the following content to the file:

       security.protocol=SASL_SSL
       ssl.truststore.location={ssl_truststore_path}
       ssl.truststore.password=dms@kafka
       ssl.endpoint.identification.algorithm=
       # If the SASL mechanism is SCRAM-SHA-512, configure as follows:
       sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
       username="**********" \
       password="**********";        
       sasl.mechanism=SCRAM-SHA-512
       # If the SASL mechanism is PLAIN, configure as follows:
       sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
       username="**********" \
       password="**********";        
       sasl.mechanism=PLAIN

       Parameter description:

       • ssl.truststore.location: path for storing the client.jks certificate. Even in Windows, you need to use slashes (/) for the certificate path. Do not use backslashes (\), which are used by default for paths in Windows. Otherwise, the client will fail to obtain the certificate.
       • ssl.truststore.password is certified by the server, which must be set to dms@kafka and cannot be changed.
       • ssl.endpoint.identification.algorithm: whether to verify the certificate domain name. This parameter must be left blank, which indicates disabling domain name verification.
       • username and password: username and password you set when enabling ciphertext access for the first time or when creating a user.
  2. Run the following command in the /bin directory of the Kafka client:

     ./kafka-consumer-groups.sh --bootstrap-server ${connection-address} --group ${group-name} --members --describe --command-config ../config/ssl-user-config.properties

     Parameter description:

     • connection-address: can be obtained from the Connection area on the Basic Information page on the Kafka console.
     • group-name: consumer group name.

     Example:

     [root@ecs-kafka bin]# ./kafka-consumer-groups.sh --bootstrap-server 192.168.xx.xx:9093,192.168.xx.xx:9093,192.168.xx.xx:9093 --group test --members --describe --command-config ../config/ssl-user-config.properties
     
     GROUP           CONSUMER-ID                                           HOST            CLIENT-ID        #PARTITIONS     
     test            console-consumer-566d0c82-07d3-4d87-9a6e-f57a9bc9fc69 /192.168.0.215  console-consumer 3               
     [root@ecs-kafka bin]#

Viewing Consumer Connection Addresses (Console)

1. Log in to the console.
2. Click in the upper left corner to select the region where your instance is located.
3. Click and choose Application > Distributed Message Service (for Kafka) to open the console of DMS for Kafka.
4. Click the desired instance to go to the instance details page.
5. In the navigation pane, choose Consumer Groups.
6. Click the desired consumer group.
7. On the Consumers tab page, view the consumer addresses.
   Figure 1 Consumer list
   

Viewing Consumer Connection Addresses (Kafka Manager)

1. Log in to Kafka Manager.
2. Click kafka_cluster to go to the cluster details page.
3. On the top menu bar, choose Consumers.
   Figure 2 Navigation bar
   

4. Click the desired consumer group to view the topics that the group has subscribed to.
   Figure 3 Consumer group list
   

5. Click the desired topic to go to the topic details page.
   Figure 4 Topics that the consumer group has subscribed to
   

6. In the Consumer Instance Owner column, view the consumer connection address.
   Figure 5 Topic details page