Kafka Manager

Prerequisites

Security group rules have been configured by referring to Table 1.

Logging In to Kafka Manager

1. Create a Windows ECS with the same VPC and security group configurations as the Kafka instance. For details, see Purchasing an ECS.

   If public access has been enabled, this step is optional. You can access the instance using the local browser. You do not need to create a Windows ECS.

2. Obtain the Kafka Manager address on the instance details page.
   • If public network access has been disabled, the Kafka Manager address is Manager Address (Private Network).
     Figure 1 Kafka Manager address (private network)
     
   • If public network access has been enabled, the Kafka Manager address is Manager Address (Public Network).
     Figure 2 Kafka Manager address (public network)
     

3. Enter the Kafka Manager address in the web browser in the Windows ECS.

   If public access is enabled, enter the Kafka Manager address in the address bar of the browser on the local PC. If public access is not enabled, log in to the ECS prepared in 1 and enter the Kafka Manager address in the address bar of the browser on the ECS.

4. Enter the username and password for logging in to Kafka Manager, which you set when creating the instance.

Viewing Information in Kafka Manager

In Kafka Manager, you can view the monitoring statistics and broker information of your Kafka clusters.

• Information about clusters
  Click Clusters to view the information about clusters. Figure 3 shows an example of the cluster information.

  • The top navigation bar provides the following functions, as shown in the red box 1 in the figure.
    • Cluster: viewing the list of clusters and cluster information.
    • Brokers: viewing information about brokers of a cluster.
    • Topic: viewing information about topics in a cluster.
    • Preferred Replica Election: electing the leader (preferred replica) of a topic. This operation is not recommended.
    • Reassign Partitions: reassigning partitions. This operation is not recommended.
    • Consumers: viewing the status of consumer groups in a cluster.
  • Red box 2 shows an example of the cluster information summary, including the number of topics and brokers in the cluster.

  Figure 3 Information about clusters
  
• Combined information about all brokers of a cluster

  This page shows statistics of brokers of a cluster. Figure 4 shows an example of the storage configuration.

  • Red box 1 shows the list of brokers, including number of incoming and outgoing bytes of different brokers.
  • Red box 2 shows the monitoring metrics of the cluster.
  Figure 4 Viewing the combined information about all brokers in a cluster
  
• Information about a specific broker

  Click the ID of a broker to view its statistics. Figure 5 shows an example of the storage configuration.

  • Red box 1 shows the statistics of the broker, including the numbers of topics, partitions, and leaders, and percentages of messages, incoming traffic, and outgoing traffic.
  • Red box 2 shows the monitoring metrics of the broker.
  Figure 5 Viewing information about a broker
  
• Topics of an instance

  In the navigation bar, choose Topic > List. The displayed page shows the list of topics and information about the topics, as shown in Figure 6.

  

  Topics starting with "__" are internal topics. To avoid service faults, do not perform any operation on these topics.

  Figure 6 Topics of an instance
  
• Details of a topic

  Click the name of a topic to view its details on the displayed page, as shown in Figure 7.

  • Red box 1: basic information about the topic, including Replication, Number of Partitions, and Sum of Partition Offsets.
  • Red box 2: information about partitions of different brokers.
  • Red box 3: consumer groups of the topic. Click the name of a consumer group name to view its details.
  • Red box 4: configurations of the topic. For details, see https://kafka.apache.org/documentation/#topicconfigs.
  • Red box 5: monitoring metrics of the topic.
  • Red box 6: information about partitions in the topic, including Latest Offset, Leader of a partition, Replicas, and In Sync Replicas.
  Figure 7 Details of a topic
  
• List of consumers

  Click Consumers to view the list of consumers in a cluster.

  

  Only consumer groups that have retrieved messages in the last 14 days are displayed.

  Figure 8 Viewing the list of consumers
  
• Details of a specific consumer

  Click the name of a consumer to view its details, including the list of topics in the consumer and the number of messages that can be retrieved in each topic (Total Lag).

  Figure 9 Viewing consumer details
  
• Details of topics in a consumer

  Click the name of a topic to view retrieval details of different partitions in the topic, including Partition, the number of messages in a partition (LogSize), progress of the retrieval (Consumer Offset), number of remaining messages in the partition that can be retrieved (Lag), and the latest consumer that retrieved from the partition (Consumer Instance Owner).

  Figure 10 Viewing details of a topic