Help Center/ Distributed Message Service for Kafka/ User Guide (Paris Region)/ Accessing a Kafka Instance/ Kafka Manager
Updated on 2024-04-07 GMT+08:00
View PDF
Share

Kafka Manager

Kafka Manager is an open-source tool for managing Kafka. It can be used only through a web browser. In Kafka Manager, you can view the monitoring statistics and broker information of your Kafka clusters.

Table 1 Kafka Manager functions on the Kafka console

Kafka Manager

Kafka Console

Viewing topics about an instance

View the topic list on the Topics page.

Viewing basic information about a topic

View the basic information (including the number of replicas, number of partitions, and aging time) about each topic on the Topics page.

Reassigning topic partitions

Reassign partitions automatically or manually on the Topics page.

Updating topic configurations

Modify topic configuration parameters on the Topics page.

Viewing the consumer group list

View the consumer group list on the Consumer Groups page.

Viewing details about a specific consumer

On the Consumer Groups page, click a consumer group name to go to the consumer group details page and view consumers and their progress.

Viewing details of topics in a consumer group

On the Consumer Groups page, click a consumer group name to go to the consumer group details page. On the Consumer Offset tab page, view the topic list of the consumer group, the number of messages accumulated in each topic, and the consumption status of each partition.

Monitoring the cluster or topics

View monitoring information on the Monitoring page.

Prerequisites

Security group rules have been configured by referring to Table 2.

Table 2 Security group rule

Direction

Protocol

Port

Source

Description

Inbound

TCP

9999

0.0.0.0/0

Access Kafka Manager.

Logging In to Kafka Manager

  1. Create a Windows ECS with the same VPC and security group configurations as the Kafka instance. For details, see Purchasing an ECS.

    If public access has been enabled, this step is optional. You can access the instance using the local browser. You do not need to create a Windows ECS.

  2. Obtain the Kafka Manager address on the instance details page.

    • If public network access has been disabled, the Kafka Manager address is Manager Address (Private Network).
      Figure 1 Kafka Manager address (private network)
    • If public network access has been enabled, the Kafka Manager address is Manager Address (Public Network).
      Figure 2 Kafka Manager address (public network)

  3. Enter the Kafka Manager address in the web browser in the Windows ECS.

    If public access is enabled, enter the Kafka Manager address in the address bar of the browser on the local PC. If public access is not enabled, log in to the ECS prepared in 1 and enter the Kafka Manager address in the address bar of the browser on the ECS.

  4. Enter the username and password for logging in to Kafka Manager, which you set when creating the instance.

Viewing Information in Kafka Manager

In Kafka Manager, you can view the monitoring statistics and broker information of your Kafka clusters.

  • Information about clusters
    Click Clusters to view the information about clusters. Figure 3 shows an example of the cluster information.
    • The top navigation bar provides the following functions, as shown in the red box 1 in the figure.
      • Cluster: viewing the list of clusters and cluster information.
      • Brokers: viewing information about brokers of a cluster.
      • Topic: viewing information about topics in a cluster.
      • Preferred Replica Election: electing the leader (preferred replica) of a topic. This operation is not recommended.
      • Reassign Partitions: reassigning partitions. This operation is not recommended.
      • Consumers: viewing the status of consumer groups in a cluster.
    • Red box 2 shows an example of the cluster information summary, including the number of topics and brokers in the cluster.
    Figure 3 Information about clusters
  • Combined information about all brokers of a cluster

    This page shows statistics of brokers of a cluster. Figure 4 shows an example of the storage configuration.

    • Red box 1 shows the list of brokers, including number of incoming and outgoing bytes of different brokers.
    • Red box 2 shows the monitoring metrics of the cluster.
    Figure 4 Viewing the combined information about all brokers in a cluster
  • Information about a specific broker

    Click the ID of a broker to view its statistics. Figure 5 shows an example of the storage configuration.

    • Red box 1 shows the statistics of the broker, including the numbers of topics, partitions, and leaders, and percentages of messages, incoming traffic, and outgoing traffic.
    • Red box 2 shows the monitoring metrics of the broker.
    Figure 5 Viewing information about a broker
  • Topics of an instance

    In the navigation bar, choose Topic > List. The displayed page shows the list of topics and information about the topics, as shown in Figure 6.

    Topics starting with "__" are internal topics. To avoid service faults, do not perform any operation on these topics.

    Figure 6 Topics of an instance
  • Details of a topic

    Click the name of a topic to view its details on the displayed page, as shown in Figure 7.

    • Red box 1: basic information about the topic, including Replication, Number of Partitions, and Sum of Partition Offsets.
    • Red box 2: information about partitions of different brokers.
    • Red box 3: consumer groups of the topic. Click the name of a consumer group name to view its details.
    • Red box 4: configurations of the topic. For details, see https://kafka.apache.org/documentation/#topicconfigs.
    • Red box 5: monitoring metrics of the topic.
    • Red box 6: information about partitions in the topic, including Latest Offset, Leader of a partition, Replicas, and In Sync Replicas.
    Figure 7 Details of a topic
  • List of consumers

    Click Consumers to view the list of consumers in a cluster.

    Only consumer groups that have retrieved messages in the last 14 days are displayed.

    Figure 8 Viewing the list of consumers
  • Details of a specific consumer

    Click the name of a consumer to view its details, including the list of topics in the consumer and the number of messages that can be retrieved in each topic (Total Lag).

    Figure 9 Viewing consumer details
  • Details of topics in a consumer

    Click the name of a topic to view retrieval details of different partitions in the topic, including Partition, the number of messages in a partition (LogSize), progress of the retrieval (Consumer Offset), number of remaining messages in the partition that can be retrieved (Lag), and the latest consumer that retrieved from the partition (Consumer Instance Owner).

    Figure 10 Viewing details of a topic