Help Center/ MapReduce Service/ Component Operation Guide (LTS)/ Using Kafka/ Enterprise-Class Enhancements of Kafka/ Configuring the Path for Extranet Clients to Access Kafka Broker
Updated on 2024-10-09 GMT+08:00
View PDF
Share

Configuring the Path for Extranet Clients to Access Kafka Broker

This section applies to MRS 3.2.0 or later.

Scenario

To access Kafka Broker deployed on a private network from the Kafka client via the Internet, enable the Kafka private and public network traffic distribution function.

Prerequisites

  • The node where Broker resides has both private and public IP addresses. Broker is bound to the private IP address and cannot be accessed via the Internet. Alternatively, the node where Broker resides has only private IP addresses, and external services access the private network through gateway mapping.
  • The ZooKeeper service is running properly.
  • The Kafka instance status and disk status are normal.

Procedure

  1. Log in to FusionInsight Manager.
  2. Choose Cluster > Services > Kafka. On the page that is displayed, click the Instance tab. On this tab page, select the target broker instance in the instance list. On the displayed page, click the Instance Configurations tab and then the All Configurations sub-tab. Enter broker.id in the search box to view and record the broker ID of the current broker instance.
  3. Repeat 2 to view and record the broker ID of each broker instance.
  4. Choose Cluster > Services > Kafka. On the page that is displayed, click the Configurations tab then the All Configurations sub-tab. On this sub-tab page, click Broker(Role) and select Server. Enter advertised and actual in the search box. The five configuration items shown in the following figure are displayed. Configure the parameters according to Table 1.

    Table 1 Parameter description

    Parameter

    Description

    Remarks

    enable.advertised.listener

    Whether to enable the advertised.listeners configuration. The default value is false.

    Set enable.advertised.listener to true.

    NOTE:

    When you install the Kafka service, do not set this parameter to true. You can set this parameter to true only after broker instances and ZooKeeper are running properly.

    advertised.broker.id.ip.map

    IP address released by Kafka. This parameter is left blank by default.

    Format: Broker ID:IP. Multiple brokers can use the same IP address. If multiple mappings are configured, use commas (,) to separate them.

    Map the broker ID of each broker instance recorded in 2 to the IP address to which the broker instance to bound.

    For example, if there are three broker instances and one IP address, the broker IDs are 1, 2, and 3, and the IP address is 10.xxx.xxx.xxx, the configuration format is 1:10.xxx.xxx.xxx,2:10.xxx.xxx.xxx,3:10.xxx.xxx.xxx.

    advertised.broker.id.port.map

    Port released by Kafka. This parameter is left blank by default.

    Format: Broker ID:Port. Port indicates the port to be bound. This port is a custom port and must be available. If multiple mappings are configured, use commas (,) to separate them.

    Map the broker ID of each broker instance recorded in 2 to the port to which the broker instance to bound.

    For example, if there are three broker instances and three ports, the broker IDs are 1, 2, and 3, and the ports are 3307, 3308, and 3309, the configuration format is 1:3307,2:3308,3:3309.

    actual.broker.id.ip.map

    IP address bound to Kafka. This parameter is left blank by default.

    Format: Broker ID:IP. If multiple mappings are configured, use commas (,) to separate them.

    Map the broker ID of each broker instance recorded in 2 to the IP address to which the broker instance to bound.

    For example, if there are three broker instances and one IP address, the broker IDs are 1, 2, and 3, and the IP address is 10.xxx.xxx.xxx, the configuration format is 1:10.xxx.xxx.xxx,2:10.xxx.xxx.xxx,3:10.xxx.xxx.xxx.

    actual.broker.id.port.map

    Port bound to Kafka. This parameter is left blank by default.

    Format: Broker ID:Port. Port indicates the port to be bound. This port is a custom port and must be available. If multiple mappings are configured, use commas (,) to separate them.

    Map the broker ID of each broker instance recorded in 2 to the port to which the broker instance to bound.

    For example, if there are three broker instances and three ports, the broker IDs are 1, 2, and 3, and the ports are 3307, 3308, and 3309, the configuration format is 1:3307,2:3308,3:3309.

  5. After the configuration is complete, click Save in the upper left corner. On the Instance tab, select the target broker instances and choose More > Instance Rolling Restart. Wait until the rolling restart is complete.
  6. (Optional) To disable this configuration, set enable.advertised.listener to false and click Save. On the Instance page of Kafka, select Kafka instances, choose More > Instance Rolling Restart, and wait until the rolling restart is complete.
  • In a cluster with Kerberos authentication enabled, after enable.advertised.listener is configured, the client supports only Kerberos authentication, but not PLAIN authentication.
  • The Port parameter in advertised.broker.id.port.map and actual.broker.id.port.map can be set to the same value.