Updated on 2023-08-03 GMT+08:00

Configuring the MapReduce Shuffle Address

Scenario

When the MapReduce shuffle service is started, it attempts to bind an IP address based on local host. If the MapReduce shuffle service is required to connect to a specific IP address, no configuration is available. The following description allows you to configure a connection to a specific IP address.

Configuration

To bind a specific IP address to the MapReduce shuffle service, set the following parameters in the mapred-site.xml configuration file on the node where the NodeManager instance is deployed. Example path: ${BIGDATA_HOME}/FusionInsight_HD_xxx/x_xx_NodeManager/etc/mapred-site.xml

Table 1 Parameter description

Parameter

Description

Default Value

mapreduce.shuffle.address

Indicates the specified address to run the shuffle service. The format is IP:PORT. The default value is empty. If this parameter is left empty, the local host IP address is bound. The default port number is 13562.

NOTE:

If the value of PORT is different from that of mapreduce.shuffle.port, the mapreduce.shuffle.port value does not take effect.

-