Updated on 2022-11-18 GMT+08:00

Yarn

Scenario

Flink runs on a Yarn cluster and JobManager runs on ApplicationMaster. Certain configuration parameters of JobManager depend on Yarn. By setting Yarn-related configuration items, Flink is enabled to run better on Yarn.

Configuration Description

The configuration items include the memory, virtual kernel, and port of the Yarn container.

Table 1 Parameter description

Parameter

Description

Default Value

Mandatory

yarn.maximum-failed-containers

Maximum number of containers the system is going to reallocate in case of a container failure of TaskManager The default value is the number of TaskManagers when the Flink cluster is started.

5

No

yarn.application-attempts

Number of ApplicationMaster restarts. The value is the maximum value in the validity interval that is set to Akka's timeout in Flink. After the restart, the IP address and port number of ApplicationMaster will change and you will need to connect to the client manually.

2

No

yarn.heartbeat-delay

Time between heartbeats with the ApplicationMaster and Yarn ResourceManager in seconds. Unit: second

5

No

yarn.containers.vcores

Number of virtual cores of each Yarn container

The default value is the number of TaskManager slots.

No

yarn.application-master.port

ApplicationMaster port number setting. A port number range is supported.

32586-32650

No