Help Center/ MapReduce Service/ Component Operation Guide (Normal)/ Using YARN/ Enterprise-Class Enhancement of YARN/ Configuring Yarn Restart
Updated on 2024-12-11 GMT+08:00
View PDF

Configuring Yarn Restart

Scenario

The Yarn Restart feature includes ResourceManager Restart and NodeManager Restart.

  • When ResourceManager Restart is enabled, the new active ResourceManager node loads the information of the previous active ResourceManager node, and takes over container status information on all NodeManager nodes to continue service running. In this way, status information can be saved by periodically executing checkpoint operations, avoiding data loss.
  • When NodeManager Restart is enabled, NodeManager locally saves information about containers running on the node. After NodeManager is restarted, the container running progress on the node will not be lost by restoring the saved status information.

Configuration Description

Go to the All Configurations page of Yarn and enter a parameter name in the search box by referring to Modifying Cluster Service Configuration Parameters.

Configure ResourceManager Restart as follows:

Table 1 Parameter description of ResourceManager Restart

Parameter

Description

Default Value

yarn.resourcemanager.recovery.enabled

Whether to enable ResourceManager to restore the status after startup. If this parameter is set to true, yarn.resourcemanager.store.class must also be set.

true

yarn.resourcemanager.store.class

State-store class used to store the application and task statuses and certificate content.

For clusters of versions earlier than MRS 3.x: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore

For clusters of MRS 3.x or later:

org.apache.hadoop.yarn.server.resourcemanager.recovery.AsyncZKRMStateStore

yarn.resourcemanager.zk-state-store.parent-path

Directory for storing ZKRMStateStore in ZooKeeper

/rmstore

yarn.resourcemanager.work-preserving-recovery.enabled

Whether to enable ResourceManager work serving. This configuration is used only for Yarn feature verification.

true

yarn.resourcemanager.state-store.async.load

Whether to apply asynchronous restoration to completed applications.

For clusters of versions earlier than MRS 3.x: false

For MRS 3.x or later: true

yarn.resourcemanager.zk-state-store.num-fetch-threads

If asynchronous restoration is enabled, increasing the number of working threads can speed up the restoration of task information stored in ZooKeeper. The value must be greater than 0.

For clusters of versions earlier than MRS 3.x: 1

For MRS 3.x or later: 20

Configure NodeManager Restart as follows:

Table 2 Parameter description of NodeManager Restart

Parameter

Description

Default Value

yarn.nodemanager.recovery.enabled

Whether to enable the function of collecting logs upon a log collection failure when NodeManager is restarted and whether to restore the unfinished application

true

yarn.nodemanager.recovery.dir

Local directory used by NodeManager to store container status It applies to clusters of MRS 3.x or later.

${SRV_HOME}/tmp/yarn-nm-recovery

yarn.nodemanager.recovery.supervised

Whether NodeManager is monitored. After this parameter is enabled, NodeManager does not clear containers after exit. NodeManager assumes that it will restart and restore containers immediately.

true