YARN Common Configuration Parameters

Capacity

Queue resource capacity (percentage). Ensure that the capacity requirement of each queue is satisfied when the system is busy. If only a few application programs are running in a queue, the remaining resource of the queue can be shared with other queues. Note that the total capacity of all queues must be smaller than 100.

20

Maximum Capacity

Maximum queue resource usage (percentage). Due to resource sharing, the resources used by a queue may exceed its capacity. The maximum resource usage can be limited using this parameter.

100

Max Master Shares(%)

Indicates the maximum percentage of resources occupied by all ApplicationMasters in the current queue.

Max Allocated vCores

Indicates the maximum number of cores that can be allocated to a single YARN container in the current queue. The default value is -1, indicating that the number of cores is not limited within the value range.

Max Allocated Memory(MB)

Indicates the maximum memory that can be allocated to a single YARN container in the current queue. The default value is -1, indicating that the memory is not limited within the value range.

Max Running Apps

Maximum number of tasks that can be executed at the same time in the current queue. The default value is -1, indicating that the number is not limited within the value range (the meaning is the same if the value is empty). The value 0 indicates that the task cannot be executed. The value ranges from -1 to 2147483647.

Max Running Apps per User

Maximum number of tasks that can be executed by each user in the current queue at the same time. The default value is -1, indicating that the number is not limited within the value range. If the value is 0, the task cannot be executed. The value ranges from -1 to 2147483647.

Max Pending Apps

Maximum number of tasks that can be suspended at the same time in the current queue. The default value is -1, indicating that the number is not limited within the value range (the meaning is the same if the value is empty). The value 0 indicates that tasks cannot be suspended. The value ranges from -1 to 2147483647.

Resource Allocation Rule

Indicates the rule for allocating resources to different tasks of a user. The rule can be FIFO or FAIR.

If a user submits multiple tasks in the current queue and the rule is FIFO, the tasks are executed one by one in sequential order; if the rule is FAIR, resources are evenly allocated to all tasks.

Default Resource Label

Indicates that tasks are executed on a node with a specified resource label.

Active

• ACTIVE: indicates that the current queue can receive and execute tasks.
• INACTIVE: indicates that the current queue can receive but cannot execute tasks. Tasks submitted to the queue are suspended.

Open

• OPEN: indicates that the current queue is opened.
• CLOSED: indicates that the current queue is closed. Tasks submitted to the queue are rejected.

yarn.log-aggregation-enable

Select whether to collect container logs to HDFS.

• If the parameter is set to true, container logs are collected to an HDFS directory. The default directory is {yarn.nodemanager.remote-app-log-dir}/${user}/{thisParam}. You can set the directory by setting the yarn.nodemanager.remote-app-log-dir-suffix parameter on the web UI.
• If this parameter is set to false, container logs will not be collected to HDFS.

After changing the parameter value, restart the Yarn service for the setting to take effect.

true

yarn.resourcemanager.max-completed-applications

Set the total number of historical jobs to be displayed on the web UI.

10000

yarn.resourcemanager.webapp.pagination.enable

Select whether to enable the job list background pagination function for the Yarn web UI.

true

yarn.resourcemanager.webapp.pagination.threshold

Set the maximum number of jobs displayed on each page after the job list background pagination function of the Yarn web UI is enabled.

5000