Configuring the MRS Tenant Queue

Prerequisites

• The IAM users have been synchronized in advance. You can do this by clicking Synchronize next to IAM User Sync on the Dashboard page of the cluster details.
• You have logged in to MRS Manager. For how to log in, see Accessing MRS Manager.

• A tenant associated with Yarn and allocated dynamic resources has been added.

Configuring a Tenant Queue on the MRS Management Console

1. Log in to the MRS console.
2. On the Active Clusters page, select a running cluster and click its name to switch to the cluster details page.
3. On the MRS details page, click Tenant.
4. Click the Queue Configuration tab.
5. In the tenant queue table, click Modify in the Operation column of the specified tenant queue.
   

   • In the tenant list on the left of the Tenant tab, click the target tenant. In the window that is displayed, choose Resource. On the page that is displayed, click to open the queue modification page.
   • A queue can be bound to only one non-default resource pool.
   • For MRS 2.x and earlier versions:

     Table 1 Queue configuration parameters

     Parameter	Description
     Maximum Applications	Specifies the maximum number of applications.
     Maximum AM Resource Percent	Specifies the maximum percentage of resources that can be used to run the ApplicationMaster in a cluster.
     Minimum User Limit Percent (%)	Specifies the minimum percentage of resources consumed by a user. The resources for each user in a queue are limited at any time. If applications of multiple users are running at the same time in a queue, the resource usage of each user fluctuates between the minimum value and the maximum value. The minimum value is determined by the number of running applications, while the maximum value is determined by this parameter. For example, assume that this parameter is set to 25. If two users submit applications to the queue, each user can use a maximum of 50% resources; if three users submit applications to the queue, each user can use a maximum of 33% resources; if four users submit applications to the queue, each user can use a maximum of 25% resources.
     User Limit Factor	Specifies the limit factor of the maximum user resource usage. The maximum user resource usage percentage can be obtained by multiplying the limit factor with the percentage of the tenant's actual resource usage in the cluster.
     Status	Specifies the current status of a resource plan.
     Default Resource Pool (Default Node Label Expression)	Specifies the resource pool used by a queue. The default value is default. If you want to change the resource pool, configure the queue capacity first. For details, see Configuring the Queue Capacity Policy of a Resource Pool.

   • For MRS cluster 3.x and later versions:

     Table 2 Queue configuration parameters

     Parameter	Description
     Max Master Shares (%)	Indicates the maximum percentage of resources occupied by all ApplicationMasters in the current queue.
     Max Allocated vCores	Indicates the maximum number of cores that can be allocated to a single YARN container in the current queue. The default value is -1, indicating that the number of cores is not limited within the value range.
     Max Allocated Memory (MB)	Indicates the maximum memory that can be allocated to a single Yarn container in the current queue. The default value is -1, indicating that the memory is not limited within the value range.
     Max Running Apps	Indicates the maximum number of tasks that can be executed at the same time in the current queue. The default value is -1, indicating that the number is not limited within the value range (the meaning is the same if the value is empty). The value 0 indicates that the task cannot be executed.
     Max Running Apps per User	Maximum number of tasks that can be executed by each user in the current queue at the same time. The default value is -1, indicating that the number is not limited within the value range. If the value is 0, the task cannot be executed.
     Max Pending Apps	Indicates the maximum number of tasks that can be suspended at the same time in the current queue. The default value is -1, indicating that the number is not limited within the value range (the meaning is the same if the value is empty). The value 0 indicates that tasks cannot be suspended.
     Resource Allocation Rule	Indicates the rule for allocating resources to different tasks of a user. The rule can be FIFO or FAIR. If a user submits multiple tasks in the current queue and the rule is FIFO, the tasks are executed one by one in sequential order. If the rule is FAIR, resources are evenly allocated to all tasks.
     Default Resource Label	Indicates that tasks are executed on a node with a specified resource label. NOTE: If you need to use a new resource pool, change the default label to the new resource pool label.
     Cross-Pool Scheduling	Indicates whether containers in the current queue support cross-pool scheduling. This parameter is available for clusters of MRS 3.3.0 or later. This function cannot be enabled for the default queue.
     Cross-Pool AM Scheduling	Indicates whether ApplicationMasters in the current queue support cross-pool scheduling. This parameter is available for clusters of MRS 3.3.0 or later. This function cannot be enabled for the default queue.
     Active	ACTIVE: indicates that the current queue can receive and execute tasks. INACTIVE: indicates that the current queue can receive but cannot execute tasks. Tasks submitted to the queue are suspended.
     Open	OPEN: indicates that the current queue is opened. CLOSED: indicates that the current queue is closed. Tasks submitted to the queue are rejected.

Configuring a Tenant Queue on Manager

For clusters of MRS 3.x and later

1. On FusionInsight Manager, choose Tenant Resources.
2. Click the Dynamic Resource Plan tab.
3. Click the Queue Configuration tab.
4. Locate the row containing the specified tenant resource name and click Modify in the Operation column. Modify the parameters based on the scheduler type in use.
   

   • YARN in a new cluster uses the Superior scheduler by default. You can switch the scheduler by referring to Switching the MRS Tenant Resource Scheduler.
   • To query the scheduler type, log in to Manager and search for the yarn.resourcemanager.scheduler.class parameter on the All Configurations page of the YARN service.
   • You can also access the Modify Queue Configuration page as follows: In the tenant list on the Tenant Resources Management page, click the target tenant, click the Resource tab, and click next to Queue Configurations (Queue name).
   • A queue can be bound to only one non-default resource pool. That is, a newly added resource pool can be bound to only one queue to serve as the default resource pool of the queue.
   • For parameters such as Max Allocated vCores, Max Allocated Memory(MB), Max Running Apps, Max Running Apps per User, and Max Pending Apps, if the value of a sub-tenant is -1, the value of the parent tenant can be set to a specific limit. If the parent tenant value is a specific limit, the sub-tenant value can be set to -1.
   • Max Allocated vCores and Max Allocated Memory(MB) must be both changed to values other than -1.
   • For queues with cross-resource-pool scheduling enabled, existing resource pools cannot be deleted during job running. Otherwise, running jobs may be continuously blocked because they cannot obtain resources. Similarly, if a new resource pool is configured for a queue during job running, the queue in the running state may not immediately use the resources in the new resource pool. The new resources are available only for jobs submitted after modification.
   • To use a Superior scheduler cluster, configure sub-tenant properties by referring to the following table.

     Table 3 Queue configuration parameters

     Parameter	Description
     Max Master Shares (%)	Indicates the maximum percentage of resources occupied by all ApplicationMasters in the current queue.
     Max Allocated vCores	Indicates the maximum number of cores that can be allocated to a single YARN container in the current queue. The default value is -1, indicating that the number of cores is not limited within the value range.
     Max Allocated Memory (MB)	Indicates the maximum memory that can be allocated to a single Yarn container in the current queue. The default value is -1, indicating that the memory is not limited within the value range.
     Max Running Apps	Indicates the maximum number of tasks that can be executed at the same time in the current queue. The default value is -1, indicating that the number is not limited within the value range (the meaning is the same if the value is empty). The value 0 indicates that the task cannot be executed.
     Max Running Apps per User	Maximum number of tasks that can be executed by each user in the current queue at the same time. The default value is -1, indicating that the number is not limited within the value range. If the value is 0, the task cannot be executed.
     Max Pending Apps	Indicates the maximum number of tasks that can be suspended at the same time in the current queue. The default value is -1, indicating that the number is not limited within the value range (the meaning is the same if the value is empty). The value 0 indicates that tasks cannot be suspended.
     Resource Allocation Rule	Indicates the rule for allocating resources to different tasks of a user. The rule can be FIFO or FAIR. If a user submits multiple tasks in the current queue and the rule is FIFO, the tasks are executed one by one in sequential order. If the rule is FAIR, resources are evenly allocated to all tasks.
     Default Resource Label	Indicates that tasks are executed on a node with a specified resource label. NOTE: If you need to use a new resource pool, change the default label to the new resource pool label.
     Cross-Pool Scheduling	Indicates whether containers in the current queue support cross-pool scheduling. This parameter is available for clusters of MRS 3.3.0 or later. This function cannot be enabled for the default queue.
     Cross-Pool AM Scheduling	Indicates whether ApplicationMasters in the current queue support cross-pool scheduling. This parameter is available for clusters of MRS 3.3.0 or later. This function cannot be enabled for the default queue.
     Active	ACTIVE: indicates that the current queue can receive and execute tasks. INACTIVE: indicates that the current queue can receive but cannot execute tasks. Tasks submitted to the queue are suspended.
     Open	OPEN: indicates that the current queue is opened. CLOSED: indicates that the current queue is closed. Tasks submitted to the queue are rejected.

   • For a cluster using the Capacity scheduler, Tenant Resource Name (Queue) indicates the tenant and queue name. Configure sub-tenants properties by referring to the following table.

     Table 4 Queue configuration parameters

     Parameter	Description
     Maximum Applications	Specifies the maximum number of applications.
     Maximum AM Resource Percent	Specifies the maximum percentage of resources that can be used to run the ApplicationMaster in a cluster.
     Minimum User Limit Percent (%)	Specifies the minimum percentage of resources consumed by a user. The resources for each user in a queue are limited at any time. If applications of multiple users are running at the same time in a queue, the resource usage of each user fluctuates between the minimum value and the maximum value. The minimum value is determined by the number of running applications, while the maximum value is determined by this parameter. For example, assume that this parameter is set to 25. If two users submit applications to the queue, each user can use a maximum of 50% resources; if three users submit applications to the queue, each user can use a maximum of 33% resources; if four users submit applications to the queue, each user can use a maximum of 25% resources.
     User Limit Factor	Specifies the limit factor of the maximum user resource usage. The maximum user resource usage percentage can be obtained by multiplying the limit factor with the percentage of the tenant's actual resource usage in the cluster.
     Status	Specifies the current status of a resource plan.
     Default Resource Pool (Default Node Label Expression)	Specifies the resource pool used by a queue. The default value is default. If you want to change the resource pool, configure the queue capacity first. For details, see Configuring the Queue Capacity Policy of a Resource Pool.

5. Click OK.

For MRS clusters of 2.x and earlier:

1. On MRS Manager, click Tenant.
2. Click the Dynamic Resource Plan tab.
3. Click the Queue Configuration tab.
4. In the tenant queue table, click Modify in the Operation column of the specified tenant queue.
   

   In the tenant list on the left of the Tenant tab, click the target tenant. In the window that is displayed, choose Resource. On the page that is displayed, click the edit icon to open the queue modification page.

   Table 5 Queue configuration parameters

   Parameter	Description
   Maximum Applications	Specifies the maximum number of applications.
   Maximum AM Resource Percent	Specifies the maximum percentage of resources that can be used to run the ApplicationMaster in a cluster.
   Minimum User Limit Percent (%)	Specifies the minimum percentage of resources consumed by a user. The resources for each user in a queue are limited at any time. If applications of multiple users are running at the same time in a queue, the resource usage of each user fluctuates between the minimum value and the maximum value. The minimum value is determined by the number of running applications, while the maximum value is determined by this parameter. For example, assume that this parameter is set to 25. If two users submit applications to the queue, each user can use a maximum of 50% resources; if three users submit applications to the queue, each user can use a maximum of 33% resources; if four users submit applications to the queue, each user can use a maximum of 25% resources.
   User Limit Factor	Specifies the limit factor of the maximum user resource usage. The maximum user resource usage percentage can be obtained by multiplying the limit factor with the percentage of the tenant's actual resource usage in the cluster.
   Status	Specifies the current status of a resource plan.
   Default Resource Pool (Default Node Label Expression)	Specifies the resource pool used by a queue. The default value is default. If you want to change the resource pool, configure the queue capacity first. For details, see Configuring the Queue Capacity Policy of a Resource Pool.