Configuring the Queue Capacity Policy of a Resource Pool

Updated on 2024-09-23 GMT+08:00

View PDF

After a resource pool is added, the capacity policies of available resources need to be configured for Yarn task queues. This ensures that tasks in the resource pool are running properly. Each queue can be configured with the queue capacity policy of only one resource pool. Users can view the queues in any resource pool and configure queue capacity policies. After the queue policies are configured, Yarn task queues and resource pools are associated.

You can configure queue policies on MRS.

Prerequisites

The IAM users have been synchronized in advance. You can do this by clicking Synchronize next to IAM User Sync on the Dashboard page of the cluster details.
You have logged in to MRS Manager. For how to log in, see Accessing MRS Manager.

A resource pool has been added.
The task queue is not associated with other resource pools except the default default resource pool.

Configuring a Queue Capacity Policy on the MRS Management Console

Log in to the MRS console.
On the Active Clusters page, select a running cluster and click its name to switch to the cluster details page.
On the MRS details page, click Tenant.
Click the Resource Distribution Policies tab.
In Resource Pools, select a specified resource pool.

Available Resource Quota: indicates that all resources in each resource pool are available for queues by default.
Locate the specified queue in the Resource Allocation table, and click Modify in the Operation column.
In Modify Resource Allocation, configure the resource capacity policy of the task queue in the resource pool.
- Capacity (%): specifies the percentage of the current tenant's computing resource usage.
- Maximum Capacity (%): specifies the percentage of the current tenant's maximum computing resource usage.
Click OK to save the settings.

Configuring the Queue Capacity Policy on Manager

For clusters of MRS 3.x and later

Log in to FusionInsight Manager.
Choose Tenant Resources > Dynamic Resource Plan and click the Resource Distribution Policy tab.
Select the name of the target cluster from Cluster and select a resource pool from Resource Pool.
Locate the row that contains the target queue in the Resource Allocation area, and click Modify in the Operation column.
Modify the resource allocation.
- Capacity scheduler
  - Capacity (%): indicates the percentage of compute resources used by the current tenant.
  - Maximum Capacity (%): indicates the maximum percentage of computing resources used by the current tenant.
- Superior Scheduler
  1. On the Resource Configuration Policy tab of the Modify Resource Allocation window, set the resource configuration policy of the queue in the resource pool.
    Figure 1 Resource configuration policy
    - Weight: The task queue with a larger weight preempts resources first when resources are insufficient. Its initial value is the same as the minimum resource percentage.
    - Minimum Resource: indicates the minimum resources that a tenant can obtain.
    - Maximum Resource: indicates the maximum resources that a tenant can obtain.
    - Reserved Resource: indicates the resources that are reserved for the tenant's queues and cannot be lent to other tenants' queues.
  2. In the Modify Resource Allocation dialog box, click the User Policy tab and set the user policy.
    NOTE:
    
    defaultUser(built-in) indicates that the policy specified by defaultUser is used if a user does not specify a policy. The default policy cannot be deleted.
    - Click Add User Policy to add a user policy.
      - Username: indicates the name of a user.
      - Weight: The task queue with a larger weight preempts resources first when resources are insufficient.
      - Max vCores: indicates the maximum number of virtual cores that the user can obtain.
      - Max Memory(MB): indicates the maximum memory that the user can obtain.
    - Click Modify in the Operation column to modify an existing user policy.
    - Click Clear in the Operation column to delete an existing user policy.
Click OK to save the settings.

For MRS clusters of 2.x and earlier:

On MRS Manager, click Tenant.
Click the Dynamic Resource Plan tab.
In Resource Pools, select a specified resource pool.

Available Resource Quota: indicates that all resources in each resource pool are available for queues by default.
Locate the specified queue in the Resource Allocation table, and click Modify in the Operation column.
In Modify Resource Allocation, configure the resource capacity policy of the task queue in the resource pool.
- Capacity (%): specifies the percentage of the current tenant's computing resource usage.
- Maximum Capacity (%): specifies the percentage of the current tenant's maximum computing resource usage.
Click OK to save the settings.