Managing MRS Tenant Resource Pools

Prerequisites

• The IAM users have been synchronized in advance. You can do this by clicking Synchronize next to IAM User Sync on the Dashboard page of the cluster details.
• You have logged in to MRS Manager. For how to log in, see Accessing MRS Manager.

• To delete a resource pool, the following requirements must be met:
  • Ensure that no queue in the cluster uses the target resource pool as its default resource pool. Before deleting the resource pool, clear this default setting. To configure an MRS tenant queue, see Configuring the MRS Tenant Queue.
  • Ensure that no queue in the cluster has a resource distribution policy configured in the target resource pool. Before deleting the resource pool, clear this policy. For details, see Clearing the MRS Tenant Queue Configuration.

Managing Resource Pools on the MRS Console

1. Log in to the MRS console.
2. On the Active Clusters page, select a running target cluster and click its name to go to the cluster details page.
3. On the cluster details page, click Tenants.
4. Click the Resource Pools tab.
5. Locate the row that contains the specified resource pool, and click Modify in the Operation column.
6. Modify hosts in a resource pool to adjust tenant compute resources. Manage hosts in a resource pool.
   • Adding a host: In the host list on the left, select the specified host name and add it to the resource pool.
   • Deleting a host: In the host list on the right, click next to a host to remove the host from the resource pool. The host list of a resource pool can contain no hosts.

7. Click OK.

   Locate the row that contains the target resource pool in the Resource Pools tab, and click Delete in the Operation column. In the displayed dialog box, click OK.

   Before deleting a resource pool, ensure that all prerequisites are met.

8. Check whether the modification takes effect: check for running resident jobs (such as Flink streaming jobs) on the target host.
   • If yes, decommission and recommission the NodeManager instances on the affected hosts to ensure smooth task migration. During decommissioning, containers on the node gradually migrate to other nodes. Ensure that the cluster has sufficient resources to handle the migrated tasks. Recommissioned nodes automatically accept new container allocations.
     Perform the following steps:

     1. On MRS Manager, choose Cluster > Services > Yarn > Instances.
     2. Select the NodeManager instances that have the same host names as the hosts whose resource pool has changed.
     3. Choose More > Decommission, enter the user password, confirm the decommissioning impact, and click OK. Wait until the decommissioning completes successfully. After successful decommissioning, the NodeManager instance status displays as Decommissioned.
     4. Select the decommissioned instances, choose More > Recommission, enter the user password, confirm the recommissioning impact, and click OK. Wait until the recommissioning completes successfully.

        After successful recommissioning, the NodeManager instance status returns to Normal, indicating that the resource pool modification has taken effect.

   • If no, no further action is required. Check whether host members have been updated in the resource pool list.

Managing Resource Pools on Manager

1. Log in to FusionInsight Manager of the MRS cluster.

   For details about how to log in to FusionInsight Manager, see Accessing MRS Manager.

2. Choose Tenant Resources > Resource Pool.

   For MRS 2.x or earlier versions, choose Tenant > Resource Pool.

3. Locate the row that contains the specified resource pool, and click Edit in the Operation column.

   For MRS 2.x and earlier versions and MRS 3.3.0 and later versions, click Modify.

4. Manage hosts in a resource pool.
   • Adding a host: Select a host from the host list on the left and click >> to add it to the resource pool.
   • Deleting a host: Select a host from the host list on the right and click << to remove it from the resource pool. The host list of a resource pool can be left blank.

5. Click OK.

   Locate the row that contains the target resource pool in the Resource Pools tab, and click Delete in the Operation column. In the displayed dialog box, click OK.

6. Check whether the modification takes effect: check for running resident jobs (such as Flink streaming jobs) on the target host.
   • If yes, decommission and recommission the NodeManager instances on the affected hosts to ensure smooth task migration. During decommissioning, containers on the node gradually migrate to other nodes. Ensure that the cluster has sufficient resources to handle the migrated tasks. Recommissioned nodes automatically accept new container allocations.
     Perform the following steps:

     1. On MRS Manager, choose Cluster > Services > Yarn > Instances.
     2. Select the NodeManager instances that have the same host names as the hosts whose resource pool has changed.
     3. Choose More > Decommission, enter the user password, confirm the decommissioning impact, and click OK. Wait until the decommissioning completes successfully. After successful decommissioning, the NodeManager instance status displays as Decommissioned.
     4. Select the decommissioned instances, choose More > Recommission, enter the user password, confirm the recommissioning impact, and click OK. Wait until the recommissioning completes successfully.

        After successful recommissioning, the NodeManager instance status returns to Normal, indicating that the resource pool modification has taken effect.

   • If no, no further action is required. Check whether host members have been updated in the resource pool list.

Helpful Links

• For details about creating a resource pool, see Adding an MRS Tenant Resource Pool.
• For details about configuring the resource distribution policy for a tenant queue, see Configuring the MRS Tenant Queue.
• For details about clearing the configured policy of a queue, see Clearing the MRS Tenant Queue Configuration.
• For details about the overall functions of MRS multi-tenant management, see Using MRS Multi-Tenancy.

FAQs

Q: What should I do if a NodeManager instance fails to decommission?

A: A decommissioning failure is usually caused by one of the following reasons:

• Unfinished container tasks on the node: Wait until tasks complete on the node, and then try again.
• Insufficient cluster resources to handle migration tasks: Add hosts to other resource pools or free up cluster resources to support task migration, and then try again.
• Abnormal NodeManager instance: Check the instance alarms on MRS Manager, fix the fault, and then try again.

Q: What should I do if a task runs abnormally after a resource pool is modified?

A: On MRS Manager, choose Cluster > Services > Yarn > Instances, and check whether the NodeManager instance on the host with the modified resource pool is normal. Ensure that decommissioning and recommissioning succeed. On the YARN ResourceManager Web UI, check the node's resource allocation and ensure that containers are scheduled correctly.