Configuring Auto Scaling for an MRS Cluster

In big data application scenarios, especially real-time data analysis and processing, the number of cluster nodes needs to be dynamically adjusted according to data volume changes to provide proper resources. The auto scaling function of MRS enables clusters to be automatically scaled out or in based on cluster load.

Auto scaling rules: You can increase or decrease Task nodes based on real-time cluster loads. Auto scaling will be triggered when the data volume changes but there may be some delays.
Resource plan (setting the task node quantity based on the time range): If the data volume changes periodically, you can create resource plans to resize the cluster before the data volume changes, thereby avoiding delays in increasing or decreasing resources.

You can configure either auto scaling rules or resource plans or both of them to trigger the auto scaling.

Scenario

The following example describes how to use both auto scaling rules and resource plans:

A real-time processing service sees an unstable increase in data volume from 7:00 to 13:00 on Monday, Tuesday, and Saturday. For example, 5 to 8 task nodes are required from 7:00 to 13:00 on Monday, Tuesday, and Saturday, and 2 to 4 are required beyond this period.

You can set an auto scaling rule based on a resource plan. When the data volume exceeds the expected value, the number of Task nodes changes with resource loads, without exceeding the node range specified in the resource plan. When a resource plan is triggered, the number of nodes changes within the specified range with minimum affect. That is, increase nodes to the upper limit and decrease nodes to the lower limit.

Video Tutorial

This video uses an MRS 3.1.0 cluster as an example to describe how to configure an auto scaling policy when you purchase a cluster and how to add an auto scaling policy to an existing cluster. For details, see Configuring Auto Scaling for an MRS Cluster.

The UI may vary according to the version. The video tutorial is for reference only.

Adding a Task Node

You can scale out an MRS cluster by manually adding task nodes.

To add a task node to a custom cluster, perform the following steps:

On the cluster details page, click the Nodes tab and click Add Node Group. The Add Node Group page is displayed.
Select Task forNode Type. Retain the default value NM for Deploy Roles. To deploy the NodeManager role, the node type must be Task. Set other parameters as required.
Figure 1 Adding a task node group

To add a task node to a non-custom cluster, perform the following steps:

On the cluster details page, click the Nodes tab and click Configure Task Node. The Configure Task Node page is displayed.
On the Configure Task Node page, set Node Type, Instance Specifications, Nodes, System Disk. In addition, if Add Data Disk is enabled, configure the storage type, size, and number of data disks.
Click OK.

Using Auto Scaling Rules and Resource Plans Together

Log in to the MRS management console.
On the Active Clusters page, and click the name of the cluster to be operated. The cluster details page is displayed.
On the page that is displayed, click the Auto Scaling tab.
Click Add Auto Scaling Policy and set Node Range to 2-4.

Figure 2 Configuring auto scaling
Configure a resource plan.
1. Click Configure Node Range for Specific Time Range under Default Range.
2. Configure the Time Range and Node Range parameters.
  Time Range: Set it to 07:00-13:00.
  
  Node Range: Set it to 5-8.
  
  Figure 3 Auto scaling
Configure an auto scaling rule.
1. Select Scale-out.
2. Click Add Rule on the right.
  Figure 4 Adding a rule
  
  Rule Name: default-expand-2.
  
  If: Select the rule objects and constraints from the drop-down list boxes, for example, YARNAppRunning is greater than 75.
  
  Last For: Set it to 1 five-minute periods.
  
  Add: Set it to 1 node.
  
  Cooldown Period: Set it to 20 minutes.
3. Click OK.
Select I agree to authorize MRS to scale out or in nodes based on the above rule.
Click OK.

Reference Information

When adding a rule, you can refer to Table 1 to configure the corresponding metrics.

Hybrid clusters support all metrics of analysis and streaming clusters.
The accuracy of different value types in Table 1 is as follows:
- Integer: integer
- Percentage: 0.01
- Ratio: 0.01

**Table 1** Auto scaling metrics
Cluster Type	Metric	Value Type	Description
Streaming cluster	StormSlotAvailable	Integer	Number of available Storm slots. Value range: 0 to 2147483646.
	StormSlotAvailablePercentage	Percentage	Percentage of available Storm slots, that is, the proportion of the available slots to total slots. Value range: 0 to 100.
	StormSlotUsed	Integer	Number of used Storm slots. Value range: 0 to 2147483646.
	StormSlotUsedPercentage	Percentage	Percentage of the used Storm slots, that is, the proportion of the used slots to total slots. Value range: 0 to 100.
	StormSupervisorMemAverageUsage	Integer	Average memory usage of the Supervisor process of Storm. Value range: 0 to 2147483646.
	StormSupervisorMemAverageUsagePercentage	Percentage	Average percentage of the used memory of the Supervisor process of Storm to the total memory of the system. Value range: 0 to 100.
	StormSupervisorCPUAverageUsagePercentage	Percentage	Average percentage of the used CPUs of the Supervisor process of Storm to the total CPUs. Value range: [0, 6000].
Analysis cluster	YARNAppPending	Integer	Number of pending tasks on Yarn. Value range: 0 to 2147483646.
	YARNAppPendingRatio	Ratio	Ratio of pending tasks on YARN, that is, the ratio of pending tasks to running tasks on YARN. Value range: 0 to 2147483646.
	YARNAppRunning	Integer	Number of running tasks on Yarn. Value range: 0 to 2147483646.
	YARNContainerAllocated	Integer	Number of containers allocated to YARN. Value range: 0 to 2147483646.
	YARNContainerPending	Integer	Number of pending containers on Yarn. Value range: 0 to 2147483646.
	YARNContainerPendingRatio	Ratio	Ratio of pending containers on Yarn, that is, the ratio of pending containers to running containers on Yarn. Value range: 0 to 2147483646.
	YARNCPUAllocated	Integer	Number of virtual CPUs (vCPUs) allocated to Yarn. Value range: 0 to 2147483646.
	YARNCPUAvailable	Integer	Number of available vCPUs on Yarn. Value range: 0 to 2147483646.
	YARNCPUAvailablePercentage	Percentage	Percentage of available vCPUs on Yarn, that is, the proportion of available vCPUs to total vCPUs. Value range: 0 to 100.
	YARNCPUPending	Integer	Number of pending vCPUs on Yarn. Value range: 0 to 2147483646.
	YARNMemoryAllocated	Integer	Memory allocated to Yarn. The unit is MB. Value range: 0 to 2147483646.
	YARNMemoryAvailable	Integer	Available memory on Yarn. The unit is MB. Value range: 0 to 2147483646.
	YARNMemoryAvailablePercentage	Percentage	Percentage of available memory on Yarn, that is, the proportion of available memory to total memory on Yarn. Value range: 0 to 100.
	YARNMemoryPending	Integer	Pending memory on Yarn. Value range: 0 to 2147483646.

When adding a resource plan, you can set parameters by referring to Table 2.

**Table 2** Configuration items of a resource plan
Parameter	Description
Effective On	The effective date of a resource plan. Daily is selected by default. You can also select one or multiple days from Monday to Sunday.
Time Range	Start time and end time of a resource plan are accurate to minutes, with the value ranging from 00:00 to 23:59. For example, if a resource plan starts at 8:00 and ends at 10:00, set this parameter to 8:00-10:00. The end time must be at least 30 minutes later than the start time. Time ranges configured for different resource plans cannot overlap.
Node Range	The number of nodes in a resource plan ranges from 0 to 500. In the time range specified in the resource plan, if the number of task nodes is less than the specified minimum number of nodes, it will be increased to the specified minimum value of the node range at a time. If the number of task nodes is greater than the maximum number of nodes specified in the resource plan, the auto scaling function reduces the number of task nodes to the maximum value of the node range at a time. The minimum number of nodes must be less than or equal to the maximum number of nodes.