Configuring Static Resources for an MRS Cluster

Impact on the System

• After a static service pool is configured, the configuration of affected services expires. You need to restart the services. Services are unavailable during restart.
• After a static service pool is configured, the maximum number of resources used by each service and role instance cannot exceed the upper limit.

Configuring Static Resources (MRS 3.x and Later)

Modify the Resource Adjustment Base

1. Log in to the MRS Manager and choose Cluster > Static Service Pool Configurations.

   For details about how to log in to MRS Manager, see Accessing MRS Manager.

2. Click Configuration in the upper right corner. The page for configuring resource pools is displayed.
3. Change the values of CPU (%) and Memory (%) in the System Resource Adjustment Base area.

   Modifying the system resource adjustment base changes the maximum physical CPU and memory usage on nodes by services. If multiple services are deployed on the same node, the maximum physical resource usage of all services cannot exceed the adjusted CPU or memory usage.

4. Click Next.

   To modify parameters again, click Previous.

Modify the Default Resource Configuration Group

5. Click default. In the Configure weight table, set CPU LIMIT(%), CPU SHARE(%), I/O(%), and Memory(%) for each service.
   Figure 1 Weight Configuration
   
   • The sum of CPU LIMIT(%) and CPU SHARE(%) values configured for all services can exceed 100%.
   • The sum of I/O(%) values configured for all services can exceed 100% but cannot be 0.
   • The sum of Memory(%) values configured for all services can exceed, fall below, or equal 100%.
   • Memory(%) settings cannot take effect dynamically and can only be modified in the default configuration group.
   • CPU LIMIT(%) specifies the proportion of CPU cores available to a service relative to the total number of cores allocated to the associated nodes.
   • CPU SHARE(%) specifies the proportion of CPU core time assigned to a service relative to the time allocated to other services sharing the same core. It represents the time ratio when multiple services contend for the same CPU core.

6. Click Generate detailed configurations based on weight configurations. FusionInsight Manager generates the actual values of the parameters in the default weight configuration table based on the cluster hardware resources and allocation information.
7. Click OK.

   In the displayed dialog box, click OK.

Add a Customized Resource Configuration Group

8. Determine whether to automatically adjust resource configurations at different time segments.
   • If yes, go to Step 9.
   • If no, use the default configurations, and no further action is required.

9. Click Configuration, change the system resource adjustment base values, and click Next.
10. Click Add to add a resource configuration group.
    Figure 2 Adding a resource configuration group
    

11. In Step 1: Scheduling Time, click Configuration. The time policy configuration page is displayed.

    Modify the following parameters based on service requirements and click OK.

    • Repeat: If selected, the resource configuration group will run repeatedly based on the scheduling period. If not selected, set the date and time when the resource configuration group can be applied.
    • Repeat Policy: You can select Daily, Weekly, or Monthly. This parameter is available only when Repeat is selected.
    • Between: indicates the time period between the start time and end time when the resource configuration is applied. Set a unique time range. If the time range overlaps with that of an existing group of resource configuration, the time range cannot be saved.

    Description

    • The default resource configuration group takes effect in all undefined time segments.
    • The newly added resource configuration group is a parameter set that takes effect dynamically in a specified time range.
    • The newly added resource configuration group can be deleted. A maximum of four resource configuration groups that take effect dynamically can be added.
    • Select a repetition policy. When the configured end time precedes the start time, the system automatically interprets the end time as occurring on the following day. For example, when a validity period is defined from 22:00 to 06:00, the resource configuration becomes effective at 22:00 on the current day and remains in effect until 06:00 on the following day.
    • When multiple configuration groups are assigned different repeat policy types, their time ranges can overlap. The policy types are prioritized in ascending order as follows: daily, weekly, and monthly. For example, consider two resource configuration groups, one using a monthly policy and the other a daily policy. Their application time ranges overlap ([04:00–07:00] and [06:00–08:00]). In such cases, the configuration of the group with the monthly policy is applied.
    • When multiple resource configuration groups are assigned the same repeat policy type, the time ranges defined for different dates can overlap. For example, consider two weekly resource configuration groups. You can configure them with identical time ranges on different days, such as 04:00–07:00 on Monday and 04:00–07:00 on Wednesday.

12. Modify the resource configuration of each service in Step 2: Weight Configuration.
13. Click Generate detailed configuration. FusionInsight Manager generates the actual values of the parameters in the default weight configuration table based on the cluster hardware resources and allocation information.
14. Click OK.

    In the displayed dialog box, click OK.

Configuring Static Resources (MRS 2.x and Earlier)

1. Modify the system resource adjustment base.
   1. Log in to MRS Manager and click System. In the Resource area, click Configure Static Service Pool.

      For details about how to log in to MRS Manager, see Accessing MRS Manager.

   2. Click Configuration. The service pool configuration group management page is displayed.
   3. In the System Resource Adjustment Base area, change the values of CPU(%) and Memory(%).

      Modifying System Resource Adjustment Base limits the maximum physical CPU and memory resource percentage of nodes that can be used by the Flume, HBase, HDFS, Impala and YARN services. If multiple services are deployed on the same node, the maximum physical resource usage of all services cannot exceed the adjusted CPU or memory usage.

   4. Click Next.

      If you need to modify the parameters again, click Previous in the lower part of the page.

2. Modify the default configuration group of the service pool.
   1. In the Service Pool Configuration table, set CPU LIMIT(%), CPU SHARE(%), I/O(%), and Memory(%) for the Flume, HBase, HDFS, Impala, and YARN services.
   • The sum of CPU LIMIT(%) used by all services can exceed 100%.
   • The sum of CPU SHARE(%) and I/O(%) used by all services must be 100%. For example, if CPU resources are allocated to the HDFS and YARN services, the total CPU resources allocated to the two services are 100%.
   • The sum of Memory(%) used by all services can be greater than, smaller than, or equal to 100%.
   • Memory(%) settings cannot take effect dynamically and can only be modified in the default configuration group.
   2. Click in the blank area of the page to complete the editing. MRS Manager generates the correct values of service pool parameters in the Detailed Configuration area based on the cluster hardware resources and allocation information.
   3. You can click the edit icon on the right of Detailed Configuration to modify the parameter values of the service pool based on service requirements.

      In the Service Pool Configuration area, click the specified service name. The Detailed Configuration area displays only the parameters of the service. Manually changing parameter values does not refresh the service resource usage. In added configuration groups, the configuration group numbers of the parameters that take effect dynamically will be displayed. For example, HBase: RegionServer: dynamic-config1.RES_CPUSET_PERCENTAGE. The parameter functions do not change.

      Table 1 Parameters of the static service pool

      Parameter	Description
      RES_CPUSET_PERCENTAGE dynamic-configX.RES_CPUSET_PERCENTAGE	Configures the service CPU percentage.
      RES_CPU_SHARE dynamic-configX.RES_CPU_SHARE	Configures the service CPU usage share.
      RES_BLKIO_WEIGHT dynamic-configX.RES_BLKIO_WEIGHT	Configures service I/O usage.
      HBASE_HEAPSIZE	Configures the maximum JVM memory for RegionServer.
      HADOOP_HEAPSIZE	Configures the maximum JVM memory of a DataNode.
      yarn.nodemanager.resource.memory-mb	Configures the memory that can be used by NodeManager on the current node.
      dfs.datanode.max.locked.memory	Configures the maximum memory that can be used by a DataNode as the HDFS cache.
      FLUME_HEAPSIZE	Configures the maximum JVM memory that can be used by each Flume instance.
      IMPALAD_MEM_LIMIT	Configures the maximum memory that can be used by an Impalad instance.

3. Add a customized resource configuration group.
   1. Determine whether to automatically adjust resource configurations based on the time.

      If yes, go to 3.b.

      If no, go to Step 4.

   2. Click to add a resource configuration group. In the Scheduling Time area, click the edit icon. The time policy configuration page is displayed.

      Modify the following parameters based on service requirements and click OK.

      • Repeat: If selected, the resource configuration group will run repeatedly based on the scheduling period. If not selected, set the date and time when the resource configuration group can be applied.
      • Repeat Policy: You can select Daily, Weekly, or Monthly. This parameter is available only when Repeat is selected.
      • Between: indicates the time period between the start time and end time when the resource configuration is applied. Set a unique time range. If the time range overlaps with that of an existing group of resource configuration, the time range cannot be saved. This parameter is available only when Repeat is selected.

      Description

      • The default resource configuration group takes effect in all undefined time segments.
      • The newly added resource configuration group is a parameter set that takes effect dynamically in a specified time range.
      • The newly added resource configuration group can be deleted. A maximum of four resource configuration groups that take effect dynamically can be added.
      • Select a repetition policy. When the configured end time precedes the start time, the system automatically interprets the end time as occurring on the following day. For example, when a validity period is defined from 22:00 to 06:00, the resource configuration becomes effective at 22:00 on the current day and remains in effect until 06:00 on the following day.
      • When multiple configuration groups are assigned different repeat policy types, their time ranges can overlap. The policy types are prioritized in ascending order as follows: daily, weekly, and monthly. For example, consider two resource configuration groups, one using a monthly policy and the other a daily policy. Their application time ranges overlap ([04:00–07:00] and [06:00–08:00]). In such cases, the configuration of the group with the monthly policy is applied.
      • When multiple resource configuration groups are assigned the same repeat policy type, the time ranges defined for different dates can overlap. For example, consider two weekly resource configuration groups. You can configure them with identical time ranges on different days, such as 04:00–07:00 on Monday and 04:00–07:00 on Wednesday.
   3. On the Service Pool Configuration page, modify the resource configuration of each service. Click the blank area on the page to complete the editing, and go to Step 4.

      You can click the edit icon on the right of Service Pool Configuration to modify the parameters. Click the edit icon in the Detailed Configuration area to manually update the parameter values generated by the system based on service requirements.

4. Saves the settings.

   Click Save. In the Save Configuration dialog box, select Restart the affected services or instances. Click OK to save the settings and restart related services.

   After the system displays "Operation succeeded", click Finish. The service is started successfully.