Help Center/ MapReduce Service/ User Guide/ Managing Clusters/ Managing MRS Cluster Components/ Adding Custom MRS Component Parameters
Updated on 2024-10-25 GMT+08:00
View PDF
Share

Adding Custom MRS Component Parameters

The big data components in the cluster fully support all parameters from the open source community. You can modify configuration parameters on the MRS console pr MRS Manager for common scenarios, but some client components may not support all open source parameters.

To modify or add configuration parameters not displayed on the page, use the configuration item customization function. The new parameters will be saved in the component's configuration file and take effect after the component is restarted.

Video Tutorial

This video demonstrates how to modify service parameters on the MRS console or FusionInsight Manager.

The UI may vary depending on the version. This tutorial is for reference only.

Prerequisites

  • You have understood the meanings of parameters to be added, configuration files that have taken effect, and the impact on components.
  • The IAM users have been synchronized in advance. You can do this by clicking Synchronize next to IAM User Sync on the Dashboard page of the cluster details.
  • You have logged in to MRS Manager. For how to log in, see Accessing MRS Manager.

Impact on the System

  • After configuring properties of a service, restart the service if the service status is Expired. During the restart, the service cannot be accessed.
  • After configuring the HBase, HDFS, Hive, Spark, YARN and MapReduce service properties, download and install the client again or download the configuration file to update the client.

Adding Custom Parameters on the Console

  1. Log in to the MRS console.
  2. On the Active Clusters page, select a running cluster and click its name to switch to the cluster details page.
  3. On the MRS cluster details page, click Components.
  4. Select the target service from the service list.
  5. Click Service Configuration.
  6. In the configuration type drop-down box on the right side, switch Basic to All.

    Figure 1 All configurations

  7. In the navigation tree on the left, locate a level-1 node and select Customization. The system displays the customized parameters of the current component.

    • You can customize parameters for services and roles as required.
    • Adding custom parameters for a single role instance is not supported.
    • The configuration files that save the newly added customized parameters are displayed in the Parameter File column. Each configuration file may support open source parameters with the same name. The effective result is determined by the order in which the component loads the configuration files.

  8. Based on the configuration files and parameter functions, locate the row where a specified parameter resides, enter the parameter name supported by the component in the Parameter column and enter the parameter value in the Value column.

    • You can click or to add or delete a custom parameter. You can delete a customized parameter only after you click for the first time.
    • If you want to cancel the modification of a parameter value, click to restore it.

  9. Click Save Configuration and operate as prompted.

Adding Custom Parameters on Manager

For MRS 3.x and later versions:

  1. Log in to FusionInsight Manager.
  2. Choose Cluster > Services.
  3. Click the specified service name on the service management page.
  4. Choose Configurations > All Configurations.
  5. In the navigation tree on the left, locate a level-1 node and select Customization. The system displays the customized parameters of the current component.

    • You can customize parameters for services and roles as required.
    • Adding custom parameters for a single role instance is not supported.
    • The configuration files that save the newly added customized parameters are displayed in the Parameter File column. Each configuration file may support open source parameters with the same name. The effective result is determined by the order in which the component loads the configuration files.

  6. Locate the row where a specified parameter resides, enter the parameter name supported by the component in the Name column and enter the parameter value in the Value column.

    You can click + or - to add or delete a customized parameter.

  7. Click Save. In the displayed Save Configuration dialog box, confirm the modification and click OK. After the system displays "Operation succeeded", click Finish. The configuration is saved.

    Restart the expired service or instance for the configuration to take effect.

    If is displayed before a parameter, this parameter takes effect dynamically. After the configuration is saved, the parameter value is automatically updated to the configuration file. is supported only in MRS 3.2.0 or later.

For MRS 2.x and earlier:

  1. On MRS Manager, click Services.
  2. Select the target service from the service list.
  3. Click Service Configuration.
  4. Set Type to All.

    Figure 2 Service configurations

  5. In the navigation tree, select Customization. The custom parameters of the current component are displayed on Manager.

    • You can customize parameters for services and roles as required.
    • Adding custom parameters for a single role instance is not supported.
    • The configuration files that save the newly added customized parameters are displayed in the Parameter File column. Each configuration file may support open source parameters with the same name. The effective result is determined by the order in which the component loads the configuration files.
    Figure 3 Custom service configurations

  6. Based on the configuration files and parameter functions, locate the row where a specified parameter resides, enter the parameter name supported by the component in the Name column and enter the parameter value in the Value column.

    • You can click or to add or delete a user-defined parameter. You can delete a custom parameter only after you click for the first time.
    • Click to cancel the change to a parameter value.

  7. Click Save Configuration and select Restart the affected services or instances. Click OK to restart the services.

    After the system displays "Operation succeeded", click Finish. The service is started.

Task Example: Adding Custom Hive Parameters

Hive depends on HDFS. By default, Hive accesses the HDFS client. The configuration parameters to take effect are controlled by HDFS in a unified manner.

For example, the HDFS parameter ipc.client.rpc.timeout affects the RPC timeout period for all clients to connect to the HDFS server. If you need to modify the timeout period for Hive to connect to HDFS, you can use the configuration customization function. After this parameter is added to the core-site.xml file of Hive, this parameter can be identified by the Hive service and its configuration overwrites the parameter configuration in HDFS.

  1. Log in to the MRS console.
  2. On the Active Clusters page, select a running cluster and click its name to switch to the cluster details page.
  3. On the MRS cluster details page, click Components.
  4. Choose Hive > Service Configuration.
  5. In the configuration type drop-down box on the right side, switch Basic to All.
  6. In the navigation tree on the left, select Customization for the Hive service. The system displays the customized service parameters supported by Hive.
  7. In core-site.xml, locate the row that contains the core.site.customized.configs parameter, enter ipc.client.rpc.timeout in the Parameter column, and enter a new value in the Value column, for example, 150000. The unit is millisecond.

    Figure 4 Configuring custom Hive parameters

  8. Click Save Configuration and operate as prompted.

    Figure 5 Saving custom configurations