Adjusting the Yarn Service Configuration

Scenario

HetuEngine depends on the resource allocation and control capabilities provided by Yarn. You need to adjust the Yarn service configuration based on the actual service and cluster server configuration to achieve the optimal performance.

Procedure

1. Log in to FusionInsight Manager.
2. Choose Cluster > Services > Yarn > Configurations > All Configurations and set Yarn service parameters by referring to Table 1.

   Table 1 Yarn configuration parameters

   Parameter	Description	Default Value	Recommended Value
   yarn.nodemanager.resource.memory-mb	Indicates the total physical memory capacity available for YARN on the node. The default value is 16384 MB. If the node has the permanent processes of other services, reduce this parameter value to reseve sufficient resources for the processes.	16384	To achieve the optimal performance, set this parameter to 90% of the minimum physical memory of the node in the cluster.
   yarn.nodemanager.resource.cpu-vcores	Number of CPU cores that can be allocated for containers.	8	To achieve the optimal performance, set this parameter to the minimum number of vCores of the node in the cluster.
   yarn.scheduler.maximum-allocation-mb	Maximum memory (in MB) allocated upon each container request in the ResourceManager. The specified amount of memory will be allocated if more memory is requested.	65536	To achieve the optimal performance, set this parameter to 90% of the minimum physical memory of the node in the cluster.
   yarn.scheduler.maximum-allocation-vcores	The maximum allocation for every container request at the ResourceManager, in terms of virtual CPU cores. Requests higher than this won't take effect, and will get capped to this value.	32	To achieve the optimal performance, set this parameter to the minimum number of vCores of the node in the cluster.

3. Click Save.
4. Choose Cluster > Services > Yarn > More > Restart Service to restart the Yarn service for the parameters to take effect.