Updated on 2022-12-14 GMT+08:00

Adjusting the Yarn Service Configuration

Scenario

HetuEngine depends on the resource allocation and control capabilities provided by Yarn. You need to adjust the Yarn service configuration based on the actual service and cluster server configuration to achieve the optimal performance.

Procedure

  1. Log in to FusionInsight Manager.
  2. Choose Cluster > Services > Yarn > Configurations > All Configurations and set Yarn service parameters by referring to Table 1.

    Table 1 Yarn configuration parameters

    Parameter

    Default Value

    Recommended Value

    yarn.nodemanager.resource.memory-mb

    16384

    To achieve the optimal performance, set this parameter to 90% of the minimum physical memory of the node in the cluster.

    yarn.nodemanager.resource.cpu-vcores

    8

    To achieve the optimal performance, set this parameter to the minimum number of vCores of the node in the cluster.

    yarn.scheduler.maximum-allocation-mb

    65536

    To achieve the optimal performance, set this parameter to 90% of the minimum physical memory of the node in the cluster.

    yarn.scheduler.maximum-allocation-vcores

    32

    To achieve the optimal performance, set this parameter to the minimum number of vCores of the node in the cluster.

  3. Click Save.
  4. Choose Cluster > Services > Yarn > More > Restart Service to restart the Yarn service for the parameters to take effect.