Updated on 2022-12-14 GMT+08:00

Adjusting Cluster Node Resource Configurations

Scenario

The default memory size and disk overflow path of HetuEngine are not the best. You need to adjust node resources in the cluster based on the actual service and server configuration of the cluster to achieve the optimal performance.

Procedure

  1. Log in to FusionInsight Manager.
  2. Choose Cluster > Services > HetuEngine > Configurations > All Configurations and adjust the cluster node resource parameters by referring to Table 1.

    Table 1 Parameters for configuring cluster node resources

    Parameter

    Default Value

    Recommended Value

    Description

    Parameter File

    yarn.hetuserver.engine.coordinator.memory

    5120

    At least 2 GB less than that of yarn.scheduler.maximum-allocation-mb

    Memory size used by a Coordinator node

    application.properties

    yarn.hetuserver.engine.coordinator.number-of-containers

    2

    2

    Number of Coordinator nodes

    application.properties

    yarn.hetuserver.engine.coordinator.number-of-cpus

    1

    At least two vCores less than yarn.scheduler.maximum-allocation-vcores

    CPU vCores used by a Coordinator node

    application.properties

    yarn.hetuserver.engine.worker.memory

    10240

    At least 2 GB less than that of yarn.scheduler.maximum-allocation-mb

    Memory size used by a worker node

    application.properties

    yarn.hetuserver.engine.worker.number-of-containers

    2

    Adjusted based on application requirements

    Number of worker nodes

    application.properties

    yarn.hetuserver.engine.worker.number-of-cpus

    1

    At least two vCores less than yarn.scheduler.maximum-allocation-vcores

    CPU vCores used by a Worker node

    application.properties

    Xmx size in the extraJavaOptions parameter

    8GB

    Max (Memory size used by a Worker node – 30 GB, Memory size used by a Worker node x 0.7)

    Maximum available memory of the worker JVM process

    worker.jvm.config

    query.max-memory-per-node

    5GB

    Worker JVM x 0.7

    Maximum available memory of a Query node

    worker.config.properties

    query.max-total-memory-per-node

    5GB

    Worker JVM x 0.7

    Maximum available memory of a Query + System node

    worker.config.properties

    memory.heap-headroom-per-node

    3 GB

    Worker JVM x 0.3

    Maximum available memory of a system heap node

    worker.config.properties

    Xmx size in the extraJavaOptions parameter

    4GB

    Max (Memory size used by a Coordinator node – 30 GB, Memory size used by a Coordinator node x 0.7)

    Maximum available memory of the Coordinator JVM process

    coordinator.jvm.config

    query.max-memory-per-node

    3GB

    Coordinator JVM x 0.7

    Maximum memory that can be used for node query

    coordinator.config.properties

    query.max-total-memory-per-node

    3GB

    Coordinator JVM x 0.7

    Maximum available memory of a Query + System node

    coordinator.config.properties

    memory.heap-headroom-per-node

    1GB

    Coordinator JVM x 0.3

    Maximum available memory of a system heap node

    coordinator.config.properties

    query.max-memory

    7GB

    Sum(query.max-memory-per-node) x 0.7

    Maximum available memory of a Query cluster

    worker.config.properties/coordinator.config.properties

    experimental.spiller-spill-path

    CONTAINER_ROOT_PATH/tmp/hetuserver/hetuserver-sqlengine/

    One or more independent SSDs

    Disk output file path

    worker.config.properties/coordinator.config.properties

    experimental.max-spill-per-node

    10 GB

    Sum(Available space of each node) x 50%

    Available disk space for storing output files of all queries on a node

    worker.config.properties/coordinator.config.properties

    experimental.query-max-spill-per-node

    10 GB

    80% of the available disk space on a node

    Available disk space for storing output files of a query on a node

    worker.config.properties/coordinator.config.properties

  3. Click Save.
  4. Choose Cluster > Services > HetuEngine > More > Restart Service to restart the HetuEngine service for the parameters to take effect.