Help Center/ MapReduce Service/ Component Operation Guide (Normal)/ Using ClickHouse/ ClickHouse Performance Tuning/ Accelerating Merge Operations
Updated on 2024-12-11 GMT+08:00
View PDF

Accelerating Merge Operations

To accelerate background tasks, adjust the ZooKeeper service configuration first. Otherwise, the ClickHouse service and background tasks will be abnormal due to insufficient ZooKeeper resources such as znodes.

  1. Adjust the ZooKeeper configuration. Log in to FusionInsight Manager and choose Cluster > Services > Zookeeper. Click Configurations then All Configurations. Click quorumpeer > System, change the value of GC_OPTS according to the following table, save the configuration, and roll restart the ZooKeeper service.

    Configuration Item

    Reference Value

    Description

    GC_OPTS

    Xmx = (Memory size of master nodes – 16 GB) x 0.65 (conservative estimation)

    JVM parameter used for garbage collection (GC). This parameter is valid only when GC_PROFILE is set to custom. Ensure that the GC_OPT parameter is set correctly. Otherwise, the process will fail to be started.

    CAUTION:

    Exercise caution when modifying this item. If this parameter is set incorrectly, the service will be unavailable.
  2. Adjust the ClickHouse configuration. On FusionInsight Manager, choose Cluster > Services > ClickHouse. Click Configurations then All Configurations. Click ClickHouseServer > Zookeeper, modify the following parameters, and save the configuration. You do not need to restart the service.

    Configuration Item

    Reference Value

    Description

    clickhouse.zookeeper.quota.node.count

    Xmx/4 GB x 1,500,000

    Node quantity quota of ClickHouse in the top directory on ZooKeeper.

    This parameter cannot be set to 0, but can be set to a minimum value of –1. –1 indicates that there is no limit on the number of ClickHouse nodes in the top directory.

    CAUTION:

    If the quantity quota is less than the actual value of the current ZooKeeper directory, the configuration can be saved but does not take effect and an alarm is reported on the GUI.

    clickhouse.zookeeper.quota.size

    Xmx/4 GB x 1 GB

    Capacity quota of ClickHouse in the top directory on ZooKeeper.

    CAUTION:

    If the quantity quota is less than the actual value of the current ZooKeeper directory, the configuration can be saved but does not take effect and an alarm is reported on the GUI.