Accelerating Merge Operations

Updated on 2024-12-09 GMT+08:00

View PDF

To accelerate background tasks, adjust the ZooKeeper service configuration first. Otherwise, the ClickHouse service and background tasks will be abnormal due to insufficient ZooKeeper resources such as znodes.

Adjust the ZooKeeper configuration. Log in to FusionInsight Manager and choose Cluster > Services > Zookeeper. Click Configurations then All Configurations. Click quorumpeer > System, change the value of GC_OPTS according to the following table, save the configuration, and roll restart the ZooKeeper service.

Configuration Item	Reference Value	Description
GC_OPTS	Xmx = (Memory size of master nodes – 16 GB) x 0.65 (conservative estimation)	JVM parameter used for garbage collection (GC). This parameter is valid only when GC_PROFILE is set to custom. Ensure that the GC_OPT parameter is set correctly. Otherwise, the process will fail to be started. CAUTION: Exercise caution when modifying this item. If this parameter is set incorrectly, the service will be unavailable.

Adjust the ClickHouse configuration. On FusionInsight Manager, choose Cluster > Services > ClickHouse. Click Configurations then All Configurations. Click ClickHouseServer > Zookeeper, modify the following parameters, and save the configuration. You do not need to restart the service.

Configuration Item	Reference Value	Description
clickhouse.zookeeper.quota.node.count	Xmx/4 GB x 1,500,000	Node quantity quota of ClickHouse in the top directory on ZooKeeper. This parameter cannot be set to 0, but can be set to a minimum value of –1. –1 indicates that there is no limit on the number of ClickHouse nodes in the top directory. CAUTION: If the quantity quota is less than the actual value of the current ZooKeeper directory, the configuration can be saved but does not take effect and an alarm is reported on the GUI.
clickhouse.zookeeper.quota.size	Xmx/4 GB x 1 GB	Capacity quota of ClickHouse in the top directory on ZooKeeper. CAUTION: If the quantity quota is less than the actual value of the current ZooKeeper directory, the configuration can be saved but does not take effect and an alarm is reported on the GUI.

Configuration Item

Reference Value

Description

clickhouse.zookeeper.quota.node.count

Xmx/4 GB x 1,500,000

Node quantity quota of ClickHouse in the top directory on ZooKeeper.

This parameter cannot be set to 0, but can be set to a minimum value of –1. –1 indicates that there is no limit on the number of ClickHouse nodes in the top directory.

CAUTION:

If the quantity quota is less than the actual value of the current ZooKeeper directory, the configuration can be saved but does not take effect and an alarm is reported on the GUI.

clickhouse.zookeeper.quota.size

Xmx/4 GB x 1 GB

Capacity quota of ClickHouse in the top directory on ZooKeeper.

CAUTION:

If the quantity quota is less than the actual value of the current ZooKeeper directory, the configuration can be saved but does not take effect and an alarm is reported on the GUI.

Parent topic: ClickHouse Performance Tuning

Previous topic: Solution to the "Too many parts" Error in Data Tables

Next topic: Accelerating TTL Operations

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot