Accelerating ClickHouse TTL Operations
When TTL is triggered in ClickHouse, a large amount of CPU and memory are consumed.
Log in to FusionInsight Manager and choose Cluster > Services > ClickHouse. Click Configurations and then All Configurations. Click ClickHouseServer > Customization, find the clickhouse-config-customize parameter, add the following parameters, save the configuration, and restart the service.
|
Configuration Item |
Description |
Reference Value |
|---|---|---|
|
merge_tree.max_replicated_merges_with_ttl_in_queue |
Number of tasks that allow TTL to merge parts concurrently in the ReplicatedMergeTree queue. If this parameter is set to 0, the system adaptively adjusts the value to a quarter of the number of CPU cores. |
Half the number of CPU cores |
|
merge_tree.max_number_of_merges_with_ttl_in_pool |
The thread pool that allows TTL to merge parts in the ReplicatedMergeTree queue. When there is more than specified number of merges with TTL entries in pool, do not assign new merge with TTL. This is to leave free threads for regular merges and avoid "Too many parts" exceptions. If this parameter is set to 0, the system adaptively adjusts the value to half the number of CPU cores. |
Number of CPU cores |
- Do not modify these configurations when the cluster writes heavily. Idle threads need to be reserved for regular Merge operations to avoid the "Too many parts" issue.
- For large tables with over 100 million records, use a scheduled task to implement data aging instead of modifying or setting TTL. To do this, run alter table table_name on cluster default_cluster drop partition partition_name.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot
