Slow Data Replication of ClickHouse Table Replicas and a Large Number of Fetch Requests in the Queue

Symptom

Run the following SQL statement and find that the data synchronization between replicas is heavily delayed:

select table,absolute_delay, queue_size, inserts_in_queue, merges_in_queue from system.replicas where absolute_delay > 0 order by absolute_delay desc limit 10

Cause Analysis

Service data is frequently inserted for dozens of times per second, and the fetch queue is stacked because of the replica synchronization logic of ClickHouse.

Solution

Modify the background_fetches_pool_size configuration item on Manager.