Pre-Caching ClickHouse Metadata to the Memory

Scenarios

RocksDB is a high performance embedded database for key-value data. It is widely used in scenarios that require fast data read and write. ClickHouse leverages RocksDB to cache metadata, greatly accelerating metadata loading.

When there are many ClickHouse service tables and large volumes of table data, metadata loading during a rolling restart may take a considerable amount of time. To speed up metadata loading, you can use RocksDB to pre-cache metadata in memory. This reduces startup time and enhances query performance.

Notes and Constraints

This section applies only to MRS 3.3.1-LTS or later.

Enabling Metadata Pre-Caching

You can set use_metadata_cache to 1 or true to cache metadata to the memory through RocksDB.

1. Install the ClickHouse client.

   For details, see Installing a Client.

2. Use the ClickHouse client to connect to the ClickHouse server by referring to ClickHouse Client Practices.
3. Enable metadata pre-caching.
   • Enable metadata pre-caching for historical tables.

     ALTER TABLE <table name> MODIFY SETTING use_metadata_cache=1;

     or

     ALTER TABLE <table name> MODIFY SETTING use_metadata_cache=true;

   • Enable metadata pre-caching when you create a table.

     CREATE TABLE <table name>
     (
     `x` UInt32,
     `y` UInt32,
     `z` UInt32,
     `t` UInt32
     )
     ENGINE = MergeTree
     PARTITION BY x % 10
     ORDER BY (x, y)
     SETTINGS index_granularity = 8192, use_metadata_cache = 1

     or

     CREATE TABLE <table name>
     (
     `x` UInt32,
     `y` UInt32,
     `z` UInt32,
     `t` UInt32
     )
     ENGINE = MergeTree
     PARTITION BY x % 10
     ORDER BY (x, y)
     SETTINGS index_granularity = 8192, use_metadata_cache = true

Parameter Tuning

Metadata pre-caching can be optimized as follows:

Log in to FusionInsight Manager, choose Cluster > Services > ClickHouse, click Configurations and then All Configurations, and modify the following parameters.