Expanding the Data Disk Capacity of a ClickHouse Node

Updated on 2024-05-28 GMT+08:00

View PDF

With the service volume increase, the data disk capacity of the ClickHouse node cannot meet service requirements and needs to be expanded.

CAUTION:

For a pay-per-use MRS cluster, the billing mode cannot be changed to yearly/monthly after the disk capacity is expanded.

This section applies only to MRS 3.1.0.

Prerequisites

The ClickHouse cluster and instances are normal.
You have evaluated the data disk capacity of the ClickHouse node to be expanded.

Expanding the Capacity of a Data Disk

Log in to the MRS console. In the left navigation pane, choose Active Clusters and click a cluster name.
Click Nodes. In the corresponding ClickHouse node group, click the name of the node to be expanded. The Disks page is displayed.
In the row of the target data disk, click Expand Capacity.

NOTE:

If only the system disk is displayed on the current page and no data disk exists, the data disk of the ClickHouse node cannot be expanded using this method.
Modify the disk capacity to be added in Add Capacity (GB) and click Next.
Read the note for capacity expansion carefully, click Expand Capacity, confirm the information about the expanded disk capacity, and click Submit.
Log in to the expanded ClickHouse node as user root and run the df -hl command to view information about the existing data directory and disk partition.

The default format of a ClickHouse data directory is /srv/BigData/dataN. The preceding figure shows that the ClickHouse data directory is /srv/BigData/data1 and the corresponding partition is /dev/vdb1.
Perform the following operations to make the new disk capacity take effect.
- To add a partition, go to 8. Adding a partition is to allocate the added disk capacity to a new partition and mount a new ClickHouse data directory to the new partition. This operation does not interrupt services.
- To extend an existing partition, go to 15. Extending an existing partition is to allocate the added disk capacity to an existing partition. Services will be interrupted during the operation. You are advised to stop services before the operation.
For details about how to add a partition, see "Creating a New MBR Partition" or "Creating a New GPT Partition" in Extending Partitions and File Systems for Data Disks (Linux).
Log in to the expanded ClickHouse node as user root and run the following commands to create a ClickHouse data directory and create a mount point for the new partition. It is recommended that the directories be numbered in ascending order based on the current number.

For example, if the current data directory is /srv/BigData/data1, the added directory should be /srv/BigData/data2.

cd /srv/BigData/

mkdir data2

cd data2

mkdir clickhouse

cd /srv/BigData/

chmod 750 -R data2

chown omm:wheel -R data2
Run the following command to mount the new partition:

mount Disk partition Mounted directory

For example, if the new partition is /dev/vdb2 and the mounted directory is /srv/BigData/data2, run the following command:

mount /dev/vdb2 /srv/BigData/data2

NOTE:

If the ECS is restarted, the mounting will become invalid. You can set automatic mounting for partitions at system start by modifying the /etc/fstab file. For details, see Configuring Automatic Mounting at System Start.
Log in to FusionInsight Manager. For details, see Accessing FusionInsight Manager (MRS 3.x or Later). Choose Cluster > ClickHouse > Configurations > All Configurations.
Search for _clickhouse.storage_configuration.disks and add the new ClickHouse data directory to the configuration item.

NOTE:

Separate multiple directories with commas (,) and ensure that each directory ends with a slash (/).

For example, add /srv/BigData/data2/clickhouse/ to /srv/BigData/data1/clickhouse/, and the new directory is /srv/BigData/data1/clickhouse/,/srv/BigData/data2/clickhouse/.
After the new directory is added, click Save to save the configuration. Click Dashboard, choose More > Synchronize Configuration, and click OK.
Log in to the expanded ClickHouse node, go to the following directory, and check whether the new data directory has been updated to the configuration file. After confirming that the information is correct, the operation is complete.

cd ${BIGDATA_HOME}/FusionInsight_ClickHouse_*/x_x_ClickHouse instance name/etc

cat config.xml

The following figure shows an example that the /srv/BigData/data2/clickhouse/ directory has been added to the config.xml file.
To extend an existing partition, ensure that the ClickHouse service has been stopped before the operation. Otherwise, services will be interrupted during the operation.
Determine the partition to be extended based on 6 and extend the partition by referring to "Extending an Existing MBR or GPT Partition" in Extending Partitions and File Systems for Data Disks (Linux).
After the existing partition is extended, run the ClickHouse service again.