ALM-45429 Table Metadata Synchronization Failed on the Added ClickHouse Node

This section applies only to MRS 3.1.2 or later.

This alarm is generated when the local table corresponding to the distributed table fails to be created during ClickHouse capacity expansion.

Alarm ID	Alarm Severity	Alarm Type	Service Type	Auto Cleared
45429	Major	Quality of service	ClickHouse	No

Type	Parameter	Description
Location Information	Source	Specifies the cluster for which the alarm is generated.
	ServiceName	Specifies the service for which the alarm is generated.
	RoleName	Specifies the role for which the alarm is generated.
	HostName	Specifies the host for which the alarm is generated.

The distributed table fails to be queried.

A node is stopped or faulty during capacity expansion.

On FusionInsight Manager, choose Cluster > Services > ClickHouse > Instance.
Check whether an instance is stopped, decommissioned, or faulty.
- If yes, go to 3.
- If no, go to 4.
Start the instance or rectify the instance fault until all instances are running properly.
On FusionInsight Manager, choose O&M > Alarm > Alarms, locate this alarm and the faulty host based on the location information.

Log in to the faulty host as user omm.
Run the following commands to initialize environment variables:

source ${BIGDATA_HOME}/FusionInsight_ClickHouse_*/*_*_ClickHouseServer/etc/ENV_VARS

source ${BIGDATA_HOME}/FusionInsight_ClickHouse_*/*_*_ClickHouseServer/etc/clickhouse-env.sh

export CLICKHOUSE_CONF_DIR=${CLICKHOUSE_CONF_DIR}
Run the following command to run the metadata synchronization tool to synchronize metadata from the existing node to the faulty node:

sh ${BIGDATA_HOME}/FusionInsight_ClickHouse_*/install/FusionInsight-ClickHouse-*/clickhouse/sbin/clickhouse-create-meta.sh true
Run the following command to view the log information and check whether the metadata has been synchronized:
vim /var/log/Bigdata/clickhouse/clickhouseServer/start.log
- If the synchronization is complete, go to 9.
- If the synchronization fails, go to 10.
On FusionInsight Manager, choose O&M > Alarm > Alarms. In the Alarm ID column, locate the corresponding alarm and click Clear in the Operation column. In the displayed dialog box, click OK to manually clear the alarm.

Collect fault information.

On FusionInsight Manager, choose O&M. In the navigation pane on the left, choose Log > Download.
Expand the Service drop-down list, select ClickHouse for the target cluster, and click OK.
Expand the Hosts drop-down list and select the target hosts.
Click the edit icon in the upper right corner, and set Start Date and End Date for log collection to 30 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
Contact O&M engineers and provide the collected logs.