ALM-45432 ClickHouse User Synchronization Process Fails

Alarm Description

The system checks the status of the ClickHouse user role synchronization process every 5 minutes. This alarm is generated when the system detects that the ClickHouse user role synchronization process is faulty or the user role synchronization fails.

This alarm is automatically cleared when the ClickHouse user role synchronization process or function becomes normal.

Alarm Attributes

Alarm ID	Alarm Severity	Auto Cleared
45432	Major	Yes

Alarm Parameters

Parameter	Description
Source	Specifies the cluster for which the alarm was generated.
ServiceName	Specifies the service for which the alarm was generated.
RoleName	Specifies the role for which the alarm was generated.
HostName	Specifies the host for which the alarm was generated.

Impact on the System

Some ClickHouseServer instances are unavailable.

Possible Causes

The ClickHouse user role synchronization process is not started properly or exits abnormally.
The ClickHouse user role synchronization process fails to synchronize user role information because the LdapServer service is faulty.

Handling Procedure

Check whether the ClickHouse user role synchronization process is normal.

Log in to FusionInsight Manager and choose O&M > Alarm > Alarms. On the page that is displayed, search for ALM-45432 ClickHouse User Synchronization Process Fails.
Check the host name and additional information in the alarm details.
- If the additional information is "Process clickhouse-ugsync is not exit.", go to 3.
- If the additional information is "Process clickhouse-ugsync sync user failed.", go to 6.
Log in to the faulty host as user omm and run the following command to check whether the ClickHouse user role synchronization process is normal:
ps -ef | grep 'clickhouse-ugsync'

Abnormal result of the synchronization process:
```
[omm@server-2110081635-0001 ~]$ ps -ef | grep 'clickhouse-ugsync'
omm      20104 13146  0 15:57 pts/7    00:00:00 grep --color=auto clickhouse-ugsync
```
- If yes, the alarm is automatically cleared. If the alarm is cleared, no further action is required. If the alarm persists, go to 8.
- If no, go to 4.
Log in to the faulty host as user omm and run the following command to check whether the crontab daemon task is correctly configured:
crontab -l

Normal setting of the crontab daemon task:
```
*/5 * * * * bash /xxxxx/clickhouse_ugsync_check.sh >/dev/null 2>&1
```
- If yes, check whether the alarm is cleared 5 minutes later. If the alarm is cleared, no further action is required. If the alarm persists, go to 8.
- If no, go to 5.
Log in to FusionInsight Manager, choose Cluster > Services > ClickHouse. On the page that is displayed, click the Instance tab. On this tab page, find the abnormal ClickHouseServer instance based on the fault information, and restart it. Wait for 5 minutes and check whether the alarm is cleared.
- If yes, no further action is required.
- If no, go to 6.

Check whether the LdapServer service is normal.

Log in to FusionInsight Manager, choose Cluster > Services, and check whether Running Status of LdapServer is Normal.
- If yes, go to 8.
- If no, go to 7.
Handle the LdapServer service unavailable alarm according to ALM-25000 LdapServer Service Unavailable.
After Running Status of LdapServer becomes Normal, check whether this alarm is cleared.
- If yes, no further action is required.
- If no, go to 8.

Collect fault information.

On FusionInsight Manager, choose O&M > Log > Download.
Expand the drop-down list next to the Service field. In the Services dialog box that is displayed, select ClickHouseServer for the target cluster.
Expand the Hosts list. In the Select Host dialog box that is displayed, select the abnormal host, and click OK.
Click in the upper right corner, and set Start Date and End Date for log collection to 1 hour ahead of and after the alarm generation time, respectively. Then, click Download.
Contact O&M personnel and provide the collected logs.