ALM-45617 Data Queued in the CDL Replication Slot Exceeds the Threshold

Alarm Description

If a large number of write-ahead logs (WALs) are stacked in the PostgreSQL/Opengauss database, the PostgreSQL/Opengauss disk space may be used up. The system checks whether the amount of data queued in the replication slot configured for a CDL job exceeds the threshold every 5 minutes. This alarm is generated when the amount of data queued in the replication slot exceeds the threshold. This alarm is cleared when the number of data queued in the replication slot falls below the threshold.

Alarm Attributes

Alarm ID	Alarm Severity	Alarm Type	Service Type	Auto Cleared
45617	Critical (default threshold: 5,120 MB) Major (default threshold: 4,196 MB)	Environment	CDL	Yes

Alarm ID

Alarm Severity

Alarm Type

Service Type

Auto Cleared

45617

Critical (default threshold: 5,120 MB)

Major (default threshold: 4,196 MB)

Environment

CDL

Yes

Alarm Parameters

Type	Parameter	Description
Location Information	Source	Specifies the cluster for which the alarm was generated.
	JobName	Specifies the job for which the alarm was generated.
	DBName	Specifies the database for which the alarm was generated.
	SlotName	Specifies the database replication slot for which the alarm was generated.
Additional Information	Lag	Specifies the volume of stacked data in the replication slot configured for a CDL task.

Impact on the System

Disk space of the source PostgreSQL or Opengauss database may be used up and the database cannot provide services.

Possible Causes

The CDL job is abnormal, and data processing stops; the source database is updated quickly, and CDL data processing is slow.

Handling Procedure

Log in to FusionInsight Manager as a user who has the CDL job creation or administrator permission.
Choose O&M. In the navigation pane on the left, choose Alarm > Alarms, click in the row where Alarm ID is 45617, and view the name of the job for which this alarm is generated in Location.
Check whether ALM-45616 CDL Job Execution Exception is displayed in the alarm list.
- If yes, handle the alarm by performing operations provided for ALM-45616 CDL Job Execution Exception.
- If no, go to 4.
Choose Cluster > Services > CDL. Click the link next to CDLService UI to go to the CDL web UI and check whether the job is displayed in the job list based on its name obtained in 2.
- If yes, check whether the job is abnormal.
  - If it is abnormal, go to 5.
  - If it is not, data processing is slow. Contact O&M engineers.
- If no, go to 7.
Click Abnormal or Failed in the row where the job is located and rectify the fault based on the error information displayed on the page.
After rectifying the fault, run the job again and check whether the job can be executed successfully.
- If yes, no further action is required.
- If no, go to 7.

Collect the fault information.

On FusionInsight Manager, choose O&M. In the navigation pane on the left, choose Log > Download.
Select CDL in the required cluster for Service.
Click the edit icon in the upper right corner, and set Start Date and End Date for log collection to 30 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
Contact O&M engineers and provide the collected logs.