Help Center/ MapReduce Service/ User Guide/ MRS Cluster O&M/ MRS Cluster Alarm Handling Reference/ ALM-45617 Data Queued in the CDL Replication Slot Exceeds the Threshold
Updated on 2024-09-23 GMT+08:00
View PDF
Share

ALM-45617 Data Queued in the CDL Replication Slot Exceeds the Threshold

Alarm Description

If too many WALs are stacked in PostgreSQL or openGauss (applicable to MRS 3.3.0 or later), the PostgreSQL or openGauss disk space may be used up. The system checks whether the amount of data queued in the replication slot configured for a CDL job exceeds the threshold every 5 minutes. This alarm is generated when the amount of data queued in the replication slot exceeds the threshold. This alarm is cleared when the number of data queued in the replication slot falls below the threshold.

Alarm Attributes

Alarm ID

Alarm Severity

Auto Cleared

45617

Major

Yes

Alarm Parameters

Parameter

Description

Source

Specifies the cluster for which the alarm was generated.

ServiceName

Specifies the service for which the alarm was generated.

JobName

Specifies the job for which the alarm was generated.

DBName

Specifies the database for which the alarm was generated.

SlotName

Specifies the database replication slot for which the alarm was generated.

Lag

Specifies the data queued in the slot.

Impact on the System

Disk space of the source PostgreSQL or openGauss database may be used up and the database cannot provide services.

Possible Causes

The CDL job is abnormal, and data processing stops; the source database is updated quickly, and CDL data processing is slow.

Handling Procedure

  1. Log in to FusionInsight Manager as a user who has the CDL job creation or administrator permission.
  2. Choose O&M. In the navigation pane on the left, choose Alarm > Alarms, click in the row where Alarm ID is 45617, and view the name of the job for which this alarm is generated in Location.
  3. Check whether ALM-45616 CDL Job Execution Exception is displayed in the alarm list.

    • If yes, handle the alarm by performing operations provided for ALM-45616 CDL Job Execution Exception.
    • If no, go to 4.

  4. Choose Cluster > Services > CDL. Click the link next to CDLService UI to go to the CDL web UI and check whether the job is displayed in the job list based on its name obtained in 2.

    • If yes, check whether the job is abnormal.
      • If it is abnormal, go to 5.
      • If it is not, data processing is slow. Contact O&M personnel.
    • If no, go to 7.

  5. Click Abnormal or Failed in the row where the job is located and rectify the fault based on the error information displayed on the page.
  6. After rectifying the fault, run the job again and check whether the job can be executed successfully.

    • If yes, no further action is required.
    • If no, go to 7.

Collect fault information.

  1. On FusionInsight Manager, choose O&M. In the navigation pane on the left, choose Log > Download.
  2. Expand the Service drop-down list, and select CDL for the target cluster.
  3. Click in the upper right corner, and set Start Date and End Date for log collection to 30 minutes ahead of and after the alarm generation time respectively. Then, click Download.
  4. Contact O&M personnel and provide the collected logs.

Alarm Clearance

This alarm is cleared when the amount of data queued in the replication slot is less than the threshold. You do not need to manually clear the alarm.

Related Information

None