Help Center/ MapReduce Service/ User Guide/ MRS Cluster O&M/ MRS Cluster Alarm Handling Reference/ ALM-12039 GaussDB Data Is Not Synchronized (For MRS 2.x or Earlier)
Updated on 2024-09-23 GMT+08:00
View PDF
Share

ALM-12039 GaussDB Data Is Not Synchronized (For MRS 2.x or Earlier)

Description

The system checks the data synchronization status between the active and standby GaussDB nodes every 10 seconds. This alarm is generated when the synchronization status cannot be queried for six consecutive times or when the synchronization status is abnormal.

This alarm is cleared when the data synchronization status is normal.

Attribute

Alarm ID

Alarm Severity

Auto Clear

12039

Critical

Yes

Parameter

Parameter

Description

ServiceName

Service for which the alarm is generated.

RoleName

Role for which the alarm is generated.

HostName

Host for which the alarm is generated.

Local GaussDB HA IP

HA IP address of the local GaussDB.

Peer GaussDB HA IP

HA IP address of the peer GaussDB.

SYNC_PERSENT

Synchronization percentage.

Impact on the System

When data is not synchronized between the active and standby GaussDBs, the data may be lost or abnormal if the active instance becomes abnormal.

Possible Causes

  • The network between the active and standby nodes is unstable.
  • The standby GaussDB is abnormal.
  • The disk space of the standby node is full.

Procedure

  1. Go to the MRS cluster details page. In the alarm list on the alarm management tab page, click the row that contains the alarm. In the alarm details, view the IP address of the standby GaussDB node.
  2. Log in to the active management node.
  3. Run the following command to check whether the standby GaussDB is reachable:

    ping heartbeat IP address of the standby GaussDB

    If yes, go to 6.

    If no, go to 4.

  4. Contact the O&M personnel to check whether the network is faulty.

    • If yes, go to 5.
    • If no, go to 6.

  5. Rectify the network fault and check whether the alarm is cleared from the alarm list.

    • If yes, no further action is required.
    • If no, go to 6.

  6. Log in to the standby GaussDB node.
  7. Run the following commands to switch the user:

    sudo su - root

    su - omm

  8. Go to the ${BIGDATA_HOME}/om-0.0.1/sbin/ directory.

    Run the following command to check whether the resource status of the standby GaussDB is normal:

    sh status-oms.sh

    In the command output, check whether the following information is displayed in the row where ResName is gaussDB: 
    10_10_10_231 gaussDB Standby_normal Normal Active_standby
    • If yes, go to 9.
    • If no, go to 15.

  1. Log in to the standby GaussDB node.
  2. Run the following commands to switch the user:

    sudo su - root

    su - omm

  3. Run the echo ${BIGDATA_DATA_HOME}/dbdata_om command to obtain the GaussDB data directory.
  4. Run the df -h command to check the system disk partition usage.
  5. Check whether the disk where the GaussDB data directory is mounted is full.

    • If yes, go to 14.
    • If no, go to 15.

  6. Contact the O&M personnel to expand the disk capacity. After capacity expansion, wait 2 minutes and check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 15.

  7. Collect fault information.

    1. On MRS Manager, choose System > Export Log.
    2. Contact the O&M engineers and send the collected logs.

Reference

None