Help Center/ MapReduce Service/ User Guide/ MRS Cluster O&M/ MRS Cluster Alarm Handling Reference/ ALM-12039 Active/Standby OMS Databases Not Synchronized
Updated on 2024-11-13 GMT+08:00
View PDF
Share

ALM-12039 Active/Standby OMS Databases Not Synchronized

Description

The system checks the data synchronization status between the active and standby OMS Databases every 10 seconds. This alarm is generated when the synchronization status cannot be queried for 30 consecutive times or when the synchronization status is abnormal.

This alarm is cleared when the data synchronization status becomes normal.

Attribute

Alarm ID

Alarm Severity

Auto Clear

12039

Critical (Versions Earlier Than MRS 3.3.1)

Major (MRS 3.3.1 and later versions)

Yes

Parameters

Name

Meaning

Source

Specifies the cluster or system for which the alarm is generated.

ServiceName

Specifies the service for which the alarm is generated.

RoleName

Specifies the role for which the alarm is generated.

HostName

Specifies the host for which the alarm is generated.

Local GaussDB HA IP

Specifies the HA IP address of the local GaussDB.

Peer GaussDB HA IP

Specifies the HA IP address of the peer GaussDB.

SYNC_PERCENT

Specifies the synchronization percentage.

Impact on the System

If the active/standby of the OMS database is not synchronized, data in the active database cannot be synchronized to the standby database. If the active instance is abnormal during the alarm reporting period, service data may be lost or data on the FusionInsight Manager may be abnormal.

Possible Causes

  • The network between the active and standby nodes is unstable.
  • The standby OMS Database is abnormal.
  • The standby node disk space is full.

Procedure

Check whether the network between the active and standby nodes is normal.

  1. Log in to FusionInsight Manager, click O&M > Alarm > Alarms, click in the row where the alarm is located, and query the standby OMS Database IP address.
  2. Log in to the active OMS Database node as user root.
  3. Run the ping Standby OMS Database heartbeat IP address command to check whether the standby OMS Database node is reachable.

    • If yes, go to 6.
    • If no, go to 4.

  4. Contact the network administrator to check whether the network is faulty.

    • If yes, go to 5.
    • If no, go to 6.

  5. Rectify the network fault and check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 6.

Check whether the standby OMS Database is normal. (Skip this check for versions later than MRS 3.1.2.)

  1. Log in to the standby OMS Database node as user root.
  2. Run the su - omm command to switch to user omm.
  3. Go to the ${BIGDATA_HOME}/om-server/om/sbin/ directory and run the ./status-oms.sh command to check whether the OMS Database resource status of the standby DBService is normal. In the command output, check whether the following information is displayed in the row where ResName is gaussDB:

    For example:

    10_10_10_231 gaussDB Standby_normal Normal Active_standby
    • If yes, go to 9.
    • If no, go to 16.

Check whether the standby node disk space is full.

  1. Log in to the standby OMS Database node as user root.
  2. Run the su - omm command to switch to user omm.
  3. Run the echo ${BIGDATA_DATA_HOME}/dbdata_om command to obtain the OMS Database data directory.
  4. Run the df -h command to view the system disk partition usage information.
  5. Check whether the disk where the OMS Database data directory is mounted is full.

    • If yes, go to 14.
    • If no, go to 16.

  6. Expand the disk capacity.
  7. After the disk capacity is expanded, wait 2 minutes and check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 16.

Collect fault information.

  1. On the FusionInsight Manager portal, choose O&M > Log > Download.
  2. Select OMMServer from the Service and click OK.
  3. Click in the upper right corner, and set Start Date and End Date for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
  4. Contact the O&M personnel and send the collected log information.

Alarm Clearing

After the fault is rectified, the system automatically clears this alarm.

Related Information

None