Updated on 2024-04-11 GMT+08:00

ALM-14000 HDFS Service Unavailable (For MRS 2.x or Earlier)

Description

The system checks the service status of NameService every 30 seconds. This alarm is generated when the system considers that the HDFS service is unavailable because all the NameService services are abnormal.

This alarm is cleared when at least one NameService service is normal and the system considers that the HDFS service recovers.

Attribute

Alarm ID

Alarm Severity

Auto Clear

14000

Critical

Yes

Parameters

Parameter

Description

ServiceName

Specifies the service for which the alarm is generated.

RoleName

Specifies the role for which the alarm is generated.

HostName

Specifies the host for which the alarm is generated.

Impact on the System

HDFS fails to provide services for HDFS service-based upper-layer components, such as HBase and MapReduce. As a result, users cannot read or write files.

Possible Causes

  • ZooKeeper is abnormal.
  • All NameService services are abnormal.

Procedure

  1. Check the ZooKeeper status.

    1. Go to the MRS cluster details page. On the Components tab page, check whether the health status of the ZooKeeper service is Good.
      • If yes, go to 1.b.
      • If no, go to 2.a.
    2. Rectify the health status of the ZooKeeper service. For details, see ALM-13000 ZooKeeper Service Unavailable (For MRS 2.x or Earlier). Then check whether the health status of the ZooKeeper service is Good.
      • If yes, go to 1.c.
      • If no, go to 3.
    3. Wait 5 minutes and check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to 2.a.

  2. Handle the NameService service exception alarm.

    1. Go to the MRS cluster details page. On the Alarms page, check whether all NameService services have abnormal alarms.
      • If yes, go to 2.b.
      • If no, go to 3.
    2. Handle the abnormal NameService services following the instructions in ALM-14010 NameService Is Abnormal (For MRS 2.x or Earlier) and check whether each NameService service exception alarm is cleared.
      • If yes, go to 2.c.
      • If no, go to 3.
    3. Wait 5 minutes and check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to 3.

  3. Collect fault information.

    1. On MRS Manager, choose System > Export Log.
    2. Contact the O&M engineers and send the collected logs.

Reference

None