Updated on 2024-04-11 GMT+08:00

ALM-18000 Yarn Service Unavailable (For MRS 2.x or Earlier)

Description

The alarm module checks the Yarn service status every 30 seconds. This alarm is generated when the Yarn service is unavailable.

This alarm is cleared when the Yarn service recovers.

Attribute

Alarm ID

Alarm Severity

Auto Clear

18000

Critical

Yes

Parameters

Parameter

Description

ServiceName

Specifies the service for which the alarm is generated.

RoleName

Specifies the role for which the alarm is generated.

HostName

Specifies the host for which the alarm is generated.

Impact on the System

The cluster cannot provide the Yarn service. Users cannot run new applications. Submitted applications cannot be run.

Possible Causes

  • ZooKeeper is abnormal.
  • HDFS is abnormal.
  • There is no active ResourceManager node in the Yarn cluster.
  • All NodeManager nodes in the Yarn cluster are abnormal.

Procedure

  1. Check the ZooKeeper status.

    1. Go to the cluster details page and choose Alarms.
    2. In the alarm list, check whether the alarm ALM-13000 ZooKeeper Service Unavailable exists.
      • If yes, go to 1.c.
      • If no, go to 2.b.
    3. Rectify the fault by following the handling procedure in ALM-13000 ZooKeeper Service Unavailable (For MRS 2.x or Earlier). Then, check whether this alarm is cleared.
      • If yes, no further action is required.
      • If no, go to 2.b.

  2. Check the HDFS status.

    1. Go to the cluster details page and choose Alarms.
    2. In the alarm list, check whether an HDFS alarm is generated.
      • If yes, go to 2.c.
      • If no, go to 3.b.
    3. Click Alarms, and handle HDFS alarms according to Alarm Help. Check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to 3.b.

  3. Check the ResorceManager status in the Yarn cluster.

    1. Go to the MRS cluster details page and click Components.
    2. Click Yarn.
    3. In Yarn Summary, check whether there is an active ResourceManager node in the Yarn cluster.
      • If yes, go to 4.b.
      • If no, go to 5.

  4. Check the NodeManager node status in the Yarn cluster.

    1. Go to the MRS cluster details page and click Components.
    2. Choose Yarn > Instances.
    3. Check Health Status of NodeManager, and check whether there are unhealthy nodes.
      • If yes, go to 4.d.
      • If no, go to 5.
    4. Rectify the fault by following the procedure provided in ALM-18002 NodeManager Heartbeat Lost (For MRS 2.x or Earlier) or ALM-18003 NodeManager Unhealthy (For MRS 2.x or Earlier). Then, check whether this alarm is cleared.
      • If yes, no further action is required.
      • If no, go to 5.

  5. Collect fault information.

    1. On MRS Manager, choose System > Export Log.
    2. Contact the O&M engineers and send the collected logs.

Reference

None