ALM-18000 Yarn Service Unavailable

Alarm Description

The alarm module checks the Yarn service status every 60 seconds. This alarm is generated when the Yarn service is unavailable.

The alarm is cleared when the Yarn service recovers.

Alarm Attributes

Alarm Parameters

Impact on the System

• The cluster cannot provide the Yarn service.
• Users cannot run new applications.
• Submitted applications cannot be run.

Possible Causes

• The ZooKeeper service is abnormal.
• The HDFS service is abnormal.
• There is no active ResourceManager instance in the Yarn cluster.
• All the NodeManager nodes in the Yarn cluster are abnormal.

Handling Procedure

Check ZooKeeper service status.

1. On the FusionInsight Manager, check whether the alarm list contains ALM-13000 ZooKeeper Service Unavailable.
   • If yes, go to Step 2.
   • If no, go to Step 3.

2. Rectify the fault by performing the operations provided in ALM-13000 ZooKeeper Service Unavailable. Then check whether the alarm is cleared.
   • If yes, no further action is required.
   • If no, go to Step 3.

Check the HDFS service status.

3. On the FusionInsight Manager, check whether the alarm list contains the HDFS alarms.
   • If yes, go to Step 4.
   • If no, go to Step 5.

4. Choose O&M > Alarm > Alarms, handle HDFS alarms based on the alarm help, and check whether the Yarn alarm is cleared.
   • If yes, no further action is required.
   • If no, go to Step 5.

Check the ResourceManager status in the Yarn cluster.

5. On FusionInsight Manager, choose Cluster > Services > Yarn.
6. On the Dashboard page, check whether there is any active ResourceManager instance in the Yarn cluster.
   • If yes, go to Step 7.
   • If no, go to Step 10.

Check the NodeManager node status in the Yarn cluster.

7. On FusionInsight Manager, choose Cluster > Services > Yarn > Instances.
8. Query the status of NodeManager and check whether there are unhealthy nodes.
   • If yes, go to Step 9.
   • If no, go to Step 10.

9. Rectify the fault by following the steps provided in ALM-18002 NodeManager Heartbeat Lost or ALM-18003 NodeManager Unhealthy. After the fault is rectified, check whether the Yarn alarm is cleared.
   • If yes, no further action is required.
   • If no, go to Step 10.

Collect fault information.

10. On the FusionInsight Manager portal of the active cluster, choose O&M > Log > Download.
11. Select Yarn in the required cluster from the Service.
12. Click in the upper right corner, and set Start Date and End Date for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
13. Contact the O&M personnel and send the collected logs.

Alarm Clearance

After the fault is rectified, the system automatically clears this alarm.

Related Information

None