ALM-17010 Abnormal Connection Between Oozie and HDFS

Alarm Description

Oozie depends on HDFS. After a task is submitted, the system checks HDFS connectivity. This alarm is generated when the service fails the check for 3 consecutive times.

This alarm is cleared when the connection between Oozie and HDFS becomes normal.

Alarm Attributes

Alarm ID	Alarm Severity	Alarm Type	Service Type	Auto Cleared
17010	Minor	Error handling	Oozie	Yes

Alarm Parameters

Type	Parameter	Description
Location Information	Source	Specifies the cluster for which the alarm was generated.
	ServiceName	Specifies the service for which the alarm was generated.
	RoleName	Specifies the role for which the alarm was generated.
	HostName	Specifies the host for which the alarm was generated.
Additional Information	Trigger Condition	Specifies the alarm triggering condition.

Impact on the System

Running scheduling tasks are blocked and new scheduling tasks cannot be submitted.

Possible Causes

The HDFS service restarts, there is a fault, or the network connectivity is abnormal.

Handling Procedure

Check the HDFS service status.

In the service list on FusionInsight Manager, check whether Running Status of HDFS is Normal.
- If yes, go to 5.
- If no, go to 2.
In the alarm list, check whether the "ALM-14000 HDFS Service Unavailable" alarm is generated.
- If yes, go to 3.
- If no, go to 5.
Rectify the fault by performing the operations provided for ALM-14000 HDFS Service Unavailable.
Wait for several minutes and check whether the alarm Abnormal Connection Between Oozie and HDFS is cleared.
- If yes, no further action is required.
- If no, go to 5.

Check the connectivity between Oozie and HDFS.

Log in to FusionInsight Manager, choose O&M > Log > Online Search, select the Oozie service, and search for the keyword [Oozie Alarm Enhancement][HDFS] in the log. View the cause in the log, and rectify the fault. In the alarm list, check whether the alarm Abnormal Connection Between Oozie and HDFS is cleared.
- If yes, no further action is required.
- If no, go to 6.

Collect fault information.

On FusionInsight Manager, choose O&M. In the navigation pane on the left, choose Log > Download.
Select Oozie for Service and click OK.
Click the edit icon in the upper right corner, and set Start Date and End Date for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
Contact O&M engineers and provide the collected logs.