ALM-45652 Flink Service Unavailable

Alarm Description

The alarm module checks the Flink status every 60 seconds. This alarm is generated when the Flink service is unavailable. This alarm is cleared when the Flink service recovers.

Alarm Attributes

Alarm Parameters

Impact on the System

Flink jobs cannot be submitted with FlinkServer and the Flink client.

Possible Causes

The ZooKeeper, HDFS, Yarn, KrbServer, or DBService service on which Flink depends is unavailable.

Handling Procedure

Check the alarm cause of Flink Service Unavailable.

If the alarm cause is ZooKeeper abnormality, check the ZooKeeper service on which Flink depends.

1. Log in to FusionInsight Manager and choose O&M > Alarm > Alarms.
2. In the alarm list, check whether "ALM-13000 ZooKeeper Service Unavailable" exists.
   • If yes, go to Step 3.
   • If no, go to Step 5.

3. Handle the alarm by referring to ALM-13000 ZooKeeper Service Unavailable.
4. After the alarm is cleared, wait a few minutes and check whether the alarm is cleared.
   • If yes, no further action is required.
   • If no, go to Step 5.

If the alarm cause is HDFS abnormality, check the HDFS service on which Flink depends.

5. On FusionInsight Manager, choose O&M > Alarm > Alarms.
6. In the alarm list, check whether "ALM-14000 HDFS Service Unavailable" exists.
   • If yes, go to Step 7.
   • If no, go to Step 9.

7. Handle the alarm by referring to ALM-14000 HDFS Service Unavailable.
8. After the alarm is cleared, wait a few minutes and check whether the alarm is cleared.
   • If yes, no further action is required.
   • If no, go to Step 9.

If the alarm cause is YARN abnormality, check the YARN service on which Flink depends.

9. On FusionInsight Manager, choose O&M > Alarm > Alarms.
10. In the alarm list, check whether "ALM-18000 Yarn Service Unavailable" exists.
    • If yes, go to Step 11.
    • If no, go to Step 13.

11. Handle the alarm by referring to ALM-18000 Yarn Service Unavailable.
12. After the alarm is cleared, wait a few minutes and check whether the alarm is cleared.
    • If yes, no further action is required.
    • If no, go to Step 13.

Collect fault information.

13. On FusionInsight Manager, choose O&M. In the navigation pane on the left, choose Log > Download.
14. Expand the Service drop-down list, and select Flink for the target cluster.
15. Click in the upper right corner, and set Start Date and End Date for log collection to 30 minutes ahead of and after the alarm generation time respectively. Then, click Download.
16. Send the collected fault logs to O&M personnel for help.

Alarm Clearance

This alarm is automatically cleared after the fault is rectified.

Related Information

None.