ALM-45652 Flink Service Unavailable

Alarm Description

The alarm module checks the Flink status every 60 seconds. This alarm is generated when the Flink service is unavailable. This alarm is cleared when the Flink service recovers.

Alarm Attributes

Alarm Parameters

Impact on the System

FlinkServer and the Flink client cannot be used to submit Flink jobs.

Possible Causes

The ZooKeeper, HDFS, Yarn, KrbServer, or DBService service on which Flink depends is unavailable.

Handling Procedure

Check whether the ZooKeeper service on which Flink depends is abnormal.

1. Log in to FusionInsight Manager and choose O&M > Alarm > Alarms.
2. In the alarm list, check whether "ALM-13000 ZooKeeper Service Unavailable" exists.
   • If yes, go to 3.
   • If no, go to 5.

3. Handle the alarm by referring to "ALM-13000 ZooKeeper Service Unavailable."
4. After the alarm is cleared, wait a few minutes and check whether the alarm is cleared.
   • If yes, no further action is required.
   • If no, go to 5.

Check whether the HDFS service on which Flink depends is abnormal.

5. On FusionInsight Manager, choose O&M > Alarm > Alarms.
6. In the alarm list, check whether "ALM-14000 HDFS Service Unavailable" exists.
   • If yes, go to 7.
   • If no, go to 9.

7. Handle the alarm by referring to "ALM-14000 HDFS Service Unavailable."
8. After the alarm is cleared, wait a few minutes and check whether the alarm is cleared.
   • If yes, no further action is required.
   • If no, go to 9.

Check whether the Yarn service on which Flink depends is abnormal.

9. On FusionInsight Manager, choose O&M > Alarm > Alarms.
10. In the alarm list, check whether "ALM-18000 Yarn Service Unavailable" exists.
    • If yes, go to 11.
    • If no, go to 13.

11. Handle the alarm by referring to "ALM-18000 Yarn Service Unavailable."
12. After the alarm is cleared, wait a few minutes and check whether the alarm is cleared.
    • If yes, no further action is required.
    • If no, go to 13.

Check whether the KrbServer service on which Flink depends is abnormal.

13. On FusionInsight Manager, choose O&M > Alarm > Alarms.
14. In the alarm list, check whether "ALM-25500 KrbServer Service Unavailable" exists.
    • If yes, go to 15.
    • If no, go to 17.

15. Handle the alarm by referring to "ALM-25500 KrbServer Service Unavailable."
16. After the alarm is cleared, wait a few minutes and check whether the alarm is cleared.
    • If yes, no further action is required.
    • If no, go to 17.

Check whether the DBService service on which Flink depends is abnormal.

17. On FusionInsight Manager, choose O&M > Alarm > Alarms.
18. In the alarm list, check whether "ALM-27001 DBService Service Unavailable" exists.
    • If yes, go to 19.
    • If no, go to 21.

19. Handle the alarm by referring to "ALM-27001 DBService Service Unavailable."
20. After the alarm is cleared, wait a few minutes and check whether the alarm is cleared.
    • If yes, no further action is required.
    • If no, go to 21.

Collect fault information.

21. On FusionInsight Manager, choose O&M. In the navigation pane on the left, choose Log > Download.
22. Expand the Service drop-down list, and select Flink for the target cluster.
23. Click in the upper right corner, and set Start Date and End Date for log collection to 30 minutes ahead of and after the alarm generation time respectively. Then, click Download.
24. Contact O&M personnel and provide the collected logs.

Alarm Clearance

This alarm is automatically cleared after the fault is rectified.

Related Information

None.