Help Center > > User Guide> Managing Active Clusters> Alarm Reference> ALM-14003 Number of Lost HDFS Blocks Exceeds the Threshold

ALM-14003 Number of Lost HDFS Blocks Exceeds the Threshold

Updated at: Mar 31, 2020 GMT+08:00


The system checks the number of lost blocks every 30 seconds and compares it with the threshold. This alarm is generated when the number of lost blocks exceeds the threshold and is cleared when the number is less than or equal to the threshold.


Alarm ID

Alarm Severity

Automatically Cleared








Specifies the service for which the alarm is generated.


Specifies the role for which the alarm is generated.


Specifies the host for which the alarm is generated.


Specifies the NameService service for which the alarm is generated.

Trigger Condition

Generates an alarm when the actual indicator value exceeds the specified threshold.

Impact on the System

Data stored in HDFS is lost. HDFS may enter the safe mode and cannot provide write services. Lost block data cannot be restored.

Possible Causes

  • The DataNode instance is abnormal.
  • Data is deleted.


  1. Check the DataNode instance.

    1. On the MRS cluster details page, choose Components > HDFS > Instance.

      For MRS 1.8.10 or earlier, log in to MRS Manager and choose Services > HDFS > Instance.

    2. Check whether the status of all DataNode instances is Good.
      • If yes, go to 3.
      • If no, go to 1.c.
    3. Restart the DataNode instance. Check whether the DataNode instance restarts successfully.
      • If yes, go to 2.b.
      • If no, go to 2.a.

  2. Delete the damaged file.

    1. Use the client on the cluster node. Run the hdfs fsck / -delete command to delete the lost file. Then rewrite the file and recover the data.
    2. Wait 5 minutes and check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to 3.

  3. Collect fault information.

    1. On MRS Manager, choose System > Export Log.
    2. Contact the O&M personnel and send the collected log information.

Related Information


Did you find this page helpful?

Submit successfully!

Thank you for your feedback. Your feedback helps make our documentation better.

Failed to submit the feedback. Please try again later.

Which of the following issues have you encountered?

Please complete at least one feedback item.

Content most length 200 character

Content is empty.

OK Cancel