ALM-14006 Number of HDFS Files Exceeds the Threshold (For MRS 2.x or Earlier)

Description

The system periodically checks the number of HDFS files every 30 seconds and compares the number of HDFS files with the threshold. This alarm is generated when the system detects that the number of HDFS files exceeds the threshold.

This alarm is cleared when the number of HDFS files is less than or equal to the threshold.

Attribute

Parameters

Impact on the System

Disk storage space is insufficient, which may result in data import failure. The performance of the HDFS system is affected.

Possible Causes

The number of HDFS files exceeds the threshold.

Procedure

1. Check whether unnecessary files exist in the system.
   1. Use the client on the cluster node and run the hdfs dfs -ls file or directory path command to check whether the file or directory can be deleted.
      • If yes, go to 1.b.
      • If no, go to 2.a.
   2. Run the hdfs dfs -rm -r file or directory path command. Delete unnecessary files, wait 5 minutes, and check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to 2.a.

2. Check the number of files in the system.
   1. On MRS Manager, choose System > Threshold Configuration.
   2. In the navigation tree on the left, choose Services > HDFS > HDFS File > Total Number of Files.
   3. In the right pane, modify the threshold in the rule based on the number of current HDFS files.

      To check the number of HDFS files, choose Services > HDFS, click Customize in the Real-Time Statistics area on the right, and select the HDFS File monitoring item.

   4. Wait 5 minutes and check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to 3.

3. Collect fault information.
   1. On MRS Manager, choose System > Export Log.
   2. Contact the O&M engineers and send the collected logs.

Reference

None