ALM-12051 Disk Inode Usage Exceeds the Threshold

Alarm Description

The system checks the disk Inode usage every 30 seconds and compares the actual Inode usage with the threshold (the default threshold is 80%). This alarm is generated when the Inode usage exceeds the threshold for several times (5 times by default) consecutively.

To change the threshold, choose O&M > Alarm > Thresholds > Host > Disk > Disk Inode Usage.

When the Trigger Count is 1, this alarm is cleared when the disk Inode usage is less than or equal to the threshold. When the Trigger Count is greater than 1, this alarm is cleared when the disk Inode usage is less than or equal to 90% of the threshold.

Alarm Attributes

Alarm Parameters

Impact on the System

Service failure: If you need to modify or use data on the disk when data cannot be written to the file system, the job may fail.

Possible Causes

Massive small files are stored in the disk.

Handling Procedure

Massive small files are stored in the disk.

1. On FusionInsight Manager, choose O&M > Alarm > Alarms and click in the row where the alarm is located in the real-time alarm list and obtain the IP address of the host and the disk partition for which the alarm is generated.
2. Log in to the host for which the alarm is generated as user root.

   For details about how to log in to a cluster node, see Logging In to an MRS Cluster Node.

3. Run the following command to check the disk Inode usage:

   df -i | grep -iE Partition name|Filesystem

   The command output is as follows:

   # df -i | grep -iE "xvda2|Filesystem"
   Filesystem            Inodes   IUsed   IFree IUse% Mounted on
   /dev/xvda2           2359296  207420 2151876    9% /

4. If the Inode usage exceeds the threshold, manually check small files stored in the disk partition and confirm whether these small files can be deleted.

   Run the for i in /*; do echo $i; find $i|wc -l; done command to query the number of files in a partition. Replace /* with the specified partition.

   # for i in /srv/*; do echo $i; find $i|wc -l; done
   /srv/BigData
   4284
   /srv/ftp
   1
   /srv/www
   13

   • If yes, delete the files and go to Step 5:

     rm -rf Path of the files or folder to be deleted

     

     Deleting a file or folder is a high-risk operation. Ensure that the file or folder is no longer required before performing this operation.

   • If no, expand the disk capacity. Then, perform Step 5.

5. Wait 5 minutes and check whether the alarm is cleared.
   • If yes, no further action is required.
   • If no, go to Step 6.

Collect fault information.

6. On the FusionInsight Manager homepage of the active cluster, choose O&M > Log > Download.
7. Select OMS from the Service and click OK.
8. Set Host to the node for which the alarm is generated and the active OMS node.
9. Click in the upper right corner, and set Start Date and End Date for log collection to 30 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
10. Contact the O&M personnel and send the collected log information.

Alarm Clearance

After the fault is rectified, the system automatically clears this alarm.

Related Information

None