Why Is My Linux ECS Not Booting and Going Into Emergency Mode?

Symptom

Your Linux ECS enters the emergency mode during startup, and displays the message "Welcome to emergency mode", asking you to enter the password of user root for maintenance.

Figure 1 Emergency mode

Possible Causes

The emergency mode allows you to recover the system even if the system fails to enter the rescue mode. In emergency mode, the system installs only the root file system for data reading. It does not attempt to install any other local file systems or activate network interfaces.

The system enters the emergency mode when:

• An error occurred in the /etc/fstab file, leading to the failure in mounting the file system.
• An error occurred in the file system.

Constraints

The operations in this section are applicable to Linux. The operations involve recovering the file system, which may lead to data loss. Therefore, back up data before recovering the file system.

Solution

1. Enter the password of user root and press Enter to enter the recovery mode.
2. Run the following command to mount the root partition in read-write mode to modify the files in the root directory:

   # mount -o rw,remount /

3. Run the following command to try to mount all unmounted file systems:

   # mount -a

   • If the message "mount point does not exist" is displayed, the mount point is unavailable. In such a case, create the mount point.
   • If the message "no such device" is displayed, the file system device is unavailable. In such a case, comment out or delete the mount line.
   • If the message "an incorrect mount option was specified" is displayed, the mount parameters have been incorrectly set. In such a case, correct the parameter setting.
   • If no error occurs and the message "UNEXPECTED INCONSISTENCY;RUN fsck MANUALLY" is displayed, the file system is faulty. In such a case, go to 7.
4. Run the following command to open the /etc/fstab file and correct the error:

   # vi /etc/fstab

   The /etc/fstab file contains the following parameters separated by space:

   [file system] [dir] [type] [options] [dump] [fsck]

   Table 1 /etc/fstab parameters

   Parameter	Description
   [file system]	Specifies the partition or storage device to be mounted. You are advised to set file system in UUID format. To obtain the UUID of a device file system, run the blkid command. Format for reference: # <device> <dir> <type> <options> <dump> <fsck> UUID=b411dc99-f0a0-4c87-9e05-184977be8539 /home ext4 defaults 0 2 UUIDs are independent from the disk order. If the sequence of storage devices is changed manually or undergoes some random changes by some BIOSs, or the storage devices are removed and installed again, UUIDs are more effective in identifying the storage devices.
   [dir]	Specifies the mount point of a file system.
   [type]	Specifies the type of the file system to which a device or partition is mounted. The following file systems are supported: ext2, ext3, ext4, reiserfs, xfs, jfs, smbfs, iso9660, vfat, ntfs, swap, and auto. If type is set to auto, the mount command will speculate on the type of the file system that is used, which is useful for mobile devices, such as CD-ROM and DVD.
   [options]	Specifies the parameters used for mounting. Some parameters are available only for specific file systems. For example, defaults indicates the default mounting parameters of a file system will be used. The default parameters of the ext4 file system are rw, suid, dev, exec, auto, nouser, and async. For more parameters, run the # man mount command to view the man manual.
   [dump]	Specifies whether file system data will be backed up. The value can be 0 or 1. 0 indicates that data will not be backed up, and 1 indicates that data will be backed up. If you have not installed dump, set the parameter to 0.
   [fsck]	Specifies the sequence of checking file systems. The parameter value can be 0, 1, or 2. 0 indicates that the file systems will not be checked by fsck. 1 indicates the highest priority of the root directory to be checked by fsck, and 2 indicates the lower priority of other systems to be checked.

5. After the modification, run the following command to check the fstab file:

   # mount -a

6. Run the following command to restart the ECS:

   # reboot

7. Run the following command to check for file system errors:

   # dmesg |egrep "ext[2..4]|xfs" |grep -i error

   

   • If the error message "I/O error... inode" is displayed, the fault is caused by a file system error.
   • If no error is found in the logs, the fault is generally caused by the damaged superblock. The superblock is the header of the file system. It records the status, size, and idle disk blocks of the file system.
   • If the superblock of a file system is damaged, for example, data is written to the superblock of the file system by mistake, the system may fail to identify the file system. As a result, the system enters the emergency mode during startup. The ext2fs file system backs up the superblock and stores the backup at the blockgroup boundary of the driver.
8. Run the following command to unmount the directory where the file system error occurred:

   # Unmount Mount point

9. Recover the damaged file system.
   

   Recovering the file system may lead to data loss. Back up data before the recovery.

   • For the ext file system, run the following command to check whether the file system is faulty:

     # fsck -n /dev/vdb1

     

     If the message "The super block Cloud no be read or does not describe a correct ext2 filesystem" is displayed, go to step 10.

     To recover the file system, run the following command:

     # fsck /dev/vdb1

   • For the xfs file system, run the following command to check whether the file system is faulty:

     # xfs_repair -n /dev/vdb1

     To recover the file system, run the following command:

     # xfs_repair /dev/vdb1

10. (Optional) If the message "The super block Cloud no be read or does not describe a correct ext2 filesystem" is displayed, the superblock is damaged. In such a case, use the superblock backup for recovery.
    Figure 2 Damaged superblock
    

    Run the following command to replace the damaged superblock with the superblock backup:

    # e2fsck -b 8193 Device name

    In the preceding command, device name is the disk name but not the partition name.

    As shown in Figure 3, the damaged superblock has been replaced.

    Figure 3 Replacing the damaged superblock
    
    

    -b 8193 indicates that the backup of superblock 8193 in the file system is used.

    The location of the superblock backup varies depending on the block size of the file system. For a file system with a 1 KB block size, locate the backup at superblock 8193; for a 2 KB block size, locate the backup at superblock 16384; for a 4 KB block size, locate the backup at superblock 32768.

11. Run the following command to restart the ECS:

    # reboot