Why Is the Amount of Migrated Data Less Than the Total Amount of Data to Be Migrated After the Migration Is Complete?

Symptom

After the migration was complete, the amount of migrated data was less than the total amount of data scanned on the source server.

Possible Causes

The Agents collects how much data to be migrated on the source serve using the df-TH command. The amount of migrated data displayed on the console is the combined size of all the migrated files.

If these two amounts are different, the possible causes are as follows:

• Possible cause 1: Some files were deleted from the source server, but the processes using these files still exist.
  The rm command or other software is often used to delete files. If a file is used by a running process when the file is deleted, the file will still be accessible to this process and will continue to occupy disk space.

  1. On the source server, run df -TH to check whether the used disk space is the same as the total amount of migrated data.

     

  2. In the root directory of the source server, run du -sh * to check the usage of disk directories.

     

     In the above figures, df reports a larger data volume than du.

  3. Run the following command on the source server:

     lsof -n / |grep deleted

     If information similar to the following is displayed, some files were deleted but are still in use by some running processes.

     

     • If the migration task can be finished and the target server can be launched, the data unmigrated has no effects on the target server, and this problem can be ignored.
     • (Optional) On the source server, run lsof -n / |grep deleted and stop all the processes returned.
     • (Optional) Restart the source server at an appropriate time to stop the processes.
• Possible cause 2: Some directories were excluded from migration.

  In Linux, the Agent does not migrate files in the following directories by default:

  /proc/*

  /sys/*

  /lost+found/*

  /var/lib/ntp/proc/*

  But these directories are counted in the total volume of source data scanned by the Agent.

  In the root directory of the source server, run du -sh * to check how much space is occupied by directories that were not migrated, such as /proc/*, /sys/*, /lost+found/*, and /var/lib/ntp/proc/*.

  If the total size of these excluded directories equals to the size of unmigrated data, the migration is successful, and this problem can be ignored.

• Possible cause 3: There were data changes on the source server during the migration.

  When a migration starts, the Agent runs the df -TH command to obtain information about directories where each partition is mounted and traverses these directories to calculate the amount of data to be migrated.

  Data that has been migrated is not affected by changes to the source data. However, if data that has not been migrated changes, the most current version is what migrated.

  The amount of data displayed on the SMS console is collected from the source server during the first scan. During the migration, if a large amount of data is changed on the source server, for example, a large amount of data is deleted before being migrated, the amount of data migrated may be less than the total amount of data scanned for the first time.

  You can run df -TH on the source and target servers for comparison.

• Possible cause 4: A large number of sparse files existed on the source server.

  Sparse files are scanned but ignored during the migration. This may lead to the amount of migrated data less than the total amount of data calculated during the first scan.

  

  There are many ways to generate sparse files in Linux, such as running the dd command.