Migrating Data Using mongodump and mongorestore

Precautions

• mongodump and mongorestore support only full migration. To ensure data consistency, stop services on the source database and stop writing data to the source database before the migration.
• You are advised to perform the migration during off-peak hours to avoid impacting services.
• The admin and local system databases cannot be migrated.
• Make sure that no service set has been created in the system databases admin and local in the source database. If there is already a service set, migrate them out of the system databases admin and local before migration.
• Before importing data, ensure that the necessary indexes are there on the source database. Delete any unnecessary indexes and create any necessary indexes before migration.
• If you choose to migrate a sharded cluster, you must create a set of shards in the destination database and configure sharding. In addition, indexes must be created before migration.
• If the backup using the mongodump tool fails (for example, an error is reported when the backup progress reaches 97%), you are advised to increase the VM storage space and reserve some redundant space before performing the backup again.
• User rwuser can only operate service database tables. You are advised to specify databases and tables to import and export only service data. Otherwise, the insufficient permission problem may occur during full import and export.

Prerequisites

1. Prepare an ECS or a device that can access GeminiDB Mongo.
   • To connect to a GeminiDB Mongo instance over a private network from an ECS, create and log in to the ECS. For details, see Purchasing an ECS and Logging In to an ECS
   • To connect to an instance through an EIP, you must:
     1. For details on how to bind an EIP, see Binding and Unbinding an EIP.
     2. Ensure that your local device can access the EIP.
2. A migration tool has been installed on the prepared ECS or the device that can access GeminiDB Mongo.

   Install the migration tool. For details, see How Can I Install a MongoDB Client?

   

   • The mongodump and mongorestore tools are part of the MongoDB client installation package.
   • The MongoDB client version must match the instance version. Otherwise, compatibility issues may occur.

Exporting Data

1. Log in to the ECS or the device that can access GeminiDB Mongo API.
2. Back up the source database data using mongodump.
   • Example command for SSL connection:

     ./mongodump --host <DB_HOST> --port <DB_PORT> --authenticationDatabase <AUTH_DB> -u <DB_USER> --ssl --sslCAFile <FILE_PATH> --sslAllowInvalidCertificates --db <DB_NAME> --collection <DB_COLLECTION> --gzip --archive=<Backup_directory>

   • Example command for unencrypted connection:

     ./mongodump --host <DB_HOST> --port <DB_PORT> --authenticationDatabase <AUTH_DB> -u <DB_USER> --db <DB_NAME> --collection <DB_COLLECTION> --gzip --archive=<Backup_directory>

   Table 1 Parameter description

   Parameter	Description
   DB_HOST	Database address
   DB_PORT	Database port
   DB_USER	Database username
   AUTH_DB	Database that stores <DB_USER> information. Generally, the value is admin.
   FILE_PATH	Path for storing the root certificate
   DB_NAME	The name of the database to be migrated.
   DB_COLLECTION	Collection in the database to be migrated
   Backup_directory	Path for storing the exported data. You can customize the path.

   Enter the database administrator password when prompted:

   Enter password:

   Example:

   ./mongodump --host 192.168.xx.xx --port 8635 --authenticationDatabase admin -u rwuser --db test --collection usertable --gzip --archive=/tmp/usertable.tar.gz

   

Importing Data

1. Log in to the ECS or the device that can access GeminiDB Mongo.
2. Upload the data to be imported to the ECS or the device that can access GeminiDB Mongo.

   Select an uploading method based on the OS you are using.

   • In Linux, for example, you can use secure copy protocol (SCP):

     scp -r <IDENTITY_DIR> <REMOTE_USER>@<REMOTE_ADDRESS>:<REMOTE_DIR>

     Table 2 Parameter description

     Parameter	Description
     <IDENTITY_DIR>	Directory where the backup folder is located.
     <REMOTE_USER>	User of the ECS OS
     <REMOTE_ADDRESS>	ECS address
     <REMOTE_DIR>	Directory of the file to be imported

   • In Windows, use a transfer tool to upload the backup file to the ECS.

3. Import the backup data to the GeminiDB Mongo instance.
   • Example command for SSL connection:

     ./mongorestore --host <DB_HOST> --port <DB_PORT> --authenticationDatabase <AUTH_DB> -u <DB_USER> --ssl --sslCAFile <FILE_PATH> --sslAllowInvalidCertificates --db <DB_NAME> --collection <DB_COLLECTION> --gzip --archive=<Backup_directory>

   • Example command for unencrypted connection:

     ./mongorestore --host <DB_HOST> --port <DB_PORT> --authenticationDatabase <AUTH_DB> -u <DB_USER> --db <DB_NAME> --collection <DB_COLLECTION> --gzip --archive=<Backup_directory>

   Table 3 Parameter description

   Parameter	Description
   DB_HOST	Database address
   DB_PORT	Database port
   AUTH_DB	The database that authenticates DB_USER. Generally, the value is admin.
   DB_USER	Account name of the database administrator. The default value is rwuser.
   FILE_PATH	Path for storing the root certificate
   DB_NAME	Name of the database to which data is to be imported
   DB_COLLECTION	Collection in the database to be imported
   Backup_directory	Directory for storing backup files

   Enter the database administrator password when prompted:

   Enter password:

   Example:

   ./mongorestore --host 192.168.xx.xx --port 8635 --authenticationDatabase admin -u rwuser --db test --collection usertable --gzip --archive=/tmp/usertable.tar.gz

   

Related Issues

When you back up the entire instance using mongodump and mongorestore, the permission verification fails.

• Cause

  The rwuser user has limited permissions on the admin and config databases of the instance. As a result, the permission verification fails.

• Solution

  Grant permissions on certain databases and tables to the user.