Migrating Data Using mongoexport and mongoimport

Precautions

• mongoexport and mongoimport support only full migration. To ensure data consistency, stop services on the source database and stop writing data to the source database before the migration.
• You are advised to perform the migration during off-peak hours to avoid the impact of migration on your services.
• The admin and local system databases cannot be migrated.
• Make sure that no service set has been created in the system databases admin and local in the source database. If there is already a service set, migrate them out of the system databases admin and local before migration.
• Before importing data, ensure that the necessary indexes are there on the source database. Delete any unnecessary indexes and create any necessary indexes before migration.
• If you choose to migrate a sharded cluster, you must create a set of shards in the destination database and configure sharding. In addition, indexes must be created before migration.

Prerequisites

1. An ECS or a device that can access GeminiDB Mongo is ready for use.
   • To connect to a GeminiDB Mongo instance over a private network from an ECS, create and log in to the ECS. For details, see Purchasing an ECS and Logging In to an ECS
   • To connect to an instance through an EIP, you must:
     1. For details on how to bind an EIP, see Binding and Unbinding an EIP.
     2. Ensure that your local device can access the EIP.
2. A migration tool has been installed on the prepared ECS.

   Install the migration tool. For details, see How Can I Install a MongoDB Client?

   

   The MongoDB client provides the mongoexport and mongoimport tools.

Exporting Data

1. Log in to the ECS or the device that can access GeminiDB Mongo API.
2. Use the mongoexport tool to transfer data from the source database to a .json file.

   An SSL connection is used as an example. If you select an unencrypted connection, delete --ssl --sslAllowInvalidCertificates from the following command.

   ./mongoexport --host <DB_ADDRESS> --port <DB_PORT> --ssl --sslAllowInvalidCertificates --type json --authenticationDatabase <AUTH_DB> -u <DB_USER> --db <DB_NAME> --collection <DB_COLLECTION> --out <DB_PATH>

   Table 1 Parameter description

   Parameter	Description
   DB_ADDRESS	Database address
   DB_PORT	Database port
   AUTH_DB	Database that stores DB_USER information. Generally, the value is admin.
   DB_USER	Database user
   DB_NAME	The name of the database to be migrated.
   DB_COLLECTION	The collection of the database to be migrated.
   DB_PATH	Path of storing the JSON file

   Enter the database administrator password when prompted:

   Enter password:

   The following is an example. After the command is executed, the exportfile.json file will be generated:

   ./mongoexport --host 192.168.1.21 --port 8635 --ssl --sslAllowInvalidCertificates --type json --authenticationDatabase admin -u rwuser --db test02 --collection Test --out /tmp/mongodb/export/exportfile.json

3. Check the results.

   If information similar to the following is displayed, the data is successfully exported. x is the number of exported data records.

   exported x records

4. Compress the exported .json file.

   gzip exportfile.json

   Compressing the file helps reduce the time needed to transmit all the data. The compressed file is exportfile.json.gz.

Importing Data

1. Log in to the ECS or the device that can access GeminiDB Mongo.
2. Upload the data to be imported to the ECS or the device that can access GeminiDB Mongo.

   Select an uploading method based on the OS you are using.

   • In Linux, for example, you can use secure copy protocol (SCP):

     scp <IDENTITY_FILE> <REMOTE_USER>@<REMOTE_ADDRESS>:<REMOTE_DIR>

     • IDENTITY_FILE indicates the directory where the exportfile.json.gz file is located. The file access permission is 600.
     • REMOTE_USER is the ECS OS user.
     • REMOTE_ADDRESS is the ECS address.
     • REMOTE_DIR is the directory of the ECS to which the exportfile.json.gz file is uploaded.
   • In Windows, upload exportfile.json.gz to the ECS using file transfer tools.

3. Decompress the package.

   gzip -d exportfile.json.gz

4. Import the JSON file to the GeminiDB Mongo database.

   An SSL connection is used as an example. If you select an unencrypted connection, delete --ssl --sslAllowInvalidCertificates from the following command.

   ./mongoimport --host <DB_ADDRESS> --port <DB_PORT> --ssl --sslAllowInvalidCertificates --type json --authenticationDatabase <AUTH_DB> -u <DB_USER> --db <DB_NAME> --collection <DB_COLLECTION> --file <DB_PATH>

   Table 2 Parameter description

   Parameter	Description
   DB_ADDRESS	IP address of the GeminiDB Mongo instance.
   DB_PORT	Database port
   AUTH_DB	The database that authenticates DB_USER. Generally, the value is admin.
   DB_USER	Database administrator account
   DB_NAME	Database to be imported
   DB_COLLECTION	Collection in the database to be imported
   DB_PATH	Path of storing the JSON file

   Enter the database administrator password when prompted:

   Enter password:

   Example:

   ./mongoimport --host 192.168.1.21 --port 8635 --ssl --sslAllowInvalidCertificates --type json --authenticationDatabase admin -u rwuser --db test02 --collection Test --file /tmp/mongodb/export/exportfile.json

5. Check the results.

   If information similar to the following is displayed, the data is successfully migrated. x is the number of imported data records.

   imported x records