Help Center/ Distributed Cache Service/ User Guide/ Migrating Instance Data/ Migrating Data from Self-Hosted Redis to DCS/ Self-Hosted Redis Migration with redis-cli (RDB)
Updated on 2024-10-28 GMT+08:00
View PDF
Share

Self-Hosted Redis Migration with redis-cli (RDB)

redis-cli is the command line tool of Redis, which can be used after you install the Redis server. redis-cli supports data export as an RDB file. If your Redis service does not support AOF file export, use redis-cli to obtain an RDB file. Then, use another tool (such as redis-shake) to import the file to a DCS instance.

  • Migrate data during off-peak hours.
  • When the source is Redis native cluster data, individually export the data of each node in the cluster, and then import the data node by node.

Prerequisites

  • If a target DCS Redis instance is not available, create one first. For details, see Buying a DCS Redis Instance.
  • If you already have a DCS Redis instance, you do not need to create one again. For comparing migration data and reserving sufficient memory, you are advised to clear the instance data before the migration. For details, see Clearing DCS Instance Data. If any data exists on the target instance, duplicate data between the source and target is overwritten. If the data exists only on the target instance, the data will be retained.
  • An Elastic Cloud Server (ECS) has been created. For details about how to create an ECS, see Purchasing a Custom ECS.
  • The source self-hosted Redis instance must support the SYNC command. Otherwise, the RDB file cannot be exported using redis-cli.

Exporting the RDB File

  1. Prepare for the export.

    For master/standby or cluster DCS instances, there is a delay in writing data into an RDB file based on the delay policies configured in the redis.conf file. Before data export, learn the RDB policy configurations of the Redis instance to be migrated, suspend your service systems, and then write the required number of test data into the Redis instance. This ensures that the RDB file is newly generated.

    For example, the default RDB policy configurations in the redis.conf file are as follows:

    save 900 1 //Writes changed data into an RDB file if there is any data change within 900s.
save 300 10 //Writes changed data into an RDB file if there are more than 10 data changes within 300s.
save 60 10000 //Writes changed data into an RDB file if there are more than 10,000 data changes within 60s.

    Based on the preceding policy configurations, after stopping your service systems from writing data into the Redis instances, you can write test data to trigger the policies, so that all service data can be synchronized to the RDB file.

    You can delete the test data after data import.

    • If there is any DB not used by your service systems, you can write test data into the DB, and run the flushdb command to clear the database after importing data into DCS.
    • Compared with master/standby instances, single-node instances without data persistence configured require a longer time for export of an RDB file, because the RDB file is temporarily generated.
  2. Log in to the ECS.
  3. Install redis-cli. The following steps assume that your client is installed on the Linux OS.
    1. Run the following command to download Redis: You can also install other Redis versions. For details, see the Redis official website. 
      wget http://download.redis.io/releases/redis-5.0.8.tar.gz
    2. Run the following command to decompress the source code package of your Redis client: 
      tar -xzf redis-5.0.8.tar.gz
    3. Run the following commands to go to the Redis directory and compile the source code of your Redis client: 
      cd redis-5.0.8
make
cd src
  4. Run the following command to export the RDB file: 
    redis-cli -h {source_redis_address} -p {port} -a {password} --rdb {output.rdb}

    {source_redis_address} is the connection address of the source Redis, {port} is the port of the source Redis, {password} is the connection password of the source Redis, and {output.rdb} is the RDB file name.

    If "Transfer finished with success." is displayed after the command is executed, the file is exported successfully.

Uploading the RDB File to Huawei Cloud ECS

To save time, you are advised to compress the RDB file and upload it to Huawei Cloud ECS using an appropriate mode (for example, SFTP mode).

Ensure that the ECS has sufficient disk space for data file decompression, and can communicate with the DCS instance. Generally, the ECS and DCS instance are configured to belong to the same VPC and subnet, and the configured security group rules do not restrict access ports. For details about how to configure a security group, see Security Group Configurations.

Importing Data

Use redis-shake to import data.

It takes 4 to 10 seconds to import an RDB file of 1 million data (20 bytes per data segment) to a VPC.

Verifying the Migration

After the data is imported successfully, access the DCS instance and run the info command to check whether the data has been successfully imported as required. Connect to Redis by referring to redis-cli.

If the import fails, check the procedure. If the import command is incorrect, run the flushall or flushdb command to clear the cache data in the target instance, modify the import command, and try again.