Self-Hosted Redis Cluster Migration with redis-shake (Online)

Notes and Constraints

• To migrate data from a self-hosted Redis Cluster instance to a DCS Redis Cluster instance online, ensure that the source Redis is connected to the target Redis, or use a transit cloud server to connect the source and target cluster instances.
• To migrate to an instance with SSL enabled, disable the SSL setting first. For details, see Transmitting DCS Redis Data with Encryption Using SSL.

Prerequisites

• If a target DCS Redis instance is not available, create one first. For details, see Buying a DCS Redis Instance.
• If you already have a DCS Redis instance, you do not need to create one again. For comparing migration data and reserving sufficient memory, you are advised to clear the instance data before the migration. For details, see Clearing DCS Instance Data. If any data exists on the target instance, duplicate data between the source and target is overwritten. If the data exists only on the target instance, the data will be retained.

• An Elastic Cloud Server (ECS) has been created. For details about how to create an ECS, see Purchasing a Custom ECS.

  Select the same VPC, subnet, and security group as the DCS Redis Cluster instance, and bind EIPs to the ECS.

• If the source self-hosted Redis Cluster is deployed on cloud servers of another cloud, allow public access to the servers.

Obtaining Information of the Source and Target Redis Nodes

1. Connect to the source and target Redis instances, respectively. Connect to Redis by referring to Connecting to Redis on redis-cli.
2. In online migration of Redis Clusters, the migration must be performed node by node. Run the following command to query the IP addresses and ports of all nodes in both the source and target Redis Clusters.

   redis-cli -h {redis_address} -p {redis_port} -a {redis_password} cluster nodes

   {redis_address} indicates the Redis connection address, {redis_port} indicates the Redis port, and {redis_password} indicates the Redis connection password.

   In the output, obtain the IP addresses and ports of all the master nodes.

   

Configuring the redis-shake Tool

1. Log in to the ECS.
2. Run the following command on the ECS to download the redis-shake: This section uses v4.3.2 as an example. You can also download other redis-shake versions as required.

   wget https://github.com/tair-opensource/RedisShake/releases/download/v4.3.2/redis-shake-v4.3.2-linux-amd64.tar.gz

3. Decompress the redis-shake file.

   mkdir redis-shake-v4.3.2
   tar -C redis-shake-v4.3.2 -xzvf redis-shake-v4.3.2-linux-amd64.tar.gz

   

4. Go to the decompressed directory.

   cd redis-shake-v4.3.2

5. Edit the shake.toml file by providing the following information of both the source and the target.

   vim shake.toml

   The modification is as follows:

   [sync_reader]
   # If the source instance type is a Redis Cluster, set the value to true.
   cluster = true
   # IP address and port of any node in the source Redis Cluster
   address = {redis_ip}:{redis_port}
   # If there is no password, skip the following parameter      
   password = {source_redis_password}
   [redis_writer]
   # If the target instance type is a Redis Cluster, set the value to true.
   cluster = true
   # IP address and port of any node in the target Redis Cluster
   address = {redis_ip}:{redis_port}
   # If there is no password, skip the following parameter      
   password = {target_redis_password}

   Press Esc to exit the editing mode and enter :wq!. Press Enter to save the configuration and exit the editing interface.

Migrating Data Online

./redis-shake shake.toml

If the following information is displayed, the full synchronization has been completed and incremental synchronization begins.

syncing aof

If the following information is displayed, no new data is incremented. You can stop the incremental synchronization by pressing Ctrl+C.

write_ops=[0.00], src-*, syncing aof, diff=[0]

Figure 1 Online migration using redis-shake

Verifying the Migration

1. After the data synchronization, connect to the Redis Cluster DCS instance by referring to Connecting to Redis on redis-cli.
2. Run the info command to check whether the data has been successfully imported as required.

   If the data has not been fully imported, run the flushall or flushdb command to clear the cached data in the target instance, and migrate data again.

3. After the verification is complete, you are advised to clear the redis-shake configuration in time.