Online Migration

Scenario

If the source and target instances are interconnected and the SYNC and PSYNC commands are supported by the source instance, data can be migrated online in full or incrementally from the source to the target.

If the SYNC and PSYNC commands are disabled on the source Redis instance, enable them before performing online migration. Otherwise, the migration fails. If you use a HUAWEI CLOUD DCS Redis instance for online migration, the SYNC command is automatically enabled.
You cannot use public networks for online migration.
During online migration, you are advised to set repl-timeout on the source instance to 300s and client-output-buffer-limit to 20% of the maximum memory of the instance.

During online migration, results of the FLUSHDB and FLUSHALL commands executed on the source will not be synchronized to the target.

Impacts on Services

During online migration, data is essentially synchronized in full to a new replica. Therefore, perform online migration during low-demand hours.

Prerequisites

Before migrating data, read through Migration Tools and Schemes to learn about the DCS data migration function and select an appropriate target instance.
By default, a Proxy Cluster instance has only one database (DB0). Before you migrate data from a single-node, read/write splitting, or master/standby instance to a Proxy Cluster instance, check whether any data exists on databases other than DB0. If yes, enable multi-DB for the Proxy Cluster instance by referring to Enabling Multi-DB.
By default, a Redis Cluster instance has only one DB (DB0). Before you migrate data from a single-node, read/write splitting, or master/standby instance to a Redis Cluster instance, check whether any data exists on databases other than DB0. If there is any, move all data to DB0 by referring to Online Migration with Rump to ensure that the migration succeeds.

Step 1: Obtain Information About the Source Redis Instance

If the source is a DCS Redis instance in the same VPC as the target Redis, obtain the name of the source Redis instance.
If the source is self-hosted Redis, Redis in another cloud, or a DCS Redis instance in a different VPC from the target Redis, obtain the source IP address or domain name and port number.

Step 2: Prepare the Target DCS Redis Instance

If a target DCS Redis instance is not available, create one first. For details, see Buying a DCS Redis Instance.
If you already have a DCS Redis instance, you do not need to create one again, but you need to clear the instance data before the migration. For details, see Clearing DCS Instance Data.
If the target instance data is not cleared before the migration and the source and target instances contain the same key, the key in the target instance will be overwritten by the key in the source instance after the migration.

Step 3: Check the Network

Check whether the source Redis instance, the target Redis instance, and the migration task are configured with the same VPC.
If yes, go to Step 4: Create an Online Migration Task. If no, go to 2.
Check whether the VPCs configured for the source Redis instance, the target Redis instance, and the migration task are connected to ensure that the VM resource of the migration task can access the source and target Redis instances.
If yes, go to Step 4: Create an Online Migration Task. If no, go to 3.
Perform the following operations to establish the network.
- If the source and target Redis instances are in the same region, create a VPC peering connection by referring to VPC Peering Connection.
- If the source and target Redis instances are on different clouds, create a connection by referring to Direct Connect documentation.

Step 4: Create an Online Migration Task

Log in to the DCS console.
In the navigation pane, choose Data Migration.
Click Create Online Migration Task.
Enter the task name and description.
Configure the VPC, subnet, and security group for the migration task.
The VPC, subnet, and security group facilitate the migration. Ensure that the migration resources can access the source and target Redis instances.
- The migration task uses a tenant IP address (Migration ECS displayed on the Basic Information page of the task.) If a whitelist is configured for the source or target instance, add the migration IP address to the whitelist or disable the whitelist.
- To allow the VM used by the migration task to access the source and target instances, set an outbound rule for the task's security group to allow traffic through the IP addresses and ports of the source and target instances. By default, all outbound traffic is allowed.

Step 5: Configure the Online Migration Task

On the Online Migration tab page, click Configure in the Operation column.

Select a migration type.

Supported migration types are Full and Full + Incremental, which are described in Table 1.

**Table 1** Migration type description
Migration Type	Description
Full	Suitable for scenarios where services can be interrupted. Data is migrated at one time. Source instance data updated during the migration will not be migrated to the target instance.
Full + incremental	Suitable for scenarios requiring minimal service downtime. The incremental migration parses logs to ensure data consistency between the source and target instances. Once the migration starts, it remains Migrating until you click Stop in the Operation column. After the migration is stopped, data in the source instance will not be lost, but data will not be written to the target instance. When the transmission network is stable, the delay of incremental migration is within seconds. The actual delay depends on the transmission quality of the network link.

Figure 1 Selecting the migration type
Click to enlarge

If Migration Type is set to Full + Incremental, you can specify a bandwidth limit.
The data synchronization rate can be kept around the bandwidth limit.
Specify Auto-Reconnect. If this option is enabled, automatic reconnections will be performed indefinitely in the case of a network exception.
Full synchronization will be triggered and requires more bandwidth if incremental synchronization becomes unavailable. Exercise caution when enabling this option.
Configure source Redis and target Redis.
1. The Redis type can be Redis in the cloud or Self-hosted Redis as required.
  - Redis in the cloud: a DCS Redis instance (source or target) that is in the same VPC as the migration task. If you select this option, specify a DCS Redis instance.
  - Self-hosted Redis: a DCS Redis instance, Redis in another cloud, or self-hosted Redis. If you select this option, enter Redis addresses.
  If the source and target Redis instances are connected but are in different regions of HUAWEI CLOUD, you can only select Self-hosted Redis for Target Redis Type and enter the instance addresses, regardless of whether the target Redis instance is self-hosted or in the cloud.
2. If the instance is password-protected, click Test Connection to check whether the instance password is correct and whether the network is connected. If the instance is not password-protected, click Test Connection directly.
Click Next.
Confirm the migration task details and click Submit.
Go back to the data migration task list. After the migration is successful, the task status changes to Successful.
- Once incremental migration starts, it remains Migrating.
- To manually stop migration, click Stop.
- After data migration, duplicate keys will be overwritten.
If the migration fails, click the migration task and check the log on the Migration Logs page.

Verifying the Migration

After the migration is complete, use redis-cli to connect the source and target Redis instances to check data integrity.

Connect to the source Redis and the target Redis.
Run the info keyspace command to check the values of keys and expires.
Calculate the difference between the values of keys and expires of the source Redis and the target Redis. If the differences are the same, the data is complete and the migration is successful.