DCS Data Migration Overview

The DCS console supports online and backup (file) migration with intuitive operations. Incremental data can be migrated online.

• Online migration is suitable when the source Redis instance supports the SYNC and PSYNC commands. Data in the source Redis instance can be migrated in full or incrementally to the target instance.

  During online migration, the PSYNC command is delivered to the source address. For details about how this works, see the replication explanation. This command will cause a fork operation at the source end, which affects latency. For details about the impact scope, see the Redis official website.

• Backup migration is suitable when the source and target Redis instances are not connected, and the source Redis instance does not support the SYNC and PSYNC commands. To migrate data, import your backup files to OBS, and DCS will read data from OBS and migrate the data to the target DCS Redis instance. Alternatively, you can import the backup files directly to the DCS instance.

Users can customize migration solutions as required based on specific Redis environment or scenarios. The data volume, source Redis deployment, and network bandwidth affect migration duration. The actual duration depends.

Before migrating an instance, analyze the cache commands (reference: Command Compatibility) used by your service systems and verify the commands one by one during the drill phase. If necessary, contact technical support.

• Currently, the data migration function is free of charge in the OBT. You will be notified when data migration starts to be charged.
• As an important and stringent task, data migration requires high accuracy and timeliness, which depends on specific services and operations.
• Cases provided in this document are for reference only. Consider your needs during actual migration.
• Some commands in this document contain the instance password, which means the passwords are recorded. Ensure that the passwords are not disclosed and clear operation records in a timely manner.
• DCS for Redis 3.0 is no longer provided. You can use DCS for Redis 4.0 or later.

DCS Data Migration Modes

• DCS for Redis refers to Redis instances provided by Huawei Cloud DCS.
• Self-hosted Redis refers to self-hosted Redis on the cloud, from other cloud vendors, or in on-premises data centers.
• √: Supported. ×: Not supported.
• You can migrate data online in full or incrementally from other cloud Redis to DCS for Redis if they are connected and the SYNC and PSYNC commands can be run on the source Redis. However, some instances provided by other cloud vendors may fail to be migrated online. In this case, migrate data through backup import or use other migration schemes. For more information, see Migration Solution Notes.

Migration Process

Figure 1 Migration flowchart

1. Evaluation
   Collect the following information about the cached data to be migrated (based on Table 2):

   • Number of instances
   • Number of databases (DBs) configured for each instance
   • Number of keys in each DB
   • DBs used for your services
   • Space occupied by each instance
   • Redis version
   • Redis instance type
   • Mapping relationships between your services and instances
   

   • The info keyspace command can be used on accessed source Redis to check whether databases have available data and the key quantity in databases. The key quantity in each database can be used for migration verification.
   • The info memory command can be used on accessed source Redis to check the source Redis data volume by the returned used_memory_human value. Check whether the following resources are sufficient: available ECS space, target instance flavor, and remaining memory (≥ source Redis data).
   Plan the following information about target DCS instances based on the collected information:

   • Number of instances to be applied for
   • Memory size of the DCS instance (≥ that of source Redis)
   • Version of the DCS instance (≥ that of source Redis)
   • DCS instance type
   • Virtual Private Clouds (VPCs), security groups, and subnets, and security groups, to which the instances and services belong
2. Preparation
   After completing the evaluation, prepare the following items:

   1. Mobile storage devices

      These devices are used to copy and transfer data in case of network disconnection (in scenarios with data centers of enterprises).

   2. Network resources

      Create VPCs and subnets based on service planning.

   3. Server resources

      Apply for ECSs to bear Redis clients. The ECSs are used to export or import cached data.

      Recommended ECS specifications are 8 vCPUs | 16 GB or higher.

   4. DCS instances

      Create DCS instances based on the migration planning. If the number of instances exceeds the default quota, submit a service ticket or contact technical support.

   5. Related tools

      Install the FTP tool and SSH tool.

   6. Information to be collected

      Collect the contact information of people involved in the migration, ECS login credentials, cache instance information, and DB information.

   7. Overall migration plan

      Formulate the overall migration plan, including the personnel arrangement, drill, migration, verification, service switchover, and rollback solutions.

      Break down each solution into executable operations and set milestones to mark the end of tasks.

3. Drill
   The drill phase aims to:

   1. Verify the feasibility of the migration tools and migration process.
   2. Discover problems that may occur during migration and make effective improvements.
   3. Evaluate the time required for migration.
   4. Optimize the migration steps and verify the feasibility of concurrent implementation of some tasks to improve migration efficiency.
4. Backup

   Before migration, back up related data, including but not limited to cached data and Redis configuration files, in case of emergency.

5. Migration

   After conducting one or two rounds of migration drill and solving problems found in the drill, start data migration.

   Break down the migration process into executable steps with specific start and end confirmation actions.

6. Data verification
   Check the following items:

   • The key distribution of each DB is consistent with the original or expected distribution.
   • Main keys.
   • Expiration time of keys.
   • Whether instances can be normally backed up and restored.
7. Service switchover
   1. After the data migration and verification, use the new instances for your services.
   2. If DB IDs are changed, modify the ID configurations for your services.
   3. If your services are migrated from data centers or cloud platforms provided by other vendors to Huawei Cloud as a whole, services and cached data can be migrated concurrently.
8. Service verification
   1. Verify the connectivity between your service applications and DCS instances.
   2. Verify whether cached data can be normally added, deleted, modified, and queried.
   3. If possible, perform pressure tests to ensure that the performance satisfies the peak service pressure.
9. Rollback

   If your services are unavailable after the data migration because unexpected problems occur and cannot be solved in the short term, roll back your services.

   Since source Redis data still exists, you only need to roll back your services and use the source Redis instances again.

   After the rollback, you can continue to restart from the drill or even preparation phase to solve the problems.