Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Situation Awareness
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive

Migrating Data Online

Updated on 2024-07-29 GMT+08:00

Scenario

If the source and target instances are interconnected and the SYNC and PSYNC commands are supported in the source instance, data can be migrated online in full or incrementally from the source to the target.

CAUTION:
  • If the SYNC and PSYNC commands are disabled on the source Redis instance, enable them before performing online migration. Otherwise, the migration fails. If you use a DCS Redis instance for online migration, the SYNC command is automatically enabled.
  • During online migration, you are advised to set repl-timeout on the source instance to 300s and client-output-buffer-limit to 20% of the maximum memory of the instance.
NOTE:

During online migration, results of the FLUSHDB and FLUSHALL commands executed on the source will not be synchronized to the target.

Impacts on Services

During online migration, data is essentially synchronized in full to a new replica. Therefore, perform online migration during low-demand hours.

Prerequisites

  • Before migrating data, read through Introduction to Migration with DCS to learn about the DCS data migration function and select an appropriate target instance.
  • To migrate data from a single-node or master/standby instance to a cluster instance, check if any data exists in DBs other than DB0 in the source instance. If yes, move the data to DB0 by using the open-source tool Rump. Otherwise, the migration will fail because a cluster instance has only one DB. For details about the migration operations, see Online Migration with Rump.

Step 1: Obtain Information About the Source Redis Instance

  • If the source is a cloud Redis instance, obtain its name.
  • If the source is self-hosted Redis, obtain its IP address or domain name and port number.

Step 2: Prepare the Target DCS Redis Instance

  • If a target DCS Redis instance is not available, create one first. For details, see Creating a DCS Redis Instance.
  • If a target instance is available, you do not need to create a new one. However, you must clear the instance data before the migration. For details, see Clearing DCS Instance Data.

    If the target instance data is not cleared before the migration and the source and target instances contain the same key, the key in the target instance will be overwritten by the key in the source instance after the migration.

Step 3: Check the Network

NOTE:
  • If the source or target of online migration is Redis in the cloud, the selected Redis instance must be in the same VPC as the migration task. Otherwise, the migration task may fail to connect to the cloud Redis instance.
  • In special scenarios, if you have enabled cross-VPC access between the migration task and the cloud Redis instance, the cloud Redis instance and the migration task can be in different VPCs.

Table 1 lists the requirements on the network between the online migration task, source Redis, and target Redis.

Table 1 Requirements on the network between the online migration task, source Redis, and target Redis

Source Redis Type

Target Redis Type

Network Requirement on Online Migration

Redis in the cloud

Redis in the cloud

When creating an online migration task, ensure that the online migration task is in the same VPC as the source and target Redis. If they are not in the same VPC, enable cross-network access between the migration task and the source and target Redis. To enable cross-network access, create a VPC peering connection by referring to section "VPC Peering Connection" in VPC User Guide.

Redis in the cloud

Self-hosted Redis

When creating an online migration task, ensure that the migration task and the source Redis are in the same VPC. Then, enable cross-network access between the migration task and the target Redis.

To enable cross-network access, create a VPC peering connection by referring to section "VPC Peering Connection" in VPC User Guide.

Self-hosted Redis

Redis in the cloud

When creating an online migration task, ensure that the migration task and the target Redis are in the same VPC. Then, enable cross-network access between the migration task and the source Redis.

To enable cross-network access, create a VPC peering connection by referring to section "VPC Peering Connection" in VPC User Guide.

Self-hosted Redis

Self-hosted Redis

After creating an online migration task, enable cross-network access between the migration task and the source and target Redis, respectively.

To enable cross-network access, create a VPC peering connection by referring to section "VPC Peering Connection" in VPC User Guide.

Step 4: Create a Migration Task

  1. Log in to the DCS console.
  2. Click in the upper left corner and select a region and a project.
  3. In the navigation pane, choose Data Migration. The migration task list is displayed.
  4. Click Create Online Migration Task.
  5. Enter the task name and description.
  6. Configure the VPC, subnet, and security group for the migration task.

    The VPC, subnet, and security group facilitate the migration. Ensure that the migration resources can access the source and target Redis instances.

    NOTICE:
    • The migration task uses a tenant IP address (Migration ECS displayed on the Basic Information page of the task.) If a whitelist is configured for the source or target instance, add the migration IP address to the whitelist or disable the whitelist.
    • To allow the VM used by the migration task to access the source and target instances, set an outbound rule for the task's security group to allow traffic through the IP addresses and ports of the source and target instances. By default, all outbound traffic is allowed.

  7. Click Next.
  8. Click Submit.

Configuring the Online Migration Task

  1. On the Online Migration tab page, click Configure in the row containing the online migration task you just created.
  2. Specify Migration Type.

    Supported migration types are Full and Full + incremental, which are described in Table 2.

    Table 2 Migration type description

    Migration Type

    Description

    Full

    Suitable for scenarios where services can be interrupted. Data is migrated at one time. Source instance data updated during the migration will not be migrated to the target instance.

    Full + incremental

    Suitable for scenarios requiring minimal service downtime. The incremental migration parses logs to ensure data consistency between the source and target instances.

    Once incremental migration starts, it remains Migrating until you click Stop in the Operation column. After the migration is stopped, data in the source instance will not be lost, but data will not be written to the target instance. When the transmission network is stable, the delay of incremental migration is within seconds. The actual delay depends on the transmission quality of the network link.

    Figure 1 Selecting the migration type

  3. If Migration Type is set to Full + Incremental, you can specify a bandwidth limit.

    The data synchronization rate can be kept around the bandwidth limit.

  4. Specify Auto-Reconnect.

    If this option is enabled, automatic reconnections will be performed indefinitely in the case of a network exception.

  5. Configure source Redis and target Redis.

    1. The Redis type can be Redis in the cloud or Self-hosted Redis as required.
      • Redis in the cloud: a DCS Redis instance that is in the same VPC as the migration task
      • Self-hosted Redis: self-hosted Redis in another cloud, or in on-premises data centers. If you select this option, enter Redis addresses.
      NOTE:

      If the source and target Redis instances are connected but are in different regions of DCS, you can only select Self-hosted Redis for Target Redis Type and enter the instance addresses, regardless of whether the target Redis instance is self-hosted or in the cloud.

    2. If the instance is password-protected, you can click Test Connection to check whether the instance password is correct and whether the network is connected. If the instance is not password-protected, click Test Connection directly.

  6. Click Next.
  7. Confirm the migration task details and click Submit.

    Go back to the data migration task list. After the migration is successful, the task status changes to Successful.

    NOTE:
    • If the migration type is full+incremental, the migration task status will remain Migrating until you click Stop.
    • After data migration, duplicate keys will be overwritten.

Verifying the Migration

After the migration is complete, use redis-cli to connect the source and target Redis instances to check data integrity.

  1. Connect to the source Redis and the target Redis.
  2. Run the info keyspace command to check the values of keys and expires.

  3. Calculate the differences between the values of keys and expires of the source Redis and the target Redis. If the differences are the same, the data is complete and the migration is successful.

During full migration, source Redis data updated during the migration will not be migrated to the target instance.

We use cookies to improve our site and your experience. By continuing to browse our site you accept our cookie policy. Find out more

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback