DCS-to-DCS Data Synchronization

Constraints

• You need to submit a service ticket to enable the whitelist for DCS professional event stream jobs.
• If the source is a DCS instance:
  • Only single-node, master/standby, and Redis Cluster Redis 5.0 instances are supported.
  • The source and target Redis instances must have the same specifications, type, and storage space.
  • Enable eventLog and disable appendonly (unavailable for single-node Redis instances) for the source and target Redis instances.
  • Disable the client IP pass-through function for the source and target Redis instances.
• If the source is a DCS address:
  • The source must be a single-node, master/standby, or Redis Cluster instance of Redis 4.0, 5.0, or 6.0. The target must be a single-node, master/standby, or Redis Cluster instance of DCS Redis 5.0.
  • If the SYNC and PSYNC commands are disabled by the source instance, enable them before synchronizing data. Otherwise, the synchronization fails.
  • If the source is a DCS Redis instance, set eventlog to no for both the source and target.
  • You are advised to set repl-timeout to 300 seconds and client-output-buffer-limit to 20% of the maximum memory of the source Redis instance.
  • The source and target Redis instances with SSL enabled do not support data synchronization. Disable SSL first.
  • If the source and target Redis instances use passwords containing single quotation marks ('), the synchronization will fail. In this case, change the passwords.
  • Ensure that the target has enough databases to match the source's highest database index that contains data. For instance, if the source databases range from DB0 to DB127 and the highest database with data is DB99, the target should have at least 100 databases.

Prerequisites

• You have created source and target Redis instances.
• Ensure that DCS, VPC, and subnet resources are sufficient. For details about how to create a VPC and subnet, see Creating a VPC.
• Ensure that no command has been written to the target Redis instance.
• Ensure that the source and target Redis instances can communicate with EG.

Procedure

1. Log in to the EG console.
2. In the navigation pane, choose Event Streams > Professional Event Stream Jobs.
3. Click Create Job in the upper right corner. The Basic Settings page is displayed.
4. Set the basic job information.

   Table 1 Basic job configuration parameters

   Parameter	Description
   Cluster	Select a created cluster. If no cluster is created, create one by referring to Professional Event Stream Clusters.
   Job Name	Enter a job name.
   Scenario	Sync is selected by default. Data across sources can be synchronized in real time.
   Description	Enter the description of the job.

5. Click Next: Configure Source and Target. The Configure Source and Target page is displayed.

   Table 2 Parameters for configuring source and target data

   Parameter	Description
   Type	Select the configuration type. Options: DCS instance and DCS address. The target data type is DCS instance by default.
   Region	Select a region.
   Project	Select a project.
   Instance	Select a DCS instance.
   Instance Type	When Type is set to DCS address, this parameter must be set in the source data. Select the instance type. Single-node Master/Standby Redis Cluster
   DCS Addresses	When Type is set to DCS address, this parameter must be set in the source data. Enter one or more DCS addresses.
   Password Protected	When Type is set to DCS address, this parameter must be set in the source data. Select whether to use password. Yes No
   Username	When Type is set to DCS address, this parameter must be set in the source data. Enter a username.
   Password	Enter the password.

6. Click Test Connectivity. After confirming that the instance connectivity of the source and target is normal, click Next: Advanced Settings. The Advanced Settings page is displayed.

   Table 3 Parameters for DCS instance

   Parameter	Description
   Sync Mode	Default: Full+Incremental.
   Limit Sync Rate	Enabled by default.
   Sync Rate (MB/s)	Set the synchronization rate. Enter a value between 1 and 20.
   Use Slave Node	Enabled by default.

   Table 4 Parameters for DCS address

   Parameter	Description
   Sync Mode	Default: Full+Incremental.
   Retry	Select a retry policy. Instantly: Retry immediately after an error is detected. As scheduled: Retry within the specified time window after an error is detected.
   Limit Retry Interval for Connection Failure	This parameter is mandatory when Retry is set to Instantly. Set retry interval for connection failure. If disabled, retries will continue without stopping.
   Retry Interval Upon Connection Failure (Minutes)	This parameter is mandatory when Retry is set to Instantly. Configure the retry interval after a connection failure.
   Limit Retry Interval for Other Failures	This parameter is mandatory when Retry is set to Instantly. Set retry interval for other failures. If disabled, retries will continue without stopping.
   Retry Interval Upon Other Failures (Minutes)	This parameter is mandatory when Retry is set to Instantly. Configure the retry interval after other failures occur.
   Retry Start Time for Connection Failure	This parameter is mandatory when Retry is set to As scheduled. Configure the start time of the retry after the connection fails.
   Retry End Time for Connection Failure	This parameter is mandatory when Retry is set to As scheduled. Configure the end time of the retry after the connection fails.
   Retry Start Time for Other Failures	This parameter is mandatory when Retry is set to As scheduled. Configure the retry start time after other failures occur.
   Retry End Time for Other Failures	This parameter is mandatory when Retry is set to As scheduled. Configure the retry end time after other failures occur.
   Use Slave Node	Enabled by default.

   

   • If the connection between the event stream cluster and the source or target instance fails after the synchronization task (with DCS addresses as the source) is started, the connection can be retried according to the configured policy. If the source or target instance is reconnected within the configured time, the task is automatically resumed. Otherwise, the synchronization task fails.

     The following are some scenarios where the connection may fail:

     1. The source or target instance is powered off, shut down, or restarted.
     2. The network connection between the event stream cluster and the source or target instance is abnormal.
     3. The source or target node or shard is faulty and cannot be accessed.
   • If other failures occur on the source or target instance after the synchronization task (with DCS addresses as the source) is started, the task can be retried according to the configured policy. If the data synchronization is restored within the configured time, the task is automatically resumed. Otherwise, the synchronization task fails.

     The following are some scenarios where non-connection failures may occur:

     1. psync is not enabled on the source.
     2. Data cannot be synchronized due to the source status, for example, the source is restarted and RDB is being loaded.
     3. The source Redis commands fail to be synchronized. For example, the source Redis commands are not supported on the target.
     4. The password of the source or target instance is changed.

7. Click Next: Pre-check. On the displayed page, click Finish.
8. Return to the professional event stream job list and click the name of the created event stream. Click Job Management to view the synchronization details.

   Table 5 Parameter description

   Parameter	Description
   Source Node	Address of the DCS instance in the source data.
   Target Node	Address of the DCS instance in the target data.
   Status	Status of DCS event stream synchronization.
   Sync Progress (%)	DCS event stream synchronization progress.
   Mode	Full, Incremental, and Full+Incremental synchronization modes are supported.
   Source Transactions Executed	Number of source data records.
   Target Transactions Executed	Number of synchronized target data records.