Updated on 2024-09-26 GMT+08:00

Real-Time Synchronization

Precautions

The performance indicators provided in this section are for reference only. The actual environment is affected by factors such as the performance of the source or destination database, network bandwidth, data model, and service model.

Specifications

Table 1 Full synchronization maximum performance

Data Flow

Reference Value of Maximum Performance of Full Synchronization (MB/s)

MySQL as the source

50

Oracle as the source

40

Redis as the source

30

GaussDB as the source

40

PostgreSQL as the source

30

DDM as the source

20

MongoDB as the source

20

  • There are many factors that affect the DRS migration speed. The current migration speed is the test data when there is no network and database performance bottlenecks and the task specifications are large. The migration speed is for reference only.
  • When the destination database is Oracle or GaussDB(DWS), the migration speed in the full phase is 30% to 50% lower than that of other types of databases due to the write mechanism of the destination database.
  • The write performance of the MongoDB database is affected by the number of indexes. The write performance decreases by 5% to 8% when there is one index. The more indexes, the slower the write speed.

Based on the incremental performance of data flow types, there are six types of specifications: micro, small, medium, large, ultra-large, and macro. Table 2 lists the performance upper limit of each specification.

Table 2 Performance upper limit

Specifications

Reference Value of Maximum Performance of Incremental Synchronization (Rows/Second)

Micro

300

Small

3,000

Medium

7,500

Large

10,000

Ultra-large

20,000

Macro

> 20,000

  • The performance of each specification is affected by factors such as the networks, source and destination database performance, and latency. The values in the table are for reference only.
  • DRS measures the performance of different specifications using the full (with flow control disabled) and incremental synchronization tasks as the standard.
  • The maximum performance (row/second) is measured by the number of transactions synchronized per second. The statement types include BEGIN, COMMIT, DML (INSERT, DELETE, and UPDATE), and DDL. You can view the destination database write frequency (apply_rows_rate) monitoring metric on Cloud Eye.
  • DRS allows you to upgrade specifications only for single-AZ synchronization tasks. Task specifications cannot be upgraded for dual-AZ tasks or downgraded. For details, see Changing Specifications.
  • If you want to compare values for a DRS task, select large or higher specifications when creating the DRS task.

Testing Models

Create a full+incremental real-time synchronization task for two RDS for MySQL instances. Table 3 shows the instance configurations.

Table 3 Instance specifications

Parameter

Source RDS for MySQL instance

Destination RDS for MySQL instance

Flavor

c6.4xlarge.4 (general-enhanced II)

c6.4xlarge.4 (general-enhanced II)

Instance specifications

Ultra-high I/O

Ultra-high I/O

Storage type

16 vCPUs|64 GB

16 vCPUs|64 GB

Storage space

300 GB

300 GB

Maximum connections

18,000

18,000

Maximum QPS

3,325

3,325

Maximum IOPS

114,152

114,152

Test model:

  • The number of test tables is 20.
  • All test tables have primary keys.
  • The record size is 1 KB.
  • Each transaction contains two DML operations and one COMMIT operation. The ratio of INSERT, UPDATE, and DELETE operations is 1:1:1.

Multiple Specifications

DRS real-time synchronization allows you to select specifications for some specified data flow tasks.

Table 4 Data flow types that support multiple specifications

Synchronization Direction

Data Flow

Multiple Specifications

Specification Upgrade

To the cloud

MySQL->MySQL

Supported

Supported only for single-AZ tasks

To the cloud

MySQL->PostgreSQL

Supported

Supported only for single-AZ tasks

To the cloud

MySQL -> GaussDB Distributed

Supported

Supported only for single-AZ tasks

To the cloud

MySQL -> GaussDB Primary/Standby

Supported

Supported only for single-AZ tasks

To the cloud

MySQL->GaussDB(DWS)

Supported

Supported only for single-AZ tasks

To the cloud

MySQL->GaussDB(for MySQL)

Supported

Supported only for single-AZ tasks

To the cloud

MySQL->MariaDB

Not supported

Not supported

To the cloud

PostgreSQL->PostgreSQL

Supported

Supported only for single-AZ tasks

To the cloud

PostgreSQL->GaussDB(DWS)

Supported

Supported only for single-AZ tasks

To the cloud

PostgreSQL -> GaussDB Primary/Standby

Not supported

Not supported

To the cloud

PostgreSQL -> GaussDB Distributed

Not supported

Not supported

To the cloud

DDM->MySQL

Supported

Supported only for single-AZ tasks

To the cloud

DDM->GaussDB(DWS)

Supported

Supported only for single-AZ tasks

To the cloud

DDM->DDM

Supported

Supported only for single-AZ tasks

To the cloud

Oracle->GaussDB(DWS)

Supported

Supported only for single-AZ tasks

To the cloud

Oracle->PostgreSQL

Supported

Supported only for single-AZ tasks

To the cloud

Oracle->MySQL

Supported

Supported only for single-AZ tasks

To the cloud

Oracle->GaussDB(for MySQL)

Supported

Supported only for single-AZ tasks

To the cloud

Oracle -> GaussDB Primary/Standby

Supported

Supported only for single-AZ tasks

To the cloud

Oracle -> GaussDB Distributed

Supported

Supported only for single-AZ tasks

To the cloud

Oracle->DDM

Supported

Supported only for single-AZ tasks

To the cloud

DB2 for LUW -> GaussDB Primary/Standby

Not supported

Not supported

To the cloud

DB2 for LUW -> GaussDB Distributed

Not supported

Not supported

To the cloud

DB2 for LUW->GaussDB(DWS)

Not supported

Not supported

To the cloud

TiDB->GaussDB(for MySQL)

Not supported

Not supported

To the cloud

Microsoft SQL Server->GaussDB(DWS)

Not supported

Not supported

To the cloud

Microsoft SQL Server -> GaussDB Primary/Standby

Not supported

Not supported

To the cloud

Microsoft SQL Server -> GaussDB Distributed

Not supported

Not supported

To the cloud

Microsoft SQL Server->Microsoft SQL Server

Not supported

Not supported

To the cloud

Microsoft SQL Server->MySQL

Not supported

Not supported

To the cloud

Microsoft SQL Server->GaussDB(for MySQL)

Not supported

Not supported

To the cloud

Microsoft SQL Server->PostgreSQL

Not supported

Not supported

To the cloud

MongoDB->DDS

Supported

Supported only for single-AZ tasks

To the cloud

MariaDB->MariaDB

Supported

Supported only for single-AZ tasks

To the cloud

MariaDB->MySQL

Not supported

Not supported

To the cloud

MariaDB->GaussDB(for MySQL)

Not supported

Not supported

To the cloud

GaussDB(for MySQL)->GaussDB(for MySQL)

Supported

Supported only for single-AZ tasks

To the cloud

Cassandra->GeminiDB Cassandra

Not supported

Not supported

To the cloud

Dynamo->GeminiDB Dynamo

Not supported

Not supported

From the cloud

MySQL->MySQL

Supported

Supported only for single-AZ tasks

From the cloud

MySQL->Kafka

Supported

Supported only for single-AZ tasks

From the cloud

MySQL->CSS/ES

Supported

Supported only for single-AZ tasks

From the cloud

MySQL->Oracle

Supported

Supported only for single-AZ tasks

From the cloud

MySQL->MariaDB

Not supported

Not supported

From the cloud

DDM->MySQL

Supported

Supported only for single-AZ tasks

From the cloud

DDM->Oracle

Supported

Supported only for single-AZ tasks

From the cloud

DDM->Kafka

Supported

Supported only for single-AZ tasks

From the cloud

DDS->MongoDB

Supported

Supported only for single-AZ tasks

From the cloud

DDS->Kafka

Supported

Supported only for single-AZ tasks

From the cloud

PostgreSQL->PostgreSQL

Supported

Supported only for single-AZ tasks

From the cloud

PostgreSQL->Kafka

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB Primary/Standby -> MySQL

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB Primary/Standby -> Oracle

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB Primary/Standby -> Kafka

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB Primary/Standby -> GaussDB(DWS)

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB Primary/Standby -> GaussDB Distributed

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB Primary/Standby -> GaussDB Primary/Standby

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB Distributed -> MySQL

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB Distributed -> Oracle

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB Distributed -> GaussDB(DWS)

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB Distributed -> Kafka

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB Distributed -> GaussDB Distributed

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB Distributed->GaussDB Primary/Standby

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB(for MySQL)->MySQL

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB(for MySQL)->GaussDB(DWS)

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB(for MySQL)->Kafka

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB(for MySQL)->CSS/ES

Supported

Supported only for single-AZ tasks

From the cloud

GaussDB(for MySQL)->Oracle

Supported

Supported only for single-AZ tasks

From the cloud

MariaDB->MariaDB

Supported

Supported only for single-AZ tasks

From the cloud

Microsoft SQL Server->Kafka

Not supported

Not supported

Self-built -> Self-built

MySQL->Kafka

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

MySQL->CSS/ES

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

MySQL -> GaussDB Primary/Standby

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

MySQL -> GaussDB Distributed

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

Oracle->Kafka

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

Oracle -> GaussDB Primary/Standby

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

Oracle -> GaussDB Distributed

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

PostgreSQL->Kafka

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

GaussDB Primary/Standby -> MySQL

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

GaussDB Primary/Standby -> Oracle

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

GaussDB Primary/Standby -> Kafka

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

GaussDB Primary/Standby -> GaussDB Distributed

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

GaussDB Primary/Standby -> GaussDB Primary/Standby

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

GaussDB Distributed -> MySQL

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

GaussDB Distributed -> Oracle

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

GaussDB Distributed -> Kafka

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

GaussDB Distributed->GaussDB Primary/Standby

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

GaussDB Distributed -> GaussDB Distributed

Supported

Supported only for single-AZ tasks

Self-built -> Self-built

DB2 for LUW -> GaussDB Primary/Standby

Not supported

Not supported

Self-built -> Self-built

DB2 for LUW -> GaussDB Distributed

Not supported

Not supported

Self-built -> Self-built

Microsoft SQL Server->Kafka

Not supported

Not supported