Before You Start
There are some constraints imposed on DRS to improve the stability and security of data synchronization. Before synchronizing data, ensure that all storage engines meet the requirements.
Tips
You are advised to start the task two to three days in advance and comply with the following operation requirements (Table 1 and Table 17) to ensure that the task is running stably.
- You are advised to run the synchronization task at a specific time point during off-peak hours due to the following reasons:
- Full synchronization increases the query workload of the source database by 50 MB/s and occupies 2 to 4 vCPUs.
- To ensure data consistency, tables to be synchronized without a primary key may be locked for 3s.
- The data being synchronized may be locked by other transactions for a long period of time, resulting in read timeout.
- Due to the inherent characteristics of MySQL, in certain scenarios the performance may be negatively affected. For example, if the CPU resources are insufficient and the storage engine is Tokudb, the read speed on tables may be decreased by 10%.
- Due to slight time difference and continuous operations on data, inconsistent comparison results may be generated, reducing the reliability and validity of the results. To obtain accurate comparison results, you are advised to start data comparison at a specified time point during off-peak hours. If it is needed, select Start at a specified time for Comparison Time.
- If you create a many-to-one synchronization task, see Constraints and Operation Suggestions on Many-to-One Scenario.
- For many-to-one synchronization tasks that involve the synchronization of the same table, DDL operations cannot be performed on source databases. Otherwise, all synchronization tasks fail.
MySQL -> RDS MySQL Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 1 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 2 apply to the entire synchronization process.
- Objects that have dependencies must be synchronized at the same time to avoid synchronization failure. Common dependencies: tables referenced by views, views referenced by views, views and tables referenced by stored procedures/functions/triggers, and tables referenced by primary and foreign keys
- The source database does not support point-in-time recovery (PITR).
- Resumable upload is supported, but data may be repeatedly inserted into a table that does not have a primary key.
- When creating a synchronization task, do not set the destination database to read-only.
- Only MySQL to MySQL synchronization supports many-to-one synchronization. During table-level many-to-one synchronization, tables without primary keys cannot exist in the source database.
- If the source and destination are RDS instances, data synchronization without database mapping is not supported.
- The source database cannot contain tables that have no primary key with the same name as those in the destination database.
- The destination database cannot be restored to a point in time when a full synchronization was being performed.
- If you create many-to-one synchronization tasks, the system automatically creates a parent task to associate multiple synchronization tasks after the tasks are started. The parent task is named in the DRS-Group-Destination DB instance name format.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the synchronization task cannot be created.
Precautions
- A data synchronization task may fail due to the change of the name, account, or port number of the source or destination database. You need to rectify the information and then retry the synchronization task on the DRS console. Generally, you are advised not to modify the preceding information during synchronization.
- If the source or destination database port is changed during data synchronization, the synchronization task fails. You can rectify the fault as follows:
- If the source database port is changed, you need to change the port on the DRS console accordingly, and then retry the task to continue data synchronization.
- If the destination database port is changed, the system automatically changes the port to the correct one. You need to retry the task for synchronization.
Generally, you are advised not to modify the port number during synchronization.
- If the source is an RDS DB instance on other cloud platform, the IP address cannot be changed. If the source database is on an RDS DB instance on the current cloud platform, the system automatically changes the IP address to the correct one. You need to retry the task to continue the synchronization. Therefore, changing the IP address is not recommended.
- Binlogs cannot be forcibly deleted. Otherwise, the synchronization task will fail.
- Data inconsistency may occur when the MyISAM table is modified during a full synchronization.
- DDL operations are not supported during full synchronization.
- Table-level incremental synchronization supports only DDL operations on tables.
- During an incremental synchronization of table-level objects, you are not advised to rename the tables.
- You are advised to set the expire_log_day parameter to a proper value to ensure that the binlog does not expire before data transfer resumes. This ensures that services can be recovered after interruption.
- DRS automatically checks the configurations and provides handling suggestions. Ensure that the environment configuration meets the following requirements.
Table 2 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- The source database account must have the following permissions: SELECT, SHOW VIEW, EVENT, LOCK TABLES, REPLICATION SLAVE, and REPLICATION CLIENT.
- The destination database account must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, and UPDATE. The root account of the RDS MySQL DB instance has the preceding permissions by default.
Synchronization object
- Only tables, primary key indexes, unique indexes, common indexes, store procedures, views, and functions can be synchronized.
- During database mapping, the source database cannot contain stored procedures, views, or functions.
- The database for mapping cannot contain objects except tables and these objects cannot be created during synchronization. Otherwise, the synchronization task fails.
- Tables with storage engine different to MyISAM and InnoDB cannot be synchronized.
Source database
- The binlog of the MySQL source database must be enabled and use the row-based format.
- If the storage space is sufficient, you are advised to store the source database binlog for as long as possible. The recommended retention period is three days.
- If the expire_logs_days value of the source database is set to 0, the synchronization may fail.
- During an incremental synchronization, the server_id value of the MySQL source database must be set. If the source database version is MySQL 5.6 or earlier, the server_id value ranges from 2 to 4294967296. If the source database is MySQL 5.7 or later, the server_id value ranges from 1 to 4294967296.
- The database name in the source database cannot contain non-ASCII characters, or the following characters: '<`>/\
- The table name and view name in the source database cannot contain non-ASCII characters, or the following characters: '<>/\
- The source database name and mapped database name cannot be ib_logfile.
- Tables with storage engine different to MyISAM and InnoDB cannot be synchronized to RDS.
- Database mapping does not support views or stored procedures. If the source database contains views or stored procedures, the synchronization may fail.
Destination database
- The destination DB instance must be available. If the destination DB instance type is primary/standby, the replication status must also be normal.
- The destination DB instance must have sufficient storage space.
- If the destination database (excluding MySQL system database) has the same name as the source database, the table structures in the destination database must be consistent with those in the source database.
- The character set of the destination database must be the same as that of the source database.
- The time zone of the destination database must be the same as that of the source database.
- During a synchronization, a large amount of data is written to the destination database. If the value of the max_allowed_packet parameter of the destination database is too small, data cannot be written. You are advised to set the max_allowed_packet parameter to a value greater than 100 MB.
- If the MyISAM tables are included in the synchronization objects, the sql_mode parameter in the destination database cannot contain the no_engine_substitution parameter. Otherwise, the synchronization fails.
- The following characters are not supported in the database names mapped to the destination database: dots (.), angle brackets (<>), backslash (\), and single quotation marks (')
MySQL -> RDS PostgreSQL Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 3 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 4 apply to the entire synchronization process.
- Associated objects must be synchronized at the same time to avoid synchronization failure due to missing associated objects. Common associations: tables or views referenced by views
- If the network is reconnected within 30 seconds, data synchronization will not be affected. If the network is interrupted for more than 30 seconds, the synchronization task will fail.
- Data can be synchronized from multiple databases to one database in mapping mode. Tables with the same name cannot exist in mapped databases. Currently, only users in the whitelist can use this function. To use this function, submit a service ticket. In the upper right corner of the management console, choose Service Tickets > Create Service Ticket to submit a service ticket.
- The source database cannot be restored.
- If a table does not have a primary key to uniquely identify every row and the network connection is unstable, data in the destination database may be inconsistent with that in the source database after synchronization.
- After different types of indexes are synchronized to the destination database, the index type changes to B-Tree.
- If the character sets of the source and destination databases are different, data may be inconsistent or synchronization may fail.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the synchronization task cannot be created.
- If the data types are incompatible, the synchronization may fail.
- Only data that violates the non-null constraint and data of the char or varchar type that exceeds the field length limit can be recorded.
Precautions
- If you change the port of the RDS source database and the synchronization task fails, you need to retry the task.
- If the source is not an RDS DB instance, the port cannot be changed.
- The IP address, account, and password cannot be changed.
- Binlogs cannot be forcibly deleted. Otherwise, the synchronization task will fail.
- Object names cannot be changed during the synchronization.
- DROP operations are not supported during synchronization.
- Incremental synchronization only supports the following DDL operations:
- Syntax for creating tables: CREATE TABLE
- Single-column syntax: ALTER TABLE [ADD| DROP|MODIFY|CHANGE] COLUMN
- Single-index syntax: ALTER TABLE [ADD| DROP|RENAME] [INDEX|UNIQUE]
- Single-constraint syntax: ALTER TABLE ADD CONSTRAINT [PRIMARY KEY|UNIQUE|FOREIGN KEY|]
- Syntax for creating a table view: CREATE VIEW
- Syntax for creating a single index: CREATE [INDEX|UNIQUE|UNIQUE CONSTRAINT].
- During an incremental synchronization of table-level objects, you are not advised to rename the tables.
- You are advised to set the expire_log_day parameter to a proper value to ensure that the binlog does not expire before data transfer resumes. This ensures that services can be recovered after interruption.
- DRS automatically checks the configurations and provides handling suggestions. Ensure that the environment configuration meets the following requirements.
Table 4 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- The source database account must have the following permissions: SELECT, SHOW VIEW, EVENT, LOCK TABLES, REPLICATION SLAVE, and REPLICATION CLIENT.
- The destination database account must have the following permissions: INSERT, SELECT, UPDATE, and DELETE. The root account of the RDS PostgreSQL DB instance has the preceding permissions by default.
Synchronization object
- Only tables, views, indexes, and constraints (primary keys, unique keys, null, not null) can be synchronized. Foreign keys, stored procedures, triggers, functions, events, and virtual columns cannot be synchronized.
- MySQL views support syntax "as select..." "from a join b where ..." but PostgreSQL does not, which may lead to the failure of synchronization.
- The following data types are not supported: XML, geometry, point, lineString, polygon, geometrycollection, multipoint, multilinestring, and multipolygon.
- Tables with storage engine different to MyISAM and InnoDB cannot be synchronized.
Source database
- The binlog of the MySQL source database must be enabled and use the row-based format.
- If the storage space is sufficient, you are advised to store the source database binlog for as long as possible. The recommended retention period is three days.
- If the expire_logs_days value of the source database is set to 0, the synchronization may fail.
- The MySQL source database server-id must be set to a value from 2 to 4294967296.
- The source database name cannot contain Chinese, non-ASCII characters, or the following characters: '.<>
- The table name and view name in the source database cannot contain non-ASCII characters, or the following characters: '.<>
Destination database
- The destination DB instance must be available. If the destination RDS DB instance type is primary/standby, the replication status must also be normal.
- The destination DB instance must have sufficient storage space.
- The time zone of the destination database must be the same as that of the source database.
PostgreSQL -> RDS PostgreSQL Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 5 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 6 apply to the entire synchronization process.
- Associated objects must be synchronized at the same time to avoid synchronization failure due to missing associated objects. Common associations: tables referenced by views, views and tables referenced by stored procedures/functions/triggers, and tables referenced by primary and foreign keys
- Only VPC, VPN, and Direct Connect networks are supported.
- In RDS for PostgreSQL synchronization, only RDS PostgreSQL Enhanced Edition is supported.
- The tables and table structures the same as those in the source database are automatically created in the destination database.
- The table to be synchronized must contain a primary key.
- The unlogged table in the source database cannot be synchronized to the destination database during incremental synchronization.
- If the primary key and unique key are created in the same column, only the primary key is synchronized.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the synchronization task cannot be created.
Precautions
- DDL of the source database cannot be replicated during incremental synchronization. DDL operations such as adding tables, dropping tables, altering table names, adding columns, and altering column types in the source database, will not be synchronized to the destination database, and data in related tables cannot be synchronized to the destination database.
- DRS automatically checks the configurations and provides handling suggestions. Ensure that the environment configuration meets the following requirements.
Table 6 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- The source database account must have the REPLICATION permission.
- The destination database account must have the CREATEDB permission.
Synchronization object
- Only tables and indexes can be synchronized.
- Temporary tables in the source database cannot be synchronized.
- Only the following fields are supported: int, serial, char, float, varchar, numeric, text, timestamp, real, bytea, boolean, smaillint, bigint, double precision, decimal, character varying, bytea, bit, timestamp, timestamptz, date, and time.
Source database
- The source database name cannot contain the following characters: +%"<>'\
- The source database name and table name cannot contain single quotation marks ('), periods (.), and quotation marks (").
- The source database does not support synchronization of tables that do not have primary keys or unique indexes.
Destination database
- The destination DB instance must be available. If the destination RDS DB instance type is primary/standby, the replication status must also be normal.
- The source and destination PostgreSQL databases (except system databases) cannot have the same names.
- The destination DB instance must have sufficient storage space.
- The character set of the destination database must be the same as that of the source database.
- The time zone of the destination database must be the same as that of the source database.
MySQL -> GaussDB(openGauss) Synchronization
- In this scenario, DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 7 Operation constraints Type
Operation Constraints
Notes
- Environment requirements apply to the entire synchronization process.
- Data synchronization across VPCs or through public networks is not supported.
- If the source database contains a duplicate primary key, the data synchronized to the destination database will be less than that in the source database. Therefore, you must check and correct the data before starting the synchronization task.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the synchronization task cannot be created.
- If the data types are incompatible, the synchronization may fail. (In this case, you may need to manually create a table in the destination database.)
- After a table in the source database is synchronized to the destination database, the table is distributed in hash mode and cannot be replicated.
Precautions
- The changes to the source database cannot be synchronized to the destination database in multiple tasks at the same time. Otherwise, data inconsistency and synchronization failure may occur.
- During an incremental synchronization, do not add, modify, or delete the username, password, and permission of the account for connecting the source and destination databases or modify the port of the source and destination databases.
- During an incremental synchronization, the source database DDL cannot be replicated. If you need to modify the source table structure to be synchronized, you must modify the table structure in the destination database. Otherwise, DDL-related data cannot be synchronized to the destination database.
- During an incremental synchronization, do not perform the restoration operation on the source database.
- The table without a primary key cannot be synchronized. If the table to be synchronized contains a table without a primary key, the synchronization fails.
- Two-phase commit is not supported.
- The ongoing synchronization task cannot be paused.
- The synchronization task in progress cannot be edited.
- Synchronization results cannot be compared.
- Data processing is not supported.
- Database name mapping, table name mapping, and column name mapping cannot be configured on the GUI.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
DDM -> RDS MySQL Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 9 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 10 apply to the entire synchronization process.
- Resumable upload is supported, but data may be repeatedly inserted into a table that does not have a primary key when the server system breaks down.
- Do not set the destination database to read-only in the synchronization task.
- Data synchronization across VPCs is not supported.
- If the source database contains a duplicate primary key or unique key, the data synchronized to the destination database will be less than that in the source database. Therefore, you must check and correct the data before starting the synchronization task.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the synchronization task cannot be created.
- If the data types are incompatible, the synchronization may fail.
Precautions
- During synchronization, do not modify or delete the username, password, and permission of the account for connecting to the source and destination databases or modify the port of the source and destination databases.
- During an incremental synchronization, do not modify the table structure to be synchronized in the source database.
- During an incremental synchronization, do not perform the restoration operation on the source database.
- DDL operations are not supported during synchronization.
- DRS automatically checks the configurations and provides handling suggestions. Ensure that the environment configuration meets the following requirements.
Table 10 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- The source sharded database account must have the following permissions: SELECT, SHOW VIEW, EVENT, LOCK TABLES, REPLICATION SLAVE, and REPLICATION CLIENT.
- The destination database account must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, and UPDATE. The root account of the RDS MySQL DB instance has the preceding permissions by default.
Synchronization object
- Only the source database data can be synchronized.
- In the destination database, you need to create table structures and indexes that correspond to the source schemas. Objects that are not created in the destination database are not to be synchronized.
- The table structure created in the destination database must be the same as that in the source database.
- The table cannot contain sharding keys of the timestamp type in the source database.
Source database
- The binlog of the MySQL source database must be enabled and use the row-based format.
- If the storage space is sufficient, you are advised to store the source database binlog for as long as possible. The recommended retention period is three days.
- During an incremental synchronization, the server_id value of the MySQL source database must be set. If the source database version is MySQL 5.6 or earlier, the server_id value ranges from 2 to 4294967296. If the source database is MySQL 5.7 or later, the server_id value ranges from 1 to 4294967296.
- The database names and table names of the source database sharding middleware cannot contain the following characters: '<>/\ and non-ASCII characters.
- You are advised to enable skip-name-resolve for the MySQL source database to reduce the possibility of connection timeout.
- You are advised to enable GTID on the source database.
Destination database
- The destination DB instance must be available. If the destination DB instance type is primary/standby, the replication status must also be normal.
- The destination DB instance must have sufficient storage space.
- If the destination database (excluding MySQL system database) has the same name as the source database, the table structures in the destination database must be consistent with those in the source database.
- The character set of the destination database must be the same as that of the source database.
- The time zone of the destination database must be the same as that of the source database.
- During a synchronization, a large amount of data is written to the destination database. If the value of the max_allowed_packet parameter of the destination database is too small, data cannot be written. You are advised to set the max_allowed_packet parameter to a value greater than 100 MB.
- Before synchronization, you need to create empty databases and tables in the destination instance.
Oracle Database -> RDS MySQL Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 11 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 12 apply to the entire synchronization process.
- Associated objects must be synchronized at the same time to avoid synchronization failure due to missing associated objects. Common association: tables referenced by primary or foreign keys
- After tables are synchronized to the destination database, their names are converted into lowercase letters. For example, the name of table ABC is changed to abc after being synchronized to the destination database. In incremental synchronization, the source database cannot contain tables with the same name but different letter cases. Otherwise, the synchronization will fail.
- The time zone settings of the source and destination database must be the same.
- If there are special characters such as Chinese and Japanese, the code used by the service to connect to the Oracle database must be the same as the code of the Oracle server. Otherwise, garbled characters are displayed in the destination database.
- After the Oracle table structure is synchronized to the MySQL database, the character set of the table is utf8mb4.
- If a table does not have a primary key to uniquely identify every row and the network connection is unstable, data in the destination database may be inconsistent with that in the source database after synchronization.
- If the length of a table structure in the Oracle database exceeds 65535, the synchronization may fail. The length of a table structure is the total length of all columns. The length of the char or varchar2 type is related to the code.
- If the source database is an RAC database and uses SCNA, the synchronization instance must be able to connect to the virtual IP addresses of all RAC nodes. Otherwise, the connection check fails. If SCAN IP is not used, the virtual IP address of a node can be used. If other nodes are abnormal, the synchronization process is not affected.
- For an Oracle RAC cluster, you are advised to use the scan IP address and service name to create a task. The SCAN IP address can provide better fault tolerance, load capability, and synchronization experience.
- If the data types are incompatible, the synchronization may fail.
- Incremental synchronization does not support Oracle 12c PDB databases.
- There are some syntax differences between Oracle and MySQL, so the syntax including but not limited to functions, expressions, and referenced system tables, may not be completely converted during the structure synchronization.
To ensure the success of synchronization, you need to manually create the table structure in the destination database.
Precautions
- During synchronization, writing data to the destination databases is not allowed. Otherwise, data inconsistency may occur.
- During synchronization, do not add, modify, or delete the username, password, and permission of the account for connecting the source and destination databases or modify the port of the source and destination databases.
- During synchronization, DDL operations cannot be performed on the source database.
- During an incremental synchronization of table-level objects, you are not advised to rename the tables.
- The names of mapped databases and tables are case-insensitive, which means no matter if the object name is uppercase or lowercase, it stays lowercase after the object is synchronized to the destination database.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Table 12 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- Source database: The source database must have the following permissions and roles: CREATE SESSION, SELECT ANY TRANSACTION, SELECT ANY TABLE, SELECT ANY DICTIONARY, and EXECUTE_CATALOG_ROLE. If the Oracle database version is 12c or later, the LOGMINING permission is required.
- The destination database account must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, UPDATE, ALTER, INDEX, EVENT, RELOAD, CREATE VIEW, CREATE ROUTINE, and TRIGGER.
Synchronization object
- Databases, table structures, primary keys, unique keys, foreign keys, normal indexes, and table data can be synchronized. Other database objects, such as stored procedures, triggers, functions, sequences, packages, synonyms, and users, cannot be synchronized.
- DDL operations cannot be synchronized during incremental synchronization.
- Full synchronization does not support the following column types: bfile, xml, sdo_geometry, urowid, and self-defined types.
- Incremental synchronization does not support the following column types: bfile, xml, interval, sdo_geometry, urowid, and self-defined types.
- Partitions in the table structure cannot be synchronized. Partition tables are changed to non-partition tables after being synchronized to the destination database.
Source database
- The maximum row length of Oracle cannot exceed 8 KB, excluding BLOB and TEXT columns because the MySQL InnoDB restricts the row length to 8 KB.
- The primary key or unique key column cannot contain values of string data types when you map the MySQL data types to the character data types in Oracle because MySQL cannot tell spaces in data. Otherwise, data inconsistency and deadlock may occur.
- The values of binary_float and binary_double cannot be set to Nan, Inf, or -Inf because MySQL does not support these values.
- MySQL does not support the synchronization of the check constraints of Oracle.
- AUTO_PK_ROW_ID cannot be used as a column name in Oracle because it is a reserved column name in MySQL 5.7 and cannot be created.
- Ensure that the precision of the number(p, s) field in the Oracle database does not exceed the precision range p: [1, 38], s:[p-65, min(p, 30)]. The value of s depends on the value of p. The lower limit is p-65, and the upper limit is the minimum value of p or 30. For example, when p is 1, the value range of s is [-64, 1]. When p is 38, the value range of s is [-27, 30].
The value of the int field cannot exceed the precision range of (65, 0). The digit range of MySQL is smaller than that of Oracle.
- The names of databases and tables cannot contain non-ASCII characters and the following special characters: .><\`|,?'!"
- The source database name cannot be ib_logfile.
- The default value of the unsupported field is an expression, for example, default(`id` + 1).
- If a function is used as the default value in the source database, the function must also be supported in the destination database.
- During an incremental synchronization from Oracle to MySQL, archive logs must be enabled on the source database.
- The source database cannot contain empty databases.
- The total index length of columns in the source database cannot exceed the length limit in the destination database. For detailed length requirements, see Index Length Description.
- The Default User statement is not supported in MySQL.
- The supplemental log level is ALL or primary key (PK) and unique index (UI).
- If the source database is an Oracle Real Application Cluster (RAC) database, you cannot add or delete nodes.
- If the source database is an RAC database and uses SCNA, the synchronization instance must be able to connect to the virtual IP addresses of all RAC nodes. Otherwise, the connection check fails.
- Only the following character sets are supported: ZHS16GBK, AL32UTF8, UTF8, US7ASCII, and WE8MSWIN1252.
Destination database
- The destination database cannot contain the database to be synchronized.
- During a synchronization, a large amount of data is written to the destination database. If the value of the max_allowed_packet parameter of the destination database is too small, data cannot be written. You are advised to set the max_allowed_packet parameter to a value greater than 100 MB.
- The time zone of the destination database must be the same as that of the source database.
- The storage space of the destination database should be about 2.5 times that of the source database.
- If the destination database version is earlier than 5.7.7, the index column length cannot exceed 767 bytes. If the destination database version is later than 5.7.7, the length cannot exceed 3072 bytes.
Oracle -> MRS Kafka Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 13 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 14 apply to the entire synchronization process.
- If there are special characters such as Chinese and Japanese, the code used by the service to connect to the Oracle database must be the same as the code of the Oracle server. Otherwise, garbled characters are displayed in the destination database.
- After data in the Oracle database is synchronized to Kafka, the character set becomes utf8.
- For an Oracle RAC cluster, you are advised to use the scan IP address and service name to create a task. The SCAN IP address can provide better fault tolerance, load capability, and synchronization experience.
- Incremental synchronization does not support Oracle 12c PDB databases.
- The columns that are not recorded in logs are represented by special values. The scenarios include but are not limited to:
- The supplemental log level of the source database is PK UI, and the operation types include insert, delete, and update.
- The supplemental log level of the source database is ALL, the last two or more consecutive columns are not updated and are null, and the operation types are insert, delete, and update.
Precautions
- During synchronization, do not delete the username, password, and permission of the account for connecting to the source and destination databases or modify the port of the destination databases.
- During an incremental synchronization of table-level objects, you are not advised to rename the tables.
- Only table-level DDL operations except the DROP COLUMN statement are supported.
- Index organized tables cannot be synchronized.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Oracle -> GaussDB(for MySQL) Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 15 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 16 apply to the entire synchronization process.
- Associated objects must be synchronized at the same time to avoid synchronization failure due to missing associated objects. Common association: tables referenced by primary or foreign keys
- After object such as tables and views are synchronized to the destination database, their names are converted into lowercase letters. For example, the name of object ABC is changed to abc after being synchronized to the destination database. In incremental synchronization, the source database cannot contain tables with the same name but different letter cases. Otherwise, the synchronization will fail.
- The time zone settings of the source and destination database must be the same.
- If there are special characters such as Chinese and Japanese, the code used by the service to connect to the Oracle database must be the same as the code of the Oracle server. Otherwise, garbled characters are displayed in the destination database.
- After the Oracle table structure is synchronized to the MySQL database, the character set of the table is utf8mb4.
- If a table does not have a primary key to uniquely identify every row and the network connection is unstable, data in the destination database may be inconsistent with that in the source database after synchronization.
- If the length of a table structure in the Oracle database exceeds 65535, the synchronization may fail. The length of a table structure is the total length of all columns. The length of the char or varchar2 type is related to the code.
- If the source database is an RAC database and uses SCNA, the synchronization instance must be able to connect to the virtual IP addresses of all RAC nodes. Otherwise, the connection check fails. If SCAN IP is not used, the virtual IP address of a node can be used. If other nodes are abnormal, the synchronization process is not affected.
- For an Oracle RAC cluster, you are advised to use the scan IP address and service name to create a task. The SCAN IP address can provide better fault tolerance, load capability, and synchronization experience.
- If the data types are incompatible, the synchronization may fail.
- Incremental synchronization does not support Oracle 12c PDB databases.
Precautions
- During synchronization, writing data to the destination databases is not allowed. Otherwise, data inconsistency may occur.
- During synchronization, do not add, modify, or delete the username, password, and permission of the account for connecting the source and destination databases or modify the port of the source and destination databases.
- During synchronization, DDL operations cannot be performed on the source database.
- During an incremental synchronization of table-level objects, you are not advised to rename the tables.
- The names of mapped databases and tables are case-insensitive, which means no matter if the object name is uppercase or lowercase, it stays lowercase after the object is synchronized to the destination database.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Table 16 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- Source database: The source database must have the following permissions and roles: CREATE SESSION, SELECT ANY TRANSACTION, SELECT ANY TABLE, SELECT ANY DICTIONARY, and EXECUTE_CATALOG_ROLE. If the Oracle database version is 12c or later, the LOGMINING permission is required.
- The destination database account must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, UPDATE, ALTER, INDEX, EVENT, RELOAD, CREATE VIEW, CREATE ROUTINE, and TRIGGER.
Synchronization object
- Databases, table structures, primary keys, unique keys, foreign keys, normal indexes, and table data can be synchronized. Other database objects, such as stored procedures, triggers, functions, sequences, packages, synonyms, and users, cannot be synchronized.
- DDL operations cannot be synchronized during incremental synchronization.
- Full synchronization does not support the following column types: bfile, xml, sdo_geometry, urowid, and self-defined types.
- Incremental synchronization does not support the following column types: bfile, xml, interval, sdo_geometry, urowid, and self-defined types.
Source database
- The maximum row length of Oracle cannot exceed 8 KB, excluding BLOB and TEXT columns because the MySQL InnoDB restricts the row length to 8 KB.
- The primary key or unique key column cannot contain values of string data types when you map the MySQL data types to the character data types in Oracle because MySQL cannot tell spaces in data. Otherwise, data inconsistency and deadlock may occur.
- The values of binary_float and binary_double cannot be set to Nan, Inf, or -Inf because MySQL does not support these values.
- MySQL does not support the synchronization of the check constraints of Oracle.
- AUTO_PK_ROW_ID cannot be used as a column name in Oracle because it is a reserved column name in MySQL 5.7 and cannot be created.
- Ensure that the precision of the number(p, s) field in the Oracle database does not exceed the precision range p: [1, 38], s:[p-65, min(p, 30)]. The value of s depends on the value of p. The lower limit is p-65, and the upper limit is the minimum value of p or 30. For example, when p is 1, the value range of s is [-64, 1]. When p is 38, the value range of s is [-27, 30].
The value of the int field cannot exceed the precision range of (65, 0). The digit range of MySQL is smaller than that of Oracle.
- The names of databases and tables cannot contain non-ASCII characters and the following special characters: .><\`|,?'!"
- The source database name cannot be ib_logfile.
- The default value of the unsupported field is an expression, for example, default(`id` + 1).
- If a function is used as the default value in the source database, the function must also be supported in the destination database.
- During an incremental-plus-full synchronization from Oracle to GaussDB(for MySQL), archive logs must be enabled on the source database.
- The source database cannot contain empty databases.
- The total index length of columns in the source database cannot exceed the length limit in the destination database. For detailed length requirements, see Index Length Description.
- The Default User statement is not supported in MySQL.
- Only the following character sets are supported: ZHS16GBK, AL32UTF8, UTF8, US7ASCII, and WE8MSWIN1252.
- The supplemental log level is ALL or primary key (PK) and unique index (UI).
- If the source database is an Oracle Real Application Cluster (RAC) database, you cannot add or delete nodes.
- If the source database is an RAC database and uses SCNA, the synchronization instance must be able to connect to the virtual IP addresses of all RAC nodes. Otherwise, the connection check fails.
Destination database
- The destination database cannot contain the database to be synchronized.
- During a synchronization, a large amount of data is written to the destination database. If the value of the max_allowed_packet parameter of the destination database is too small, data cannot be written. You are advised to set the max_allowed_packet parameter to a value greater than 100 MB.
RDS MySQL -> MySQL Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
RDS MySQL -> Kafka Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 18 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 19 apply to the entire synchronization process.
- The source database does not support point-in-time recovery (PITR).
- Resumable upload is supported, but data may be repeatedly inserted into a table that does not have a primary key.
- When creating a synchronization task, do not set the destination database to read-only.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the synchronization task cannot be created.
- If the data types are incompatible, the synchronization may fail.
Precautions
- A synchronization task may fail due to the change of the username and password of the source database. You need to rectify the information and then retry the synchronization task on the DRS console. Generally, you are advised not to modify the preceding information during synchronization.
- If the source database port is changed during synchronization, the synchronization task fails. If the destination database port is changed, the system automatically changes the port to the correct one. You need to retry the task for synchronization. Generally, you are advised not to modify the port number during synchronization.
- During data synchronization, if the synchronization task fails due to the change of the IP address, the system automatically updates the IP address to the correct one. You need to retry the task to continue the synchronization. Therefore, changing the IP address is not recommended.
- Binlogs cannot be forcibly deleted. Otherwise, the synchronization task will fail.
- Data inconsistency may occur when the MyISAM table is modified during synchronization.
- During synchronization of table-level objects, you are not advised to rename the tables.
- You are advised to set the expire_log_day parameter to a proper value to ensure that the binlog does not expire before data transfer resumes. This ensures that services can be recovered after interruption.
- SSL connection is not supported.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Table 19 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- The source database account must have the following permissions: SELECT, LOCK TABLES, REPLICATION SLAVE, and REPLICATION CLIENT.
Synchronization object
- The table data can be synchronized.
- Tables with storage engine different to MyISAM and InnoDB cannot be synchronized.
Source database
- The binlog of the MySQL source database must be enabled and use the row-based format.
- If the storage space is sufficient, you are advised to store the source database binlog for as long as possible. The recommended retention period is three days.
- If the expire_logs_days value of the source database is set to 0, the synchronization may fail.
- During an incremental synchronization, the server_id value of the MySQL source database must be set. If the source database version is MySQL 5.6 or earlier, the server_id value ranges from 2 to 4294967296. If the source database is MySQL 5.7 or later, the server_id value ranges from 1 to 4294967296.
- The database and table names in the source database cannot contain non-ASCII characters, or the following characters: '<`>/\
- The source database name cannot be ib_logfile.
Destination database
The isolation.level parameter is set to read_committed during consumption.
DDM -> Oracle Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 20 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 21 apply to the entire synchronization process.
- If the source database contains a duplicate primary key, the data synchronized to the destination database will be less than that in the source database. Therefore, you must check and correct the data before starting the synchronization task.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the synchronization task cannot be created.
- If the data types are incompatible, the synchronization may fail.
Precautions
- During an incremental synchronization, do not add, modify, or delete the username, password, and permission of the account for connecting the source and destination databases or modify the port of the source and destination databases.
- During an incremental synchronization, if you need to modify the structure of the table to be synchronized in the source database, you must modify the table structure in the destination database.
- During an incremental synchronization, do not perform the restoration operation on the source database.
- DDL operations are not supported during synchronization.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Table 21 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- The source sharded database account must have the following permissions: SELECT, SHOW VIEW, EVENT, LOCK TABLES, REPLICATION SLAVE, and REPLICATION CLIENT.
- The destination database account must have the following permissions on each table: ALTER ANY INDEX, ALTER ANY TABLE, ALTER SESSION, COMMENT ANY TABLE, CREATE ANY INDEX, CREATE ANY TABLE, CREATE SESSION, DELETE ANY TABLE, DROP ANY TABLE, INSERT ANY TABLE, SELECT ANY TABLE, SELECT ANY DICTIONARY, SELECT ANY TRANSACTION, UPDATE ANY TABLE, ANALYZE ANY, and RESOURCE.
Synchronization object
- The source database can be synchronized in real time.
- The source database table structure supports only full synchronization.
- Database objects other than table structures, indexes, and constraints cannot be synchronized.
Source database
- The binlog of the source sharded database must be enabled and use the row-based format.
- If the storage space is sufficient, you are advised to store the binlogs for as long as possible. The recommended retention period is three days.
- During an incremental synchronization, the server_id value of the MySQL source database must be set. If the source database version is MySQL 5.6 or earlier, the server_id value ranges from 2 to 4294967296. If the source database is MySQL 5.7 or later, the server_id value ranges from 1 to 4294967296.
- The database names and table names of the source database sharding middleware cannot contain the following characters: '<>/\ and non-ASCII characters.
- You are advised to enable skip-name-resolve for the source sharded database to reduce the possibility of connection timeout.
- You are advised to enable GTID on the source sharded database.
- The source sharded database does not support data synchronization of the enumerated and set types.
- The source database table name and field name cannot exceed 30 characters.
- Tables without primary keys cannot be synchronized.
- The source databases to be synchronized must have the RESOURCE permission.
- The default value in the timestamp column of the source database must be within the range allowed by the destination database. Otherwise, the synchronization fails.
Destination database
- The destination DB instance is running properly.
- The destination DB instance must have sufficient storage space.
- The time zone of the destination database must be the same as that of the source database.
- The destination database (account) must have the RESOURCE permission.
MySQL -> MRS Kafka Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 22 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 23 apply to the entire synchronization process.
- Objects that have dependencies must be synchronized at the same time to avoid synchronization failure. Common dependencies: tables referenced by views, views referenced by views, views and tables referenced by stored procedures/functions/triggers, and tables referenced by primary and foreign keys
- The source database does not support point-in-time recovery (PITR).
- The destination database cannot be restored to a point in time when a full synchronization was being performed.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the synchronization task cannot be created.
Precautions
- During synchronization, do not delete the username, password, and permission of the account for connecting to the source and destination databases or modify the port of the destination databases.
- During an incremental synchronization of table-level objects, you are not advised to rename the tables.
- Only table-level DDL operations except the DROP COLUMN statement are supported.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Table 23 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- The source database account must have the following permissions: SELECT, SHOW VIEW, EVENT, LOCK TABLES, REPLICATION SLAVE, REPLICATION CLIENT, and RELOAD.
- The destination database account must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, and UPDATE. The root account of the RDS MySQL DB instance has the preceding permissions by default.
Synchronization object
- Only tables, primary key indexes, unique indexes, common indexes, store procedures, views, and functions can be synchronized.
- During database mapping, the source database cannot contain stored procedures, views, or functions.
- The database for mapping cannot contain objects except tables and these objects cannot be created during synchronization. Otherwise, the synchronization task fails.
- Tables with storage engine different to MyISAM and InnoDB cannot be synchronized.
Source database
- The binlog of the MySQL source database must be enabled and use the row-based format.
- If the storage space is sufficient, you are advised to store the source database binlog for as long as possible. The recommended retention period is three days.
- If the expire_logs_days value of the source database is set to 0, the synchronization may fail.
- During an incremental synchronization, the server_id value of the MySQL source database must be set. If the source database version is MySQL 5.6 or earlier, the server_id value ranges from 2 to 4294967296. If the source database is MySQL 5.7, the server_id value ranges from 1 to 4294967296.
- The database name in the source database cannot contain non-ASCII characters, or the following characters: '<`>/\
- The table name and view name in the source database cannot contain non-ASCII characters, or the following characters: '<>/\
- The source database name and mapped database name cannot be ib_logfile.
- Tables with storage engine different to MyISAM and InnoDB cannot be synchronized to RDS.
- Database mapping does not support views or stored procedures. If the source database contains views or stored procedures, the synchronization may fail.
Destination database
The destination database is Kafka of the MapReduce service.
GaussDB(openGauss) -> DMQ Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 24 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 25 apply to the entire synchronization process.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the synchronization task cannot be created.
Precautions
- A synchronization task may fail due to the change of the username and password of the source database or network disconnection. You need to rectify the information and then retry the synchronization task on the DRS console. Generally, you are advised not to modify the preceding information during synchronization.
- If the source database port is changed during synchronization, the synchronization task fails. Generally, you are advised not to modify the port number during synchronization.
- Changing the IP address will cause synchronization tasks to fail. You are advised not to change the IP address.
- Data definition language (DDL) statements cannot be synchronized due to the limitation of the openGauss logical replication function. Generally, you are not advised to perform DDL operations during synchronization.
- Column-store tables, compression tables, and unlogged tables cannot be synchronized in real time due to the limitation of the openGauss logical replication function.
- The consistency of distributed transactions is not ensured.
- Online cluster capacity expansion is not supported due to the limitation of the openGauss logical replication function.
- The size of a single record cannot exceed 1 MB due to DMQ constraints. If the size of a record exceeds 1 MB, it will be ignored.
- The tables that have no primary keys and whose replication data is not full cannot be synchronized.
- SSL connection is not supported.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Table 25 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- The source database account must have the replication or sysadmin role.
Synchronization object
- The table data can be synchronized.
- Data in column-store tables, compression tables, unlogged tables, and temporary tables cannot be synchronized.
- DDL cannot be synchronized.
- The supported data types are INTEGER, BIGINT, SMALLILNT, TINYINT, SERIAL, SMALLSERIAL, BIGSERIAL, FLOAT, DOUBLEPRECISION, DATE, TIME WITHOUT TIME ZONE, TIMESTAMP WITHOUT TIME ZONE, CHAR(n), VARCHAR(n) and TEXT.
Source database
- The wal_level value of the GaussDB(openGauss) source database must be logical.
- The GaussDB(openGauss) table to be synchronized must have a primary key. If the table does not have a primary key, set the replication attribute of the table to full.
- The database name in the source database cannot contain non-ASCII characters, or the following characters: +%"<>'\
- The table name in the source database cannot contain non-ASCII characters, or the following characters: +%"<>'\
Destination database
The destination database is on DMQ of the consumer cloud.
GaussDB(openGauss) -> MySQL Synchronization
- DRS has the following constraints on common operations in the case of synchronization failures caused by unpredictable errors or sudden environment changes.
Table 26 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 27 apply to the entire synchronization process.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the synchronization task cannot be created.
Precautions
- A synchronization task may fail due to the change of the username and password of the source database or network disconnection. You need to rectify the information and then retry the synchronization task on the DRS console. Generally, you are advised not to modify the preceding information during synchronization.
- If the source database port is changed during synchronization, the synchronization task fails. Generally, you are advised not to modify the port number during synchronization.
- Changing the IP address will cause synchronization tasks to fail. You are advised not to change the IP address.
- Data definition language (DDL) statements cannot be synchronized due to the limitation of the openGauss logical replication function. Generally, you are not advised to perform DDL operations during synchronization.
- Column-store tables, compression tables, and unlogged tables cannot be synchronized in real time due to the limitation of the openGauss logical replication function.
- The consistency of distributed transactions is not ensured during synchronization.
- Tables that have no primary keys and whose replication data is not full cannot be synchronized.
- SSL connection is not supported.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Table 27 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- The source database account must have the replication or sysadmin role.
Synchronization object
- Only table data can be synchronized.
- Data in column-store tables, compression tables, unlogged tables, and temporary tables cannot be synchronized.
- DDL cannot be synchronized.
- Tables that have no primary keys and whose replication data is not full cannot be synchronized.
- The supported data types are INTEGER, BIGINT, SMALLILNT, TINYINT, SERIAL, SMALLSERIAL, BIGSERIAL, FLOAT, DOUBLEPRECISION, DATE, TIME WITHOUT TIME ZONE, TIMESTAMP WITHOUT TIME ZONE, CHAR(n), VARCHAR(n) and TEXT.
Source database
- The wal_level value of the GaussDB(openGauss) source database must be logical.
- The GaussDB(openGauss) table to be synchronized must have a primary key. If the table does not have a primary key, set the replication attribute of the table to full.
- The database name in the source database cannot contain non-ASCII characters, or the following characters: +%"<>'\
- The table name in the source database cannot contain non-ASCII characters, or the following characters: +%"<>'\
Destination database
- The destination database is an on-premises MySQL database.
- Before synchronization, create tables in the destination database.
- The DDM destination database user must have the following permissions: CREATE, DROP, ALTER, INDEX, INSERT, DELETE, UPDATE, and SELECT. In addition, grant the select permission on all tables.
Last Article: Task Status
Next Article: Creating a Synchronization Task
Did this article solve your problem?
Thank you for your score!Your feedback would help us improve the website.