Synchronization objects | - DML operations INSERT, UPDATE, and DELETE can be synchronized.
- The DDL operation of adding columns can be synchronized.
- Only MyISAM and InnoDB tables can be synchronized.
- Views, foreign keys, stored procedures, triggers, functions, events, virtual columns, unique constraints, and unique indexes cannot be synchronized.
- Foreign keys that contain reference operations such as CASCADE, SET NULL, and SET DEFAULT cannot be synchronized. These operations will cause the update or deletion of rows in parent tables and affect records in child tables. Also, operations related to child tables are not recorded in Binlogs.
|
Usage | - General:
- During real-time synchronization, the IP addresses, ports, accounts, and passwords cannot be changed.
- The source database cannot be restored.
- Binlogs must be enabled for the source database and the value of binlog_row_image must be full.
- It is recommended that MySQL binlogs be retained for more than three days. Binlogs cannot be forcibly cleared.
When a job is restored after an exception, the recorded binlog location may expire. As a result, the job fails to be restored. It is important to monitor the duration of job exceptions and the binlog retention period. - During real-time synchronization, the source MySQL database cannot be upgraded across major versions. Otherwise, data may become inconsistent or the synchronization task may fail (data, table structures, and keywords may cause compatibility changes after the cross-version upgrade). You are advised to create a synchronization task again if the source MySQL database is upgraded across major versions.
- Views, foreign keys, stored procedures, triggers, functions, events, virtual columns, unique constraints, and unique indexes cannot be synchronized.
- Synchronizing tables that have no primary keys may cause duplicate data. Exercise caution when performing this operation.
- If the source database is MySQL 8.0.23 or a later version and the data to be synchronized or migrated contains hidden columns, data in these columns may fail to be obtained and may be lost.
- Full synchronization phase:
During task startup and full data synchronization, do not perform DDL operations on the source database. Otherwise, the task may fail. - Incremental synchronization phase:
- During incremental synchronization, executing non-idempotent DDL statements (for example, ALTER TABLE ddl_test ADD COLUMN c3 timestamp default now();) may cause data inconsistency.
- During incremental synchronization, the following DDL types can be identified: creating a table, deleting a table, adding a column, deleting a column, renaming a table, renaming a column, modifying the column type, and clearing a table. For details about the DDL types supported by different links, see the notes and constraints of each link.
- Foreign keys that contain reference operations such as CASCADE, SET NULL, and SET DEFAULT cannot be synchronized. These operations will cause the update or deletion of rows in parent tables and affect records in child tables. Also, operations related to child tables are not recorded in Binlogs.
- The position cannot be set before DDL operations that involves table structure changes, such as table modification and column addition.
- After incremental data is synchronized to Hudi MOR tables in MRS clusters earlier than 3.2.0-LTS1.5, CDM or Spark SQL cannot be used to write data. You need to perform compaction before writing data.
- Database/Table sharding:
In the database and table sharding scenario, you must stop service operations before renaming columns. Otherwise, data inconsistency may occur. In the database and table sharding scenario, you are advised to synchronize only the DDL for adding columns. If you synchronize other DDL operations, the job may fail or data may be inconsistent because the destination table has been modified. In the database and table sharding scenario, ensure that the types of columns to be added to each table are the same. Otherwise, the task may fail.
- Extraction from the standby database:
Before extracting data from the standby database, ensure that a value can be returned after the SHOW MASTER STATUS command is executed on the standby database node. Otherwise, data jobs may be abnormal and data may be lost.
|
Other | Tables in the destination database can contain more columns than those in the source database. The following failures must be avoided: Assume that extra columns in the destination database cannot be null and have no default values. If newly inserted data records are synchronized from the source database to the destination database, the extra columns will become null, which does not meet the requirements of the destination database and will cause the task to fail. |