Help Center/ Data Replication Service/ Real-Time Synchronization/ To the Cloud/ From Oracle to DDM
Updated on 2023-04-26 GMT+08:00
View PDF

From Oracle to DDM

Supported Source and Destination Databases

Table 1 Supported databases

Source DB

Destination DB
  • On-premises databases
  • ECS databases
  • DDM instances

Prerequisites

  • You have logged in to the DRS console.
  • Your account balance is greater than or equal to $0 USD.
  • For details about the DB types and versions supported by real-time synchronization, see Supported Databases.
  • If a subaccount is used to create a DRS task, ensure that an agency has been added. To create an agency, see Agency Management.

Suggestions

  • When a task is being started or in the full synchronization phase, do not perform DDL operations on the source database. Otherwise, the task may be abnormal.
  • To keep data consistency before and after the synchronization, ensure that no data is written to the destination database during the synchronization.
  • The success of database synchronization depends on environment and manual operations. To ensure a smooth synchronization, perform a synchronization trial before you start the synchronization to help you detect and resolve problems in advance.
  • Start your synchronization task during off-peak hours. A less active database is easier to synchronize successfully. If the data is fairly static, there is less likely to be any severe performance impacts during the synchronization.
    • If network bandwidth is not limited, the query rate of the source database increases by about 50 MB/s during full synchronization, and two to four CPUs are occupied.
    • The data being synchronized may be locked by other transactions for a long period of time, resulting in read timeout.
    • When DRS concurrently reads data from a database, it will use about 6 to 10 sessions. The impact of the connections on services must be considered.
    • If you read a table, especially a large table, during the full migration, the exclusive lock on that table may be blocked.
    • For more information about the impact of DRS on databases, see What Is the Impact of DRS on Source and Destination Databases?
  • Data-Level Comparison

    To obtain accurate comparison results, compare data at a specified time point during off-peak hours. If it is needed, select Start at a specified time for Comparison Time. Due to slight time difference and continuous operations on data, data inconsistency may occur, reducing the reliability and validity of the comparison results.

Precautions

Before creating a synchronization task, read the following notes:

  • You are advised to create an independent database account for DRS task connection to prevent task failures caused by database account password modification.
  • After changing the account passwords for the source or destination databases, modify the connection information in the DRS task as soon as possible to prevent automatic retry after a task failure. Automatic retry will lock the database accounts.
Table 2 Precautions

Type

Restrictions

Database permissions
  • Source database:
    • Full synchronization requires the following permissions: CREATE SESSION, SELECT ANY DICTIONARY, and SELECT for single tables (GRANT SELECT <userName.tbName> to drsUser)
    • Full+Incremental synchronization:

      For 12c and later versions, you must have the following permissions: CREATE SESSION, SELECT ANY DICTIONARY, SELECT for a single table (GRANT SELECT <userName.tbName> to drsUser), EXECUTE_CATALOG_ROLE, SELECT ANY TRANSACTION, LOGMINING, SELECT ON SYS.COL$, SELECT ON SYS.OBJ$, and SET CONTAINER (GRANT SET CONTAINER TO <userName> CONTAINER=ALL).

      To synchronize a pluggable database (PDB) of Oracle 12c or later, you must have the following permissions: CREATE SESSION, SELECT ANY DICTIONARY, SELECT for a single table (GRANT SELECT <userName.tbName> to drsUser), EXECUTE_CATALOG_ROLE, SELECT ANY TRANSACTION, LOGMINING, SELECT ON SYS.COL$, and SELECT ON SYS.OBJ$, and CREATE SESSION, SELECT ANY DICTIONARY, EXECUTE_CATALOG_ROLE, SELECT ANY TRANSACTION, LOGMINING and SET CONTAINER (GRANT SET CONTAINER TO <userName> CONTAINER=ALL) permissions for a CDB.

      For Oracle 11g and earlier versions, the following permissions are required for database synchronization: CREATE SESSION, SELECT ANY DICTIONARY, SELECT for a single table (GRANT SELECT <userName.tbName> to drsUser), EXECUTE_CATALOG_ROLE, SELECT ANY TRANSACTION, SELECT ON SYS.COL$, and SELECT ON SYS.OBJ$.

    • During incremental synchronization, enable PK, UK, or ALL supplemental logging for the source Oracle database at the database level or table level. If supplemental logging is enabled at table level, enable supplemental logging again after you rebuild or rename tables. During the synchronization, ensure that the preceding settings are always enabled.
    • Oracle 12c or later does not support incremental synchronization using accounts whose ORACLE_MAINTAINED is Y (except system/sys), because accounts with this attribute do not have the permission to parse logs.
  • The destination database user must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, UPDATE, ALTER, INDEX, EVENT, RELOAD and CREATE VIEW.

Synchronization object
  • DDL operations cannot be synchronized during incremental synchronization.
  • In the full synchronization phase, bfile, xml, sdo_geometry, urowid, interval (precision greater than 6 digits), and user-defined types are not supported.
  • In the incremental synchronization phase, bfile, xml, interval, sdo_geometry, urowid, timestamp (precision greater than 6 digits), and user-defined types are not supported.
  • During the incremental synchronization, if the source database is a physical standby Oracle database, data of the LOB type cannot be parsed (the data dictionary cannot be generated). If the table to be synchronized contains data of the LOB type, the incremental synchronization will fail.
  • The source database data can be synchronized, but the source table structure and other objects cannot be synchronized.
  • Create table structures and indexes in the destination database that corresponds to the schema of the source database. Objects that are not created in the destination database are not to be synchronized.
  • During synchronization, create a table structure in the destination database. The table structure of the destination database must contain all columns in the source database, and the primary keys must be the same.
  • The default values of the source database can be to_date and sys_guid functions. To use other functions as default values, ensure that the destination database has the same functions. If the destination database does not have the corresponding function, the following results may be displayed:
    • The default value may be left empty.
    • The table fails to be created. As a result, the object comparison result is inconsistent or the task fails.
  • If the table contains only LOB columns, data inconsistency may occur.
  • If the empty function of the LOB type is used to write data in the Oracle database, the value queried through JDBC is an empty string. Whether the value is an empty string or NULL after being written to the destination database depends on the processing of the empty string in the destination database.
  • For a table that does not have a primary key or index, the number of columns of non-large fields must be greater than 3. Otherwise, incremental synchronization may fail because all columns cannot be matched.
  • Tables whose default values contain expressions of functions cannot be synchronized.
  • Temporary tables in the source database cannot be synchronized.

Source database
  • The maximum row length of Oracle cannot exceed 8 KB, excluding BLOB and TEXT columns which will be automatically converted to the text and blob types of MySQL. The reason is that the MySQL InnoDB restricts the row length to 8 KB.
  • The primary key or unique key column cannot contain values of the string data type when you map the MySQL data types to the character data types in Oracle because MySQL cannot tell spaces in data. Otherwise, data inconsistency and deadlock may occur.
  • The values of binary_float and binary_double cannot be set to Nan, Inf, or -Inf because MySQL does not support these values. DRS converts the three values to 0 and saves them by default.
  • AUTO_PK_ROW_ID cannot be used as a column name in Oracle because it is a reserved column name in MySQL 5.7 and cannot be created.
  • Ensure that the precision of the number(p, s) field in the Oracle database does not exceed the precision range p: [1, 38], s:[p-65, min(p, 30)]. The value of s depends on the value of p. The lower limit is p-65, and the upper limit is the minimum value of p or 30. For example, when p is 1, the value range of s is [-64, 1]. When p is 38, the value range of s is [-27, 30].

    The value of the int field cannot exceed the precision range of (65, 0). The digit range of MySQL is smaller than that of Oracle.

  • Tables whose names contain special characters except underscores (_) cannot be synchronized.
  • The source database name or mapped name cannot start with ib_logfile or be ib_buffer_pool, ib_doublewrite, ibdata1 or ibtmp1.
  • An empty source database cannot be synchronized.
  • Only the following character sets are supported: ZHS16GBK, AL32UTF8, UTF8, US7ASCII, WE8MSWIN1252, WE8ISO8859P1, WE8ISO8859P2, WE8ISO8859P4, WE8ISO8859P5, WE8ISO8859P7, WE8ISO8859P9, WE8ISO8859P13, WE8ISO8859P15.

Destination database
  • Before synchronization, you must create databases, tables, columns, indexes, and constraints with the same names (in lowercase) as the corresponding source objects to be synchronized in the destination end.
  • During a synchronization, a large amount of data is written to the destination database. If the value of the max_allowed_packet parameter of the destination database is too small, data cannot be written. You are advised to set the max_allowed_packet parameter to a value greater than 100 MB.
  • Do not use foreign keys for tables during synchronization. Otherwise, the sequence of writing data to different tables may be inconsistent with that in the source database, which may trigger foreign key constraints and cause synchronization failures.
  • The destination table can contain more columns than the source table. However, the following failures must be avoided:
    • Assume that extra columns on the destination cannot be null or have default values. If newly inserted data records are synchronized from the source to the destination, the extra columns will become null, which does not meet the requirements of the destination and will cause the task to fail.
    • Assume that extra columns on the destination must be fixed at a default value and have a unique constraint. If newly inserted data records are synchronized from the source to the destination, the extra columns will contain multiple default values. That does not meet the unique constraint of the destination and will cause the task to fail.

Operations
  • Table names are converted to lowercase letters after the tables are synchronized to the destination database. For example, ABC is converted to abc. In incremental synchronization, the source database cannot contain tables with the same name but different letter cases. Otherwise, the synchronization will fail.
  • The table without a primary key lacks a unique identifier for rows. When the network is unstable, you may need to retry the task several times, or data inconsistency may occur.
  • If there are special characters such as Chinese and Japanese in the Oracle database, the code of the destination Oracle database must be the same as the code of the source Oracle database. Otherwise, garbled characters are displayed in the destination database.
  • After the Oracle table structure is synchronized to DDM, the character set of the table is utf8mb4.
  • If the length of a table structure in the Oracle database exceeds 65,535 bytes, the synchronization may fail. The length of a table structure is the total length of all columns. The length of the char or varchar2 type is related to the code.
  • The size of an Oracle archive log file must be greater than the maximum size of a single data record to prevent incremental data parsing exceptions caused by cross-file (more than two log files) of a single data record.
  • For an Oracle RAC cluster, use the scan IP address and service name to create a task. The SCAN IP address can provide better fault tolerance, load capability, and synchronization experience.
  • If the source is an Oracle RAC database and the SCAN IP address is used to configure a DRS task, ensure that the SCAN IP address and DRS node IP address can communicate with all virtual IP addresses of the source database. Otherwise, the connectivity check fails. If the SCAN IP address is not used, the virtual IP address of a node can be used. In this case, DRS logs are parsed only on the RAC node specified by the virtual IP address.
  • If the source is an RAC database, all RAC nodes must be online when incremental synchronization is started for the first time. Otherwise, an error occurs during incremental synchronization.
  • If the source is an RAC database, the number of nodes cannot be increased or decreased during incremental synchronization to avoid incremental synchronization exceptions and ensure strong data consistency.
  • If the data types are incompatible, the synchronization may fail.
  • If the Oracle character set is WE8MSWIN1252, the CLOB column synchronized to the destination database may contain garbled characters. You can change the character set of the source database to AL32UTF8 before the synchronization.
  • During synchronization, do not modify or delete the usernames, passwords, permissions, or ports of the source and destination databases.
  • During synchronization, DDL operations cannot be performed on the source database.
  • During the synchronization, do not perform the resetlogs operation on the source Oracle database. Otherwise, data cannot be synchronized and tasks cannot be restored.
  • During synchronization, the rollback operation of the LOB type is not supported. Otherwise, the synchronization task fails.
  • During synchronization, writing data to the destination databases is not allowed. Otherwise, data inconsistency may occur.
  • In a full synchronization for the table structure, the length of the char and varchar2 characters in the source database is automatically increased by at least 1.5 times by byte in the destination database (because the length of the destination database is in the unit of byte). The increase multiple depends on the character set of the source and destination databases. For example, if the character set is UTF8, increase the length (byte) by three times by default. If the character set is GBK, increase the length (byte) by two times by default.
  • During full synchronization of the partitioned table structure, the table is converted to a non-partitioned table. During incremental synchronization, operations related to the partitioned table in the source database may fail to be executed in the destination database.
  • During an incremental synchronization, 0x00 at the end of BLOB and the spaces at the end of CLOB are truncated.
  • During an incremental synchronization of table-level objects, renaming tables is not recommended.
  • The source database username is consistent with the destination database name.
  • When the source database user and table structure are synchronized to the destination database, the corresponding information is converted into lowercase letters. For example, tables Ab and AB are changed to table ab after being synchronized to the destination database.
  • Index organized tables cannot be synchronized.
  • Before a full synchronization task starts, if a transaction is not committed for a long time, data may be lost.
  • When editing the task to add a new table, ensure that transactions of the new table have been committed. Otherwise, transactions that are not committed may fail to be synchronized to the destination database. You are advised to add tables during off-peak hours.
Parent topic: To the Cloud