Help Center/ Data Replication Service/ Real-Time Synchronization/ To the Cloud/ From Oracle to GaussDB Distributed
Updated on 2024-12-01 GMT+08:00
View PDF
Share

From Oracle to GaussDB Distributed

Supported Source and Destination Databases

Table 1 Supported databases

Source DB

Destination DB
  • On-premises databases (Oracle 10g, 11g, 12c, 18c, 19c, and 21c)
  • Self-built databases on ECS (Oracle 10g, 11g, 12c, 18c, 19c, and 21c)

GaussDB Distributed (1.0.0 and later versions)

Database Account Permission Requirements

To start a synchronization task, the source and destination database users must meet the requirements in the following table. Different types of synchronization tasks require different permissions. For details, see Table 2. DRS automatically checks the database account permissions in the pre-check phase and provides handling suggestions.

Table 2 Database account permissions

Type

Full Synchronization

Full+Incremental Synchronization and Incremental Synchronization

Source database user
  • To synchronize a pluggable database (PDB) of Oracle 12c or later in tenant mode, the user must have the following permissions:
    • To synchronize a PDB as a PDB user, the user must have the CREATE SESSION, SELECT ANY DICTIONARY and SELECT ANY TRANSACTION permissions for the PDB and the SELECT permission for a single table.

      Authorization statement: GRANT SELECT ON <userName.tbName> to drsUser;)

    • To synchronize a PDB as a CDB user, the user must have the following permissions: CREATE SESSION, SELECT ANY DICTIONARY, SELECT ANY TRANSACTION permissions for the PDB, the SELECT permission for a single table, and CREATE SESSION, SELECT ANY DICTIONARY, SELECT ANY TRANSACTION, and SET CONTAINER permissions for the CDB.
    • Authorization statement: GRANT SET CONTAINER TO <userName> CONTAINER=ALL;)
  • In other cases:

    The user must have the CREATE SESSION and SELECT ANY DICTIONARY permissions. The SELECT permission on a single table is also required.

    Authorization statement: GRANT SELECT ON <userName.tbName> to drsUser;
  • Oracle 12c or later in tenant mode:
    • To synchronize a container database (CDB) of Oracle 12c or later, the user must have the following permissions: CREATE SESSION, SELECT ANY DICTIONARY, SELECT for a single table (GRANT SELECT ON <userName.tbName> to drsUser), EXECUTE_CATALOG_ROLE, SELECT ANY TRANSACTION, and LOGMINING.
    • To synchronize a pluggable database (PDB) of Oracle 12c or later, the user must have the following permissions: CREATE SESSION, SELECT ANY DICTIONARY, SELECT for a single table (GRANT SELECT ON <userName.tbName> to drsUser), EXECUTE_CATALOG_ROLE and SELECT ANY TRANSACTION permissions for a PDB, and CREATE SESSION, SELECT ANY DICTIONARY, EXECUTE_CATALOG_ROLE, SELECT ANY TRANSACTION, LOGMINING and SET CONTAINER (GRANT SET CONTAINER TO <userName> CONTAINER=ALL) permissions for a CDB.
  • Oracle 12c or later in non-tenant mode:

    The user must have the following permissions: CREATE SESSION, SELECT ANY DICTIONARY, SELECT for a single table (GRANT SELECT ON <userName.tbName> to drsUser), EXECUTE_CATALOG_ROLE, SELECT ANY TRANSACTION, and LOGMINING.

  • To synchronize a database of Oracle 11g or earlier, the user must have the following permissions: CREATE SESSION, SELECT ANY DICTIONARY, SELECT for a single table (GRANT SELECT ON <userName.tbName> to drsUser), EXECUTE_CATALOG_ROLE, and SELECT ANY TRANSACTION.

Destination database user
  • Database-level permissions: Log in to the postgres base database as user root or user DATABASE with the Sysadmin role, and grant the CREATE and CONNECT permissions to user DATABASE.

    Statement: GRANT CREATE, CONNECT ON DATABASE <database> TO <user>

  • Schema-level permission: Log in to the database as user root or user DATABASE with the Sysadmin role, or the owner of the database, and grant the CREATE and USAGE permissions of the schema to the user.

    Statement: GRANT CREATE, USAGE ON SCHEMA <schema> TO <user>

  • Table-level permission: Log in to the database as user root or user DATABASE with the Sysadmin role, or the owner of the database, and grant the DML permission for tables in the SCHEMA to the user. (The SELECT permission is required only for tables without primary keys.)

    Statement for granting the DML permission on all tables in the SCHEMA: GRANT SELECT, UPDATE, INSERT, DELETE, INDEX, ALTER ON ALL TABLES IN SCHEMA <schema> TO <user>

    Statement for granting the DML permission on a specified table in the SCHEMA: GRANT SELECT, UPDATE, INSERT, DELETE, INDEX, ALTER ON TABLE <schema.table> TO <user>

  • If gs_loader is used to create system catalogs (such as public.pgxc_copy_error_log and public.gs_copy_summary) in the destination database, the all privilege permission is required for accessing the system catalogs.For details, see gs_loader.
  • You are advised to create an independent database account for DRS task connection to prevent task failures caused by database account password modification.
  • After changing the account passwords for the source and destination databases, modify the connection information of the DRS task by referring to Modifying Connection Information to prevent automatic retry after a task failure. Automatic retry will lock the database accounts.

Supported Synchronization Objects

Table 3 lists the objects that can be synchronized in different scenarios. DRS will automatically check the objects you selected before the synchronization.

Table 3 Supported synchronization objects

Type

Precautions

Synchronization objects
  • During full synchronization, tables, common indexes, primary key and unique constraints, and data can be synchronized. Tables can be synchronized in real time during incremental synchronization.
  • Full synchronization does not support the following column types: bfile, sdo_geometry, urowid, and user-defined types. Incremental synchronization does not support the following column types: bfile, sdo_geometry, urowid, interval year to month, interval day to second, and user-defined types.
  • The maximum precision supported by timestamp and interval day to second is 6.
  • During full synchronization, the table structure does not support bitmap indexes, inverted indexes, and function indexes.
  • For incremental synchronization, the LOB type supports only the BasicFiles attribute. It does not support the SecureFiles attribute. The size of the LOB type must be less than 10 MB.
  • For the TIMESTAMP WITH TIME ZONE data type, the value cannot be greater than 9999-12-31 23:59:59.999999 after being converted based on the time zone of the destination database.
  • Tables whose default values contain expression functions cannot be synchronized.
  • Temporary tables in the source database cannot be synchronized.
  • Tables with virtual columns in the source database cannot be synchronized.
  • A user-defined table that has no primary key and is a partitioned table cannot be synchronized. Otherwise, the task may fail.

Precautions

To ensure tasks can run normally, DRS provides automatic pre-check. Before starting a DRS task, DRS checks the configurations and conditions of the source and destination databases. For details about the main check items and handling suggestions, see Pre-check Items. In addition to the pre-check items, you need to pay attention to the items listed in Table 4.

Table 4 Precautions

Type

Restrictions

Restrictions on the source database
  • The total index length of columns in the source database cannot exceed the length limit in the destination database. For detailed length requirements, see Index Length Description.

Restrictions on usage

General

  • Do not modify or delete the usernames, passwords, and permissions of the source and destination databases or change the ports of the source and destination databases.
  • Do not perform the resetlogs operation on the source Oracle database. Otherwise, data cannot be synchronized and tasks cannot be restored.
  • The rollback operation of the LOB type is not supported. Otherwise, the synchronization task fails.
  • The username (schema name) of the source Oracle database cannot be changed, including the scenarios where the schema name is changed by modifying the USER$ dictionary table in versions earlier than 11.2.0.2 and by using ALTER USER username RENAME TO new_username in versions later than 11.2.0.2.

Full synchronization

  • Do not write data to the destination database table during full synchronization. Otherwise, data will be inconsistent.
  • Full synchronization consists of two phases: table structure synchronization (including indexes) and data synchronization. If the structure of a table is created in the destination database, data synchronization starts. If a table fails to be synchronized, you can restart the task to synchronize the table data. However, the table structure is not synchronized, so you must manually create the table in the destination database.
  • During table structure synchronization in a full synchronization, if the destination database has a constraint with the same name as the source database, the table fails to be created.
  • During table structure synchronization in a full synchronization, the column name cannot be CTID, XMIN, CMIN, XMAX, CMAX, TABLEOID, XC_NODE_ID, TID, OID, GS_TUPLE_UID or TABLEBUCKETID. Otherwise, the table fails to be created.
  • In a full synchronization for the table structure, the length of the char and varchar2 types in the source database automatically increases by at least 1.5 times by byte in the destination database (because the length of the destination database is in the unit of byte). The increase multiple depends on the character set of the source and destination databases. For example, if the character set is UTF8, increase the length (byte) by three times by default. If the character set is GBK, increase the length (byte) by two times by default.
  • During table structure synchronization in the full phase, if there is a schema with the same name as a user in the destination database and another user is used to synchronize the table structure to the schema, run the grant [role] to [role] command to grant permissions to the user. Otherwise, the synchronization may fail because the table structure cannot be created.
  • When the structure of a partitioned table is synchronized during a full synchronization, the partitioned table is converted to a non-partitioned table. During incremental synchronization, operations on partitioned tables in the source database may fail to be synchronized to the destination database or fail to be executed in the destination database.
  • During table structure synchronization in a full synchronization, only default value constraints of the character string and number types are supported. Default value constraints of the function and sequence types are not supported.

Incremental synchronization

  • During an incremental synchronization, some DDL operations are supported. DDL conversion of heterogeneous databases requires semantic analysis and syntax compatibility. Only some DDL operations can be synchronized when the conversion is successful and the following conditions are met. If a task is abnormal due to DDL synchronization in other cases, you need to manually execute the DDL operations in the destination database.
    • Table-level synchronization supports alter table add column, alter table drop column, alter table rename column, alter table modify column, and truncate table. The modification of default values is not supported.
    • The character length of new fields in incremental DDLs is not automatically extended.
    • During incremental synchronization, tables cannot be renamed, tables cannot be redefined online, and partitions cannot be swapped.
  • During incremental synchronization, modify the start point of a capture task to resynchronize data.
    • After the capture point is modified, the previous object-level comparison result is not displayed.
    • If the start point of a capture task is changed separately, the change is synchronized to the start point of the replay task. That is, the start point of the replay task is the same as that of the capture task. This does not affect the change of the start point of the replay task.
  • During an incremental synchronization, 0x00 at the end of BLOB and the spaces at the end of CLOB are truncated.
  • During incremental synchronization, you are not advised to select a hybrid partition table because DML logs are not generated when data in the external partition of the hybrid partition table changes. DRS cannot obtain the changes during incremental synchronization, which may cause data inconsistency.
  • During incremental synchronization, the incremental parsing supports only data within the valid range of the Oracle time type. Data beyond the valid range may cause task exceptions. For example, data truncation occurs if the year value is greater than 9999.
  • For incremental synchronization, the LOB type supports only the BasicFiles attribute. It does not support the SecureFiles attribute. The size of the LOB type must be less than 10 MB.
  • During incremental synchronization, if the source or destination database is abnormal, the task fails. After the database is recovered, the task is globally started. The status of the original capture or replay component is ignored, and the replay component is started from the capture interruption point.
  • In the incremental phase, Oracle extended characters are not supported. The standard character set cannot parse Oracle customized extended characters.

Troubleshooting

  • If any problem occurs during task creation, startup, full synchronization, incremental synchronization, or completion, rectify the fault by referring to Troubleshooting.

Other restrictions
  • The following functions can be used as default values during table structure synchronization: decode, nvl, nvl2, instr, substr, to_char, to_date, to_timestamp, length, lengthb, sysdate, trunc, nullif, next_day, regexp_instr, add_months, systimestamp, to_number, empty_clob and empty_blob. To use these functions as default values, ensure that the destination database has the same functions. If the destination database does not have the corresponding function, the following results may be displayed:
    • The default value may be left empty.
    • The table fails to be created. As a result, the object comparison result is inconsistent or the task fails.
  • If a table contains only LOB columns, data inconsistency or task failure may occur.
  • If the empty function of the LOB type is used to write data in the Oracle database, the value queried through JDBC is an empty string. Whether the value is an empty string or NULL after being written to the destination database depends on the processing of the empty string in the destination database.
  • By default, the Oracle-to-GaussDB synchronization task does not support loopback and cascading synchronization. That is, data cannot be synchronized from instance A to instance B and then from instance B to instance C.
  • For a table that does not have a primary key or index, the number of columns of non-large fields must be greater than 3. Otherwise, incremental synchronization may fail because all columns cannot be matched.
  • Incremental synchronization does not support distributed transactions (XA transactions) and PARALLEL DML on an Oracle database.
  • When you manually create a table structure, the time type in the destination database must be the same as that in the source database. Otherwise, time data may be inconsistent after time zone conversion.
  • Ensure that a database named in lowercase letters has been created in destination database.
  • The destination table can contain more columns than the source table. However, the following failures must be avoided:
    • Assume that extra columns on the destination cannot be null or have default values. If newly inserted data records are synchronized from the source to the destination, the extra columns will become null, which does not meet the requirements of the destination and will cause the task to fail.
    • Assume that extra columns on the destination must be fixed at a default value and have a unique constraint. If newly inserted data records are synchronized from the source to the destination, the extra columns will contain multiple default values. That does not meet the unique constraint of the destination and will cause the task to fail.
  • Cascading one-way synchronization is not supported. For example, data cannot be synchronized from instance A to instance B and then from instance B to instance C.
  • Do not use an imprecise value type as the primary key in the database. This feature affects the synchronization of UPDATE and DELETE statements in the DRS incremental scenario and causes value comparison to be unavailable. For details about the data types supported by the primary key, see Mapping Data Types.
  • The size of an Oracle archive log file must be greater than the maximum size of a single data record to prevent incremental data parsing exceptions caused by cross-file (more than two log files) storage of a single data record.
  • If the Oracle character set is WE8MSWIN1252 or WE8ISO8859P1, the CLOB column synchronized to the destination database may contain garbled characters. You can change the character set of the source database to AL32UTF8 before the synchronization.
  • If the PDB database is used for synchronization, all PDBs must be enabled during incremental synchronization due to the restrictions of the Oracle LogMiner component.
  • In Oracle 12.2 and later versions, due to the restrictions of the Oracle LogMiner component, a table or column name contains no more than 30 characters during an incremental synchronization.
  • If the total length of the storage data in each row exceeds the upper limit (8192 bytes by default) of GaussDB, the task may fail.
  • For an Oracle RAC cluster, use the scan IP address and service name to create a task. The SCAN IP address can provide better fault tolerance, load capability, and synchronization experience.
  • If the source is an Oracle RAC database and the SCAN IP address is used to configure a DRS task, ensure that the SCAN IP address and DRS node IP address can communicate with all virtual IP addresses of the source database. Otherwise, the connectivity check fails. If the SCAN IP address is not used, the virtual IP address of a node can be used. In this case, DRS logs are parsed only on the RAC node specified by the virtual IP address.
  • If the source is an RAC database, all RAC nodes must be online when incremental synchronization is started for the first time. Otherwise, an error occurs during incremental synchronization.
  • If the source is an RAC database, the number of nodes cannot be increased or decreased during incremental synchronization to avoid incremental synchronization exceptions and ensure strong data consistency.
  • The table without a primary key lacks a unique identifier for rows. When the network is unstable, you may need to retry the task several times, or data inconsistency may occur.
  • If the character sets of the source database are different from those of the destination database, adjust the field length of the destination database based on the site requirements. For example, the source character set is ZHS16GBK and the destination character set is UTF8, a Chinese character of ZHS16GBK occupies two bytes, and a Chinese character of UTF8 occupies three bytes. When CHAR or VARCHAR data type is synchronized to the destination database, the field length may exceed the defined length, so you need to increase the field length to 1.5 times of that in the source database.
  • If you use DRS to synchronize table structures, the tables, constraints, and indexes in the same schema cannot have the same name with different letter cases. For example, table A contains an index named inx1, and table B contains an index name a. Table A and index a have the same name with different letter cases, which will cause table structure synchronization to fail. If multiple schemas are mapped to one schema, the source schemas cannot contain tables, constraints, and indexes with the same name but different letter cases.
  • In a full+incremental or incremental synchronization, the PDB database cannot be directly connected. You need to provide the service name/SID, username, and password of the CDB.
  • Full and incremental synchronization does not support hidden columns (UNUSED, INVISIBLE).
  • If the destination database is a distributed GaussDB database, the update operation performed on the source distribution column will fail to be synchronized to the GaussDB database during incremental synchronization, causing data inconsistency. Therefore, avoid updating a distribution column.
  • You are advised to disable the global secondary index (GSI) function for the destination database. Otherwise, incremental synchronization may fail.
  • You are not advised to use the LOB type and extended character type (the length exceeds 4000 bytes) as incremental data filtering conditions. Oracle logs may not record the old value of update.
  • When editing the task to add a new table, ensure that transactions of the new table have been committed. Otherwise, transactions that are not committed may fail to be synchronized to the destination database. You are advised to add tables during off-peak hours.
  • If the synchronization table in the destination database has been associated with a sequence when the task stops, DRS automatically resets the sequence value of the destination database. The auto-increment sequence value is the source database sequence value plus the security margin, and the auto-decrement sequence value is the source database sequence value minus the security margin. The default security margin is 10,000.
  • When a dual-AZ task is switched over (for example, the subtask in the primary AZ fails), if the primary task has a long delay or there are transactions that have been there for a long time without being submitted in the source database, the task may fail to be started after the switchover because the source database logs are cleared, or synchronization exceptions may occur after the switchover due to DDL changes during replication delay or long transactions.
  • If the supplemental logging level of the source Oracle database is not set to All and the incremental update data of the source database is not found in the destination database, the complete data of the source database cannot be written to the destination database even if the conflict policy is set to Overwrite. To write all data to the destination database, set the supplemental logging level of the source database to All.
  • When you update a unique constraint column in an Oracle database and multiple rows are hit, data conflicts may occur. In extreme scenarios, there may be data inconsistency.
  • The XMLTYPE type is converted to CLOB or BLOB in the Oracle database. The usage restrictions are the same as those of CLOB and BLOB.
  • Only Oracle 11g and later versions support XMLTYPE data synchronization.

Procedure

  1. On the Data Synchronization Management page, click Create Synchronization Task.
  2. On the Create Synchronization Instance page, select a region and project, specify the task name, description, and the synchronization instance details, and click Create Now.

    • Task information description
      Figure 1 Synchronization task information
      Table 5 Task information

      Parameter

      Description

      Region

      The region where the replication instance is deployed. You can change the region.

      Project

      The project corresponds to the current region and can be changed.

      Task Name

      The task name must start with a letter and consist of 4 to 50 characters. It can contain only letters, digits, hyphens (-), and underscores (_).

      Description

      The description consists of a maximum of 256 characters and cannot contain special characters !=<>'&"\
    • Synchronization instance details
      Figure 2 Synchronization instance details
      Table 6 Synchronization instance settings

      Parameter

      Description

      Data Flow

      Select To the cloud.

      Source DB Engine

      Select Oracle.

      Destination DB Engine

      Select GaussDB Distributed.

      Network Type

      Public network is used as an example. Available options: VPC, Public network and VPN or Direct Connect

      • VPC is suitable for data synchronization between cloud databases of the same account in the same region and VPC.
      • Public network is suitable for data synchronization from on-premises or external cloud databases to the destination databases bound with an EIP.
      • VPN or Direct Connect is suitable for data synchronization from on-premises databases to cloud databases, between databases of different accounts in the same region on the cloud, or between databases across regions on the cloud using a VPN, Direct Connect, Cloud Connect, VPCEP, or a VPC peering connection.

      DRS Task Type

      Type of the DRS task. The value can be Single-AZ or Dual-AZ.

      • Dual-AZ: This architecture provides HA, improving the reliability of DRS tasks. After a dual-AZ task is created, DRS creates two subtasks, each running in the primary and standby AZs. If the subtask in the primary AZ fails, DRS automatically starts the subtask in the standby AZ to continue the synchronization. This deployment is for scenarios where there is a lot of service data, long-term synchronization is required, and there are strict limits on how much service downtime can be tolerated.
      • Single-AZ: Single-node deployment is used. The synchronization task will be created on only one node to save money. This deployment is for scenarios where there is a small amount of service data, short-term synchronization is required, and there is no requirement on service downtime.

      This option is available only in specific scenarios. For details, see Performing a Switchover for a Dual-AZ Task.

      Destination DB Instance

      The GaussDB distributed instance you created.

      Synchronization Instance Subnet

      Select the subnet where the synchronization instance is located. You can also click View Subnets to go to the network console to view the subnet where the instance resides.

      By default, the DRS instance and the destination DB instance are in the same subnet. You need to select the subnet where the DRS instance resides, and there are available IP addresses for the subnet. To ensure that the synchronization instance is successfully created, only subnets with DHCP enabled are displayed.

      Synchronization Mode

      The synchronization mode supported by a DRS task. Full+Incremental is used as an example. For details about the underlying working principles for full or incremental synchronization, see Product Architecture and Function Principles.

      Specify EIP

      This parameter is available when you select Public network for Network Type. Select an EIP to be bound to the DRS instance. DRS will automatically bind the specified EIP to the DRS instance and unbind the EIP after the task is complete.

      If DRS Task Type is set to Dual-AZ, you need to specify the primary and standby IP addresses.

      For details about the data transfer fee generated using a public network, see EIP Price Calculator.
    • Task Type
      Figure 3 Task type
      Table 7 Task type information

      Parameter

      Description

      Specifications

      DRS instance specifications. Different specifications have different performance upper limits. For details, see Real-Time Synchronization.

      NOTE:

      DRS allows you to upgrade specifications only for single-AZ synchronization tasks. Task specifications cannot be downgraded. For details, see Changing Specifications.

      AZ

      Select the AZ where you want to create the DRS task. Selecting the one housing the source or destination database can provide better performance.

      If DRS Task Type is set to Dual-AZ, you can specify Primary AZ and Standby AZ.

      Figure 4 AZ
    • Enterprise Project and Tags
      Figure 5 Enterprise Project and Tags

      Table 8 Enterprise Project and Tags

      Parameter

      Description

      Enterprise Project

      An enterprise project you would like to use to centrally manage your cloud resources and members. Select an enterprise project from the drop-down list. The default project is default.

      For more information about enterprise project, see Enterprise Management User Guide.

      To customize an enterprise project, click Enterprise in the upper right corner of the console. The Enterprise Project Management Service page is displayed. For details, see Creating an Enterprise Project in Enterprise Management User Guide.

      Tags
      • Tags a task. This configuration is optional. Adding tags helps you better identify and manage your tasks. Each task can have up to 20 tags.
      • If your organization has configured tag policies for DRS, add tags to tasks based on the policies. If a tag does not comply with the policies, task creation may fail. Contact your organization administrator to learn more about tag policies.
      • After a task is created, you can view its tag details on the Tags tab. For details, see Tag Management.

    If a task fails to be created, DRS retains the task for three days by default. After three days, the task automatically stops.

  3. On the Configure Source and Destination Databases page, specify source and destination database information and click Test Connection for both the source and destination databases to check whether they have been connected to the synchronization instance. After the connection tests are successful, click Next.

    Figure 6 Source database information
    Table 9 Source database settings

    Parameter

    Description

    IP Address or Domain Name

    The IP address or domain name of the source database.

    NOTE:

    For a RAC cluster, use a Scan IP address to improve access performance.

    Port

    The port of the source database. Range: 1 – 65535

    Database Service Name

    Enter a database service name (Service Name/SID). The client can connect to the Oracle database through the database service name. For details about how to query the database service name, see the prompt on the GUI.

    PDB Name

    Container database (CDB) and pluggable database (PDB) are new features in Oracle 12c and later versions. This function is optional, but it must be enabled if you want to migrate only PDB tables.

    Enter the service name, SID, username, and password of the CDB that contains the PDB tables to be migrated.

    Database Username

    The username for accessing the source database.

    Database Password

    The password for the database username.

    SSL Connection

    SSL encrypts the connections between the source and destination databases. If SSL is enabled, upload the SSL CA root certificate.

    NOTE:
    • The maximum size of a single certificate file that can be uploaded is 500 KB.
    • If SSL is disabled, your data may be at risk.
    Figure 7 Destination database information
    Table 10 Destination database settings

    Parameter

    Description

    DB Instance Name

    The GaussDB distributed DB instance selected during synchronization task creation. This parameter cannot be changed.

    Database Username

    The username for accessing the destination database.

    Database Password

    The password for the database username. The database username and password are encrypted and stored in the system and will be cleared after the task is deleted.

  4. On the Set Synchronization Task page, select the synchronization object, enter the name of the destination database, and click Next.

    Figure 8 Synchronization mode
    Table 11 Synchronization Object

    Parameter

    Description

    Flow Control

    You can choose whether to control the flow. Flow Control takes effect in the full phase only.

    • Yes

      You can customize the maximum synchronization speed. During the full synchronization, the synchronization speed of each task (or each subtask in multi-task mode) does not exceed the value of this parameter.

      In addition, you can set the time range based on your service requirements. The traffic rate setting usually includes setting of a rate limiting time period and a traffic rate value. Flow can be controlled all day or during specific time ranges. The default value is Always. A maximum of 10 time ranges can be set, and they cannot overlap.

      The flow rate must be set based on the service scenario and cannot exceed 9,999 MB/s.

      Figure 9 Flow control
    • No
      The synchronization speed is not limited and the outbound bandwidth of the source database is maximally used, which will increase the read burden on the source database. For example, if the outbound bandwidth of the source database is 100 MB/s and 80% bandwidth is used, the I/O consumption on the source database is 80 MB/s.
      NOTE:
      • The flow control mode takes effect only in the full synchronization phase.
      • You can also change the flow control mode after creating a task. For details, see Modifying the Flow Control Mode.

    Sharding Mode

    Select Sharding by primary key or Sharding by row ID as needed.

    • Sharding by primary key

      Tables with primary keys are sharded using primary key values. The sharding efficiency is lower than that of Sharding by row ID.

    • Sharding by row ID

      Sharding by row ID is recommended for synchronizing large tables without primary keys, improving sharding efficiency.

      Restrictions on Sharding by row ID: During full synchronization, do not perform operations that will cause row ID changes on the source database, such as Export/import of the table, ALTER TABLE XXXX MOVE, ALTER TABLE XXXX SHRINK SPACE, FLASHBACK TABLE XXXX, Splitting a partition, Updating a value so that it moves to a new partition, or Combining two partitions. Otherwise, data may be inconsistent. You can run ALTER TABLE XXXX DISABLE ROW MOVEMENT to disable these operations.

    Synchronize

    Incremental DDL synchronization is supported. You can determine whether to synchronize DDLs based on service requirements.

    Synchronize DML

    Select the DML operations to be synchronized. By default, all DML operations are selected.

    If you do not select Delete, DELETE statements in the incremental data of the source database will not be synchronized, which may cause a data inconsistency. As a result, there may be a data conflict or the task may fail.

    Incremental Read Mode

    Select the mode of reading logs in the incremental synchronization phase.

    • LogMiner: uses the official Oracle interface to read redo logs. This mode is stable.
    • Binary Reader: uses DRS-developed method to directly read and parse original redo logs. The performance is high, and the Oracle resource consumption is low.

    LogMiner is recommended.

    Synchronize DDLs

    This parameter is available when Incremental DDLs is selected for Synchronize. Select DDL type for incremental synchronization. You can select Default or Custom based on your service requirements.

    • Default

      During table-level synchronization, only DDLs of the selected tables will be synchronized to the destination. The supported DDL statements are CREATE_TABLE, ALTER_TABLE, and TRUNCATE_TABLE.

    • Custom: You can select the DDL type to be synchronized as required. The DDL types supported by different data flow types are displayed on the GUI.

    Synchronization Object

    The left pane displays the source database objects, and the right pane displays the selected objects. You can select Tables or Import object file for Synchronization Object as required.

    NOTE:
    • To quickly select the desired database objects, you can use the search function.
    • If there are changes made to the source databases or objects, click in the upper right corner to update the objects to be synchronized.
    • If an object name contains spaces, the spaces before and after the object name are not displayed. If there are two or more consecutive spaces in the middle of the object name, only one space is displayed.
    • The name of the selected synchronization object cannot contain spaces.

  5. On the Process Data page, set the filtering rules for data processing.

    • If data processing is not required, click Next.
    • If you need process columns or filter out data, set the corresponding rules by referring to Processing Data.
    Figure 10 Processing data

  6. On the Check Task page, check the synchronization task.

    • If any check fails, review the cause and rectify the fault. After the fault is rectified, click Check Again.

      For details about how to handle check failures, see Solutions to Failed Check Items in Data Replication Service User Guide.

    • If all check items are successful, click Next.

      You can proceed to the next step only when all checks are successful. If there are any items that require confirmation, view and confirm the details first before proceeding to the next step.

  7. On the displayed page, specify Start Time, Send Notifications, SMN Topic, Delay Threshold (s), and Stop Abnormal Tasks After, confirm that the configured information is correct, select the check box before the agreement, and click Submit to submit the task.

    Figure 11 Task startup settings

    Table 12 Task startup settings

    Parameter

    Description

    Start Time

    Set Start Time to Start upon task creation or Start at a specified time based on site requirements.

    NOTE:

    After a synchronization task is started, the performance of the source and destination databases may be affected. You are advised to start a synchronization task during off-peak hours.

    Send Notifications

    This parameter is optional. After enabled, select a SMN topic. If the status, latency metric, or data of the migration task is abnormal, DRS will send you a notification.

    SMN Topic

    This parameter is available only after you enable Send Notifications and create a topic on the SMN console and add a subscriber.

    For details, see Simple Message Notification User Guide.

    Delay Threshold (s)

    During an incremental synchronization, a synchronization delay indicates a time difference (in seconds) of synchronization between the source and destination database.

    If the synchronization delay exceeds the threshold you specify, DRS will send alarms to the specified recipients. The value ranges from 0 to 3,600. To avoid repeated alarms caused by the fluctuation of delay, an alarm is sent only after the delay has exceeded the threshold for six minutes.

    NOTE:
    • If the delay threshold is set to 0, no notifications will be sent to the recipient.
    • In the early stages of an incremental synchronization, the synchronization delay is long because a large quantity of data is awaiting synchronization. In this case, no notifications will be sent.
    • Before setting the delay threshold, enable Send Notifications.

    Data Exception Notification

    This parameter is optional. After enabled, DRS will send a notification if the task data is abnormal.

    Stop Abnormal Tasks After

    Number of days after which an abnormal task is automatically stopped. The value must range from 14 to 100. The default value is 14.

    NOTE:
    • You can set this parameter only for pay-per-use tasks.
    • Tasks in the abnormal state are still charged. If tasks remain in the abnormal state for a long time, they cannot be resumed. Abnormal tasks run longer than the period you set (unit: day) will automatically stop to avoid unnecessary fees.

  8. After the task is submitted, you can view and manage it on the Data Synchronization Management page.

    • You can view the task status. For more information about task status, see Task Statuses.
    • You can click in the upper right corner to view the latest task status.
    • By default, DRS retains a task in the Configuration state for three days. After three days, DRS automatically deletes background resources, but the task status remains unchanged. When you configure the task again, DRS applies for resources for the task again. In this case, the IP address of the DRS instance changes.
    • For a public network task, DRS needs to delete background resources after you stop the task. The EIP bound to the task cannot be restored to the Unbound state until background resources are deleted.

Parent topic: To the Cloud