Before You Start
There are some constraints imposed on DRS to improve the stability and security of data migration. Before using DRS, ensure that all storage engines meet the given constraints.
Tips
- During a full migration, stop writing data to the source and destination databases.
- If you perform a full plus incremental migration task, you can still write service data to the source database. It is recommended that you start the migration task 2 to 3 days in advance and comply with the following constraints (Table 1 and Table 19) to ensure a successful migration.
- Full migration Due to the following reasons, you are advised to start a migration task during off-peak hours. If you have to migrate data during peak hours, enable Flow Control to adjust the migration speed.
- Full migration poses certain workload on the source database.
- To ensure data consistency, tables to be migrated without a primary key may be locked for 3s.
- The data being migrated may be locked by other transactions for a long period of time, resulting in read timeout.
- Due to the inherent characteristics of MySQL, in certain scenarios the read and write performance may be negatively affected. For example, if the CPU resources are insufficient and the storage engine is Tokudb, the read speed on tables may be decreased by 10%.
- Data-level comparison
Due to slight time difference and continuous operations on data, inconsistent comparison results may be generated, reducing the reliability and validity of the results. To obtain accurate comparison results, you are advised to start data comparison at a specified time point during off-peak hours. If it is needed, select Start at a specified time for Comparison Time.
- Full migration
MySQL -> RDS MySQL
- DRS has the following constraints on common operations in the case of migration failures caused by unpredictable errors or sudden environment changes.
Table 1 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 2 apply to the entire migration process.
- Objects that have dependencies must be migrated at the same time to avoid migration failure. Common dependencies: tables referenced by views, views referenced by views, views and tables referenced by stored procedures/functions/triggers, and tables referenced by primary and foreign keys
- Due to the MySQL constraints, if the one-time event triggering time of the source database is earlier than the migration start time, the event will not be migrated to the destination database.
- When creating multiple migration tasks in the many-to-one scenario, ensure that the read and write settings of the destination database are consistent in these tasks.
- Incremental migration filters the DDL operations such as creating users, deleting users, and modifying user permissions.
- If a table does not have a primary key to uniquely identify each row and the network connection is unstable, data in the destination database may be inconsistent with that in the source database after migration.
- The destination database cannot be restored to a point in time when a full migration was being performed.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the migration task cannot be created.
Precautions
- During full migration, do not perform DDL operations such as modifying or deleting tables in the source database.
- The selected events and triggers are migrated while the migration task proceeds to the final stage. Before a task is completed, ensure that the source and destination databases are connected and pay attention to the migration status reported by the migration log.
- During migration, do not modify or delete the usernames, passwords, permissions, or ports of the source and destination databases.
- During an incremental migration, do not perform the point-in-time recovery (PITR) operation on the source database.
- During an incremental migration, if distributed transactions exist in the source database, the migration may fail.
- To ensure data consistency, you are not allowed to modify the destination database (including but not limited to DDL and DML operations) during migration.
- During an incremental migration, resumable upload is supported. However, data may be repeatedly inserted into a non-transactional table that does not have a primary key when the server system breaks down.
- During migration, do not write the statement-based binlog into the source database.
- During migration, do not clear the binlog in the source database.
- During an incremental migration of table-level objects, you are not advised to rename the tables.
- If the source database is an on-premises database and has Percona Server for MySQL 5.6.x or Percona Server for MySQL 5.7.x installed, the memory manager must use Jemalloc to prevent the problem that the database is running out of memory caused by frequent query of system tables.
- During migration, do not create a database named ib_logfile in the source database.
- You are advised to set the expire_log_day parameter to a proper value to ensure that the binlog does not expire before data transfer resumes. This ensures that services can be recovered after interruption.
- DRS automatically checks the configurations and provides handling suggestions. Ensure that the environment configuration meets the following requirements.
Table 2 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- Full migration:
- The source database user must have the SELECT, SHOW VIEW, and EVENT permissions.
- The destination database user must have the following permissions: SELECT, CREATE, ALTER, DROP, DELETE, INSERT, UPDATE, INDEX, EVENT, CREATE VIEW, CREATE ROUTINE, TRIGGER, and WITH GRANT OPTION.
- Full+incremental migration:
- The source database user must have the following permissions: SELECT, SHOW VIEW, EVENT, LOCK TABLES, REPLICATION SLAVE, and REPLICATION CLIENT.
- The destination database user must have the following permissions: SELECT, CREATE, ALTER, DROP, DELETE, INSERT, UPDATE, INDEX, EVENT, CREATE VIEW, CREATE ROUTINE, TRIGGER, and WITH GRANT OPTION.
- Account migration:
- If the source database is a non-Alibaba Cloud database, the account must have the SELECT permission of mysql.user. If the source database is an Alibaba Cloud database, the account must have the SELECT permission of mysql.user and mysql.user_view.
- The destination database user must have the SELECT, INSERT, UPDATE, and DELETE permissions of the MySQL database.
Migration objects
- Supported objects: databases, tables, views, indexes, constraints, functions, stored procedures, triggers, and events.
- The system database and event statuses cannot be migrated.
- Tables with storage engine different to MyISAM and InnoDB tables cannot be migrated.
Source database
- The source database names cannot contain non-ASCII characters, or the following characters: '<`>/\
- The names of the source tables and views cannot contain non-ASCII characters, or the following characters: '<>/\
- The source database name cannot be ib_logfile.
- The binlog of the MySQL source database must be enabled and use the row-based format.
- If the storage space is sufficient, you are advised to store the source database binlog for as long as possible. The recommended retention period is three days.
- If the expire_logs_days value of the source database is set to 0, the migration may fail.
- During an incremental migration, the server_id value of the MySQL source database must be set. If the source database version is MySQL 5.6 or earlier, the server_id value ranges from 2 to 4294967296. If the source database is MySQL 5.7 or later, the server_id value ranges from 1 to 4294967296.
- You are advised to enable skip-name-resolve for the MySQL source database to reduce the possibility of connection timeout.
- You are advised to enable GTID on the source database.
- The source database does not support the mysql binlog dump command.
- The character sets of the source and destination databases must be the same. Otherwise, the migration fails.
- The log_slave_updates parameter of the source database must be enabled. Otherwise, the migration fails.
- The binlog_row_image parameter value of the source database must be set to FULL. Otherwise, the migration fails.
- Currently, the source MySQL 8.0 database does not support migrations with lower_case_table_names set to 0.
Destination database
- Data cannot be migrated from a newer version database to an older version database.
- You are advised to use the row-based binlog in the destination MySQL database. Otherwise, an error may occur during an incremental migration.
- The destination DB instance is running properly.
- The destination DB instance must have sufficient storage space.
- The destination DB instance cannot contain databases with the same name as the source database (except MySQL system databases).
- The destination database isolation level must be set to at least read committed.
- During migration, a large amount of data is written to the destination database. If the value of the max_allowed_packet parameter of the destination database is too small, data cannot be written. You are advised to set the max_allowed_packet parameter to a value greater than 100 MB.
- You are advised to enable GTID on the destination database.
- If the SERVER_UUID values of the source and destination databases are the same, the incremental migration fails.
- The collation_server values of the source and destination databases must be the same. Otherwise, the migration fails.
- The tables referenced by the foreign key must be migrated with the selected migration objects. Otherwise, the migration fails.
- The time_zone values of the source and destination databases must be the same. Otherwise, the migration fails.
- The sql_mode values of the source and destination databases must be the same. Otherwise, the migration fails.
- If the MyISAM tables are included in the migration objects, the sql_mode parameter in the destination database cannot contain the no_engine_substitution parameter. Otherwise, the migration fails.
- The innodb_strict_mode values of the source and destination databases must be the same. Otherwise, the migration fails.
- The lower_case_table_names values of the source and destination databases must be the same. Otherwise, the migration fails.
- The log_bin_trust_function_creators parameter value of the destination database must be set to on. Otherwise, the migration fails.
- Full migration:
MySQL -> DDM
- DRS has the following constraints on common operations in the case of migration failures caused by unpredictable errors or sudden environment changes.
Table 3 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 4 apply to the entire migration process.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the migration task cannot be created.
- If the data types are incompatible, the migration may fail.
Precautions
- During migration, do not modify or delete the usernames, passwords, permissions, or ports of the source and destination databases.
- During an incremental migration, do not modify the table structure to be migrated in the source database.
- During an incremental migration, do not perform the restoration operation on the source database.
- During an incremental migration of table-level objects, you are not advised to rename the tables.
- DDL operations are not supported during the migration.
- You are advised to set the expire_log_day parameter to a proper value to ensure that the binlog does not expire before data transfer resumes. This ensures that services can be recovered after interruption.
- If the source database is an on-premises database and has Percona Server for MySQL 5.6.x or Percona Server for MySQL 5.7.x installed, the memory manager must use Jemalloc to prevent the problem that the database is running out of memory caused by frequent query of system tables.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Table 4 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- Full migration:
- The source database user must have the SELECT, SHOW VIEW, and EVENT permissions.
- The DDM destination database user must have the following permissions: CREATE, DROP, ALTER, INDEX, INSERT, DELETE, UPDATE, and SELECT. In addition, grant the select permission on all tables.
- The DDM destination database user must have the permission on the database to be migrated.
- Full+incremental migration:
- The source database user must have the following permissions: SELECT, SHOW VIEW, EVENT, LOCK TABLES, REPLICATION SLAVE, and REPLICATION CLIENT.
- The DDM destination database user must have the following permissions: CREATE, DROP, ALTER, INDEX, INSERT, DELETE, UPDATE, and SELECT. In addition, grant the select permission on all tables.
- The DDM destination database user must have the permission on the database to be migrated.
Migration objects
- Only the source database data can be migrated to DDM.
- In the destination database, create table structures and indexes that correspond to the source schemas. If the objects that correspond to the source objects are not created in the destination database, the source objects will not be migrated.
- The table structure created in the destination database must be the same as that in the source database.
- Tables with storage engine different to MyISAM and InnoDB tables cannot be migrated.
Source database
- The binlog of the MySQL source database must be enabled and use the row-based format during incremental migration.
- If the storage space is sufficient during the incremental migration, you are advised to store the source database binlog as long as possible. The recommended retention period is three days.
- If the expire_logs_days value of the source database is set to 0, the migration may fail.
- During an incremental migration, the server-id value of the MySQL source database must be set. If the source database version is MySQL 5.6 or earlier, the server_id value ranges from 2 to 4294967296. If the source database is MySQL 5.7 or later, the server_id value ranges from 1 to 4294967296.
- The names of the source databases and tables cannot contain non-ASCII characters, or the following characters: '.<>/\
- You are advised to enable skip-name-resolve for the MySQL source database to reduce the possibility of connection timeout.
- You are advised to enable GTID on the source database.
Destination database
- Ensure that the destination database is empty before starting the migration. Otherwise, data in the destination may be overwritten during incremental migration.
- The destination DB instance and associated RDS DB instance are running properly. If the RDS DB instance is a primary/standby instance, the replication status must be normal.
- The associated RDS DB instance must have sufficient storage space.
- The character set of the associated RDS database must be the same as that of the source database.
- If the destination DB instance uses columns of the TIMESTAMP or DATETIME data type as its sharding key, the seconds precision of the column is removed after the migration.
- The value of AUTO_INCREMENT of a table in the destination database cannot be less than that of AUTO_INCREMENT of a table in the source database.
- Full migration:
PostgreSQL -> RDS PostgreSQL
- DRS has the following constraints on common operations in the case of migration failures caused by unpredictable errors or sudden environment changes.
Table 5 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 6 apply to the entire migration process.
- Full migration:
- The destination database version must be later than PostgreSQL 9.4.
- Data cannot be migrated from the database of a later major version to the database of an earlier major version.
- In the same migration task, multiple source databases cannot be migrated to the same destination database.
- Objects that have dependencies must be migrated at the same time to avoid migration failure. Common dependencies: tables referenced by views, views referenced by views, views and tables referenced by stored procedures/functions/triggers, and tables referenced by primary and foreign keys
- Database objects cannot be selected.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the migration task cannot be created.
Precautions
- During migration, do not modify or delete the usernames, passwords, permissions, or ports of the source and destination databases.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Table 6 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- Full migration:
- The source database account must have the following permissions: SELECT, REFERENCES, TRIGGER, EXECUTE, and USAGE
Migration objects
Full migration:
Supported objects: tables, indexes, foreign keys, stored procedures, functions, views, constraints, triggers, modes, sorting rules, code conversion information, data types, aggregate functions, operators, sequences, materialized views, statistical extensions, rules, event triggers, text search parsers, text search dictionaries, text search template, and text search configurations.
Supported field types: digit, currency, character, binary, date/time, Boolean, enumeration, geometry, network address, bit, text search, UUID, XML, JSON, array, compound, and range.
Source database
- The source database name and table name cannot contain the following characters: "<>
- The source database name cannot contain dots (.), angle brackets (<) and (>), and single quotation marks (').
- If the source database version is 9.5, the wal_level parameter value must be set to hot_standby. If the source database version is 9.6, the wal_level parameter value must be set to replica. Otherwise, the migration fails.
- The max_replication_slots value of the source database must be greater than 0. Otherwise, the migration may fail.
- You are advised to set the wal_keep_segments value of the source database to a value greater than or equal to the value that is calculated as follows: (Source database size x 5% / 16). If the source database has a relatively heavy workload, increase that value accordingly.
Destination database
- Data cannot be migrated from a newer version database to an older version database.
- The destination DB instance is running properly.
- The destination DB instance must have sufficient storage space.
- Many-to-one migration is not supported.
- The root user password of the destination database is required.
- The destination DB instance cannot contain databases with the same name as the source database (except system databases).
- The plugin versions of the destination database must be later than those of the source database. Otherwise, the migration fails.
- The lc_monetary values of the source and destination databases must be the same. Otherwise, the migration fails.
- If the values of max_connections, max_worker_processes, max_prepared_transactions, and max_locks_per_transaction of the destination database must be greater than those of the source database. Otherwise, the migration fails.
- The values of wal_block_size and block_size of the source and destination databases must be the same. Otherwise, the migration fails.
- The destination RDS database version must be 2.5.9 or later. Otherwise, the migration of triggers may fail.
- After the full migration of materialized views, if the destination database needs to use the materialized views, run the following statement to update the views:
- Full migration:
MongoDB -> DDS
- DRS has the following constraints on common operations in the case of migration failures caused by unpredictable errors or sudden environment changes.
Table 7 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 8 apply to the entire migration process.
- Objects that have dependencies must be migrated at the same time to avoid migration failure. Common dependencies: collections referenced by views, and views referenced by views
- MongoDB replica set instances must be available and have primary nodes.
- Source database from a single node instance on other clouds cannot be migrated.
- If the source database is not on a cluster instance, the following operations are supported during incremental migration:
- Creating and deleting databases
- Adding, deleting, and updating documents
- Creating and deleting collections
- Creating and deleting indexes
- Creating and deleting views
- The convertToCapped, collMod, and renameCollection commands are supported.
- During a full plus incremental migration between clusters, the objects to be migrated cannot be deleted. Otherwise, the migration task will fail.
- If a Time-to-Live (TTL) index already exists in the collection of the source database or is created during an incremental migration, data consistency cannot be ensured when source and destination databases are in different time zone.
- If the MongoDB service of the source database is deployed with other services on the same server, you need to set the value of the cacheSizeGB parameter to the half of the minimum idle cache that WiredTiger will use for all data.
- DCC does not support DDS DB instances.
- During an incremental migration of collections, you are advised not to rename the collections.
- If the source database is on a replica set instance, enter information about all primary and secondary nodes to reduce the impact of a primary/secondary switchover on the migration task. If you enter information about multiple primary and secondary nodes, ensure that all nodes belong to the same replica set instance.
- If the source database is on a cluster instance, enter information about multiple mongos nodes to reduce the impact of single-node failure on the migration task. In addition, ensure that all mongos nodes belong to the same cluster instance. For an incremental migration of a cluster instance, you are advised to enter the information about all primary and secondary shard nodes in the same cluster to reduce the impact of a primary/secondary switchover on the migration task.
Precautions
- To ensure data consistency, you are not allowed to modify the destination database (including but not limited to DDL and DML operations) or perform DDL operations on the source database during the entire migration process.
- During migration, do not modify or delete the usernames, passwords, permissions, or ports of the source and destination databases.
- During migration, data rollback caused by a primary/standby switchover of the source database is not supported.
- During an incremental migration of collections, you are advised not to rename the collections.
- To accelerate the migration, you are advised to delete unnecessary indexes from the source database and retain only necessary indexes before the migration. You are advised not to create indexes for the source database during the migration. If indexes must be created, create them in the background.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Table 8 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
Source database
- Full migration:
- Replica set: The source database user must have the readAnyDatabase permission on the admin database.
- Cluster: The source database user must have the readAnyDatabase permission on the admin database and the read permission on the config database.
- Single node: The source database user must have the readAnyDatabase permission on the admin database.
- To migrate accounts and roles of the source database, the source database user must have the read permission on the system.users and system.roles system tables of the admin database.
- Full+incremental migration:
- Replica set: The source database user must have the readAnyDatabase permission on the admin database and have the read permission on the local database.
- Single node: The source database user must have the readAnyDatabase permission on the admin database and have the read permission on the local database.
- Cluster: The source database user that connects to the mongos node must have the readAnyDatabase permission on the admin database and have the read permission on the config database. The source database user that connects to the shard nodes must have the readAnyDatabase permission on the admin database and have the read permission on the local database.
- To migrate accounts and roles of the source database, the source database user must have the read permission on the system.users and system.roles system tables of the admin database.
Permission requirements for the destination database: The destination database user must have the readAnyDatabase permission on the admin database and the readWrite permission on the destination database.
Migration objects
- Replica set: Only collections (including validator and capped collections), indexes, and views can be migrated.
- Cluster: Only collections (including validator and capped collections), shard keys, indexes, and views can be migrated.
- Single node: Only collections (including validator and capped collections), indexes, and views can be migrated.
- Only user data and source database account information can be migrated. The system database and system collection cannot be migrated. If service data is stored in the system database, run the renameCollection command to move the service data to the user database.
- Collections that contain the _id field without indexes are not supported.
Source database
- The names of the source databases, collections, and views cannot contain the following characters: '<>
- If the incremental source data cluster is migrated, the source database balancer must be disabled.
Destination database
- The destination DB instance is running properly.
- The destination DB instance must have sufficient storage space.
- When multiple source databases are migrated to the same destination database, the name of the database to be migrated must be unique.
- DRS supports full migration between cluster instances. If the source cluster instance is not sharded, ensure that the size of the primary shard on the destination database is greater than that of the source database.
- Data cannot be migrated from a newer version database to an older version database.
- Full migration:
MongoDB -> GaussDB(for Mongo)
- DRS has the following constraints on common operations in the case of migration failures caused by unpredictable errors or sudden environment changes.
Table 9 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 10 apply to the entire migration process.
- Objects that have dependencies must be migrated at the same time to avoid migration failure. Common dependencies: collections referenced by views, and views referenced by views
- MongoDB replica set instances must be available and have primary nodes.
- If the source database is not on a cluster instance, the following operations are supported during incremental migration:
- Creating and deleting databases
- Adding, deleting, and updating documents
- Creating and deleting collections
- Creating and deleting indexes
- Creating and deleting views
- The convertToCapped, collMod, and renameCollection commands are supported.
- During a full plus incremental migration from a replica set to a cluster or between clusters, the objects to be migrated cannot be deleted. Otherwise, the migration task will fail.
- If a Time-to-Live (TTL) index already exists in the collection of the source database or is created during an incremental migration, data consistency cannot be ensured when source and destination databases are in different time zone.
- If the MongoDB service of the source database is deployed with other services on the same server, you need to set the value of the cacheSizeGB parameter to the half of the minimum idle cache that WiredTiger will use for all data.
- During an incremental migration of collections, you are advised not to rename the collections.
- If the source database is on a replica set instance, enter information about all primary and secondary nodes to reduce the impact of a primary/secondary switchover on the migration task. If you enter information about multiple primary and secondary nodes, ensure that all nodes belong to the same replica set instance.
If the source database is on a cluster instance, enter information about multiple mongos nodes to reduce the impact of single-node failure on the migration task. In addition, you need to ensure that all mongos nodes belong to the same cluster instance. For an incremental migration of a cluster instance, you are advised to enter the information about all primary and secondary shard nodes in the same cluster to reduce the impact of a primary/secondary switchover on the migration task.
- The maximum number of collections that can be created in a GaussDB(for Mongo) cluster is calculated as follows: Maximum number of chunks x 4 x Number of shards
Maximum number of chunks for different GaussDB(for Mongo) instance class:
1 vCPUs, 4 GB -> 50
2 vCPUs, 8 GB -> 100
4 vCPUs, 16 GB -> 200
8 vCPUs, 32 GB -> 400
16 vCPUs, 64 GB -> 800
32 vCPUs, 128 GB -> 1600
Check whether the destination database meets the requirements based on the number of source database collections.
Precautions
- To ensure data consistency, you are not allowed to modify the destination database (including but not limited to DDL and DML operations) or perform DDL operations on the source database during the entire migration process.
- During migration, do not modify or delete the usernames, passwords, permissions, or ports of the source and destination databases.
- During migration, data rollback caused by a primary/standby switchover of the source database is not supported.
- During an incremental migration of collections, you are advised not to rename the collections.
- To accelerate the migration, you are advised to delete unnecessary indexes from the source database and retain only necessary indexes before the migration. You are advised not to create indexes for the source database during the migration. If indexes must be created, create them in the background.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Table 10 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
Source database
- Full migration:
- Replica set: The source database user must have the readAnyDatabase permission on the admin database.
- Cluster: The source database user must have the readAnyDatabase permission on the admin database and the read permission on the config database.
- Single node: The source database user must have the readAnyDatabase permission on the admin database.
- To migrate accounts and roles of the source database, the source database user must have the read permission on the system.users and system.roles system tables of the admin database.
- Full+incremental migration:
- Replica set: The source database user must have the readAnyDatabase permission on the admin database and have the read permission on the local database.
- Single node: The source database user must have the readAnyDatabase permission on the admin database and have the read permission on the local database.
- Cluster: The source database user that connects to the mongos node must have the readAnyDatabase permission on the admin database and have the read permission on the config database. The source database user that connects to the shard nodes must have the readAnyDatabase permission on the admin database and have the read permission on the local database.
- To migrate accounts and roles of the source database, the source database user must have the read permission on the system.users and system.roles system tables of the admin database.
Permission requirements for the destination database: The destination database user must have the readAnyDatabase permission on the admin database and the readWrite permission on the destination database.
Migration objects
- Replica set: Only collections (including validator and capped collections), indexes, and views can be migrated.
- Cluster: Only collections (including validator and capped collections), shard keys, indexes, and views can be migrated.
- Single node: Only collections (including validator and capped collections), indexes, and views can be migrated.
- Collections that contain the _id field without indexes are not supported.
Source database
- The names of the source databases, collections, and views cannot contain the following characters: '<>
- If the incremental source data cluster is migrated, the source database balancer must be disabled.
- The source database cannot be a GaussDB(for Mongo) instance.
Destination database
- The destination DB instance is running properly.
- The destination DB instance must have sufficient storage space.
- When multiple source databases are migrated to the same destination database, the name of the database to be migrated must be unique.
- DRS supports full migration between cluster instances. If the source cluster instance is not sharded, ensure that the size of the primary shard on the destination database is greater than that of the source database.
- Data cannot be migrated from a newer version database to an older version database.
- Full migration:
Oracle Database -> RDS MySQL
- DRS has the following constraints on common operations in the case of migration failures caused by unpredictable errors or sudden environment changes.
Table 11 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 12 apply to the entire migration process.
- Objects that have dependencies must be migrated at the same time to avoid migration failure. Common dependencies: tables referenced by primary or foreign keys
- After objects such as tables are migrated to the destination database, their names are converted into lowercase letters. For example, the name of object ABC is changed to abc after being migrated to the destination database.
- The time zone settings of the source and destination database must be the same.
- If there are special characters such as Chinese and Japanese, the code used by the service to connect to the Oracle database must be the same as the code of the Oracle server. Otherwise, garbled characters are displayed in the destination database.
- After the Oracle table structure is migrated to the MySQL database, the character set of the table is utf8mb4.
- If a table does not have a primary key to uniquely identify each row and the network connection is unstable, data in the destination database may be inconsistent with that in the source database after migration.
- If the length of a table structure in the Oracle database exceeds 65535, the migration may fail. The length of a table structure is the total length of all columns. The length of the char or varchar2 type is related to the code.
- If the source database is an Oracle RAC environment, you cannot use SCANIP to create tasks. You can use a node as the source database. Except the node used for task configuration, other nodes of the source database can still be migrated if they become faulty.
- If the data types are incompatible, the migration may fail.
Precautions
- During the migration, writing data to the destination databases is not allowed. Otherwise, data inconsistency may occur.
- During migration, do not modify or delete the usernames, passwords, permissions, or ports of the source and destination databases.
- During migration, DDL operations cannot be performed on the source database.
- During an incremental migration of table-level objects, you are not advised to rename the tables.
- During migration, do not create a database named ib_logfile in the source database.
- DRS automatically checks the configurations and provides handling suggestions. Ensure that the environment configuration meets the following requirements.
Table 12 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- Full migration:
- Source database: The source database user must have the CREATE SESSION, SELECT ANY TRANSACTION, SELECT ANY TABLE, and SELECT ANY DICTIONARY permissions.
- The destination database account must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, UPDATE, ALTER, INDEX, EVENT, RELOAD, CREATE VIEW, CREATE ROUTINE, and TRIGGER.
- Full+incremental migration:
- Source database: The source database must have the following permissions and roles: CREATE SESSION, SELECT ANY TRANSACTION, SELECT ANY TABLE, SELECT ANY DICTIONARY, and EXECUTE_CATALOG_ROLE. If the Oracle database version is 12c or later, the LOGMINING permission is required.
- The destination database account must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, UPDATE, ALTER, INDEX, EVENT, RELOAD, CREATE VIEW, CREATE ROUTINE, and TRIGGER.
Migration objects
- Databases, table structures, primary keys, unique keys, foreign keys, normal indexes, and table data can be migrated. Other database objects, such as stored procedures, triggers, functions, sequences, packages, synonyms, and users, cannot be migrated.
- Incremental migration does not support DDL synchronization.
- Full migration does not support the following column types: bfile, xml, sdo_geometry, urowid, and self-defined types.
- Incremental migration does not support the following column types: bfile, xml, interval, sdo_geometry, urowid, and self-defined types.
Source database
- The maximum row length of Oracle cannot exceed 8 KB, excluding BLOB and TEXT columns because the MySQL InnoDB restricts the row length to 8 KB.
- The primary key or unique key column cannot contain values of the string data type when you map the MySQL data types to the character data types in Oracle because MySQL cannot tell spaces in data. Otherwise, data inconsistency and deadlock may occur.
- The values of binary_float and binary_double cannot be set to Nan, Inf, or -Inf because MySQL does not support these values.
- MySQL does not support the migration of the check constraints of Oracle.
- AUTO_PK_ROW_ID cannot be used as a column name in Oracle because it is a reserved column name in MySQL 5.7 and cannot be created.
- The values of the NUMBER and INT fields in Oracle cannot exceed the precision and scale of (65, 30) and (65, 0), respectively because the precision permitted in MySQL is smaller than that of Oracle.
- The names of databases and tables cannot contain non-ASCII characters and the following special characters .><\`|,?'!"
- The source database name cannot be ib_logfile.
- The default value of the unsupported field is an expression, for example, default(`id` + 1).
- If a function is used as the default value in the source database, the function must also be supported in the destination database.
- During incremental migration from Oracle to MySQL, archive logs must be enabled on the source Oracle database.
- The total index length of columns in the source database cannot exceed the length limit in the destination database. For detailed length requirements, see Index Length Description.
- The Default User statement is not supported in MySQL.
- The source database cannot contain empty databases.
- Currently, only the following character sets are supported: ZHS16GBK, AL32UTF8, UTF8, US7ASCII, and WE8MSWIN1252.
Destination database
- The destination database cannot contain the database to be migrated.
- During a synchronization, a large amount of data is written to the destination database. If the value of the max_allowed_packet parameter of the destination database is too small, data cannot be written. You are advised to set the max_allowed_packet parameter to a value greater than 100 MB.
- Full migration:
Oracle -> RDS PostgreSQL
- DRS has the following constraints on common operations in the case of migration failures caused by unpredictable errors or sudden environment changes.
Table 13 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 14 apply to the entire migration process.
- Objects that have dependencies must be migrated at the same time to avoid migration failure. Common dependencies: tables referenced by primary or foreign keys
- Only full migration is supported.
- You can migrate one database (owner) at a time in a single migration task. To migrate multiple databases, you need to create multiple tasks.
- After object such as tables are migrated to the destination database, their names are converted into lowercase letters. For example, the name of object ABC is changed to abc after being migrated to the destination database.
- The time zone settings of the source and destination database must be the same.
- If there are special characters such as Chinese and Japanese, the code used by the service to connect to the Oracle database must be the same as the code of the Oracle server. Otherwise, garbled characters are displayed in the destination database.
- If a table does not have a primary key to uniquely identify each row and the network connection is unstable, data in the destination database may be inconsistent with that in the source database after migration.
- If the data types are incompatible, the migration may fail.
- The default value cannot be a function.
Precautions
- During the migration, writing data to the source and destination databases is not allowed. Otherwise, data inconsistency may occur.
- During migration, do not modify or delete the usernames, passwords, permissions, or ports of the source and destination databases.
- During migration, DDL operations cannot be performed on the source database.
- DRS automatically checks the configurations and provides handling suggestions. Ensure that the environment configuration meets the following requirements.
Table 14 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- Source database: The source database user must have the CREATE SESSION, SELECT ANY TRANSACTION, SELECT ANY TABLE, SELECT ANY DICTIONARY, and SELECT ANY SEQUENCE permissions.
- Destination database: You must have the permission to create databases and tables in the destination database.
Migration objects
- Tables, indexes, constraints, sequences, and data can be migrated. Other database objects, such as stored procedures, cannot be migrated.
- Full migration does not support the following column types: bfile, xml, sdo_geometry, urowid, and self-defined types.
- Functions and indexes cannot be migrated.
create index idx_t on t(substr(dt, 1, 8));
Source database
- PostgreSQL and Oracle database structures are different. The former has a schema. Therefore the statement for creating views cannot contain db.Table. Otherwise, the view migration fails.
Example: Change the statement 1 to statement 2.
Statement 1:
create view v1 as select id from db1.t1;
Statement 2:
create view v1 as select id from t1;
- The maximum precision supported by timestamp and interval day to second is 6.
- The data type cannot be bfile, xmltype, sdo_geometry, or user-defined.
- The source database cannot contain tables with same names but different letter cases.
- The names of databases, tables, and views cannot contain non-ASCII characters and the following special characters: .><\`|,?'!"
- If the destination database is PostgreSQL Enhanced Edition, the trigger name in the source database must be unique.
- The source database cannot contain empty databases.
- Currently, only the following character sets are supported: ZHS16GBK, AL32UTF8, UTF8, US7ASCII, and WE8MSWIN1252.
Destination database
- The destination database must be in the RDS PostgreSQL Enhanced Edition DB instance.
- Before the migration, create an empty database in the destination instance and name it as the source database name in lowercase letter.
- The destination database cannot contain objects whose names are lowercase version of the source database object names.
MySQL Partition Table -> DDM
- DRS has the following constraints on common operations in the case of migration failures caused by unpredictable errors or sudden environment changes.
Table 15 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 16 apply to the entire migration process.
- If DCC does not support 4 vCPUs | 8 GB or larger instance specifications, the migration task cannot be created.
- If the data types are incompatible, the migration may fail.
Precautions
- During migration, do not modify or delete the usernames, passwords, permissions, or ports of the source and destination databases.
- During an incremental migration, do not modify the table structure to be migrated in the source database.
- During an incremental migration, do not perform the restoration operation on the source database.
- During an incremental migration of table-level objects, you are not advised to rename the tables.
- DDL operations are not supported during the migration.
- You are advised to set the expire_log_day parameter to a proper value to ensure that the binlog does not expire before data transfer resumes. This ensures that services can be recovered after interruption.
- If the source database is an on-premises database and has Percona Server for MySQL 5.6.x or Percona Server for MySQL 5.7.x installed, the memory manager must use Jemalloc to prevent the problem that the database is running out of memory caused by frequent query of system tables.
- DRS automatically checks the configurations and provides handling suggestions. Ensure that the environment configuration meets the following requirements.
Table 16 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- Full migration:
- The source sharded database user must have the SELECT, SHOW VIEW, and EVENT permissions.
- The DDM destination database user must have the following permissions: CREATE, DROP, ALTER, INDEX, INSERT, DELETE, UPDATE, and SELECT. In addition, grant the select permission on all tables.
- The DDM destination database user must have the permission on the database to be migrated.
- Full+incremental migration:
- The source sharded database user must have the following permissions: SELECT, SHOW VIEW, EVENT, LOCK TABLES, REPLICATION SLAVE, and REPLICATION CLIENT.
- The DDM destination database user must have the following permissions: CREATE, DROP, ALTER, INDEX, INSERT, DELETE, UPDATE, and SELECT. In addition, grant the select permission on all tables.
- The DDM destination database user must have the permission on the database to be migrated.
Migration objects
- Only the source database data can be migrated to DDM.
- In the destination database, create table structures and indexes that correspond to the source schemas. If the objects that correspond to the source objects are not created in the destination database, the source objects will not be migrated.
- The table structure created in the destination database must be the same as that in the source database.
- If the source database is a DDM database, the table cannot contain sharding keys of the timestamp type.
- Tables with storage engine different to MyISAM and InnoDB tables cannot be migrated.
Source database
- The binlog of the MySQL source database must be enabled and use the row-based format during incremental migration.
- If the storage space is sufficient during the incremental migration, you are advised to store the source database binlog as long as possible. The recommended retention period is three days.
- If the expire_logs_days value of the source database is set to 0, the migration may fail.
- During an incremental migration, the server-id value of the MySQL source database must be set. If the source database version is MySQL 5.6 or earlier, the server_id value ranges from 2 to 4294967296. If the source database is MySQL 5.7 or later, the server_id value ranges from 1 to 4294967296.
- The database names and table names of the source database sharding middleware cannot contain the following characters: '<>/\ and non-ASCII characters.
- You are advised to enable skip-name-resolve for the MySQL source database to reduce the possibility of connection timeout.
- You are advised to enable GTID on the source database.
Destination database
- Ensure that the destination database is empty before starting the migration. Otherwise, data in the destination may be overwritten during incremental migration.
- The destination DB instance and associated RDS DB instance are running properly. If the RDS DB instance is a primary/standby instance, the replication status must be normal.
- The associated RDS DB instance must have sufficient storage space.
- The character set of the associated RDS database must be the same as that of the source database.
- If the destination DB instance uses columns of the TIMESTAMP or DATETIME data type as its sharding key, the seconds precision of the column is removed after the migration.
- The value of AUTO_INCREMENT of a table in the destination database cannot be less than that of AUTO_INCREMENT of a table in the source database.
- Full migration:
MySQL -> GaussDB(for MySQL)
- DRS has the following constraints on common operations in the case of migration failures caused by unpredictable errors or sudden environment changes.
Table 17 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 18 apply to the entire migration process.
- Supported objects: databases, tables, views, indexes, constraints, functions, stored procedures, triggers, and events.
- The system database and event statuses cannot be migrated.
- Encrypted tables cannot be migrated.
- Tables with storage engine different to MyISAM and InnoDB tables cannot be migrated.
Precautions
- When a migration task is completed, the system begins to migrate the selected events and triggers. You must therefore check the status of the migration log to ensure database integrity.
- During migration, do not modify or delete the usernames, passwords, permissions, or ports of the source and destination databases.
- During an incremental migration, do not perform the point-in-time recovery (PITR) operation on the source database.
- During an incremental migration, if distributed transactions exist in the source database, the migration may fail.
- To ensure data consistency, you are not allowed to modify the destination database (including but not limited to DDL and DML operations) during migration.
- During an incremental migration, resumable upload is supported. However, data may be repeatedly inserted into a non-transactional table that does not have a primary key when the server system breaks down.
- Do not disconnect the source database from the public network before the migration task is complete.
- During migration, do not write the statement-based binlog into the source database.
- During migration, do not clear the binlog in the source database.
- To prevent data being migrated from being locked by other transactions, you are advised to migrate data during off-peak hours.
- During migration, TaurusDB automatically converts the MyISAM table to the InnoDB table. If the conversion fails, the migration fails.
- During an incremental migration of table-level objects, you are not advised to rename the tables.
- You are advised to set the expire_log_day parameter to a proper value to ensure that the binlog does not expire before data transfer resumes. This ensures that services can be recovered after interruption.
- If the source database is an on-premises database and has Percona Server for MySQL 5.6.x or Percona Server for MySQL 5.7.x installed, the memory manager must use Jemalloc to prevent the problem that the database is running out of memory caused by frequent query of system tables.
- DRS automatically checks the configurations and provides handling suggestions. Ensure that the environment configuration meets the following requirements.
Table 18 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- Full migration:
- The source database user must have the SELECT, SHOW VIEW, and EVENT permissions.
- The destination database user must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, UPDATE, INDEX, EVENT, CREATE VIEW, CREATE ROUTINE, TRIGGER, and WITH GRANT OPTION.
- Full+incremental migration:
- The source database user must have the following permissions: SELECT, SHOW VIEW, EVENT, LOCK TABLES, REPLICATION SLAVE, and REPLICATION CLIENT.
- The destination database user must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, UPDATE, INDEX, EVENT, CREATE VIEW, CREATE ROUTINE, TRIGGER, and WITH GRANT OPTION.
Migration objects
- Supported objects: databases, tables, views, indexes, constraints, functions, stored procedures, triggers, and events.
- The system database and event statuses cannot be migrated.
- Encrypted tables cannot be migrated.
- Tables with storage engine different to MyISAM and InnoDB tables cannot be migrated.
Source database
- The names of the source databases, tables, and views cannot contain non-ASCII characters, or the following characters: '.<>/\
- The binlog of the MySQL source database must be enabled and use the row-based format.
- If the storage space is sufficient, you are advised to store the source database binlog for as long as possible. The recommended retention period is three days.
- If the expire_logs_days value of the source database is set to 0, the migration may fail.
- During an incremental migration, the server_id value of the MySQL source database must be set. If the source database version is MySQL 5.6 or earlier, the server_id value ranges from 2 to 4294967296. If the source database is MySQL 5.7 or later, the server_id value ranges from 1 to 4294967296.
- You are advised to enable skip-name-resolve for the MySQL source database to reduce the possibility of connection timeout.
- You are advised to enable GTID on the source database.
- The source database cannot contain empty databases.
Destination database
- You are advised to use the row-based binlog in the destination MySQL database. Otherwise, an error may occur during an incremental migration.
- The destination DB instance is running properly.
- The destination DB instance cannot contain databases with the same name as the source database (except MySQL system databases).
- The destination database isolation level must be set to at least read committed.
- Full migration:
Oracle -> GaussDB(for MySQL) scenario
- DRS has the following constraints on common operations in the case of migration failures caused by unpredictable errors or sudden environment changes.
Table 19 Operation constraints Type
Operation Constraints
Notes
- Requirements in Table 20 apply to the entire migration process.
- Objects that have dependencies must be migrated at the same time to avoid migration failure. Common dependencies: tables referenced by primary or foreign keys
- After object such as tables are migrated to the destination database, their names are converted into lowercase letters. For example, the name of object ABC is changed to abc after being migrated to the destination database.
- The time zone settings of the source and destination database must be the same.
- If there are special characters such as Chinese and Japanese, the code used by the service to connect to the Oracle database must be the same as the code of the Oracle server. Otherwise, garbled characters are displayed in the destination database.
- After the Oracle table structure is migrated to the GaussDB(for MySQL) database, the character set of the table is utf8mb4.
- If a table does not have a primary key to uniquely identify each row and the network connection is unstable, data in the destination database may be inconsistent with that in the source database after migration.
- If the length of a table structure in the Oracle database exceeds 65535, the migration may fail. The length of a table structure is the total length of all columns. The length of the char or varchar2 type is related to the code.
- If the data types are incompatible, the migration may fail.
Precautions
- During the migration, writing data to the destination databases is not allowed. Otherwise, data inconsistency may occur.
- During migration, do not modify or delete the usernames, passwords, permissions, or ports of the source and destination databases.
- During migration, DDL operations cannot be performed on the source database.
- During an incremental migration of table-level objects, you are not advised to rename the tables.
- Ensure that the environment configuration meets the following constraints. DRS automatically checks the configurations and provides handling suggestions.
Table 20 Environment constraints Type
Usage Constraints (DRS Automatic Check)
Database permissions
- Full migration:
- Source database: The source database user must have the CREATE SESSION, SELECT ANY TRANSACTION, SELECT ANY TABLE, and SELECT ANY DICTIONARY permissions.
- The destination database account must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, UPDATE, ALTER, INDEX, EVENT, RELOAD, CREATE VIEW, CREATE ROUTINE, and TRIGGER.
- Full+incremental migration:
- Source database: The source database must have the following permissions and roles: CREATE SESSION, SELECT ANY TRANSACTION, SELECT ANY TABLE, SELECT ANY DICTIONARY, and EXECUTE_CATALOG_ROLE. If the Oracle database version is 12c or later, the LOGMINING permission is required.
- The destination database account must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, UPDATE, ALTER, INDEX, EVENT, RELOAD, CREATE VIEW, CREATE ROUTINE, and TRIGGER.
Migration objects
- Databases, table structures, primary keys, unique keys, foreign keys, normal indexes, and table data can be migrated. Other database objects, such as stored procedures, triggers, functions, sequences, packages, synonyms, and users, cannot be migrated.
- Incremental migration does not support DDL synchronization.
- Incremental migration does not support the following column types: bfile, xml, interval, sdo_geometry, and self-defined types.
Source database
- The maximum row length of Oracle cannot exceed 8 KB, excluding BLOB and TEXT columns because the InnoDB restricts the row length to 8 KB.
- The primary key or unique key column cannot contain values of the string data type when you map the GaussDB(for MySQL) data types to the character data types in Oracle because GaussDB(for MySQL) cannot tell spaces in data. Otherwise, data inconsistency and deadlock may occur.
- The values of binary_float and binary_double cannot be set to Nan, Inf, or -Inf because GaussDB(for MySQL) does not support these values.
- The value of the NUMBER field in Oracle cannot exceed the precision and scale of (65, 30). The value of the INT field cannot exceed the precision and scale of (65, 0). The digit range of GaussDB(for MySQL) is smaller than that of Oracle.
- The names of databases, tables, and views cannot contain non-ASCII characters and the following special characters: .><\`|,?'!"
- The default value of the unsupported field is an expression, for example, default(`id` + 1).
- If a function is used as the default value in the source database, the function must also be supported in the destination database.
- During an incremental migration, archive logs must be enabled on the source database.
- Currently, only the following character sets are supported: ZHS16GBK, AL32UTF8, UTF8, US7ASCII, and WE8MSWIN1252.
- The total index length of columns in the source database cannot exceed the length limit in the destination database. For detailed length requirements, see Index Length Description.
- The Default User statement is not supported in GaussDB(for MySQL).
- The source database cannot contain empty databases.
Destination database
- The destination database cannot contain the database to be migrated.
- During a synchronization, a large amount of data is written to the destination database. If the value of the max_allowed_packet parameter of the destination database is too small, data cannot be written. You are advised to set the max_allowed_packet parameter to a value greater than 100 MB.
- Full migration:
Last Article: Migrating Data to the Cloud
Next Article: Creating a Migration Task
Did this article solve your problem?
Thank you for your score!Your feedback would help us improve the website.