Supported Data Sources

Offline data integration jobs support the following synchronization modes: from a table to another table, from a database to a table, and from a database/table partition to a table. The supported data sources vary depending on the synchronization mode.

Table synchronization: synchronization of table or file data to a data lake or migration of such data to the cloud. For details about the supported data sources, see Data Sources Supported for Table/File Synchronization.
Database/table partition synchronization: synchronization of multiple databases or tables to a data lake or migration of them to the cloud. For details about the supported data sources, see Data Sources Supported for Database/Table Partition Synchronization.
Entire DB migration: import of data into a data lake or migration of data from on-premises databases to the cloud. For details, see Data Sources Supported for Entire DB Synchronization.

The supported data sources vary depending on the CDM cluster version.

Different CDM clusters support different data sources.

Data Sources Supported for Table/File Synchronization

Table/File synchronization can synchronize data in tables or files.

The following data sources are supported for single-table synchronization (they can serve as the source or destination in different links):

Data warehouse: DWS and DLI
Hadoop: MRS Hive, Apache Hive, MRS HBase, MRS Hudi, OpenSource ClickHouse, Apache HDFS, and Doris
Object-based storage: Object Storage Service (OBS)
File systems: FTP and SFTP
Relational databases: RDS (MySQL), RDS (PostgreSQL), RDS (SQL Server), RDS (DM), RDS (SAP HANA), Oracle, and GBASE

You can create a connection to your on-premises database, such as MySQL, PostgreSQL, SQL Server, Dameng (DM), and SAP HANA by selecting RDS(MySQL), RDS(PostgreSQL), RDS(SQL Server), RDS(DM), and RDS(SAP HANA), respectively on the page for creating a data connection.
Non-relational databases: MongoDB and Redis
Message system: DMS Kafka
Search: Elasticsearch
Other: Rest Client

Data Sources Supported for Database/Table Partition Synchronization

Database/Table partition synchronization is applicable to synchronizing data from on-premises data centers or ECS-hosted databases to database services or big data services on the cloud. It is suitable for the synchronization of multiple databases and tables.

The data sources supported for database/table partition synchronization are as follows:

If the migration source is RDS (MySQL), database/table partition synchronization is supported.

Data Sources Supported for Entire DB Synchronization

Entire DB synchronization is used when an on-premises data center or a database created on an ECS needs to be synchronized to a database service or big data service on the cloud. It is suitable for offline database synchronization but not online real-time synchronization.

The following data sources are supported for entire DB synchronization (they can serve as the source or destination in different links):