Supported Data Sources

Offline data integration jobs support the following synchronization modes: from a table to another table, from a database to a table, and from a database/table partition to a table. The supported data sources vary depending on the synchronization mode.

Table synchronization: synchronization of table or file data to a data lake or migration of such data to the cloud. For details about the supported data sources, see Data Sources Supported for Table/File Synchronization.
Database/table partition synchronization: synchronization of multiple databases or tables to a data lake or migration of them to the cloud. For details about the supported data sources, see Data Sources Supported for Database/Table Partition Synchronization.
Entire DB migration: import of data into a data lake or migration of data from on-premises databases to the cloud. For details, see Data Sources Supported for Entire DB Synchronization.

The supported data sources vary depending on the CDM cluster version.

Different CDM clusters support different data sources.

Data Sources Supported for Table/File Synchronization

Table/File synchronization can synchronize data in tables or files.

Table 1 lists the data sources that support single table synchronization.

**Table 1** Reading and writing of a single table from different data sources in offline jobs
Category	Data Source	Read	Write	Description
Data warehouse	GaussDB(DWS) and DLI	Supported	Supported	The GaussDB(DWS) physical machine management mode is not supported.
Hadoop	MRS Hive, MRS Hudi, Doris, MRS ClickHouse, and MRS HBase	Supported	Supported	Recommended MRS ClickHouse version: 21.3.4.X Recommended MRS HBase versions: 2.1.X 1.3.X MRS Hive and MRS Hudi 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X
Object storage	OBS	Supported	Supported	-
File system	FTP and SFTP	Supported	Not supported	-
Relational database	RDS for MySQL, RDS for PostgreSQL, RDS for SQL Server, Oracle, RDS for SAP HANA, and GBase 8a	Supported	Supported	You can create a connection to your on-premises database, such as MySQL, PostgreSQL, SQL Server, Dameng (DM), and SAP HANA by selecting RDS(MySQL), RDS(PostgreSQL), RDS(SQL Server), RDS(DM), and RDS(SAP HANA), respectively on the page for creating a data connection. SAP HANA: Only the 2.00.050.00.1592305219 version is supported.
Relational database	RDS for Dameng database	Not supported	Not supported	-
Non-relational database	MongoDB and Redis	Supported	Supported	Recommended MongoDB version: 4.2 MongoDB data sources with SSL enabled are not supported.
Message system	Apache HDFS and DMS Kafka	Supported	Supported	Apache HDFS can only be the source. Recommended Apache HDFS versions: 2.8.X 3.1.X
	LTS	Supported	Not supported	-
	Apache RocketMq	Not supported	Supported	-
Search	Elasticsearch	Supported	Supported	-
Other	Rest Client	Supported	Not supported	-
Other	OpenGauss (GaussDB)	Supported	Supported	-

Data Sources Supported for Database/Table Partition Synchronization

Database/Table partition synchronization is applicable to synchronizing data from on-premises data centers or ECS-hosted databases to database services or big data services on the cloud. It is suitable for the synchronization of multiple databases and tables.

The data sources supported for database/table partition synchronization are as follows:

If the migration source is RDS for MySQL, database/table partition synchronization is supported.

Data Sources Supported for Entire DB Synchronization

Entire DB synchronization is used when an on-premises data center or a database created on an ECS needs to be synchronized to a database service or big data service on the cloud. It is suitable for offline database synchronization but not online real-time synchronization.

The following data sources are supported for entire DB synchronization (they can serve as the source or destination in different links):