Supported Data Sources (2.10.0.300)

CDM provides the following migration modes which support different data sources:

Table/File migration in the import of data into a data lake or migration of data to the cloud. For details, see Data Sources Supported by Table/File Migration.
Entire DB migration in the import of data into a data lake or migration of data to the cloud. For details, see Supported Data Sources in Entire DB Migration.

This section describes the data sources supported by CDM clusters of version 2.10.0.300. The supported data sources vary depending on the CDM cluster version.

Data Sources Supported by Table/File Migration

Table/File migration can migrate data in tables or files.

Table 1 describes the supported data sources.

**Table 1** Supported data sources during table/file migration
Category	Source	Destination	Description
Data warehouse	GaussDB(DWS)	Data warehouse: GaussDB(DWS), Data Lake Insight (DLI), Doris, and MRS ClickHouse Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, RDS for SQL Server, MySQL, PostgreSQL, Microsoft SQL Server, and Oracle Search: Elasticsearch In OBT: Cloud Search Service (CSS) and CloudTable	The DWS physical machine management mode is not supported.
	Data Lake Insight (DLI)	Data warehouse: GaussDB(DWS), Data Lake Insight (DLI), Doris, and MRS ClickHouse Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, RDS for SQL Server, MySQL, PostgreSQL, Microsoft SQL Server, and Oracle NoSQL: MongoDB Search: Elasticsearch In OBT: Cloud Search Service (CSS) and CloudTable	Recommended MongoDB version: 4.2
	MRS ClickHouse	Data warehouse: MRS ClickHouse and Data Lake Insight (DLI)	Recommended MRS ClickHouse version: 21.3.4.X MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
	Doris	Data warehouses: Doris	MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
Hadoop	MRS HDFS	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, RDS for SQL Server, MySQL, PostgreSQL, Microsoft SQL Server, and Oracle Search: Elasticsearch In OBT: Cloud Search Service (CSS) and CloudTable	Supported by local storage. Only MRS Hive and MRS Hudi are supported in storage-compute decoupling scenarios. Only MRS Hive is supported in Ranger scenarios. Not supported if SSL is enabled for ZooKeeper Recommended MRS HDFS versions: 2.8.X 3.1.X Recommended MRS HBase versions: 2.1.X 1.3.X MRS Hive and MRS Hudi 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
	MRS HBase
	MRS Hive	Data warehouse: GaussDB(DWS), Data Lake Insight (DLI),, Doris, and MRS ClickHouse Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, RDS for SQL Server, MySQL, PostgreSQL, Microsoft SQL Server, and Oracle Search: Elasticsearch In OBT: Cloud Search Service (CSS), CloudTable, and SAP HANA
	MRS Hudi	Data warehouse: GaussDB(DWS) Hadoop: MRS HBase
	Apache HBase	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Search: Elasticsearch In OBT: Cloud Search Service (CSS) and CloudTable	Apache cannot serve as the destination. Supported only by local storage and not in storage-compute decoupling scenarios Not supported by Ranger Not supported if SSL is enabled for ZooKeeper Recommended Apache HBase versions: 2.1.X 1.3.X Apache Hive 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X Recommended Apache HDFS versions: 2.8.X 3.1.X
	Apache Hive
	Apache HDFS
Object storage	Object Storage Service (OBS)	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Search: Elasticsearch In OBT: Cloud Search Service (CSS) and CloudTable	Object Storage Migration Service (OMS) is recommended for migration between object storage services. Binary files cannot be imported to a database or NoSQL.
File system	FTP	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Search: Elasticsearch Object storage: Object Storage Service (OBS) In OBT: Cloud Search Service (CSS) and CloudTable	The file system cannot serve as the destination. Only text files such as CSV files can be migrated from FTP or SFTP servers to search services. Binary files cannot. Only binary files can be migrated from FTP or SFTP servers to OBS. obsutil is recommended for migrating data from HTTP servers to OBS. For details, see Introduction to obsutil.
	SFTP
	HTTP	Hadoop: MRS HDFS
Relational database	RDS for MySQL	Data warehouse: GaussDB(DWS), Data Lake Insight (DLI), and Doris Hadoop: MRS HDFS, MRS HBase, MRS Hive, and MRS Hudi Object storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, and RDS for SQL Server Search: Elasticsearch In OBT: Cloud Search Service (CSS), CloudTable, and SAP HANA	You are advised to use Data Replication Service (DRS) to migrate data between OLTP databases. Recommended Microsoft SQL Server version: 2005 or later The KingBase database and GaussDB can be connected using the PostgreSQL connector. The supported source and destination are the same as those of the PostgreSQL data source.
	RDS for SQL Server	Data warehouse: GaussDB(DWS) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, and RDS for SQL Server Search: Elasticsearch In OBT: Cloud Search Service (CSS) and CloudTable
	RDS for PostgreSQL
	MySQL	Data warehouse: GaussDB(DWS) Hadoop: MRS HDFS, MRS HBase, MRS Hive, and MRS Hudi Object storage: Object Storage Service (OBS) Search: Elasticsearch In OBT: Cloud Search Service (CSS) and CloudTable
	PostgreSQL
	Oracle
	Microsoft SQL Server	Data warehouse: GaussDB(DWS) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Search: Elasticsearch In OBT: Cloud Search Service (CSS) and CloudTable
NoSQL	Distributed Cache Service (DCS)	Hadoop: MRS HDFS, MRS HBase, and MRS Hive	NoSQL cannot serve as the destination. For how to migrate data from Redis to DCS, see Migrating Data from Self-Hosted Redis to DCS.
	Redis
	MongoDB
Message system	Data Ingestion Service (DIS)	In OBT: Cloud Search Service (CSS)	The message system cannot serve as the destination.
	Apache Kafka
	DMS Kafka
	MRS Kafka	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, and RDS for SQL Server Search: Elasticsearch In OBT: CloudTable and Cloud Search Service (CSS)	MRS Kafka cannot serve as the destination. Supported only by local storage and not in storage-compute decoupling scenarios Not supported by Ranger Not supported if SSL is enabled for ZooKeeper MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
Search	Elasticsearch	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, and RDS for SQL Server Search: Elasticsearch In OBT: CloudTable and Cloud Search Service (CSS)	Only the non-security mode is supported.
In OBT	CloudTable HBase	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, RDS for SQL Server, MySQL, PostgreSQL, Microsoft SQL Server, and Oracle Search: Elasticsearch In OBT: CloudTable and Cloud Search Service (CSS)	-
	Cloud Search Service (CSS)	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, and RDS for SQL Server Search: Elasticsearch In OBT: CloudTable and Cloud Search Service (CSS)	You are advised to use Logstash to import data to CSS. For details, see Using Logstash to Import Data to Elasticsearch
	SAP HANA	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS Hive	SAP HANA data sources have the following restrictions: SAP HANA cannot serve as the destination. Only the 2.00.050.00.1592305219 version is supported. Only the Generic Edition is supported. BW/4 FOR HANA is not supported. Only database names, table names, and column names consisting of English letters are supported. Special characters such as spaces and symbols are not allowed. The following data types are supported: date, digit, Boolean, and character (except SHORTTEXT). Other data types such as binary are not supported. During migration, tables cannot be automatically created at the destination.
	FusionInsight HDFS	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Search: Elasticsearch In OBT: Cloud Search Service (CSS) and CloudTable	FusionInsight cannot serve as the destination. Supported only by local storage and not in storage-compute decoupling scenarios Not supported by Ranger Not supported if SSL is enabled for ZooKeeper Recommended FusionInsight HDFS versions: 2.8.X 3.1.X Recommended FusionInsight HBase versions: 2.1.X 1.3.X Recommended FusionInsight Hive versions: 1.2.X 3.1.X
	FusionInsight HBase
	FusionInsight Hive
	Database shard	Data warehouse shard: Data Lake Insight (DLI) Hadoop: MRS HBase and MRS Hive Search: Elasticsearch Object storage: Object Storage Service (OBS) In OBT: Cloud Search Service (CSS)	Database shards cannot serve as the destination.
	Dameng database	Data warehouse: GaussDB(DWS) Hadoop: MRS Hive and MRS Hudi	-
	ShenTong	Hadoop: MRS Hive and MRS Hudi	-
	Document Database Service (DDS)	Hadoop: MRS HDFS, MRS HBase, and MRS Hive	-
	Cassandra	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object storage: Object Storage Service (OBS) Search: Elasticsearch In OBT: Cloud Search Service (CSS) and CloudTable	-
	GBASE8S	Hadoop: MRS HDFS and MRS HBase Message system: DMS Kafka	-
	GBASE8A	Hadoop: MRS HDFS, MRS Hive, and MRS HBase Message system: DMS Kafka	-

In the preceding table, the non-cloud data sources, such as MySQL, include on-premises MySQL, MySQL built on ECSs, or MySQL on the third-party cloud.

Supported Data Sources in Entire DB Migration

Entire DB migration is used when an on-premises data center or a database created on an ECS needs to be synchronized to a database service or big data service on the cloud. It is suitable for offline database migration but not online real-time migration.

Table 2 lists the data sources supporting entire DB migration using CDM.

**Table 2** Supported data sources in entire DB migration
Category	Data Source	Read	Write	Description
Data warehouse	GaussDB(DWS)	Supported	Supported	-
Hadoop (available only for local storage, and not for storage-compute decoupling, Ranger, or ZooKeeper for which SSL is enabled)	MRS HBase	Supported	Supported	Entire DB migration only to MRS HBase Recommended versions: 2.1.X 1.3.X MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
	MRS Hive	Supported	Supported	Entire DB migration only to a relational database 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
	Apache HBase	Supported	Not supported	Recommended versions: 2.1.X 1.3.X
	Apache Hive	Supported	Not supported	Entire DB migration only to a relational database 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X
	MRS Hudi	Supported	Supported	Supported only by local storage and in storage-compute decoupling scenarios 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
Relational database	RDS for MySQL	Supported	Supported	Migration from OLTP to OLTP is not supported. In this scenario, you are advised to use the Data Replication Service (DRS).
	RDS for PostgreSQL	Supported	Supported
	RDS for SQL Server	Supported	Supported
	MySQL	Supported	Not supported
	PostgreSQL	Supported	Not supported
	Microsoft SQL Server	Supported	Not supported
	Oracle	Supported	Not supported
NoSQL	Distributed Cache Service (DCS)	Not supported	Supported	Only migration from MRS to DCS is supported.
In OBT	CloudTable	Supported	Supported	-
	FusionInsight HBase	Supported	Not supported	Recommended versions: 2.1.X 1.3.X
	FusionInsight Hive	Supported	Not supported	Entire DB migration only to a relational database 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X
	SAP HANA	Supported	Not supported	Only the 2.00.050.00.1592305219 version is supported. Only the Generic Edition is supported. BW/4 FOR HANA is not supported. Only database names, table names, and column names consisting of English letters are supported. Special characters such as spaces and symbols are not allowed. The following data types are supported: date, digit, Boolean, and character (except SHORTTEXT). Other data types such as binary are not supported. During migration, tables cannot be automatically created at the destination.
	Dameng database	Supported	Not supported	Only to DWS and Hive
	Document Database Service (DDS)	Supported	Supported	Only migration between DDS and MRS is supported.