Supported Data Sources (2.9.3.300)

CDM provides the following migration modes which support different data sources:

Table/File migration in the import of data into a data lake or migration of data to the cloud. For details, see Data Sources Supported by Table/File Migration.
Entire DB migration in the import of data into a data lake or migration of data to the cloud. For details, see Supported Data Sources in Entire DB Migration.

This section describes the data sources supported by CDM clusters of version 2.9.3.300. The supported data sources vary depending on the CDM cluster version.

Data Sources Supported by Table/File Migration

Table/File migration can migrate data in tables or files.

Table 1 describes the supported data sources.

**Table 1** Supported data sources during table/file migration
Category	Source	Destination	Description
Data warehouse	Data Warehouse Service	Data warehouse: GaussDB(DWS), Data Lake Insight (DLI), and MRS ClickHouse Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object-based storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, RDS for SQL Server, MySQL, PostgreSQL, Microsoft SQL Server, and Oracle NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS)	The DWS physical machine management mode is not supported.
	Data Lake Insight (DLI)	Data warehouse: GaussDB(DWS), Data Lake Insight (DLI), and MRS ClickHouse Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object-based storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, RDS for SQL Server, MySQL, PostgreSQL, Microsoft SQL Server, and Oracle NoSQL: CloudTable and MongoDB Search: Elasticsearch and Cloud Search Service (CSS)	Recommended MongoDB version: 4.2
	MRS ClickHouse	Data warehouse: MRS ClickHouse and Data Lake Insight (DLI)	Recommended MRS ClickHouse version: 21.3.4.X MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
Hadoop	MRS HDFS	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object-based storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, RDS for SQL Server, MySQL, PostgreSQL, Microsoft SQL Server, and Oracle NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS)	Supported by local storage. Only MRS Hive and MRS Hudi are supported in storage-compute decoupling scenarios. Only MRS Hive is supported in Ranger scenarios. Not supported if SSL is enabled for ZooKeeper Recommended MRS HDFS versions: 2.8.X 3.1.X Recommended MRS HBase versions: 2.1.X 1.3.X MRS Hive and MRS Hudi 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
	MRS HBase
	MRS Hive	Data warehouse: GaussDB(DWS), Data Lake Insight (DLI), and MRS ClickHouse Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object-based storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, RDS for SQL Server, MySQL, PostgreSQL, Microsoft SQL Server, and Oracle NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS)
	MRS Hudi	Data warehouse: GaussDB(DWS) Hadoop: MRS HBase
	FusionInsight HDFS	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object-based storage: Object Storage Service (OBS) NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS)	FusionInsight cannot serve as the destination. Supported only by local storage and not in storage-compute decoupling scenarios Not supported by Ranger Not supported if SSL is enabled for ZooKeeper Recommended FusionInsight HDFS versions: 2.8.X 3.1.X Recommended FusionInsight HBase versions: 2.1.X 1.3.X Recommended FusionInsight Hive versions: 1.2.X 3.1.X
	FusionInsight HBase
	FusionInsight Hive
	Apache HBase	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object-based storage: Object Storage Service (OBS) NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS)	Apache cannot serve as the destination. Supported only by local storage and not in storage-compute decoupling scenarios Not supported by Ranger Not supported if SSL is enabled for ZooKeeper Recommended Apache HBase versions: 2.1.X 1.3.X Apache Hive 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X Recommended Apache HDFS versions: 2.8.X 3.1.X
	Apache Hive
	Apache HDFS
Object storage	Object Storage Service (OBS)	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS)	Object Storage Migration Service (OMS) is recommended for migration between object storage services. Binary files cannot be imported to a database or NoSQL.
File system	FTP	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS) Object-based storage: Object Storage Service (OBS)	The file system cannot serve as the destination. Only text files such as CSV files can be migrated from FTP or SFTP servers to search services. Binary files cannot. Only binary files can be migrated from FTP or SFTP servers to OBS. obsutil is recommended for migrating data from HTTP servers to OBS. For details, see Introduction to obsutil.
	SFTP
	HTTP	Hadoop: MRS HDFS
Relational database	RDS for MySQL	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, MRS Hive, and MRS Hudi Object-based storage: Object Storage Service (OBS) NoSQL: CloudTable Relational database: RDS for MySQL, RDS for PostgreSQL, and RDS for SQL Server Search: Elasticsearch and Cloud Search Service (CSS)	RDS for MySQL does not support the SSL mode. Recommended Microsoft SQL Server version: 2005 or later The KingBase database and GaussDB can be connected using the PostgreSQL connector. The supported source and destination are the same as those of the PostgreSQL data source.
	RDS for SQL Server	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object-based storage: Object Storage Service (OBS) NoSQL: CloudTable Relational database: RDS for MySQL, RDS for PostgreSQL, and RDS for SQL Server Search: Elasticsearch and Cloud Search Service (CSS)
	RDS for PostgreSQL
	MySQL	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, MRS Hive, and MRS Hudi Object-based storage: Object Storage Service (OBS) NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS)
	PostgreSQL
	Oracle
	Microsoft SQL Server	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object-based storage: Object Storage Service (OBS) NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS)
	SAP HANA	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS Hive	SAP HANA data sources have the following restrictions: SAP HANA cannot serve as the destination. Only the 2.00.050.00.1592305219 version is supported. Only the Generic Edition is supported. BW/4 FOR HANA is not supported. Only database names, table names, and column names consisting of English letters are supported. Special characters such as spaces and symbols are not allowed. The following data types are supported: date, digit, Boolean, and character (except SHORTTEXT). Other data types such as binary are not supported. During migration, tables cannot be automatically created at the destination.
	Database Sharding	Data warehouse: Data Lake Insight (DLI) Hadoop: MRS HBase and MRS Hive Search: Elasticsearch and Cloud Search Service (CSS) Object-based storage: Object Storage Service (OBS)	Database shards cannot serve as the destination.
	ShenTong	Hadoop: MRS Hive and MRS Hudi	-
NoSQL	Distributed Cache Service (DCS)	Hadoop: MRS HDFS, MRS HBase, and MRS Hive	NoSQL except CloudTable cannot serve as the destination. For how to migrate data from Redis to DCS, see Migrating Data from Self-Hosted Redis to DCS.
	Redis
	Document Database Service
	MongoDB
	CloudTable HBase	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object-based storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, RDS for SQL Server, MySQL, PostgreSQL, Microsoft SQL Server, and Oracle NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS)
	Cassandra	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object-based storage: Object Storage Service (OBS) NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS)
Message system	Data Ingestion Service (DIS)	Search: Cloud Search Service (CSS)	The message system cannot serve as the destination.
	Apache Kafka
	DMS Kafka
	MRS Kafka	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object-based storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, and RDS for SQL Server NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS)	MRS Kafka cannot serve as the destination. Supported only by local storage and not in storage-compute decoupling scenarios Not supported by Ranger Not supported if SSL is enabled for ZooKeeper MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
Search	Elasticsearch	Data warehouse: GaussDB(DWS) and Data Lake Insight (DLI) Hadoop: MRS HDFS, MRS HBase, and MRS Hive Object-based storage: Object Storage Service (OBS) Relational database: RDS for MySQL, RDS for PostgreSQL, and RDS for SQL Server NoSQL: CloudTable Search: Elasticsearch and Cloud Search Service (CSS)	Only the non-security mode is supported.
Search	Cloud Search Service (CSS)		You are advised to use Logstash to import data to CSS. For details, see Using Logstash to Import Data to Elasticsearch

In the preceding table, the non-cloud data sources, such as MySQL, include on-premises MySQL, MySQL built on ECSs, or MySQL on the third-party cloud.

Supported Data Sources in Entire DB Migration

Entire DB migration is used when an on-premises data center or a database created on an ECS needs to be synchronized to a database service or big data service on the cloud. It is suitable for offline database migration but not online real-time migration.

Table 2 lists the data sources supporting entire DB migration using CDM.

**Table 2** Supported data sources in entire DB migration
Category	Data Source	Read	Write	Description
Data warehouse	Data Warehouse Service	Supported	Supported	-
Hadoop (available only for local storage, and not for storage-compute decoupling, Ranger, or ZooKeeper for which SSL is enabled)	MRS HBase	Supported	Supported	Entire DB migration only to MRS HBase Recommended versions: 2.1.X 1.3.X MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
	MRS Hive	Supported	Supported	Entire DB migration only to a relational database 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
	FusionInsight HBase	Supported	Not supported	Recommended versions: 2.1.X 1.3.X
	FusionInsight Hive	Supported	Not supported	Entire DB migration only to a relational database 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X
	Apache HBase	Supported	Not supported	Recommended versions: 2.1.X 1.3.X
	Apache Hive	Supported	Not supported	Entire DB migration only to a relational database 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X
	MRS Hudi	Supported	Supported	Supported only by local storage and in storage-compute decoupling scenarios 2.x versions are not supported. The following versions are recommended: 1.2.X 3.1.X MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
Relational database	RDS for MySQL	Supported	Supported	Migration from OLTP to OLTP is not supported. In this scenario, you are advised to use the Data Replication Service (DRS).
	RDS for PostgreSQL	Supported	Supported
	RDS for SQL Server	Supported	Supported
	MySQL	Supported	Not supported
	PostgreSQL	Supported	Not supported
	Microsoft SQL Server	Supported	Not supported
	Oracle	Supported	Not supported
	SAP HANA	Supported	Not supported	Only the 2.00.050.00.1592305219 version is supported. Only the Generic Edition is supported. BW/4 FOR HANA is not supported. Only database names, table names, and column names consisting of English letters are supported. Special characters such as spaces and symbols are not allowed. The following data types are supported: date, digit, Boolean, and character (except SHORTTEXT). Other data types such as binary are not supported. During migration, tables cannot be automatically created at the destination.
	Dameng database	Supported	Not supported	Only to DWS and Hive
NoSQL	Distributed Cache Service (DCS)	Not supported	Supported	Only migration from MRS to DCS is supported.
	Document Database Service (DDS)	Supported	Supported	Only migration between DDS and MRS is supported.
	CloudTable	Supported	Supported	-