Overview

DataArts Migration (offline jobs) uses an architecture with decoupled read and write. It supports read and write of any data sources, enabling flexible synchronization between data sources.

Decoupled read and write of any data sources
DataArts Migration (offline jobs) can read and write data sources separately. Any data source that can be read can function as the source of data synchronization, and any data source that can be written can function as the destination of data synchronization.

You can connect any data source that can be read to any data source that can be written to form a cross-source synchronization link, for example, from a relational database to a lakehouse storage or from a file system to a real-time data warehouse.
Differences from DataArts Migration (CDM jobs)
Compared with DataArts Migration (CDM jobs), DataArts Migration (offline jobs) uses an optimized architecture and supports cross-source synchronization with real read and write decoupling and through any link.

The traditional CDM architecture does not support read and write decoupling or synchronization between any data sources. Instead, it only supports synchronization between some predefined and fixed data sources. DataArts Migration (offline jobs) supports decoupled read and write capabilities and combination of data sources, meeting more extensive data transfer and integration requirements and adapting to various scenarios.
Synchronization of a single table, database and table shards, and entire databases
Offline data migration jobs support the following synchronization modes: from a table to another table, from a database to a table, and from a database/table shard to a table. The supported data sources vary depending on the synchronization mode.
- Single table synchronization: DataArts Migration can synchronize tables and files in data ingestion into a data lake or data migration to the cloud.
- Database and table shard synchronization: DataArts Migration can synchronize data from multiple tables in multiple databases in data ingestion into a data lake or data migration to the cloud.
- Entire database migration: DataArts Migrations can synchronize data from an on-premises database in data ingestion into a data lake or data migration to the cloud.

The supported data sources vary depending on the CDM cluster version.

Supported Data Sources

Example: If data can be read from a table on a FTP server and written to a table in Hudi, you can configure a task for synchronizing file data on the FTP server to a Hudi table.

The feasibility of cross-source synchronization depends only on whether data can be read from the source and written to the destination. The source and destination can be different types of data, and there is no limit on the source and destination pair.

Table 1 lists the data sources that support single table synchronization.

**Table 1** Read and write of different data sources
Category	Data Source	Single Table Read	Single Table Write	Database/Table Shard Read	Database/Table Shard Write	Entire Database Read	Entire Database Write
Relational database	MySQL	Supported	Supported	Supported	Supported	Supported	x
	PostgreSQL	Supported	Supported	√ (supported in some regions)	Supported	Supported	x
	SQL Server	Supported	Supported	√ (supported in some regions)	Supported	x	x
	Oracle	Supported	Supported	x	Supported	x	x
	SAP HANA	Supported	Supported	x	Supported	x	x
	GBase	Supported	Supported	x	Supported	x	x
Data warehouse	DWS	Supported	Supported	x	Supported	Supported	x
	DLI	Supported	Supported	x	Supported	x	x
	Apache Hive	Supported	Supported	x	Supported	x	√ (supported in some regions)
	MRS Hive	Supported	Supported	x	Supported	x	√ (supported in some regions)
	MRS Hudi	Supported	Supported	x	Supported	x	x
	Doris	Supported	Supported	x	Supported	x	x
	MRS ClickHouse	Supported	Supported	x	Supported	x	x
	Open ClickHouse	Supported	Supported	x	Supported	x	x
Semi-structured data source	MRS HBase	Supported	Supported	x	Supported	x	x
	MongoDB	Supported	Supported	x	Supported	x	x
	Redis	Supported	Supported	x	Supported	x	x
	Elasticsearch	Supported	Supported	x	Supported	x	x
	DMS Kafka	Supported	Supported	x	Supported	x	x
	Apache RocketMq	x	Supported	x	Supported	x	x
	LTS	√ (supported in some regions)	x	x	x	x	x
	Rest Client	Supported	x	x	x	x	x
File system	OBS	Supported	Supported	x	Supported	x	x
	FTP	Supported	√ (supported in some regions)	x	√ (supported in some regions)	x	x
	SFTP	Supported	√ (supported in some regions)	x	√ (supported in some regions)	x	x
	Apache HDFS	Supported	Supported	x	Supported	x	x