Updated on 2023-09-21 GMT+08:00

Connecting to Data Sources

Overview

Before creating a data integration task, you need to connect data sources at the source and destination to ROMA Connect so that data can be read from the source and written to the destination.

Prerequisites

  • Before accessing a data source, ensure that the ROMA Connect instance can connect to the network where your data source resides.
    • Same VPC: The instance can directly access the data source.
    • Two VPCs in the same region: Connect the instance and the data source using a peering connection. For details, see VPC Peering Connection.
    • Two VPCs in two regions: Connect the instance and the data source using a peering connection. For details, see Network Communications Among VPCs Across Regions.
    • Communication over the public network: Ensure that the ROMA Connect instance has been bound with an EIP.
  • To enable cross-VPC communication over the private network, configure the routes between the instance and the subnet where the data source resides.

Connecting to a Data Source

The configuration for data source connection varies depending on data source types. After a data source is connected, click the data source name to view the data source details, which include relevant task information.

  • ROMA Connect data integration is applicable to data type conversion and integration between heterogeneous data sources. DRS is recommended for data migration and synchronization between mainstream databases. In scenarios where data such as relational database data, big data, and text is migrated to the data lake, CDM is recommended.
Table 1 Configurations for data source connections

Data Source Connection

Common Task Integration Mode (Source)

Common Task Integration Mode (Destination)

Composite Task Integration Mode (Source)

Composite Task Integration Mode (Destination)

Connecting to an API Data Source

Scheduled

Scheduled, Real-Time

-

-

Connecting to an ActiveMQ Data Source

Real-Time

Scheduled, Real-Time

-

-

Connecting to an ArtemisMQ Data Source

Real-Time

Scheduled, Real-Time

-

-

Connecting to a DB2 Data Source

Scheduled

Scheduled, Real-Time

-

-

Connecting to a DIS Data Source

Real-Time

Scheduled, Real-Time

-

-

Connecting to a DWS Data Source

Scheduled

Scheduled, Real-Time

-

-

Connecting to the DM Data Source

Scheduled

Scheduled, Real-Time

-

-

Connecting to a Gauss100 Data Source

Scheduled

Scheduled, Real-Time

-

-

Connecting to an FTP Data Source

Scheduled

Scheduled

-

-

Connecting to an HL7 Data Source

Real-Time

Scheduled, Real-Time

-

-

Connecting to a HANA Data Source

Scheduled

Scheduled, Real-Time

Scheduled

Scheduled

Connecting to an LDAP Data Source

Scheduled

-

-

-

Connecting to an IBM MQ Data Source

Real-Time

Scheduled, Real-Time

-

-

Connecting to a Kafka Data Source

Real-Time

Scheduled, Real-Time

-

Real-Time

Connecting to a MySQL Data Source

Scheduled

Scheduled, Real-Time

Scheduled, Real-Time

Scheduled, Real-Time

Connecting to a MongoDB Data Source

Scheduled

Scheduled, Real-Time

-

-

Connecting to an MQS Data Source

Real-Time

Scheduled, Real-Time

-

-

Connecting to an MRS Hive Data Source

Scheduled

Scheduled, Real-Time

-

-

Connecting to an MRS HDFS Data Source

Scheduled

Scheduled, Real-Time

-

-

Connecting to an MRS HBase Data Source

Scheduled

Scheduled, Real-Time

-

-

Connecting to an MRS Kafka Data Source

Real-Time

Scheduled, Real-Time

-

-

Connecting to an OBS Data Source

Scheduled

Scheduled

-

-

Connecting to an Oracle Data Source

Scheduled

Scheduled, Real-Time

Scheduled, Real-Time

Scheduled, Real-Time

Connecting to a PostgreSQL/openGauss Data Source

Scheduled

Scheduled, Real-Time

Scheduled

Scheduled, Real-Time

Connecting to a Redis Data Source

-

Scheduled, Real-Time

-

-

Connecting to a RabbitMQ Data Source

Real-Time

Scheduled, Real-Time

-

-

Connecting to a RocketMQ Data Source

Real-Time

Scheduled, Real-Time

-

-

Connecting to an SAP Data Source

Scheduled

-

-

-

Connecting to an SNMP Data Source

Scheduled

-

-

-

Connecting to a SQL Server Data Source

Scheduled

Scheduled, Real-Time

Scheduled, Real-Time

Scheduled, Real-Time

Connecting to a GaussDB(for MySQL) Data Source

Scheduled

Scheduled, Real-Time

-

-

Connecting to a WebSocket Data Source

Real-Time

-

-

-

Connecting to a Custom Data Source

Scheduled

Scheduled

-

-