Data Source

Overview

You can select or add Data Source components. In a data integration task, the Data Source component is mandatory. In a typical service integration task, the Data Source component is also used.

Data Source types include Relational Database, Big Data Storage, Message System, NoSQL, Semi-structured Storage, and Application Program. Table 1 lists the supported data source types.

Table 1 Data source types

Category

Data Source Type

Relational database

DB2, Gauss200, MySQL, Oracle, PostgreSQL, and SQL Server

Big Data Storage

DWS

Message System

ActiveMQ, ArtemisMQ, Kafka, and RabbitMQ

No SQL

MongoDB

Semi-structured Storage

OBS

Application program

API

Data Integration Tasks

Data integration tasks support all types of data source components. After selecting a data source in the Data Source component, connect the data source and task scheduler and configure line parameters.

  • If the data source is located at the source of the data integration task, configure line parameters. For details, see Table 1.
  • If the data source is located at the destination of the data integration task, configure line parameters. For details, see Table 2.

Service Integration Task

The service integration task supports only the Data Source component of the relational database type. After selecting Data Source, configure the database operation on the connection line between the previous component and the Data Source component.

You can configure the database in either of the following ways:

  1. Using SQL statements

    Click Insert SQL and enter an SQL statement in the Statement text box to perform operations on the database. In the text box, you can reference the data in the previous step. For details, see Data Reference.

  2. Not using SQL statements

    Disable Insert SQL. You can set SQL Operation, Table, Select Table Field, and Condition to perform database operations.

    • SQL Operation: Includes four basic operations of the database table: Select (to query), Update, Insert, and Delete.
      • By clicking Select, you can select the field name to be queried in the dialog box displayed.
      • By clicking Update, you can select the field name to be updated in the dialog box displayed and enter the new field value.
      • By clicking Insert, you can select the field name to be inserted in the dialog box displayed and enter the field value.
    • Table: After you select a data table and click Select Table Field, the Data Table Field dialog box is displayed.
    • Condition: You can add, update, and delete query conditions. The condition types include and and or. Select the data table fields to be filtered by conditions, select the condition operator, and enter the condition value.