RawFormDataRequest

sourceSelectedDatasource

No

SourceDatasourceConfig

Data source at the source.

targetSelectedDatasource

No

TargetDatasourceConfig

Data source at the destination.

mappingTable

No

List<MappingTable>

Mapping between the source and destination.

mappingTableProperties

No

List<MappTableProperties>

Property in the mapping between the source and destination. This property is used for relational data sources. For other types of data sources, leave this parameter blank.

mappingSourceTextOrNot

No

List<Flag>

Whether the text mode is used for the mapping between the source and destination. The default value is false.

mappingInput

No

List<MappingInput>

Metadata mapping between the source and destination.

mappingRequired

No

Boolean

Whether a mapping is required. The default value is true.

- true: A mapping is required.

- false: A mapping is not required.

error_handler

No

List<ErrorHandler>

Fault information storage configuration. This parameter is mandatory only if fault information needs to be stored.

afterImplementation

No

List<AfterImplementation>

After the task is complete, the system sends messages to the Kafka or MRS Kafka based on the configuration. The messages can also be sent to a third-party platform.

source_column

No

String

Name of a field in the source table.

source_column_type

No

String

Type of a field in the source table.

source_column_length

No

String

Length of a field in the source table.

target_column

No

String

Name of the destination field.

target_column_type

No

String

Type of a field in the destination table.

target_column_length

No

String

Length of a field in the destination table.

is_add_mapping

No

String

Whether a mapping is added. The default value is false.

keys

No

String

Property name.

value

No

String

Property value.

targetNeedCreateTable

No

Boolean

Whether a table is automatically created.

Currently, this parameter is supported only for the PostgreSQL data source. For other types of data sources, set this parameter to false.

isSupportClearTab

No

Boolean

Whether to clear the table. If this parameter is set to true, the scheduling task clears the destination table first. The default value is false.

If the data source type is Oracle, retain the default value false.

If the data source type is DWS, you can set this parameter to either true or false.

targetTable

No

String

Name of the destination table.

Set this parameter if the data source type is DWS or Oracle.

target_schema

No

String

Mode of the destination database. This parameter is used for the GaussDB 100 database.

batchNumber

No

String

Batch number field.

Select a field in the destination table as the batch number field. This field will stores a random number that is used to identify the data of this batch.

A field whose type is string and length is greater than 14 characters in the destination table can be selected as the batch number field. In addition, the batch number field cannot be the same as the destination field (target_column) in mapping information.

Set this parameter if the data source type is DWS or Oracle.

targetSelectedColumnList

No

List<Column>

List of field names selected at the destination. Set this parameter if the data source type is DWS and Oracle.

supportPartUpdate

No

Boolean

Whether to update only the changed fields. If this option is enabled, only the table fields whose values are changed are updated. If this option is disabled, all table fields are updated.

Set this parameter if the data source type is DWS or Oracle.

meta_data

No

List<MetaData>

Metadata definition object of the data source. This parameter is mandatory only if is_parse is set to true.

For the FTP data source type, the metadata must be entered in the sequence of the fields in the file.

Set this parameter if the data source type is API or FTP.

path

No

String

File path. This parameter is mandatory for file data sources.

write_mode

No

String

Write mode. This parameter is mandatory for file data sources.

file_name_charset

No

String

Character code of the file name. Set this parameter if the data source type is FTP.

file_content_type

No

String

File content type.

If this parameter is textFile, file_type must be csv or txt. If this parameter is binaryFile, file_type must be xls or xlsx.

- binaryFile: binary file

- textFile: text file

This parameter is mandatory for file data sources.

file_type

No

String

File type.

If file_content_type is textFile, set this parameter to csv or txt.

If file_content_type is binaryFile, set this parameter to xls or xlsx.

This parameter is mandatory for file data sources.

file_name_prefix

No

String

File name prefix. This parameter is mandatory for file data sources.

null_format

No

String

Space format character, which defines the format of spaces in a file.

This parameter is mandatory for file data sources.

remark_date_format

No

String

File name extension. This parameter is mandatory for file data sources.

add_header

No

Boolean

Whether to add a file header. This parameter is mandatory for file data sources.

file_content_charset

No

String

Character encoding format of the file content. This parameter is mandatory for file data sources.

field_delimiter

No

String

File separator. By default, files are separated by commas (,).

This parameter is mandatory for file data sources.

source_nullable

No

String

Whether the source table field can be left blank.

- 0: The source table field cannot left blank.

- 1: The source table field can be left blank.

source_unique

No

String

Whether the source table field is unique.

- 0: The source table field is unique.

- 1: The source table field is not unique.

target_nullable

No

String

Whether the destination table field can be left blank.

- 0: The destination table field cannot left blank.

- 1: The destination table field can be left blank.

target_unique

No

String

Whether the destination table field is unique.

- 0: The destination table field is unique.

- 1: The destination table field is not unique.

need_inc_migration

No

Boolean

Whether to enable incremental migration

This parameter is applicable to API data sources.

- true: Incremental collection is used.

- false: Incremental collection is not used. (Full collection is used for each task execution.)

incr_migration_type

No

String

Type of the period for incremental collection.

This parameter is applicable to API data sources.

- systemDefault: system default

- selfDefine: custom

advanceStartTime

No

Integer

Number of days before the start time of data collection. The parameter is applicable to API data sources.

This parameter is mandatory only if incr_migration_type is set to systemDefault.

If data generated at the source changes in real time, such as alarm data, you can collect the data by setting this parameter. Start time of data collection = Data source system time – Start time offset

period_granularity

No

String

Period granularity unit. The parameter is applicable to API data sources.

This parameter is mandatory only if incr_migration_type is set to selfDefine.

- hour: hour

- day: day

- week: week

- month: month

period_assignment

No

String

Specified period. The parameter is applicable to API data sources.

This parameter is mandatory only if incr_migration_type is set to selfDefine.

- currentPeriod: current period

- lastPeriod: previous period

period_right_boarder

No

String

Right boundary of a period. The parameter is applicable to API data sources.

This parameter is mandatory only if incr_migration_type is set to selfDefine.

- off: closed interval

- on: open interval

timeZone

No

String

Time zone used for incremental collection. In normal cases, set this parameter to GMT+08:00. The parameter is applicable to API data sources.

This parameter is mandatory only if need_inc_migration is set to true.

dataTimestamp

No

Integer

Initial timestamp of incremental collection. The parameter is applicable to API data sources.

This parameter is mandatory only if need_inc_migration is set to true.

compensation_time

No

Integer

Time compensation of incremental collection. The parameter is applicable to API data sources.

If compensation is not required, set this parameter to 0. This parameter is mandatory only if need_inc_migration is set to true.

If the data generated at the source is delayed, you can adjust the time based on the compensation time. The end time of data query is the current system time minus the compensation time.

date_form

No

String

Time format of incremental collection. The parameter is applicable to API data sources.

This parameter is mandatory only if need_inc_migration is set to true.

isNeedPageSrcApi

No

Boolean

Whether to enable pagination query.

This parameter is applicable to API data sources.

- true: Pagination query is enabled.

- false: Pagination query is disabled.

pageKeySrcApi

No

String

Name of the corresponding page number field during pagination query.

This parameter is mandatory only if isNeedPageSrcApi is set to true.

This parameter is applicable to API data sources.

pageSizeKeySrcApi

No

String

Pagination size field name during pagination query.

This parameter is mandatory only if isNeedPageSrcApi is set to true.

This parameter is applicable to API data sources.

current_page_num

No

Integer

Page number in pagination query. In normal cases, set this parameter to 1. This parameter is mandatory only if isNeedPageSrcApi is set to true.

This parameter is applicable to API data sources.

page_size

No

Integer

Size of each page in pagination query. This parameter is mandatory if isNeedPageSrcApi is set to true.

This parameter is applicable to API data sources.

pageNumLimit

No

Integer

Maximum number of pages the can be queried in a single query task. If this parameter is left blank or set to 0, the number of pages is not limited. If this parameter is set to a value other than 0, the query stops when the number of pages exceeds the specified value.

This parameter is applicable to API data sources.

pagingEndType

No

Boolean

Mode used for stopping pagination query.

This parameter is applicable to API data sources.

- paging-list: If no data record is returned, pagination query is stopped is stopped. This mode is used by default.

- paging-total: If the number of records to be queried reaches the maximum limit allowed, pagination query is stopped. In this mode, pagingEndFieldPath must be used to specify the key value of the total number of records. The key value is returned by the API server.

pagingEndFieldPath

No

String

Key value returned by the API service, which indicates the total data volume.

This parameter is mandatory only if pagingEndType is set to paging-total.

This parameter is applicable to API data sources.

beginDateColSrcApi

No

String

Query key value of the start time when incremental collection is used.

This parameter is mandatory only if need_inc_migration is set to true.

This parameter is applicable to API data sources.

endDateColSrcApi

No

String

Query key value of the end time when incremental collection is used.

This parameter is mandatory only if need_inc_migration is set to true.

This parameter is applicable to API data sources.

sourceParams

No

List<KeyValuePair>

Parameter in a request.

This parameter is applicable to API data sources.

sourceHeaders

No

List<KeyValuePair>

Header in a request.

This parameter is applicable to API data sources.

sourceBodyType

No

String

Body type in a request.

This parameter is applicable to API data sources.

- none

- formData

- x-www-form-urlencoded

- raw-json (default value)

- raw-sorted-json

- raw-text

bodyRaw

No

String

Request body.

Set this parameter when sourceBodyType is set to raw-json.

This parameter is applicable to API data sources.

sourceBodyFormData

No

List<KeyValuePair>

Request body in form format.

This parameter is mandatory only if sourceBodyType is set to formData.

This parameter is applicable to API data sources.

sourceResponseType

No

String

Data type of a response.

This parameter is applicable to API data sources.

- json

- xml

isParse

No

Boolean

Whether to parse the request result.

This parameter is applicable to API data sources.

- true

- false

rootPath

No

String

Root path for parsing the response result.

This parameter is applicable to API data sources.

sourceMetaData

No

List<MetaData>

Metadata of the data source type at the source.

This parameter is applicable to API data sources.

path

No

String

File path.

Enter the path of the folder to be accessed on the FTP server, for example, /data/FDI.

file_name

No

String

File name.

File to be accessed on the FTP server.

file_name_charset

No

String

File name encoding.

Set this parameter if the data source type is FTP.

max_file_size

No

Integer

Maximum file size, in unit of MB.

Set this parameter if the data source type is FTP.

file_content_charset

No

String

File content encoding.

Set this parameter if the data source type is FTP.

field_delimiter

No

String

File separator.

Set this parameter if the data source type is FTP.

null_format

No

String

Space format character, which defines the format of spaces in a file.

Set this parameter if the data source type is FTP.

batch_num

No

Integer

Amount of data to be migrated.

Enter the number of data rows to be parsed if is_parse is set to true. The maximum value is 1500000.

Enter the number of files if is_parse is set to false. The maximum value is 20000.

Set this parameter if the data source type is FTP.

skip_header

No

Boolean

Whether to skip the file header.

This parameter is mandatory only if is_parse is set to true.

This parameter specifies whether to skip the file header in the data file. The file header is the first line or several lines at the beginning of a file, which helps identify and distinguish the file content.

Set this parameter if the data source type is FTP.

skip_header_rows

No

Integer

Number of file header lines to be skipped.

Set this parameter if the data source type is FTP.

extend_meta_data

No

List<MetaData>

Metadata of the data source type at the source.

Set this parameter if the data source type is FTP.

instanceId

No

String

Instance ID.

isAddSql

No

Boolean

Whether to add SQL statements. This parameter is used for relational databases.

- true: SQL statements are added.

- false: SQL statements are not added.

querySql

No

String

SQL query statement, which can be left blank. This parameter is used for relational databases.

sourceTable

No

String

Source table name. This parameter is used for relational databases.

sourceSchema

No

String

Mode of the source database. This parameter is used for the GaussDB 100 database.

sourceSelectedColumnList

No

List<Column>

List of field names selected at the source. This parameter is used for relational databases.

orderColumn

No

String

List of field names used for sorting at the source. This parameter is used for relational databases.

orderType

No

String

Sorting type at the source, which can be left empty. This parameter is used for relational databases.

- ASC: ascending order

- DESC: descending order

sourceDateColumn

No

String

Column name used for incremental migration at the source. This parameter is used for relational databases.

need_inc_migration

No

Boolean

Whether to enable incremental migration.

This parameter is used for relational databases.

- true: Incremental collection is used.

- false: Incremental collection is not used. (Full collection is used for each task execution.)

timeZone

No

String

Time zone used for incremental collection. In normal cases, set this parameter to GMT+08:00. This parameter is used for relational databases.

This parameter is mandatory only when need_inc_migration is set to true.

dataTimestamp

No

Integer

Initial timestamp during incremental collection. This parameter is used for relational databases.

This parameter is mandatory only when need_inc_migration is set to true.

compensation_time

No

Integer

Time compensation value for incremental collection. This parameter is used for relational databases.

If compensation is not required, set this parameter to 0. This parameter is mandatory only when need_inc_migration is set to true.

If the data generated at the source is delayed, you can adjust the time based on the compensation time. The end time of data query is the current system time minus the compensation time.

where

No

List<Where>

Filter criteria statement of the relational or Hive data source at the source. This parameter can be left blank.

whereOptions

No

List<WhereOptions>

Criteria used for filtering types at the source. This parameter can be left blank. This parameter is used for relational databases.

- equal

- notEqual

- less

- lessEqual

- more

- moreEqual

- isNull

- notNull

- between

- notBetween

- inList

- notInList

flag

No

Boolean

Whether the text mode is used. The default value is false.

- true: The text mode is used.

- false: The text mode is not used.

dataSource

No

List<DataSource>

Mapping information of the data source.

index

No

String

Field index.

name

No

String

Field name.

type

No

String

Field type.

format

No

String

Field format.

path

No

String

Parsing path.

field_name

No

String

Field name.

field_type

No

String

Field type.

field_length

No

String

Field length.

null_able

No

String

Whether the field can be left blank.

- 0: The field cannot be left blank.

- 1: The field can be left blank.

unique

No

String

Whether the field is unique.

- 0: The field is unique.

- 1: The field is not unique.

dataSource

No

List<DataSource>

Mapping information of the data source.

mode

No

String

Content mode, for example, keyvalue.

sourceKey

No

String

Source value.

targetKey

No

String

Destination value.

path

No

String

Save path.

datasource_id

No

String

Data source ID.

mode

No

String

Data source type.

app_id

No

String

Application ID.

projectId

No

String

Project ID.

actionType

No

String

Operation type.

appId

No

String

Application ID.

datasourceId

No

String

Data source ID.

topic

No

String

Topic name.

messageContent

No

String

Custom message.