Updated on 2022-08-17 GMT+08:00

From Kafka/DMS Kafka

If the source link of a job is the Link to Kafka or Link to DMS Kafka, configure the source job parameters based on Table 1.

Table 1 Parameter description

Parameter

Description

Example Value

Topics

One or more topics can be entered.

est1,est2

Offset

Initial offset parameter

  • Latest: Maximum offset, indicating that the latest data will be extracted.
  • Earliest: Minimum offset, indicating that the earliest data will be extracted.
  • Submitted: data that has been submitted
  • Time Range: data within a specified time range

Latest

Permanent Running

Whether a job runs permanently.

Yes

Consumer Group ID

Consumer group ID

If you export data from DMS Kafka, enter any value for Kafka Platinum but a valid consumer group ID for Kafka Basic.

sumer-group

Data Format

Format used for parsing data. The options are as follows:

  • Binary: Data is transferred directly. It is not converted to another format. This setting is suitable for file migration.
  • CSV: Source data will be migrated after being converted in CSV format.
  • JSON: Source data will be migrated after being converted in JSON format.
  • CDC (DRS_JSON): Source data will be migrated after being converted in DRS_JSON format.

Binary

Field Delimiter

The default value is space. To set the Tab key as the delimiter, set this parameter to \t.

,

Max. Poll Records

(Optional) Maximum number of records per poll

100

Max. Poll Interval

(Optional) Maximum interval between polls (seconds)

100