Help Center/DataArts Studio/User Guide/DataArts Migration (CDM Jobs)/Creating a Job in a CDM Cluster/Configuring CDM Destination Job Parameters/To Doris
Updated on 2026-03-20 GMT+08:00
View PDF
Share

To Doris

Table 1 lists the destination job parameters when the destination link is a Doris link.

Table 1 Parameter description

Type

Parameter

Description

Example Value

Basic parameters

Schema/Tablespace

Name of the database to which data will be written. The schema can be automatically created. Click the icon next to the text box to select a schema or tablespace.

schema

Table Name

Name of the table to which data will be written. Click the icon next to the text box. The dialog box for selecting the table is displayed.

This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. For details, see Incremental Synchronization Using the Macro Variables of Date and Time.

NOTE:

If you have configured a macro variable of date and time and schedule a CDM job through DataArts Factory of DataArts Studio, the system replaces the macro variable of date and time with (Planned start time of the data development job – Offset) rather than (Actual start time of the CDM job – Offset).

table

Clear Data Before Import

Whether to clear the data in the destination table before data import. The options are as follows:
  • Do not clear: The data in the destination table is not cleared before data import. The imported data is just added to the table.
  • Clear all data: All data is cleared from the destination table before data import.
  • Clear part of data: Part of the data in the destination table is cleared before data import. If you select Clear part of data, you must configure WHERE Clause to specify which part will be deleted.

Clear part of data

WHERE Clause

If Clear Data Before Import is set to Clear part of data, data in the destination table will be deleted based on the WHERE clause after the configuration is complete and before the import starts.

age > 18 and age <=60

Write Mode

STREAM_LOAD is recommended.

  • STREAM_LOAD: Data is imported using stream load. To use this mode, ensure that the stream load port is enabled and the port number has been configured in the connection.
  • JDBC: JDBC is used to execute SQL statements to write data. JDBC is not recommended because it may cause Doris audit logs.

STREAM_LOAD

stream load config properties

Set stream load parameters when you set Write Mode to STREAM_LOAD.

The parameters are included in the request header for writing data using stream load. The following parameters are supported:

  • Public parameters:
    • format: format of the data to be imported. CSV and JSON are supported. CSV is recommended because it delivers better performance.
    • partitions: partitions to which data is to be imported. If this parameter is not specified, data is imported to all partitions of the Doris table by default. Use commas (,) to separate partitions, such as p1, p2.
    • strict_mode: whether to enable the strict mode. The default value is true. If this parameter is set to false, the field conversion fails and the value is set to NULL. This will cause data loss.
    • timeout: timeout interval for the data import job. The value ranges from 1 to 259200, in seconds. The default value is 600.
    • compress_type: format that CSV data is compressed into by Doris. The value can be gz, lz4, or none.
  • Parameters supported for the JSON format:

    strip_outer_array: whether to strip the outermost array structure. The default value is true.

  • Parameters supported for the CSV format:
    • line_separator: line separator in the Doris data source. The default value is \n for the CSV format.
    • row_delimiter: row delimiter for the StarRocks data source. The default value is \n for the CSV format.
    • column_separator: field separator. The default value is \t for the CSV format.

-

Advanced attributes

Prepare for Data Import

The SQL statement that is first executed before a task is executed. Currently, only one SQL statement can be executed.

create temp table

Complete Statement After Data Import

The SQL statement that is executed after a task is executed. Currently, only one SQL statement can be executed.

merge into

Loader Threads

Number of threads started in each loader. A larger number allows more concurrent write operations.

The unique model or aggregation function replace have requirements on the insertion sequence. When they are used, do not use the concurrency capability.

Conflict handling policies do not support "replace into" or "on duplicate key update".

1