Updated on 2023-12-22 GMT+08:00

To MRS ClickHouse

If the destination link of a job is an MRS ClickHouse link, configure the destination job parameters based on Table 1.

If the source link of the job is an MRS ClickHouse, DWS, or Hive link:

  • If the int or float fields are null, set the field type to nullable() when creating an MRS ClickHouse table. Otherwise, the value written to MRS ClickHouse is 0.
  • Check whether the destination table engine is ReplicatedMergeTree. This engine has a deduplication mechanism, in which the data to be deduplicated cannot be predicted accurately. If this engine is used, ensure that data is unique. Otherwise, non-unique data will be ignored and not written, or ReplicatedMergeTree will be replaced by other types of table engines such as MergeTree.
Table 1 Parameter description

Parameter

Description

Example Value

Schema/Tablespace

Click the icon next to the text box to select a schema or tablespace.

schema

Table Name

Destination table name.

Click the icon next to the text box. The dialog box for selecting the table is displayed.

This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. For details, see Incremental Synchronization Using the Macro Variables of Date and Time.

NOTE:

If you have configured a macro variable of date and time and schedule a CDM job through DataArts Studio DataArts Factory, the system replaces the macro variable of date and time with (Planned start time of the data development jobOffset) rather than (Actual start time of the CDM jobOffset).

table

Clear Data Before Import

Whether to clear the data in the destination table before data import. The options are as follows:
  • Do not clear: The data in the destination table is not cleared before data import. The imported data is just added to the table.
  • Clear all data: All data is cleared from the destination table before data import.
  • Clear part of data: Part of the data in the destination table is cleared before data import. If you select Clear part of data, you must configure WHERE Clause to specify which part will be deleted.

Clear part of data

Whether On Cluster

This parameter is displayed when Clear Data Before Import is set to Clear part of data or Clear all data. If this parameter is set to Yes, all or part of data on all the nodes in the cluster will be cleared.

Yes

WHERE Clause

If Clear Data Before Import is set to Clear part of data, data in the destination table will be deleted based on the WHERE clause after the configuration is complete and before the import starts.

age > 18 and age <= 60