Updated on 2024-10-23 GMT+08:00

From Elasticsearch

Table 1 Parameter description

Type

Parameter

Description

Mandatory

Example Value

Basic parameters

Index

It is similar to the schema or name of a relational database. Multiple indexes for entire DB migration are separated by commas (,).

You can enter index aliases.

You can also enter an expression containing wildcard characters, for example, asterisks (*). If you select multiple indexes, their structure must be the same.

This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. For details, see Incremental Synchronization Using the Macro Variables of Date and Time.

NOTE:

If you have configured a macro variable of date and time and schedule a CDM job through DataArts Studio DataArts Factory, the system replaces the macro variable of date and time with (Planned start time of the data development jobOffset) rather than (Actual start time of the CDM jobOffset).

Yes

index_sample

Type

It is similar to the schema or name of a relational database. Multiple indexes for entire DB migration are separated by commas (,).

You can enter index aliases.

You can also enter an expression containing wildcard characters, for example, asterisks (*). If you select multiple indexes, their structure must be the same.

Yes

type_example

Advanced attributes

Split Nested Field

Whether to split the JSON content of nested fields, for example, splitting a:{ b:{ c:1, d:{ e:2, f:3 } } } into [a.b.c], [a.b.d.e], and [a.b.d.f].

No

Yes

Filter Conditions

Condition used to filter source data. The q syntax of Elasticsearch is used.

No

last_name:Smith

Meta-field Extraction

Whether to extract meta-fields from the index. Currently, only _index, _type, _id, and _score are supported.

Examples: _index, _type, _id, and _score

No

_index

Page Size

Number of records displayed on each page

No

1000

ScrollId Time Out

ScrollId timeout duration, which is five minutes by default.

No

5

Retries

Number of retries upon a request failure. The maximum number of retries is 10.

No

3