From Elasticsearch
Type | Parameter | Description | Mandatory | Example Value |
|---|---|---|---|---|
Basic parameters | Index | It is similar to the schema or name of a relational database. Multiple indexes for entire DB migration are separated by commas (,). You can enter index aliases. You can also enter an expression containing wildcard characters, for example, asterisks (*). If you select multiple indexes, their structure must be the same. This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. For details, see Incremental Synchronization Using the Macro Variables of Date and Time. NOTE: If you have configured a macro variable of date and time and schedule a CDM job through DataArts Studio DataArts Factory, the system replaces the macro variable of date and time with (Planned start time of the data development job – Offset) rather than (Actual start time of the CDM job – Offset). | Yes | index_sample |
Type | It is similar to the schema or name of a relational database. Multiple indexes for entire DB migration are separated by commas (,). You can enter index aliases. You can also enter an expression containing wildcard characters, for example, asterisks (*). If you select multiple indexes, their structure must be the same. | Yes | type_example | |
Advanced attributes | Split Nested Field | Whether to split the JSON content of nested fields, for example, splitting a:{ b:{ c:1, d:{ e:2, f:3 } } } into [a.b.c], [a.b.d.e], and [a.b.d.f]. | No | Yes |
Filter Conditions | Condition used to filter source data. The q syntax of Elasticsearch is used. | No | last_name:Smith | |
Meta-field Extraction | Whether to extract meta-fields from the index. Currently, only _index, _type, _id, and _score are supported. Examples: _index, _type, _id, and _score | No | _index | |
Page Size | Number of records displayed on each page | No | 1000 | |
ScrollId Time Out | ScrollId timeout duration, which is five minutes by default. | No | 5 | |
Retries | Number of retries upon a request failure. The maximum number of retries is 10. | No | 3 |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.
