Flink SQL Elasticsearch Development Suggestions
Job Parameters
Parameter |
Mandatory (Yes/No) |
Data Type |
Description |
---|---|---|---|
connector |
Yes |
String |
Connector to be interconnected. For example, elasticsearch-7 means a cluster of Elasticsearch 7.x or later is interconnected. |
hosts |
Yes |
String |
IP addresses of one or multiple Elasticsearch hosts to be connected. Example: 'http://10.10.10.10:24100;http://10.10.10.10:24100' |
index |
Yes |
String |
Index of each record in Elasticsearch The index can be a static one (for example, 'myIndex') or a dynamic one (for example, 'index-{log_ts|yyyy-MM-dd}'). |
document-id.key-delimiter |
No |
String |
Delimiter of a composite key. The default value is _. If this parameter is set to $, the document ID is KEY1$KEY2$KEY3. |
username |
No |
String |
Username for connecting to the Elasticsearch instance. |
password |
No |
String |
Password for connecting to the Elasticsearch instance. This parameter must be a non-null string if username is set. |
failure-handler |
No |
String |
Policy for handling Elasticsearch request failures. Value options include:
|
sink.flush-on-checkpoint |
No |
Boolean |
|
sink.bulk-flush.max-actions |
No |
Integer |
Maximum number of buffer operations for each batch request. The default value is 1000. You can set this parameter to 0 to disable this function. |
sink.bulk-flush.max-size |
No |
MemorySize |
Maximum size of the buffer operations for each batch request in the memory. The default value is 2 MB. The unit must be MB. You can set this parameter to 0 to disable this function. |
sink.bulk-flush.interval |
No |
Duration |
Interval for buffer operations. The default value is 1s. You can set this parameter to 0 to disable this function. |
sink.bulk-flush.backoff.strategy |
No |
String |
Retry strategy if any flush operations fail due to a temporary request error. Value options include:
|
sink.bulk-flush.backoff.max-retries |
No |
Integer |
Maximum number of backoff retries. |
sink.bulk-flush.backoff.delay |
No |
Duration |
Delay between backoff retries. The options include:
|
connection.path-prefix |
No |
String |
Prefix string to be added to each REST communication, for example, '/v1'. |
format |
No |
String |
Format supported by the Elasticsearch connector. The default value is json. |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot