Updated on 2024-09-30 GMT+08:00
Flink Job Overview
DLI supports two types of Flink jobs:
- Flink OpenSource SQL job:
- It is fully compatible with Flink of the community edition, ensuring that jobs can run smoothly on these Flink versions.
- DLI Flink has expanded the support for connectors based on Flink of the community edition, supporting Redis and GaussDB(DWS) as new data source types. With this expansion, you can now utilize a wider range of data source types, providing greater flexibility and convenience when working with datasets.
- Flink OpenSource SQL jobs are ideal for scenarios where stream processing logic can be defined and executed through SQL statements. This simplifies stream processing, allowing developers to focus more on implementing service logic.
For how to create a Flink OpenSource SQL job, see Creating a Flink OpenSource SQL Job.
- Flink Jar job:
- DLI allows you to submit Flink jobs compiled into JAR files, providing higher flexibility and customization capabilities. It is applicable to scenarios where complex data processing is required.
- If the connectors provided by Flink of the community edition cannot meet specific needs, you can use Jar jobs to implement custom connectors or data processing logic.
- It is ideal for scenarios where user-defined functions (UDFs) or specific library integration are required. You can use the Flink ecosystem to implement advanced stream processing logic and status management.
For how to create a Flink Jar job, see Creating a Flink Jar Job.
Parent topic: Submitting a Flink Job Using DLI
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
The system is busy. Please try again later.
For any further questions, feel free to contact us through the chatbot.
Chatbot