Updated on 2024-08-30 GMT+08:00

Flink Specification Overview

Scope

This document describes the design and development rules of using the MRS Flink component for data lake and data house integration and unified stream and batch data processing. The following specifications are provided:

  • Data table design
  • Resource configurations
  • Performance tuning
  • Fault handling
  • Typical parameter settings

Terms

This section uses the following terms for description:

  • Rule: a principle that must be observed during programming.
  • Suggestion: a principle that must be considered during programming.
  • Description: an explanation of the rule or suggestion in question.
  • Example: an example for a rule or suggestion.

Application Scope

  • Design, develop, test, and maintain data storage and data processing jobs based on MRS-Flink data storage.
  • Use this specification for MRS 3.2.0 and later versions.
  • Parameter optimization guide is available for MRS 3.2.0 and later versions.
  • For any inconsistency between this document and the open source community, use the specifications in this document.

Reference

Flink open-source community: https://nightlies.apache.org/flink/flink-docs-stable/.