Deze pagina is nog niet beschikbaar in uw eigen taal. We werken er hard aan om meer taalversies toe te voegen. Bedankt voor uw steun.

On this page
Help Center/ MapReduce Service/ Service Overview/ Components/ Flink/ Relationships Between Flink and Other Components

Relationships Between Flink and Other Components

Updated on 2025-01-22 GMT+08:00

Relationship Between Flink and Yarn

Flink supports Yarn-based cluster management mode. In this mode, Flink serves as an application of Yarn and runs on Yarn.

Figure 1 shows the Yarn-based Flink cluster deployment.

Figure 1 Yarn-based Flink cluster deployment
  1. The Flink Yarn Client first checks whether there are sufficient resources for starting the Yarn cluster. If yes, the Flink Yarn client uploads JAR files and configuration files to HDFS.
  2. Flink Yarn client communicates with Yarn ResourceManager to request a container for starting ApplicationMaster. After all Yarn NodeManagers finish downloading the JAR file and configuration files, the ApplicationMaster is started.
  3. During the startup, the ApplicationMaster interacts with the Yarn ResourceManager to request the container for starting a TaskManager. After the container is ready, the TaskManager process is started.
  4. In the Flink Yarn cluster, the ApplicationMaster and Flink JobManager are running in the same container. The ApplicationMaster informs each TaskManager of the RPC address of the JobManager. After TaskManagers are started, they register with the JobManager.
  5. After all TaskManagers has registered with the JobManager, Flink starts up in the Yarn cluster. Then, the Flink Yarn client can submit Flink jobs to the JobManager, and Flink can perform mapping, scheduling, and computing for the jobs.
Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback