Flink Job Engine

Flink web UI provides a web-based visual development platform. You only need to compile SQL statements to develop jobs, slashing the job development threshold. In addition, the exposure of platform capabilities allows service personnel to compile SQL statements for job development to quickly respond to requirements, greatly reducing the Flink job development workload.

This section applies to only MRS 3.1.0 or later.

Flink Web UI Features

The Flink web UI has the following features:

Enterprise-class visual O&M: GUI-based O&M management, job monitoring, and standardization of Flink SQL statements for job development.
Quick cluster connection: After configuring the client and user credential key file, you can quickly access a cluster using the cluster connection function.
Quick data connection: You can access a component by configuring the data connection function. If Data Connection Type is set to HDFS, you need to create a cluster connection. If Authentication Mode is set to KERBEROS for other data connection types, you need to create a cluster connection. If Authentication Mode is set to SIMPLE, you do not need to create a cluster connection.

If Data Connection Type is set to Kafka, Authentication Type cannot be set to KERBEROS.
Visual development platform: The input/output mapping table can be customized to meet the requirements of different input sources and output destinations.
Easy to use GUI-based job management

Key Web UI Capabilities

Table 1 shows the key capabilities provided by Flink web UI.

**Table 1** Key web UI capabilities
Item	Description
Batch-Stream convergence	Batch jobs and stream jobs can be processed with a unified set of Flink SQL statements.
Flink SQL kernel capabilities	Flink SQL supports customized window size, stream compute within 24 hours, and batch processing beyond 24 hours. Flink SQL supports reading data from Kafka and HDFS, writing data to Kafka and HDFS. A job can define multiple Flink SQL jobs, and multiple metrics can be combined into one job for computing. If a job contains same primary keys as well as same inputs and outputs, the job supports the computing of multiple windows. The AVG, SUM, COUNT, MAX, and MIN statistical methods are supported.
Flink SQL functions on the console	Cluster connection management allows you to configure clusters where services such as Kafka and HDFS reside. Data connection management allows you to configure services such as Kafka and HDFS. Data table management allows you to define data tables accessed by SQL statements and generate DDL statements. Flink SQL job definition allows you to verify, parse, optimize, convert a job into a Flink job, and submit the job for running based on the entered SQL statements.
Flink job visual management	Stream jobs and batch jobs can be defined in a visual manner. Job resources, fault recovery policies, and checkpoint policies can be configured in a visual manner. Status monitoring of stream and batch jobs are supported. The Flink job O&M is enhanced, including redirection of the native monitoring page.
Performance and reliability	Stream processing supports 24-hour window aggregation computing and millisecond-level performance. Batch processing supports 90-day window aggregation computing, which can be completed in minutes. Invalid data of stream processing and batch processing can be filtered out. When HDFS data is read, the data can be filtered based on the calculation period in advance. If the job definition platform is faulty or the service is degraded, jobs cannot be redefined, but the computing of existing jobs is not affected. The automatic restart mechanism is provided for job failures. You can configure restart policies.

Flink Web UI Application Process

The Flink web UI application process is shown as follows:

Figure 1 Flink web UI application process

**Table 2** Description of the Flink web UI application process
Step	Description	Reference
Creating an application	Applications can be used to isolate different upper-layer services.	Creating a FlinkServer Application
Creating a cluster connection	Different clusters can be accessed by configuring the cluster connection.	Creating a FlinkServer Cluster Connection
Creating a Data Connection	Through data connections, you can access different data services, including HDFS and Kafka.	Creating a FlinkServer Data Connection
Creating a stream table	Data tables can be used to define basic attributes and parameters of source tables, dimension tables, and output tables.	Creating a FlinkServer Stream Table Source
Creating a SQL/JAR job (stream/batch job)	APIs can be used to define Flink jobs, including Flink SQL and Flink Jar jobs.	Creating a FlinkServer Job
Managing jobs	A created job can be managed, including starting, developing, stopping, deleting, and editing the job.	Creating a FlinkServer Job