Updated on 2023-08-31 GMT+08:00

Flink Sample Project

To obtain an MRS sample project, visit https://github.com/huaweicloud/huaweicloud-mrs-example and switch to the branch that matches the MRS cluster version. Download the package to the local PC and decompress it to obtain the sample project of each component.

MRS provides the following Flink sample projects. For the security mode, the path is flink-examples/flink-examples-security, and the path for common mode is flink-examples/flink-examples-normal.
Table 1 Flink sample projects

Project

Description

FlinkCheckpointJavaExample

Asynchronous checkpoint

Assume that you want to collect data volume in a 4-second time window every seconds and the status of operators must be strictly consistent. That is, if an application recovers from a failure, the status of all operators must be the same. For more information about the scenario, see Asynchronous Checkpoint Mechanism

FlinkCheckpointScalaExample

FlinkKafkaJavaExample

Kafka data producing and consuming

Call APIs of the flink-connector-kafka module to produce and consume data. For details about service scenario, see Interconnecting with Kafka.

FlinkKafkaScalaExample

FlinkPipelineJavaExample

Job pipeline

For details about the service scenario, see Job Pipeline Program.

The publisher job generates 10,000 data records per second and sends the data to downstream systems through the NettySink operator of the job. The other two jobs function as subscribers of each data copy and print data.

FlinkPipelineScalaExample

FlinkSqlJavaExample

SQL job submission through JAR jobs on the client

FlinkStreamJavaExample

DataStream

For details about the service scenario, see DataStream Application.

Assume that there are a log file of time on site during weekends of an online shopping website and a CSV table of user information. The Flink application can be used to collect real-time statistics on detailed information about the female users who spend more than 2 hours on online shopping.

FlinkStreamScalaExample

FlinkStreamSqlJoinExample

Stream SQL Join

For details about the service scenario, see Stream SQL Join Program.

Assume that a Flink service receives a message every second. The message records the basic information about a user, including the name, gender, and age. Another Flink service (service 2) receives a message irregularly, and the message records the name and career information about the user. The application performs joint query on the two messages in real time using the user name recorded in the message of service 2 as the keyword.