Updated on 2024-08-16 GMT+08:00

Kafka Application Development Process

Kafka client roles include Producer and Consumer, which share the same application development process.

Figure 1 and Table 1 show each phase of the development process.

Figure 1 Kafka client application development process
Table 1 Kafka client application development process description

Phase

Description

Reference

Understand basic concepts.

Before developing an application, learn basic concepts of Kafka, and determine whether the desired role is Producer or Consumer based on the actual scenario.

Common Concepts of Kafka Application Development

Prepare the development environment.

The Java language is recommended for the development of Kafka client applications, and Maven is recommended for constructing projects.

Preparing the Maven and JDK

Prepare the operating environment.

The running environment of the Kafka sample application consists of nodes of the VPC cluster where the MRS service is deployed.

-

Prepare a project.

Kafka provides sample projects for different scenarios. You can download a sample project for learning. You can also create a Kafka project according to the guide.

Importing and Configuring Kafka Sample Projects

Develop a project based on the scenario.

Producer and Consumer API usage samples are provided and cover old APIs, new APIs, and multi-thread usage scenarios, helping you quickly know Kafka APIs well.

Kafka Development Plan

Compile and run an application.

Compile and Compress a developed application. Then upload it to a Linux node in the VPC cluster for running.

Commissioning a Kafka Application

View application running results.

Application running results can be output to the Linux CLI page. You can also use a Linux client to consume the topic data to check whether the application running results are successfully written.

Commissioning a Kafka Application