Updated on 2023-08-31 GMT+08:00

Development Process

Kafka client roles include Producer and Consumer, which share the same application development process.

Figure 1 and Table 1 show each phase of the development process.

Figure 1 Kafka client application development process
Table 1 Kafka client development process

Phase

Description

Reference Document

Preparing the development environment

Before you start development, prepare the development environment. You are advised to use the Java language and IntelliJ IDEA tool to develop applications, and complete initial configurations of JDK and Maven.

Preparing the Development Environment

Preparing the configuration files for connecting to the cluster

During the development or a test run of the program, you need to use the cluster configuration files to connect to an MRS cluster. The configuration files usually contain the cluster component information file and user files used for security authentication. You can obtain the required information from the created MRS cluster.

Nodes used for program debugging or running must be able to communicate with the nodes within the MRS cluster, and the hosts domain name must be configured.

Preparing the Development Environment

Configuring and importing sample projects

Kafka provides sample projects in various scenarios. sample projects can be imported for studying.

Configuring and Importing Sample Projects

Configuring security authentication

If you are using an MRS cluster with Kerberos authentication enabled, security authentication is required.

Preparing for Security Authentication

Writing program code for a service scenario

Producer and Consumer API usage samples are provided and old APIs, new APIs, and multi-thread usage scenarios are covered, helping you quickly know Kafka APIs well.

Compile and run the program. You can debug and run the program in the local Windows development environment, or compile the program into a JAR package and submit it to a Linux node.

Developing an Application

Compiling and running the program

This phase provides guidance for users to submit and run a developed program and then view the result.

Application Commissioning