Updated on 2023-08-31 GMT+08:00

Development process description

Figure 1 and Table 1 describe the development process.

Figure 1 MapReduce application development process
Table 1 Description of the MapReduce development process

Step

Description

Reference

Preparing the development environment

Before you start development, prepare the development environment. You are advised to use the Java language and IntelliJ IDEA tool to develop applications, and complete initial configurations of JDK and Maven.

Preparing for Development Environment

Preparing the configuration files for connecting to the cluster

During the development or a test run of the program, you need to use the cluster configuration files to connect to an MRS cluster. The configuration files usually contain the cluster component information file and user files used for security authentication. You can obtain the required information from the created MRS cluster.

Nodes used for program debugging or running must be able to communicate with the nodes within the MRS cluster, and the hosts domain name must be configured.

Preparing the Configuration Files for Connecting to the Cluster

Configuring and importing sample projects

HBase provides a range of sample projects for different scenarios. You can obtain a sample project and import it to the local development environment.

Configuring and Importing Sample Projects

Configuring security authentication

If you are using an MRS cluster with Kerberos authentication enabled, security authentication is required.

Preparing for Security Authentication

Write program code for a service scenario.

Develop the program based on service requirements and call component APIs to implement required functions.

Developing the Project

Compile and run the program.

Compile and run the program. You can debug and run the program in the local Windows development environment, or compile the program into a JAR package and submit it to a Linux node.

Debugging the Application