Help Center/ MapReduce Service/ Developer Guide (Normal_Earlier Than 3.x)/ Introduction to MRS Application Development
Updated on 2024-08-16 GMT+08:00

Introduction to MRS Application Development

MRS is a unified platform for enterprise-level big data storage, query, and analysis. It helps enterprises quickly build massive data processing systems and discover new value points and business opportunities by analyzing and mining massive data.

MRS provides sample programs for common service scenarios of each component. Developers can develop and compile data applications based on the sample projects. The JAR packages on which the sample projects depend can be downloaded from the HUAWEI CLOUD open-source image site. The JAR packages of other communities can be downloaded from the Maven public repository.

Developer Capability Requirements

  • You have a certain understanding of the components in the big data domain.
  • You are familiar with how to use the ECS and the MRS development components.
  • You have a certain understanding of the Maven build mode and usage methods.
  • You already have some knowledge of the Java syntax.

MRS Application Development Process

The following figure shows the MRS component application development process. For details about the development and compilation operations of each component application, see the corresponding chapter in the component development guide.

Figure 1 MRS Application Development Process
Table 1 MRS Application Development Process

stage

Description

Preparing the Development Environment

Before developing applications, prepare the development environment. The IntelliJ IDEA tool is recommended. In addition, you need to configure the JDK and Maven locally.

Preparing the Configuration File for Connecting to the Cluster

During application development or running, you need to connect to the MRS cluster through the cluster configuration file. The configuration file contains user files used for security authentication. You can obtain related content from the created MRS cluster.

Nodes used for program commissioning or running must be able to communicate with nodes in the MRS cluster.

Configuring and Importing a Sample Project

MRS provides multiple sample programs for different component scenarios. You can obtain sample projects and import them to the local development environment for program learning.

Configuring Security Authentication

When connecting to the MRS cluster with Kerberos authentication enabled, configure the user with the related resource access permission in the application for security authentication.

Develop programs based on business scenarios.

Develop programs based on the actual service scenario and invoke component interfaces to implement corresponding functions.

Compiling and Running the Program

Compile and run the developed program. You can debug and run the program in the local Windows development environment or compile the program into a JAR package and submit it to the Linux node for running.