Compiling and Running the Application

Updated on 2022-07-11 GMT+08:00

View PDF

Scenario

After the code development is complete, you can run an application in the Linux development environment.

Procedure

Go to the local root directory of the project and run the following command in Windows cmd to compress the package:

mvn -s "{maven_setting_path}" clean package
NOTE:
- In the preceding command, {maven_setting_path} is the path of the setting.xml file of the local maven.
- After the package is successfully packed, obtain the JAR package, for example, MRTest-XXX.jar, from the target subdirectory in the root directory of the project. The name of the JAR package varies according to the actual package.
Upload MRTest-XXX.jar to the Linux client, such as /opt/client/conf, the same directory where the configuration files are located in.
Submit the MapReduce job in the Linux environment and execute the sample project.
- Run the following command for MapReduce statistics sample project to configure parameters and submit jobs:
  yarn jar MRTest-XXX.jar com.huawei.bigdata.mapreduce.examples.FemaleInfoCollector <inputPath> <outputPath>
  
  <inputPath> indicates the input path and <outputPath> indicates the output path in HDFS.
  NOTE:
  - Before running the yarn jar MRTest-XXX.jar com.huawei.bigdata.mapreduce.examples.FemaleInfoCollector <inputPath> <outputPath> command, upload the log1.txt and log2.txt files to the <inputPath> directory in HDFS. See Typical Scenarios.
  - Before running the yarn jar MRTest-XXX.jar com.huawei.bigdata.mapreduce.examples.FemaleInfoCollector <inputPath> <outputPath> command, ensure that the <outputPath> directory is deleted. Otherwise, an error will occur.
  - Do not restart the HDFS service during the running of MapReduce tasks. Otherwise, the tasks may fail.
- In MapReduce accessing multi-components sample project, perform the following operations:
  1. Obtain the hbase-site.xml, hiveclient.properties and hive-site.xml configuration files, create a folder for example /opt/client/conf, and save the configurations files.
    NOTE:
    
    The file hbase-site.xml is acquired from the HBase client, hiveclient.properties and hive-site.xml are acquired from the Hive client.
  2. Add the classpath required for sample projects in the Linux environment, Example of classpath:
    export YARN_USER_CLASSPATH=/opt/client/conf:/opt/client/HBase/hbase/lib/*:/opt/client/HBase/hbase/lib/client-facing-thirdparty/*:/opt/client/Hive/Beeline/lib/*
  3. Submit MapReduce jobs. Run the following command to run the sample project:
    yarn jar MRTest-XXX.jar com.huawei.bigdata.mapreduce.examples.MultiComponentExample

Parent topic: Commissioning the Application in the Linux Environment

Previous topic: Commissioning the Application in the Linux Environment

Next topic: Checking the Commissioning Result

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot