Compiling and Running Applications
Scenario
You can run applications in the Windows environment after application code development is complete. If the local and cluster service planes can communicate with each other, you can perform the commissioning on the local host.
- If the IBM JDK is used in the windows OS, applications cannot be directly run in the windows OS.
- Do not restart the HDFS service during the running of MapReduce jobs. Otherwise, the jobs may fail.
Prerequisites
- The configuration file obtained in Preparing the Configuration Files for Connecting to the Cluster has been stored in the ../src/mapreduce-example-security/conf directory of the MapReduce sample project.
- You have uploaded the data to be processed to HDFS by referring to Planning MapReduce Statistics Sample Program Data.
Running the Statistics Sample Program
- Ensure that all JAR packages on which the sample project depends have been obtained.
- In the IntelliJ IDEA development environment, open the LocalRunner.java project in the sample project, right-click the project, and choose Run LocalRunner.main() from the shortcut menu to run the application project.
Running Sample Applications About Multi-Components
- Create the jaas_mr.conf file in the ../src/mapreduce-example-security/conf directory where the MapReduce project example is stored and add the following content to the file:
Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="user.keytab" principal="test@<System domain name>" useTicketCache=false storeKey=true debug=true; };
- test: indicates the name of the user created in User Information for Cluster Authentication, for example, developuser.
- System domain name: Log in to FusionInsight Manager, choose System > Permission > Domain and Mutual Trust, and view the value of Local Domain, which is the current system domain name.
- Ensure that all Hive and HBase JAR packages on which the sample project depends have been obtained.
- In the IntelliJ IDEA development environment, select the MultiComponentLocalRunner.java project and click to run the corresponding application project. Alternatively, right-click the project and choose Run MultiComponentLocalRunner.main() from the shortcut menu to run the application project.
If ZooKeeper SSL is enabled in the cluster, check the mapred-site.xml configuration file before running the sample. (Obtained from the conf configuration file directory of the sample project in the Preparing the Configuration Files of the Running Environment) Check whether the configuration items mapreduce.admin.map.child.java.opts and mapreduce.admin.reduce.child.java.opts of the contain the following information:
-Dzookeeper.client.secure=true -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty
If no, add the preceding content to the end of the configuration item.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot