Downloading and Importing Sample Projects
Scenario
Download sample projects and import them to the IDEA on Windows for learning.
Prerequisites
- A developer account, for example, developuser, has been prepared by following instructions in Preparing Development and Operating Environment, and the user authentication credential file has been downloaded to the local host.
The user has the common user permission of Oozie, HDFS access permission, Hive table read and write permission, HBase read and write permission, and Yarn queue submission permission.
- A complete cluster client has been installed in the Linux environment.
- The URL of a running Oozie server (any node) has been obtained. The URL is the target address to which the client submits a workflow job.
The URL format is https://Oozie node service IP address:21003/oozie. The port number is the value of OOZIE_HTTPS_PORT, which is 21003 by default.
for example, https://10.10.10.176:21003/oozie.
Procedure
- Obtain the OozieMapReduceExample, OozieSparkHBaseExample, and OozieSparkHiveExample sample projects from the sample project folder ooziesecurity-examples in the src\oozie-examples directory where the sample code is decompressed. For details, see Obtaining the MRS Application Development Sample Project.
- Copy the keytab file user.keytab and user authentication credential file krb5.conf obtained in Preparing a Developer Account to the \src\main\resources directory of the OozieMapReduceExample, OozieSparkHBaseExample, and OozieSparkHiveExample sample projects.
- In an application development environment, import the sample projects to the IDEA development environment.
- Choose File > Open.
The Browse dialog box is displayed.
- Select a sample project folder, and click OK.
- Choose File > Open.
- Modify the parameters in the sample project. For details, see Table 1.
Table 1 Parameters to be modified File Name
Parameter
Value
Example Value
\src\main\resources\job.properties
userName
User who submits a job.
developuser
\src\main\resources\application.properties
submit_user
User who submits a job.
developuser
oozie_url_default
https://Oozie service IP address:21003/oozie
https://10.10.10.176:21003/oozie
- Select the sample project to be run.
- For the OozieMapReduceExcample sample project, go to 6.
- For the OozieSparkHBaseExample and OozieSparkHiveExample sample projects, see Scheduling Spark2x to Access HBase and Hive Using Oozie.
- Use the client to upload the examples folder of Oozie to HDFS.
- Log in to the node where the client is installed, and switch to the directory of the client, for example /opt/client.
cd /opt/client
- Run the following command to configure environment variables:
- Run the following command to perform user authentication. Change the password upon the first login.
- Run the following commands to create a directory in HDFS and upload the sample project to the directory:
hdfs dfs -mkdir /user/developuser
hdfs dfs -put -f /opt/client/Oozie/oozie-client-*/examples /user/developuser
In the preceding command, oozie-client-* indicates the version number. Replace it with the actual version number.
- Log in to the node where the client is installed, and switch to the directory of the client, for example /opt/client.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot