Downloading and Importing Sample Projects

Scenario

Download sample projects and import them to the IDEA on Windows for learning.

Prerequisites

• A complete client has been installed in the Linux environment.
• The URL of a running Oozie server (any node) has been obtained. The URL is the target address to which the client submits a workflow job.

  The URL format is https://Oozie service IP address:21003/oozie, for example, https://10.10.10.176:21003/oozie.

Procedure

1. Obtain the OozieMapReduceExample, OozieSparkHBaseExample, and OozieSparkHiveExample sample projects from the sample project folder oozienormal-examples in the src\oozie-examples directory where the sample code is decompressed. For details, see Obtaining Sample Projects from Huawei Mirrors.
2. In an application development environment, import the sample projects to the IDEA development environment.
   1. Choose File > Open.

      The Browse dialog box is displayed.

   2. Select a sample project folder, and click OK.

3. Modify the parameters in the sample project. For details, see Table 1.

   Table 1 Parameters to be modified

   File Name	Parameter	Value	Example Value
   \src\main\resources\job.properties	userName	User who submits a job.	developuser
   \src\main\resources\application.properties	submit_user	User who submits a job.	developuser
   	oozie_url_default	https://Oozie service IP address:21003/oozie	https://10.10.10.176:21003/oozie

4. Select the sample project to be run.
   • For the OozieMapReduceExcample sample project, go to 5.
   • For the OozieSparkHBaseExample and OozieSparkHiveExample sample projects, see Scheduling Spark2x to Access HBase and Hive Using Oozie.

5. Use the client to upload the example files of Oozie to HDFS.
   1. Log in to the node where the client is installed, and switch to the directory of the client, for example /opt/client.

      cd /opt/client

   2. Run the following command to configure environment variables:

      source bigdata_env

   3. Run the following commands to create a directory in HDFS and upload the sample project to the directory:

      hdfs dfs -mkdir /user/developuser

      hdfs dfs -put -f /opt/client/Oozie/oozie-client-*/examples /user/developuser

      

      In the preceding command, oozie-client-* indicates the version number. Replace it with the actual version number.