Help Center/ MapReduce Service/ Component Operation Guide (ME-Abu Dhabi Region)/ Using Oozie/ Using Hue to Submit an Oozie Job/ Submitting a Workflow Job/ Submitting a DistCp Job
Updated on 2024-07-19 GMT+08:00
View PDF
Share

Submitting a DistCp Job

Scenario

This section describes how to submit an Oozie job of the DistCp type on the Hue web UI.

Procedure

  1. Create a workflow. For details, see Creating a Workflow.
  2. On the workflow editing page, select next to Distcp and drag it to the operation area.
  3. Determine whether the current DistCp operation is performed across clusters.

    • If yes, go to 4.
    • If no, go to 7.

  4. Establish cross-Manager mutual trust between two clusters.
  5. In the Distcp window that is displayed, set the value of Source, for example, to hdfs://hacluster/user/admin/examples/input-data/text/data.txt. Set Destination, for example, to hdfs://target_ip:target_port/user/admin/examples/output-data/distcp-workflow/data.txt. Click Add.
  6. Click the configuration button in the upper right corner. On the Properties tab page, click PROPERTIES+, enter the attribute name oozie.launcher.mapreduce.job.hdfs-servers in the text box on the left, enter the attribute value hdfs://source_ip:source_port,hdfs://target_ip:target_port in the text box on the right, and go to 8.

    source_ip: service address of the HDFS NameNode in the source cluster

    source_port: port number of the HDFS NameNode in the source cluster.

    target_ip: service address of the HDFS NameNode in the target cluster

    target_port: port number of the HDFS NameNode in the target cluster.

  7. In the Distcp window that is displayed, set the value of Source, for example, to /user/admin/examples/input-data/text/data.txt. Set Destination, for example, to /user/admin/examples/output-data/distcp-workflow/data.txt. Click Add.
  8. Click in the upper right corner. On the configuration page that is displayed, click Delete+ and add the directory to be deleted, for example, /user/admin/examples/output-data/distcp-workflow.

  9. Click in the upper right corner of the Oozie editor.

    If you need to modify the job name before saving the job (default value: My Workflow), click the name directly for modification, for example, Distcp-Workflow.

  10. After the configuration is saved, click , and submit the job.

    After the job is submitted, you can view the related contents of the job, such as the detailed information, logs, and processes, on Hue.