Help Center/ MapReduce Service/ Component Operation Guide (LTS)/ Using Hive/ Enterprise-Class Enhancements of Hive/ Switching the Hive Execution Engine to Tez
Updated on 2024-10-25 GMT+08:00
View PDF
Share

Switching the Hive Execution Engine to Tez

Scenario

Hive can use the Tez engine to process data computing tasks. Before executing a task, you can manually switch the execution engine to Tez.

Prerequisites

The TimelineServer role of the Yarn service has been installed in the cluster and is running properly.

Switching the Execution Engine on the Client to Tez

  1. Install and log in to the Hive client. For details, see Using the Hive Client.
  2. Run the following commands for MRS 3.1.2 to switch the engine and enable the yarn.timeline-service.enabled parameter:

    set hive.execution.engine=tez;

    set yarn.timeline-service.enabled=true;
    • After yarn.timeline-service.enabled is enabled, you can view the details about the tasks executed by the Tez engine on TezUI. After this function is enabled, task information will be reported to TimelineServer. If the TimelineServer instance is faulty, the task will fail.
    • Tez uses the ApplicationMaster buffer pool. Therefore, yarn.timeline-service.enabled must be enabled before Tez tasks are submitted. Otherwise, this parameter cannot take effect and you need to log in to the client again to configure it.
    • When the execution engine needs to be switched to another engine, you need to run the set yarn.timeline-service.enabled=false command on the client to disable the yarn.timeline-service.enabled parameter.
    • To specify a Yarn running queue, run the set tez.queue.name=default command on the client.

  3. For MRS 3.2.0 and later versions, run the following command to switch the engine:

    set hive.execution.engine=tez;

    To specify a running Yarn queue, run the set tez.queue.name=default command on the client.

  4. Submit and execute the Tez tasks.
  5. Log in to FusionInsight Manager. Choose Cluster > Services > Tez > TezUI(host name) to view the task execution status on the TezUI page.

Switching the Default Execution Engine of Hive to Tez

  1. Log in to FusionInsight Manager. Choose Cluster > Services > Hive > Configurations > All Configurations > HiveServer(Role), and search for hive.execution.engine.

  2. Set hive.execution.engine to tez.
  3. Choose Hive(Service) > Customization and search for yarn.site.customized.configs for MRS 3.1.2.
  4. Add custom parameter yarn.timeline-service.enabled to yarn.site.customized.configs and set it to true for MRS 3.1.2.

    • After yarn.timeline-service.enabled is enabled, you can view the details about the tasks executed by the Tez engine on TezUI. After this function is enabled, task information will be reported to TimelineServer. If the TimelineServer instance is faulty, the task will fail.
    • Tez uses the ApplicationMaster buffer pool. Therefore, yarn.timeline-service.enabled must be enabled before Tez tasks are submitted. Otherwise, this parameter cannot take effect and you need to log in to the client again to configure it.
    • When the execution engine needs to be switched to another one, you need to set the value of parameter yarn.timeline-service.enabled to false.

  5. Click Save. In the displayed confirmation dialog box, click OK.
  6. Choose Dashboard > More > Restart Service, enter the password of the current user, and click OK to restart the Hive service.

  7. Install and log in to the Hive client. For details, see Using the Hive Client.
  8. Submit and execute the Tez tasks.
  9. Log in to FusionInsight Manager and choose Cluster > Services > Tez > TezUI(host name). On the displayed TezUI page, view the task execution status.