Configuring Hive Transform

Scenarios

The Hive Transform function allows users to embed external scripts (such as Python and Java) in Hive queries to perform complex data conversion and processing. This enables users to execute custom data processing logic that cannot be implemented using standard Hive functions. The Transform operation transfers data to external scripts for processing and then returns the results to Hive, implementing data cleaning, conversion, and enrichment. This function extends the capabilities of Hive to meet more complex ETL requirements.

The Transform function is disabled by default in open-source Hive. However, MRS Hive allows you to configure this feature. When Kerberos authentication is enabled for the cluster (in security mode), the function is disabled by default to match the open-source community version. You can manually toggle the switch to enable it. Exercise caution when enabling the Transform function, as it may pose security risks to your cluster.

Notes and Constraints

When Kerberos authentication is disabled for the cluster (in normal mode), the Transform function is enabled by default, and the operations in this section do not apply. However, once Ranger authentication is enabled for Hive, the function is disabled by default. You can then enable or disable it by following the instructions in this section.

Procedure

1. Log in to FusionInsight Manager, choose Cluster > Services > Hive, and click Configurations and then All Configurations.

   

2. Modify or add the hive.security.transform.disallow parameter.
   • Kerberos authentication is enabled for the cluster (the cluster is in security mode)

     Search for hive.security.transform.disallow in the search box, change the parameter value to true or false, save the changes, and restart all HiveServer instances.

   • Kerberos authentication is disabled for the cluster (the cluster is in normal mode), and Ranger authentication is enabled for Hive.

     In the navigation pane, choose HiveServer(Role) > Customization. Add a custom parameter to the hivemetastore-site.xml file. Set Name to hive.security.transform.disallow and Value to true or false. Save the settings and restart all HiveServer instances.

   

   • When hive.security.transform.disallow is set to true, the Transform function is disabled. This configuration aligns with the default behavior of the open-source community version.
   • When hive.security.transform.disallow is set to false, the Transform function is enabled. Note that enabling this function can pose security risks.