Help Center > > User Guide> Storage-Compute Decoupling Operation Guide> Using a Storage-Compute Decoupled Cluster> Interconnecting Hive with OBS

Interconnecting Hive with OBS

Updated at: Oct 21, 2021 GMT+08:00

Before performing the following operations, ensure that you have configured a storage-compute decoupled cluster by referring to Configuring a Storage-Compute Decoupled Cluster (Agency) or Configuring a Storage-Compute Decoupled Cluster (AK/SK).

When creating a table, set the table location to an OBS path.

  1. Log in to the client installation node as the client installation user.
  2. Run the following command to initialize environment variables:

    source ${client_home}/bigdata_env

  3. For a security cluster, run the following command to perform user authentication (the user must have the permission to perform Hive operations). If Kerberos authentication is not enabled for the current cluster, you do not need to run this command.

    kinit User performing Hive operations

  4. Log in to FusionInsight Manager and choose Cluster > Services > Hive > Configurations > All Configurations.

    In the left navigation tree, choose Hive > Customization. In the customized configuration items, add dfs.namenode.acls.enabled to the hdfs.site.customized.configs parameter and set its value to false.

  5. Save the configurations and restart Hive.
  6. Log in to the beeline client and set Location to the OBS file system path when creating a table.

    beeline

    create table test(name string) location "obs://OBS parallel file system name/user/hive/warehouse/test";

    You need to add the component operator to the URL policy in the Ranger policy. Set the URL to the complete path of the object on OBS. Select the Read and Write permissions.

Setting the Default Location of the Created Hive Table to the OBS Path

  1. Log in to FusionInsight Manager and choose Cluster > Services > Hive > Configurations > All Configurations.
  2. In the left navigation tree, choose MetaStore > Customization. Add hive.metastore.warehouse.dir to the hive.metastore.customized.configs parameter and set it to the OBS path.

    Figure 1 hive.metastore.warehouse.dir configuration

  3. In the left navigation tree, choose HiveServer > Customization. Add hive.metastore.warehouse.dir to the hive.metastore.customized.configs and hive.server.customized.configs parameters and set it to the OBS path.

    Figure 2 hive.metastore.warehouse.dir configuration

  4. Save the configurations and restart Hive.
  5. Log in to the beeline client, create a table, and check whether the location is the OBS path.

    beeline

    create table test(name string);

    desc formatted test;

Did you find this page helpful?

Submit successfully!

Thank you for your feedback. Your feedback helps make our documentation better.

Failed to submit the feedback. Please try again later.

Which of the following issues have you encountered?







Please complete at least one feedback item.

Content most length 200 character

Content is empty.

OK Cancel