Interconnecting HetuEngine with OBS

Updated on 2024-11-29 GMT+08:00

View PDF

Interconnecting with OBS

In an MRS cluster, Location can be set to an OBS file system path during HetuEngine table creation and HetuEngine can connect to OBS through Hive Metastore.

Setting Location to the OBS file system path when creating a table
1. If a HetuEngine compute instance is running, restart it.
  Log in to FusionInsight Manager as a user who has permission to access the HetuEngine web UI. Choose Cluster > Services > HetuEngine. In the Basic Information area in the Dashboard tab, click the link next to HSConsole WebUI. On the displayed HSConsole page, click Compute Instance. In the instance list, click Restart in the Operation column and operate as prompted.
2. Log in to the node where the HetuEngine service client is located as the client installation user and run the following command:
  source Client installation directory/bigdata_env
3. Log in to the HetuEngine client based on the cluster authentication mode.
  - Kerberos authentication has been enabled for the cluster (security mode): Run the following command to complete user authentication and log in to the HetuEngine client:
    kinit User performing HetuEngine operations
    
    hetu-cli --catalog hive --tenant default --schema default
    
    For details about how to assign permissions to users in the Ranger, see Configuring Ranger Permissions.
  - Kerberos authentication is not enabled for the cluster (normal mode): Run the following command to log in to the HetuEngine client:
    hetu-cli --catalog hive --tenant default --schema default --user User performing HetuEngine operations
4. Set Location to the OBS file system path when creating a table.
  create table test(name string) with (location = 'obs://Name of the OBS parallel file system/user/hive/warehouse/test');
Interconnecting with OBS through Hive Metastore
1. Complete the configurations by referring to Interconnecting Hive with OBS using MetaStore.
2. Log in to FusionInsight Manager, choose Cluster > Services > HetuEngine. On the displayed page, choose More > Synchronize Configuration. After the synchronization is complete, choose More > Synchronize Configuration again and then restart the HetuEngine service as prompted.
  NOTICE:
  
  If a HetuEngine compute instance is running, stop it before restarting the service. After the service is restarted, start this compute instance.
3. No location needs to be specified when you log in to the HetuEngine client to create a schema or table. The schema or table is stored on OBS by default.

Configuring Ranger Permissions

For HetuEngine clusters with Kerberos authentication enabled (security mode), the methods to grant Ranger permission are the same for both storage-compute decoupled architecture and and storage-compute coupled architecture.

Parent topic: Interconnecting Components with OBS Using Guardian

Previous topic: Interconnecting Hudi with OBS

Next topic: Interconnecting HDFS with OBS

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot