Configuring Component Isolation to Access Hive MetaStore

Updated on 2024-10-25 GMT+08:00

View PDF

Scenario

This function restricts components in a cluster to connect to specified Hive Metastore instances. By default, components can connect to all Metastore instances. This function applies only to clusters whose version is MRS 3.2.0 or later.

Currently, only HetuEngine, Hive, Loader, Metadata, Spark2x and Flink can connect to Metastore in a cluster. The Metastore instances can be allocated in a unified manner.

NOTE:

This function only limits the Metastore instances accessed by component servers. Metadata is not isolated.
Currently, Flink tasks can only connect to Metastore instances through the client.
When spark-sql is used to execute tasks, the client is directly connected to Metastore. The client needs to be updated for the isolation to take effect.
This function supports only isolation in the same cluster. If HetuEngine is deployed in different clusters, unified isolation configuration is not supported. You need to modify the HetuEngine configuration to connect to the specified Metastore instance.
You are advised to configure at least two Metastore instances for each component to ensure availability during isolation configuration.

Prerequisites

The Hive service has been installed in the cluster and is running properly.

Procedure

Log in to FusionInsight Manager and choose Cluster > Services > Hive. On the displayed page, click the Configurations tab and then All Configurations, and search for the HIVE_METASTORE_URI parameter.
Set the value of HIVE_METASTORE_URI_DEFAULT to the URI connection string of all Metastore instances.
Connect a component to a specified Metastore instance. Copy the value in 2, modify the configuration items based on the component name, save the modification, and restart the component.
The following example shows how Spark2x connects to only two Metastore instances of Hive.
1. Log in to FusionInsight Manager and choose Cluster > Services > Hive. On the displayed page, click the Configurations tab and then All Configurations, and search for the HIVE_METASTORE_URI parameter.
2. Copy the default configuration of HIVE_METASTORE_URI_DEFAULT to the URI configuration item of Spark2x. If Spark2x needs to connect to only two Metastore instances, retain two nodes as required. Click Save.
3. Choose Cluster > Services > Spark2x. On the displayed page, click the Instance tab, select the instances whose configuration has expired, and choose More > Restart Instance. In the dialog box that is displayed, enter the password and click OK to restart the instances.

Parent topic: Hive Load Balancing

Previous topic: Configuring User Lease Isolation to Access HiveServer on a Specified Node

Next topic: Configuring Load Balancing for HiveMetaStore Client Connections

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot