Preparing Environment

Before interconnecting open source component Hive with LakeFormation, complete the following operations:

Prepare an available open-source Hive environment (Hive 2.3 and Hive 3.1 are supported) and install the Git environment.
Prepare a LakeFormation instance. For details, see Creating an Instance.
Create a LakeFormation access client in the same VPC and subnet as Hive. For details, see Managing Clients.
Prepare the development environment. For details, see the part "Preparing the Environment" inPreparing the Development Environment. You can choose whether to install and configure IntelliJ IDEA.
Download the LakeFormation client.
- Method 1: downloading the release version
  Download link: https://gitee.com/HuaweiCloudDeveloper/huaweicloud-lake-formation-lakecat-sdk-java/releases
  
  Download the corresponding client based on the Spark and Hive versions. For example, if the Hive version is 2.3.9, download lakeformation-lakecat-client-hive2.3-spark3.1-1.0.0.jar.
- Method 2: compiling the client locally
  1. Obtain the client code.
    Download link: https://gitee.com/HuaweiCloudDeveloper/huaweicloud-lake-formation-lakecat-sdk-java
    
    Run the following command in Git to switch the branch to master_dev:
    
    git checkout master_dev
  2. Configure the Maven source. For details, see Obtaining the SDK and Configuring Maven.
  3. Obtain the following JAR package and corresponding POM file and save the file to the local Maven repository.
    For example, if the local repository directory is D:\maven\repository, save the file in the D:\maven\repository\com\huaweicloud\hadoop-huaweicloud\3.1.1-hw-53.8 path.
    - JAR package: https://github.com/huaweicloud/obsa-hdfs/blob/master/release/hadoop-huaweicloud-3.1.1-hw-53.8.jar
    - POM file: https://github.com/huaweicloud/obsa-hdfs/blob/master/hadoop-huaweicloud/pom.xml (rename the downloaded file hadoop-huaweicloud-3.1.1-hw-53.8.pom)
  4. Package and upload the client code.
    Go to the client project directory and run the following command to package the client code:
    
    mvn clean install -DskipTests=true -P"${HIVE_PROFILE}"
    
    HIVE_PROFILE: Set this parameter to hive-2.3 or hive-3.1.
  5. After the packaging is complete, obtain the lakeformation-lakecat-client-1.0.0.jar package from the target directory of lakeformation-lakecat-client.
Download the JAR packages required by the Hive kernel.
- Method 1: downloading the JAR packages needed for Hive pre-building.
  Download link: https://gitee.com/HuaweiCloudDeveloper/huaweicloud-lake-formation-lakecat-sdk-java/releases
  
  Download the corresponding client based on the Hive versions. For example, if the Hive version is 2.3.9, download hive-exec-2.3.9.jar and hive-common-2.3.9.jar.
- Method 2: modifying Hive-related JAR packages locally
  In Windows, you need to perform Maven operations in the WSL development environment.
  1. Download the Hive source code based on the Hive version.
    For example, if the Hive kernel version is 2.3.9, the download link is https://github.com/apache/hive/tree/rel/release-2.3.9.
  2. Apply the patch in the LakeFormation client code to the Hive source code.
    1. Switch the Hive source code branch as required. For example, if the Hive kernel version is 2.3.9, run the following command:
      git checkout rel/release-2.3.9
    2. Run the following command to apply the patch file to the Hive source code project after the branch is switched:
      mvn patch:apply -DpatchFile=${your patch file location}
      
      In the command, your patch file location indicates the storage path of hive-2_3_for_lakeformation.patch or hive-3_1_for_lakeformation.patch. The patch file can be obtained from the client project, as shown in the following figure.
    3. Run the following command to recompile the Hive kernel source code:
      mvn clean install -DskipTests=true

Prepare and replace JAR packages required by the Hive kernel.

Obtain the JAR packages listed in the following table and supplement or replace them in the lib directory in the Hive installation environment.

**Table 1** JAR packages required by the Hive environment
No.	JAR Package	How to Obtain
1	spring-web-5.3.24.jar	https://mirrors.huaweicloud.com/repository/maven/org/springframework/spring-web/5.3.24/
2	spring-core-5.3.24.jar	https://mirrors.huaweicloud.com/repository/maven/org/springframework/spring-core/5.3.24/
3	spring-context-5.3.24.jar	https://mirrors.huaweicloud.com/repository/maven/org/springframework/spring-context/5.3.24/
4	spring-beans-5.3.24.jar	https://mirrors.huaweicloud.com/repository/maven/org/springframework/spring-beans/5.3.24/
5	caffeine-2.9.3.jar	https://mirrors.huaweicloud.com/repository/maven/com/github/ben-manes/caffeine/caffeine/2.9.3/
6	mapstruct-1.5.3.Final.jar	https://mirrors.huaweicloud.com/repository/maven/org/mapstruct/mapstruct/1.5.3.Final/
7	http-core-4.4.13.jar (If the Hive kernel version is 3.1, this JAR package is not required.)	https://mirrors.huaweicloud.com/repository/maven/org/apache/httpcomponents/httpcore/4.4.13/
8	jca-1.0.4.jar	https://mirrors.huaweicloud.com/repository/maven/org/openeuler/jca/1.0.4/
9	Replace commons-codec-1.4.jar with commons-codec-1.15.jar. (If the Hive kernel version is 3.1, this JAR package is not required.)	https://mirrors.huaweicloud.com/repository/maven/commons-codec/commons-codec/1.15/
10	java-sdk-core-3.2.4.jar (If only Custom Authentication Information Obtaining Class is used for token authentication, this JAR package is not required.)	https://mirrors.huaweicloud.com/repository/maven/huaweicloudsdk/com/huawei/apigateway/java-sdk-core/
11	hadoop-huaweicloud-3.1.1-hw-53.8.jar	https://github.com/huaweicloud/obsa-hdfs/blob/master/release/hadoop-huaweicloud-3.1.1-hw-53.8.jar
12	lakeformation-lakecat-client-1.0.0.jar	Obtain the package by referring to the operation in 5.
13	hive-exec-${version}.jar	Obtain the package by referring to the operation in 6.
14	hive-common-${version}.jar	Obtain the package by referring to the operation in 6.