Help Center/ MapReduce Service/ User Guide (Ankara Region)/ Troubleshooting/ Using Sqoop/ An Error Is Reported When sqoop import Is Executed to Import PostgreSQL Data to Hive
Updated on 2024-11-29 GMT+08:00
View PDF
Share

An Error Is Reported When sqoop import Is Executed to Import PostgreSQL Data to Hive

Background

The sqoop import command is executed to extract data from open-source PostgreSQL to MRS HDFS or Hive.

Issue

The sqoop command can be executed to query the PostgreSQL database table, but an error is reported when the sqoop import command is executed to import data.

The authentication type 5 is not supported. Check that you have configured the pg_hba.conf file to include the client's IP address or subnet.

Cause Analysis

  1. MD5 authentication for connecting to PostgreSQL fails. A whitelist needs to be configured in the pg_hba.cnf file.
  2. When the sqoop import command is executed, a MapReduce job is started. The PostgreSQL driver package gsjdbc4-*.jar exists in the MRS Hadoop installation directory /opt/Bigdata/FusionInsight_HD_*/1_*_DataNode/install/hadoop/share/hadoop/common/lib, which is incompatible with the open-source PostgreSQL service. As a result, an error is reported.

Procedure

  1. Configure a whitelist in the pg_hba.cnf file.
  2. Delete the gsjdbc4-*.jar packages from all core nodes, and add the PostgreSQL JAR package to sqoop/lib.

    mv /opt/Bigdata/FusionInsight_HD_*/1_*_DataNode/install/hadoop/share/hadoop/common/lib/gsjdbc4-*.jar /tmp

Parent topic: Using Sqoop