Failed to Run the Application Developed Based on the Hive JDBC Code Case

Symptom

After a user develops a service application by referring to the jdbc-examples sample project of the Hive component, the application fails to be executed. The application reports the following exception:

..........

2017-05-11 14:33:52.174 ERROR   --- [           main] o.a.thrift.transport.TSaslTransport      : SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(Unknown Source)
at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1711)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:260)
at org.apache.hive.jdbc.HiveConnection.createClient(HiveConnection.java:213)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:178)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at com.xxx.bigdata.hive.example.JDBCExample.main(JDBCExample.java:107)
Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Unknown Source)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Unknown Source)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Unknown Source)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(Unknown Source)
at sun.security.jgss.GSSContextImpl.initSecContext(Unknown Source)
at sun.security.jgss.GSSContextImpl.initSecContext(Unknown Source)
... 17 common frames omitted
......

Cause Analysis

1. It is suspected that service interaction is performed before Kerberos authentication is complete.
2. Further analyze the logs. The log contains "com.xxx.bigdata.security.LoginUtil - Login success!!!!!!!!!!!!!!" but not "org.apache.hadoop.security.UserGroupInformation : Login successful...".

   Analyze the code. It is found that:

   /*      */   @InterfaceAudience.Public
   /*      */   @InterfaceStability.Evolving
   /*      */   public static synchronized void loginUserFromKeytab(String user, String path)
   /*      */     throws IOException
   /*      */   {
   /*  958 */     if (!isSecurityEnabled()) {
   /*  959 */       return;
   /*      */     }
   ......

3. Analyze isSecurityEnabled() and check whether hadoop.security.authentication is set to kerberos in the configuration.

   This Hive service application is not correctly configured. Therefore, the system determines that Kerberos authentication is not required.

   Analyze the jdbc-examples sample project of the Hive component. This problem does not occur in the sample project because the core-site.xml configuration file exists in the classpath directory of the project and hadoop.security.authentication is set to kerberos in the configuration file.

Solution

Use any of the following methods to solve the problem:

• Method 1:

  Save the core-site.xml configuration file in the classpath directory by referring to the jdbc-examples sample project of the Hive component.

• Method 2:
  In the code, explicitly load the configuration file core-site.xml:

  ......
      conf = new Configuration();
      String userdir = System.getProperty("user.dir") + File.separator + "conf" + File.separator;
      conf.addResource(new Path(userdir + "core-site.xml"));
  ......

• Method 3:
  In the code, set hadoop.security.authentication to kerberos:

  ......    
      CONF = new Configuration();
      CONF.set("hadoop.security.authentication", "kerberos");
  ......