Help Center/ MapReduce Service/ Troubleshooting/ Using Spark/ Authentication Fails When Spark Connects to Other Services
Updated on 2023-11-30 GMT+08:00
View PDF
Share

Authentication Fails When Spark Connects to Other Services

Symptom

  • When Spark connects to HBase, an authentication failure message is displayed or the HBase table cannot be connected.
  • When Spark connects to HBase, a message is displayed indicating that the JAR package cannot be found.

Cause Analysis

  • Symptom 1: HBase does not obtain the authentication information of the current task. As a result, the authentication fails when HBase is connected, and the corresponding data cannot be read.
  • Symptom 2: By default, Spark does not load the HBase JAR package. You need to use --jars to add the JAR package to the task.

Procedure

  • Symptom 1: Enable the HBase authentication function by running the spark.yarn.security.credentials.hbase.enabled=true command. However, do not replace hbase-site.xml on the Spark client with hbase-site.xml on the HBase client because they are not completely consistent.
  • Symptom 2: Use --jars to upload the HBase JAR package.
Parent topic: Using Spark