Help Center/ MapReduce Service/ Troubleshooting/ Using Spark/ Failed to Start spark-sql and spark-shell Due to JDK Version Mismatch
Updated on 2023-11-30 GMT+08:00

Failed to Start spark-sql and spark-shell Due to JDK Version Mismatch

Symptom

The JDK version does not match. As a result, the client fails to start spark-sql and spark-shell.

Cause Analysis

  1. The following error information is displayed on the Driver:
    Exception Occurs: BadPadding 16/02/22 14:25:38 ERROR Schema: Failed initialising database. Unable to open a test connection to the given database. JDBC url = jdbc:postgresql://ip:port/sparkhivemeta, username = spark. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). 
  2. When a SparkSQL task is used, DBService needs to be accessed to obtain metadata information. On the client, the ciphertext needs to be decrypted for access. During the use, the user does not follow the process or configure environment variables, and the default JDK version exists in the environment variables of the client. As a result, the decryption program invoked during decryption is abnormal, and the user is locked.

Solution

  1. Run the which java command to check whether the default Java command is the Java command of the client.
  2. If it is not, go to the next step.

    source ${client_path}/bigdata_env

    Run the kinit username command and enter the password corresponding to the username to start the task.