Spark Shuffle Exception Handling

Question

In some scenarios, the following exception occurs in the Spark shuffle phase:

Solution

For JDBC:

Log in to FusionInsight Manager, change the value of the JDBCServer parameter spark.authenticate.enableSaslEncryption to false, and restart the corresponding instance.

For client jobs:

When the client submits the application, change the value of spark.authenticate.enableSaslEncryption in the spark-defaults.conf file to false.