Help Center/ MapReduce Service/ Troubleshooting/ Using Spark/ Authentication Fails When Spark Connects to Kafka
Updated on 2023-11-30 GMT+08:00
View PDF
Share

Authentication Fails When Spark Connects to Kafka

Symptom

In an MRS 2.1.0 cluster, when Spark consumes Kafka data, an authentication failure occurs one day later.

Cause Analysis

The cluster version does not match the version of the running program package.

Procedure

  1. Complete the configuration by referring to Running the "Connecting Spark Streaming to Kafka0-10" Sample Application.
  2. Obtain the spark-streaming-kafka-0-10_2.11-2.3.2-mrs-2.1.jar package if you need to use the sample programs provided by Maven, because the current cluster version is MRS 2.1.0.
Parent topic: Using Spark