Help Center/ MapReduce Service/ User Guide (Kuala Lumpur Region)/ Troubleshooting/ Using Spark/ Submission Status of the Spark Job API Is Error
Updated on 2022-12-14 GMT+08:00
View PDF
Share

Submission Status of the Spark Job API Is Error

Issue

After a Spark job is submitted using an API, the job status is displayed as error.

Issue Type

Job management

Symptom

After the log level in /opt/client/Spark/spark/conf/log4j.properties is changed and a job is submitted using API V1.1, the job status is displayed as error.

Cause Analysis

The executor monitors the job log output and determines the job execution result. After the execution result is changed to error, the output result cannot be detected. Therefore, the executor determines that the job status is abnormal after the job expires.

Procedure

Change the log level in the /opt/client/Spark/spark/conf/log4j.properties file to info.

Summary and Suggestions

You are advised to use the V2 API to submit jobs.

Parent topic: Using Spark