What Can I Do When Receiving java.lang.AbstractMethodError in the Spark Job?
The Spark 2.3 has changed the behavior of the internal interface Logging. If the user code directly inherits the Logging and the earlier version Spark is used during compilation, the java.lang.AbstractMethodError is reported when the application runs in the Spark 2.3 environment.
Solutions are as follows:
- You can recompile the application based on Spark 2.3.
- You can use the sl4j+log4j to implement the log function instead of inheriting the internal interface Logging of the Spark. Details are described as follows:
<dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <version>1.7.16</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-log4j12</artifactId> <version>1.7.16</version> </dependency> <dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId> <version>1.2.17</version> </dependency> private val logger = LoggerFactory.getLogger(this.getClass) logger.info("print log with sl4j+log4j")
Job O&M Errors FAQs
- What Can I Do When Receiving java.lang.AbstractMethodError in the Spark Job?
- Why Are Errors "ResponseCode: 403" and "ResponseStatus: Forbidden" Reported When a Spark Job Accesses OBS Data?
- Why Is Error "verifyBucketExists on XXXX: status [403]" Reported When I Use a Spark Job to Access an OBS Bucket That I Have Access Permission?
- Why Is a Job Running Timeout Reported When a Spark Job Runs a Large Amount of Data?
- Why Does the Job Fail to Be Executed and the Log Shows that the File Directory Is Abnormal When I Use a Spark Job to Access Files in SFTP?
- Why Does the Job Fail to Be Executed Due to Insufficient Database and Table Permissions?
- Why Can't I Find the global_temp Database in the Spark 3.x Job Log?
- Why Does the DataSource Syntax Fail to Create an OBS Table in Avro Format When Spark 2.3.x Is Used?
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.
more