An Error Occurs When SparkSQL Reads the ORC Table

Symptom

An error occurs when SparkSQL is used to read an ORC table created by Hive.

Cause Analysis

This problem is caused by an open-source issue which is described at https://issues.apache.org/jira/browse/HIVE-11102. It may be triggered when the open-source Hive 1.2.1 package is used.

This problem has been resolved in the Hive for Spark module of MRS.

Procedure

Use the Hive for Spark packages provided by the MRS cluster.

hive-beeline-1.2.1.spark_2.2.1-mrs-x.x.x.jar
hive-cli-1.2.1.spark_2.2.1-mrs-x.x.x.jar
hive-common-1.2.1.spark_2.2.1-mrs-x.x.x.jar
hive-exec-1.2.1.spark_2.2.1-mrs-x.x.x.jar
hive-jdbc-1.2.1.spark_2.2.1-mrs-x.x.x.jar
hive-metastore-1.2.1.spark_2.2.1-mrs-x.x.x.jar

Obtain the sample project from the Maven repository by referring to Obtaining Sample Projects from Huawei Mirrors.