An Error Occurs When SparkSQL Reads the ORC Table

Issue

An error occurs when SparkSQL is used to read an ORC table created by Hive.

Symptom

An error occurs when SparkSQL is used to read an ORC table created by Hive.

Cause Analysis

This problem is caused by an open-source bug which is described at https://issues.apache.org/jira/browse/HIVE-11102. It may be triggered when the open-source Hive 1.2.1 package is used.

This bug has been fixed in the Hive for Spark code of MRS.

Procedure

Use the Hive for Spark packages provided by the cluster.

hive-beeline-1.2.1.spark_2.2.1-mrs-x.x.x.jar
hive-cli-1.2.1.spark_2.2.1-mrs-x.x.x.jar
hive-common-1.2.1.spark_2.2.1-mrs-x.x.x.jar
hive-exec-1.2.1.spark_2.2.1-mrs-x.x.x.jar
hive-jdbc-1.2.1.spark_2.2.1-mrs-x.x.x.jar
hive-metastore-1.2.1.spark_2.2.1-mrs-x.x.x.jar

Obtain the sample project from the Maven repository by referring to Obtaining Sample Projects from Huawei Mirrors.