Help Center/ MapReduce Service/ Troubleshooting/ Cluster Management/ An Error Is Reported When the Insert Command Is Executed on the Hive Beeline CLI
Updated on 2023-11-30 GMT+08:00
View PDF
Share

An Error Is Reported When the Insert Command Is Executed on the Hive Beeline CLI

Symptom

When the insert into statement is executed in Beeline of MRS Hive, the following error is reported:

Mapping run in Tez on Hive transactional table fails when data volume is high with error: "org.apache.hadoop.hive.ql.lockmgr.LockException Reason: Transaction... already aborted, Hive SQL state [42000]."

Cause Analysis

This problem is caused by improper cluster configuration and Tez resource setting.

Procedure

This problem can be solved by setting configuration parameters on Beeline.

  1. Set the following properties to optimize performance (you are advised to change them at the cluster level):

    • Set hive.auto.convert.sortmerge.join to true.
    • Set hive.optimize.bucketmapjoin to true.
    • Set hive.optimize.bucketmapjoin.sortedmerge to true.

  2. Modify the following content to adjust the resources of Tez:

    • Set hive.tez.container.size to the size of the Yarn container.
    • Set hive.tez.container.size to the Yarn container size yarn.scheduler.minimum-allocation-mb or a smaller value (for example, a half or quarter of the Yarn container size). Ensure that the value does not exceed the yarn.scheduler.maximum-allocation-mb value.

Parent topic: Cluster Management