Help Center/ MapReduce Service/ User Guide (ME-Abu Dhabi Region)/ Troubleshooting/ Cluster Management/ Error Occurs When MRS Executes the Insert Command Using Beeline
Updated on 2022-02-22 GMT+08:00
View PDF
Share

Error Occurs When MRS Executes the Insert Command Using Beeline

Issue

An error occurs when MRS executes the insert command using Beeline.

Symptom

When the insert into statement is executed in Beeline of Hive, the following error is reported:

Mapping run in Tez on Hive transactional table fails when data volume is high with error: "org.apache.hadoop.hive.ql.lockmgr.LockException Reason: Transaction... already aborted, Hive SQL state [42000]."

Cause Analysis

This problem is caused by improper cluster configuration and Tez resource setting.

Procedure

This problem can be solved by setting configuration parameters on Beeline.

  1. Set the following properties to optimize performance (you are advised to change them at the cluster level):

    • Set hive.auto.convert.sortmerge.join to true.
    • Set hive.optimize.bucketmapjoin to true.
    • Set hive.optimize.bucketmapjoin.sortedmerge to true.

  2. Modify the following content to adjust the resources of Tez:

    • Set hive.tez.container.size to the size of the Yarn container.
    • Set hive.tez.container.size to the Yarn container size yarn.scheduler.minimum-allocation-mb or a smaller value (for example, a half or quarter of the Yarn container size). Ensure that the value does not exceed yarn.scheduler.maximum-allocation-mb.

Parent topic: Cluster Management