An Error Is Reported When Oozie Schedules HiveSQL Jobs

Symptom

In an MRS 3.x cluster, Oozie fails to schedule Hive jobs. The HiveSQL logs show that the scheduling task is executed successfully but the Yarn job fails. The error information is as follows:

java.io.ioException:output.properties data exceeds its limit [2048]

Cause Analysis

A large number of jobs are submitted in the HiveSQL script at a time, and the size of the job data exceeds the maximum value (2 KB, 2048 bytes) allowed by Oozie Launcher. Therefore, you need to increase the default value. Adjusting this parameter value does not affect the cluster performance.

Procedure

1. Log in to Manager, choose Cluster > Services > Oozie > Configurations > All Configurations > Customization, and add parameter oozie.action.max.output.data and value 204800 to the oozie-site.xml file.

   

2. Save the settings and restart the Oozie service.
3. Execute the Oozie scheduling job again.