ALM-43007 Non-Heap Memory Usage of the JobHistory Process Exceeds the Threshold (For MRS 2.x or Earlier)

Description

The system checks the JobHistory process status every 30 seconds. The alarm is generated when the non-heap memory usage of the JobHistory process exceeds the threshold (90% of the maximum memory).

Attribute

Parameters

Impact on the System

If the available JobHistory process non-heap memory is insufficient, a memory overflow occurs and the service breaks down.

Possible Causes

The non-heap memory of the JobHistory process is overused or the non-heap memory is inappropriately allocated.

Procedure

1. Check non-heap memory usage.
   1. Go to the cluster details page and choose Alarms.
   2. Select the alarm whose Alarm ID is 43007 and view the IP address and role name of the instance in Location.
   3. Choose Components > Spark > Instance > JobHistory (IP address of the instance for which the alarm is generated) > Customize > Non-Heap Memory Statistics of the JobHistory Process. Click OK to view the non-heap memory usage.
   4. Check whether the non-heap memory usage of JobHistory has reached the threshold (90% of the maximum memory).
      • If yes, go to 1.e.
      • If no, go to 2.
   5. Choose Components > Spark > Service Configuration. Set Type to All and choose JobHistory > Default. Increase the value of -XX:MaxMetaspaceSize in SPARK_DAEMON_JAVA_OPTS as required.
   6. Check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to 2.

2. Collect fault information.
   1. On MRS Manager, choose System > Export Log.
   2. Contact the O&M engineers and send the collected logs.

Reference

None