Help Center> MapReduce Service> User Guide (Kuala Lumpur Region)> FusionInsight Manager Operation Guide (Applicable to 3.x)> Alarm Reference (Applicable to MRS 3.x)> ALM-19018 HBase Compaction Queue Exceeds the Threshold
Updated on 2022-08-12 GMT+08:00
View PDF

ALM-19018 HBase Compaction Queue Exceeds the Threshold

Description

The system checks the compaction queue length of the HBase service every 300 seconds. This alarm is generated when the system detects that the compaction queue length of the HBase service exceeds the alarm threshold (100 by default). This alarm is cleared when the compaction queue length is less than the alarm threshold.

If the multi-instance function is enabled in the cluster and multiple HBase services are installed, determine the HBase service for which the alarm is generated based on the value of ServiceName in Location of the alarm. For example, if Service name is HBase-1 in Location, change the operation object in the handling procedure from HBase to HBase-1.

Attribute

Alarm ID

Alarm Severity

Automatically Cleared

19018

Minor

Yes

Parameters

Name

Meaning

Source

Specifies the cluster for which the alarm is generated.

ServiceName

Specifies the service for which the alarm is generated.

RoleName

Specifies the role for which the alarm is generated.

HostName

Specifies the host for which the alarm is generated.

Impact on the System

The compaction queue length of the HBase service exceeds the threshold. If this alarm is not handled in a timely manner, the cluster performance may deteriorate, affecting data read and write.

Possible Causes

  • The number of HBase RegionServers is too small.
  • There are too many regions on a single RegionServer of HBase.
  • The HBase RegionServer heap size is small.
  • Resources are insufficient.
  • The related parameters are set improperly.

Procedure

Check whether the related configuration is correct.

  1. On the FusionInsight Manager, choose O&M > Alarm > Alarms and check whether alarms "19011" exist.

    • If yes, click View Help on the right of the alarm and rectify the fault by referring to the help document. Then, go to 3.
    • If no, go to 2.

  2. On FusionInsight Manager, choose Cluster > Name of the desired cluster > Services > HBase > Configurations > All Configurations. Search for hbase.hstore.compaction.min, hbase.hstore.compaction.max, hbase.hstore.compactionThreshold, hbase.regionserver.thread.compaction.small, and hbase.regionserver.thread.compaction.throttle, and increase their values.
  3. Check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 4.

Collect fault information.

  1. On FusionInsight Manager, choose O&M > Log > Download.
  2. In the Service area, select HBase of the cluster to be operated.
  3. Click in the upper right corner, and set Start Date and End Date for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
  4. Contact the O&M personnel and provide the collected logs.

Alarm Clearing

After the fault is rectified, the system automatically clears this alarm.

Related Information

None