Help Center/ MapReduce Service/ User Guide/ MRS Cluster O&M/ MRS Cluster Alarm Handling Reference/ ALM-45001 Faulty HetuEngine Compute Instances
Updated on 2024-09-23 GMT+08:00
View PDF
Share

ALM-45001 Faulty HetuEngine Compute Instances

This alarm applies only to MRS 3.2.0 or later.

Alarm Description

The system checks the HetuEngine compute instance status every 60 seconds. This alarm is generated when a HetuEngine compute instance is faulty.

This alarm is cleared when all faulty HetuEngine compute instances are restored.

Alarm Attributes

Alarm ID

Alarm Severity

Auto Cleared

45001

Major

Yes

Alarm Parameters

Parameter

Description

Source

Specifies the cluster for which the alarm was generated.

ServiceName

Specifies the service for which the alarm was generated.

RoleName

Specifies the role for which the alarm was generated.

HostName

Specifies the host for which the alarm was generated.

Impact on the System

SQL tasks submitted to the faulty compute instance of HetuEngine fail to be executed.

Possible Causes

  • The HDFS service is abnormal.
  • The Yarn service is abnormal.
  • Yarn queue resources are insufficient.
  • The process of compute instances is faulty.

Handling Procedure

Check the HDFS service status.

  1. In the alarm list, check whether the "ALM-14000 HDFS Service Unavailable" alarm is generated.

    • If yes, go to 2.
    • If no, go to 4.

  2. Clear "ALM-14000 HDFS Service Unavailable" according to the alarm help.
  3. In the alarm list, check whether the "ALM-45001 Faulty HetuEngine Compute Instances" alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 4.

Check the YARN service status.

  1. In the alarm list, check whether the "ALM-18000 YARN Service Unavailable" alarm is generated.

    • If yes, go to 5.
    • If no, go to 7.

  2. Clear "ALM-18000 YARN Service Unavailable" according to the alarm help.
  3. In the alarm list, check whether the "ALM-45001 Faulty HetuEngine Compute Instances" alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 7.

Check the YARN queue resource status.

  1. In the alarm list, check whether the "ALM-18022 Insufficient YARN Queue Resources" alarm is generated.

    • If yes, go to 8.
    • If no, go to 10.

  2. Clear "ALM-18022 Insufficient YARN Queue Resources" according to the alarm help.
  3. In the alarm list, check whether the "ALM-45001 Faulty HetuEngine Compute Instances" alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 10.

Check the HetuEngine compute instance status.

  1. Log in to FusionInsight Manager as an administrator who can access the HetuEngine web UI and choose Cluster > Services > HetuEngine.
  2. In the Basic Information area on the Dashboard tab page, click the link next to HSConsole WebUI to access the HSConsole page.
  3. On the compute instance page, check whether any compute instances are in the FAULT state.

    • If yes, go to 13.
    • If no, go to 14.

  1. In the Operation column of the target compute instance, click Start and wait until the instance is started.
  2. In the alarm list, check whether the "ALM-45001 Faulty HetuEngine Compute Instances" alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 15.

Collect fault information.

  1. On FusionInsight Manager, choose O&M > Log > Download.
  2. Expand the Service drop-down list. In the Services dialog box that is displayed, select HetuEngine under the target cluster name, and click OK.
  3. Expand the Hosts drop-down list. In the Select Host dialog box that is displayed, select the hosts to which the role belongs, and click OK.
  4. Click in the upper right corner, and set Start Date and End Date for log collection to 30 minutes ahead of and after the alarm generation time respectively. Then, click Download.
  5. Contact O&M personnel and provide the collected logs.

Alarm Clearance

This alarm is automatically cleared after the fault is rectified.

Related Information

None