Help Center/ MapReduce Service/ User Guide/ MRS Cluster O&M/ MRS Cluster Alarm Handling Reference/ ALM-38000 Kafka Service Unavailable
Updated on 2024-09-23 GMT+08:00
View PDF
Share

ALM-38000 Kafka Service Unavailable

Description

The system checks the Kafka service status every 30 seconds. This alarm is generated when the Kafka service is unavailable.

This alarm is cleared when the Kafka service recovers.

Attribute

Alarm ID

Alarm Severity

Automatically Cleared

38000

Critical

Yes

Parameters

Name

Meaning

Source

Specifies the cluster for which the alarm is generated.

ServiceName

Specifies the service for which the alarm is generated.

RoleName

Specifies the role for which the alarm is generated.

HostName

Specifies the host for which the alarm is generated.

Impact on the System

The cluster cannot provide the Kafka service, and users cannot perform new Kafka tasks.

Possible Causes

  • The KrbServer service is abnormal.(Skip this step if the normal mode is used.)
  • The ZooKeeper service is abnormal or does not respond.
  • The Broker instance in the Kafka cluster are abnormal.

Procedure

Check the status of the KrbServer service. (Skip this step if the normal mode is used.)

  1. On the FusionInsight Manager portal, choose Cluster > Name of the desired cluster > Services > KrbServer.
  2. Check whether the running status of the KrbServer service is Normal.

    • If yes, go to 5.
    • If no, go to 3.

  3. Rectify the fault by following the steps provided in ALM-25500 KrbServer Service Unavailable.
  4. Perform 2 again.

Check the status of the ZooKeeper cluster.

  1. Check whether the running status of the ZooKeeper service is Normal.

    • If yes, go to 8.
    • If no, go to 6.

  2. If ZooKeeper service is stopped, start it, else rectify the fault by following the steps provided in ALM-13000 ZooKeeper Service Unavailable.
  3. Perform 5 again.

Check the Broker status.

  1. Choose Cluster > Name of the desired cluster > Services > Kafka > Instance to go to the Kafka instances page.
  2. Check whether all instances in Roles are running properly.

    • If yes, go to 11.
    • If no, go to 10.

  3. Select all Broker instances, choose More > Restart Instance, and check whether the instances restart successfully.

    During the restart of the Broker instance, if the current Topic is a single copy and is on the current Broker node, the Kafka service will be interrupted. Otherwise, the Kafka service will not be affected.

    • If yes, go to 11.
    • If no, go to 13.

  4. Choose Cluster > Name of the desired cluster > Services > Kafka to check whether the running status is Normal.

    • If yes, go to 12.
    • If no, go to 13.

  5. Wait for 30 seconds and check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 13.

Collecting Fault Information

  1. On the FusionInsight Manager portal, choose O&M > Log > Download.
  2. Select Kafka in the required cluster from the Service drop-down list.
  3. Click in the upper right corner, and set Start Date and End Date for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
  4. Contact the O&M personnel and send the collected logs.

Alarm Clearing

After the fault is rectified, the system automatically clears this alarm.

Related Information

None