Updated on 2024-11-29 GMT+08:00

ALM-38000 Kafka Service Unavailable

Alarm Description

The system checks the Kafka service status every 30 seconds. This alarm is generated when the Kafka service is unavailable.

This alarm is cleared when the Kafka service recovers.

Alarm Attributes

Alarm ID

Alarm Severity

Alarm Type

Service Type

Auto Cleared

38000

Critical

Quality of service

Kafka

Yes

Alarm Parameters

Type

Parameter

Description

Location Information

Source

Specifies the cluster for which the alarm is generated.

ServiceName

Specifies the service for which the alarm is generated.

RoleName

Specifies the role for which the alarm is generated.

HostName

Specifies the host for which the alarm is generated.

Impact on the System

The cluster cannot provide the Kafka service, and users cannot perform new Kafka tasks.

Possible Causes

  • The KrbServer service is abnormal.(Skip this step if the normal mode is used.)
  • The ZooKeeper service is abnormal or does not respond.
  • The Broker instance in the Kafka cluster are abnormal.

Handling Procedure

Check the status of the KrbServer service. (Skip this step if the normal mode is used.)

  1. On the FusionInsight Manager portal, choose Cluster > Name of the desired cluster > Services > KrbServer.
  2. Check whether the running status of the KrbServer service is Normal.

    • If yes, go to 5.
    • If no, go to 3.

  3. Rectify the fault by following the steps provided in ALM-25500 KrbServer Service Unavailable.
  4. Perform 2 again.

Check the status of the ZooKeeper cluster.

  1. Check whether the running status of the ZooKeeper service is Normal.

    • If yes, go to 8.
    • If no, go to 6.

  2. If ZooKeeper service is stopped, start it, else rectify the fault by following the steps provided in ALM-13000 ZooKeeper Service Unavailable.
  3. Perform 5 again.

Check the Broker status.

  1. Choose Cluster > Name of the desired cluster > Services > Kafka > Instance to go to the Kafka instances page.
  2. Check whether all instances in Roles are running properly.

    • If yes, go to 11.
    • If no, go to 10.

  3. Select all Broker instances, choose More > Restart Instance, and check whether the instances restart successfully.

    • If yes, go to 11.
    • If no, go to 13.

  4. Choose Cluster > Name of the desired cluster > Services > Kafka to check whether the running status is Normal.

    • If yes, go to 12.
    • If no, go to 13.

  5. Wait for 30 seconds and check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 13.

Collecting Fault Information

  1. On the FusionInsight Manager portal, choose O&M > Log > Download.
  2. Select Kafka in the required cluster from the Service drop-down list.
  3. Click in the upper right corner, and set Start Date and End Date for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
  4. Contact the O&M engineers and send the collected logs.

Alarm Clearance

After the fault is rectified, the system automatically clears this alarm.

Related Information

None.