Help Center > > User Guide> FusionInsight Manager Operation Guide (Applicable to 3.x)> Alarm Reference (Applicable to MRS 3.x)> ALM-14010 NameService Service Is Abnormal

ALM-14010 NameService Service Is Abnormal

Updated at: Oct 21, 2021 GMT+08:00

Description

The system checks the NameService service status every 180 seconds. This alarm is generated when the NameService service is unavailable.

This alarm is cleared when the NameService service recovers.

Attribute

Alarm ID

Alarm Severity

Auto Clear

14010

Major

Yes

Parameters

Name

Meaning

Source

Specifies the cluster for which the alarm is generated.

ServiceName

Specifies the name of the service for which the alarm is generated.

RoleName

Specifies the name of the role for which the alarm is generated.

HostName

Specifies the host for which the alarm is generated.

NameServiceName

Specifies the NameService for which the alarm is generated.

Impact on the System

HDFS fails to provide services for upper-layer components based on the NameService service, such as HBase and MapReduce. As a result, users cannot read or write files.

Possible Causes

  • The KrbServer service is abnormal.
  • The JournalNode is faulty.
  • The DataNode is faulty.
  • The disk capacity is insufficient.
  • The NameNode enters safe mode.

Procedure

Check the KrbServer service status.

  1. On FusionInsight Manager, choose Cluster > Name of the desired cluster > Services.
  2. Check whether the KrbServer service exists.

    • If yes, go to 3.
    • If no, go to 6.

  3. Click KrbServer.
  4. Click Instances. On the KrbServer management page, select the faulty instance, and choose More > Restart Instance. Check whether the instance successfully restarts.

    • If yes, go to 5.
    • If no, go to 24.

  5. Choose O&M > Alarm > Alarms and check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 6.

Check the JournalNode instance status.

  1. On FusionInsight Manager, choose Cluster > Name of the desired cluster > Services.
  2. Choose HDFS > Instances.
  3. Check whether the Running Status of the JournalNode is Normal.

    • If yes, go to 11.
    • If no, go to 9.

  4. Select the faulty JournalNode, and choose More > Restart Instance. Check whether the JournalNode successfully restarts.

    • If yes, go to 10.
    • If no, go to 24.

  5. Choose O&M > Alarm > Alarms and check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 11.

Check the DataNode instance status.

  1. On FusionInsight Manager, choose Cluster > Name of the desired cluster > Services > HDFS.
  2. Click Instances and check whether Running Status of all DataNodes is Normal.

    • If yes, go to 15.
    • If no, go to 13.

  3. Click Instances. On the DataNode management page, select the faulty instance, and choose More > Restart Instance. Check whether the DataNode successfully restarts.

    • If yes, go to 14.
    • If no, go to 15.

  4. Choose O&M > Alarm > Alarms and check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 15.

Check disk status.

  1. On FusionInsight Manager, choose Cluster > Name of the desired cluster > Host.
  2. In the Disk column, check whether the disk space is insufficient.

    • If yes, go to 17.
    • If no, go to 19.

  3. Expand the disk capacity.
  4. Choose O&M > Alarm > Alarms and check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 19.

Check whether NameNode is in the safe mode.

  1. On the FusionInsight Manager portal, choose Cluster > Name of the desired cluster > Services > HDFS. Click NameNode(Active) of the abnormal NameService. The NameNode web UI is displayed.

    By default, the admin user does not have the management rights of other components. If the page cannot be opened or the content is not completely displayed due to insufficient permission when you access the native page of a component, you can manually create a user with the management rights of the corresponding component to log in to the component.

  2. On the NameNode web UI, check whether "Safe mode is ON." is displayed.

    Information behind Safe mode is ON is alarm information and is displayed based actual conditions.

    • If yes, go to 21.
    • If no, go to 24.
    Figure 1 Overview

  3. Log in to the client as user root. Run the cd command to go to the client installation directory and run the source bigdata_env command. If the cluster uses the security mode, perform security authentication. Run the kinit hdfs command and enter the password as prompted. The password can be obtained from the administrator. If the cluster uses the non-security mode, log in as user omm and run the command. Ensure that user omm has the client execution permission.
  4. Run hdfs dfsadmin -safemode leave.
  5. Choose O&M > Alarm > Alarms and check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 24.

Collect fault information.

  1. On the FusionInsight Manager portal, choose O&M > Log > Download.
  2. In the Service area, select the following nodes of the desired cluster.

    • ZooKeeper
    • HDFS

  3. Click in the upper right corner, and set Start Date and End Date for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
  4. Contact O&M personnel and send the collected logs.

Alarm Clearing

After the fault that triggers the alarm is rectified, the alarm is automatically cleared.

Related Information

None

Did you find this page helpful?

Submit successfully!

Thank you for your feedback. Your feedback helps make our documentation better.

Failed to submit the feedback. Please try again later.

Which of the following issues have you encountered?







Please complete at least one feedback item.

Content most length 200 character

Content is empty.

OK Cancel