ALM-14044 XceiverCount Value Exceeds the Threshold

Alarm Description

The system checks the XceiverCount value of each DataNode every 30 seconds. This alarm is generated when a DataNode's XceiverCount value exceeds 95% of the dfs.datanode.max.transfer.threads value. This parameter defines the maximum number of threads allowed for data transmission between DataNodes, with a default value of 8192. This alarm is cleared when the XceiverCount value on the DataNode falls to or below the configured threshold.

This section applies to MRS 3.6.0-LTS and later versions.

Alarm Attributes

Alarm Parameters

Impact on the System

The write speed in HDFS decreases, affecting the overall HDFS performance.

Possible Causes

• The DataNode's XceiverCount value is not properly configured.
• There is an excessive number of write requests in HDFS.

Handling Procedure

Check whether the DataNode's XceiverCount value is properly configured.

1. Log in to MRS Manager and choose Cluster > Services > HDFS. Click Configurations and then All Configurations. Search for the dfs.datanode.max.transfer.threads parameter.
2. Check whether the value is much smaller than the default value.
   • If yes, go to Step 3.
   • If no, go to Step 6.

3. Reset the parameter value to its default setting and click Save.
4. Click Instances, select all DataNode instances, and choose More > Restart Instance to restart all DataNode instances.
5. Wait 5 minutes and check whether the alarm is automatically cleared.
   • If yes, no further action is required.
   • If no, go to Step 6.

Expand the DataNode capacity.

6. Expand the DataNode capacity.
7. Wait 5 minutes and check whether the alarm is automatically cleared.
   • If yes, no further action is required.
   • If no, go to Step 8.

Collect fault information.

8. On MRS Manager, choose O&M. In the navigation pane on the left, choose Log > Download.
9. Expand the Service drop-down list, select the HDFS services for the target cluster, and click OK.
10. Click the edit icon in the upper right corner and select a time span starting 10 minutes before and ending 10 minutes after when the alarm was generated. Then, click Download to collect the logs.
11. Contact O&M engineers and provide the collected logs.

Alarm Clearance

This alarm is automatically cleared after the fault is rectified.

Related Information

None