ALM-12050 Network Write Throughput Rate Exceeds the Threshold

Alarm Description

The system checks the network write throughput rate every 30 seconds and compares the actual throughput rate with the threshold (the default threshold is 80%). This alarm is generated when the system detects that the network write throughput rate exceeds the threshold for several times (5 times by default) consecutively.

To change the threshold, choose O&M > Alarm > Thresholds > Name of the desired cluster > Host > Network Writing > Write Throughput Rate.

When the Trigger Count is 1, this alarm is cleared when the network write throughput rate is less than or equal to the threshold. When the Trigger Count is greater than 1, this alarm is cleared when the network write throughput rate is less than or equal to 90% of the threshold.

Alarm Attributes

Alarm Parameters

Impact on the System

• Latency: When the host network write throughput exceeds the threshold, the request response is slowed down and services are delayed.
• Service failure: When the host network write throughput exceeds the threshold, requests cannot be responded or times out, which may cause job running failures.

Possible Causes

• The alarm threshold is set improperly.
• The network port rate cannot meet the current service requirements.

Handling Procedure

Check whether the threshold is set properly.

1. On the FusionInsight Manager, choose O&M > Alarm > Thresholds > Name of the desired cluster > Host > Network Writing > Write Throughput Rate and check whether the alarm threshold is set properly. (By default, 80% is a proper value. However, users can configure the value as required.)
   • If yes, go to 4.
   • If no, go to 2.

2. Based on actual usage condition, choose O&M > Alarm > Thresholds > Name of the desired cluster > Host > Network Writing > Write Throughput Rate and click Modify in the Operation column to modify the alarm threshold.

   For details, see Figure 1.

   Figure 1 Setting alarm thresholds
   

3. Wait for 5 minutes, and check whether the alarm is cleared.
   • If yes, no further action is required.
   • If no, go to 4.

Check whether the network port rate can meet the service requirements.

4. On FusionInsight Manager, click in the row where the alarm is located in the real-time alarm list and obtain the IP address of the host and the network port name for which the alarm is generated.
5. Log in to the host for which the alarm is generated as user root.
6. Run the ethtoolnetwork port name command to check the maximum speed of the current network port.
   

   In the VM environment, you cannot run a command to query the network port rate. It is recommended that you contact the system administrator to confirm whether the network port rate meets the requirements.

7. If the network write throughput rate exceeds the threshold, contact the system administrator to increase the network port rate.
8. Check whether the alarm is cleared.
   • If yes, no further action is required.
   • If no, go to 9.

Collect fault information.

9. On the FusionInsight Manager home page of the active cluster, choose O&M > Log > Download.
10. Select OMS from the Service and click OK.
11. Set Host to the node for which the alarm is generated and the active OMS node.
12. Click in the upper right corner, and set Start Date and End Date for log collection to 30 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
13. Contact the O&M engineers and send the collected log information.

Alarm Clearance

After the fault is rectified, the system automatically clears this alarm.

Related Information

None.