ALM-33007 ALB TPS of Containers Exceeds the Threshold

Alarm Description

This alarm is generated when the ALB TPS exceeds the threshold (tps.threshold) set by the cluster administrator in ALB configuration file alb.properties.

This alarm is automatically cleared when the ALB TPS is less than or equal to the threshold.

Alarm Attributes

Alarm ID	Alarm Severity	Alarm Type	Service Type	Auto Cleared
33007	Warning	Quality of service	Containers	Yes

Alarm Parameters

Type	Parameter	Description
Location Information	Source	Specifies the cluster or system for which the alarm is generated.
	HostName	Specifies the name of the host accommodating the ALB instance for which the alarm is generated.
	Container_ID	Identifies the container accommodating the ALB instance for which the alarm is generated.
	BLU_ID	Identifies the BLU of the ALB instance for which the alarm is generated.
	BLU_Version	Specifies the BLU version of the ALB instance for which the alarm is generated.
	ServerAddress	Specifies the external socket address of the ALB instance for which the alarm is generated.
	TPS_Threshold	Specifies the TPS alarm threshold configured by the administrator.
Additional Information	Trigger Condition	Specifies the alarm triggering condition.

Impact on the System

ALB request forwarding is slow, and service calls are responded slowly.

Possible Causes

If the number of requests forwarded by each ALB instance is relatively even, ALB instances may be not enough for external requests.
If the number of requests forwarded by each ALB instance differs greatly, the TPS of some ALB instances may exceed the threshold.

Handling Procedure

Check whether ALB instances are not enough for external requests if the number of requests forwarded by each ALB instance is relatively even.

Check whether the number of requests forwarded to each ALB instance is even based on how external messages are connected to the cluster.

For example, when a forwarder is used to send messages to ALB instances, check the number of messages forwarded by the forwarder to each socket address (one socket address corresponds to one ALB instance).
- If yes, go to 2.
- If no, go to 6.

Log in to FusionInsight Manager, choose Cluster > Services > Containers, and click the Business Manager tab.
Choose ALB in the navigation pane and click the ALB name. On the page that is displayed, click BLU Instance and check whether there are only a few ALB instances.
- If yes, go to 4.
- If no, go to 6.
Click Add Instance, select the location of new instance, and click OK.
Check whether the alarm is cleared.
- If yes, no further action is required.
- If no, go to 8.

Check whether the TPS of some ALB instances exceeds the threshold if the number of requests forwarded by each ALB instance differs greatly.

Resolve this problem based on the actual condition because the policy cannot be modified in the cluster. For example, adjust the forwarding policy of the forwarder to decrease requests sent to overloaded ALB instances (especially those that have received alarms).
Check whether the alarm is cleared.
- If yes, no further action is required.
- If no, go to 8.

Collect fault information.

On FusionInsight Manager, choose O&M. In the navigation pane on the left, choose Log > Download.
Expand the Service drop-down list and select Containers.
Expand the Hosts drop-down list. In the Select Host dialog box that is displayed, select the hosts to which the role belongs, and click OK.
Click the edit icon in the upper right corner, and set Start Date and End Date for log collection to 1 hour ahead of and after the alarm generation time, respectively. Then, click Download.
Contact O&M personnel and provide the collected logs.