ALM-27005 Database Connection Usage Exceeds the Threshold

Alarm Description

The system checks the database connection usage of the DBServer node every 30 seconds and compares the actual database connection usage with the threshold. This alarm is generated when the database connection usage exceeds the threshold for five consecutive times (configurable, five by default).

The trigger count is configurable. When the value is 1 and the data connections are no more than the threshold, the alarm is cleared. When the trigger count is greater than 1 and the data connections are no more than 90% of the threshold, the alarm is cleared.

Alarm Attributes

Alarm Parameters

Impact on the System

Upper-layer services may fail to connect to the DBService database, affecting services.

Possible Causes

• There are too many database connections in use.
• The maximum number of database connections is set improperly.
• The alarm threshold or alarm trigger count is improperly configured.

Handling Procedure

Check whether there are too many database connections in use.

1. On FusionInsight Manager, click DBService in the service list on the left. The DBService monitoring page is displayed.
2. View the number of connections used by the database user, as shown in Figure 1. You can reduce the number of connections based on service requirements.
   Figure 1 Connections used by the database user
   

3. Wait 2 minutes and check whether the alarm is cleared.
   • If yes, no further action is required.
   • If no, go to 4.

Check whether the maximum number of database connections is set properly.

4. Log in to FusionInsight Manager, choose Cluster > Name of the desired cluster > Service > DBService, click Configuration and then All Configurations. Increase the maximum number of database connections allowed based on service requirements. After the modification, click Save. On the page that is displayed, click OK.
   Figure 2 Setting maximum database connections
   

5. After changing the maximum number of database connections, restart DBService (do not restart its upper-layer services).

   Procedure: Log in to FusionInsight Manager, choose Cluster > Name of the desired cluster > Service > DBService. On the displayed page, click More > Restart Service in the upper right corner, enter the password of the current login user, and click OK. Do not select Restart upper-layer services. Click OK.

6. Wait 2 minutes and check whether the alarm is automatically cleared.
   • If yes, no further action is required.
   • If no, go to 7.

Check whether the alarm threshold or alarm trigger count is properly configured.

7. Log in to FusionInsight Manager and change the alarm threshold and alarm smoothing times based on the number of database connections. Click O&M and choose Alarm > Thresholds in the navigation pane on the left. On the displayed page, click the name of the desired cluster > DBService > Database > Database Connection Usage (DBServer). Click the pencil icon next to Trigger Count to change the value.
   

   If the value of Trigger Count exceeds the threshold, the alarm is generated.

   Figure 3 Setting the alarm trigger count
   

   Click O&M, and choose Alarm > Thresholds in the navigation pane on the left. On the displayed page, click the name of the desired cluster > DBService > Database > Database Connection Usage (DBServer) in the list on the left. Click Modify in the Operation column. Modify the rule and click OK. The modification takes effect immediately.

   Figure 4 Configuring the alarm threshold
   

8. Wait 2 minutes and check whether the alarm is automatically cleared.
   • If yes, no further action is required.
   • If no, go to 9.

Collect fault information.

9. On FusionInsight Manager, choose O&M. In the navigation pane on the left, choose Log > Download.
10. Expand the Service drop-down list, and select DBService for the target cluster.
11. Specify Hosts for collecting logs, which is optional. By default, all hosts are selected.
12. Click in the upper right corner, and set Start Date and End Date for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
13. Contact O&M engineers and provide the collected logs.

Alarm Clearance

This alarm is automatically cleared after the fault is rectified.

Related Information

None.