Updated on 2026-08-18 GMT+08:00

Connection Exhaustion

Scenarios

When the number of connections to the database reaches the upper limit, a connection exhaustion alarm is generated, or new connections fail to be created on the client due to timeout.

Cause Analysis

Table 1 Cause analysis for connection exhaustion

Cause

Description

Impact

Unused or misconfigured connection pool

The connection pool is not used. Database connections are directly created and closed.

In high-concurrency scenarios, the number of database connections increases sharply, resources are exhausted, and system performance deteriorates.

The maximum number of connections in the connection pool is set too high.

A large number of connections are maintained during low load, wasting database resources and possibly affecting other services.

The minimum number of idle connections in the connection pool is too large.

A large number of idle connections are maintained during low load, increasing database resource consumption.

Idle connections are not reclaimed in a timely manner (for example, the timeout interval or reclamation policy is not configured).

Idle connections accumulate, eventually exhausting the database connection pool. New requests cannot obtain connections.

Burst service traffic

Activities such as flash sales and promotions cause a sudden surge in concurrent requests, with the connection creation speed far exceeding the release speed.

The number of database connections is quickly exhausted, causing system response delays or even breakdown.

Slow SQL statements and long-running transactions

The SQL statement execution takes an extended period, and the occupied connections are not released in a timely manner.

The number of available connections in the connection pool decreases, forcing applications to create more connections and intensifying resource contention.

Transactions are not committed in a timely manner, occupying connections for an extended period.

The number of available connections in the connection pool decreases, causing system performance to deteriorate and potentially leading to deadlocks or timeouts.

Connection leakage

Database connections are not properly closed.

The number of available connections in the connection pool continuously decreases, eventually exhausting connection resources and preventing the system from processing new requests.

Preventive Measures

  • Set the following parameters properly:
    • max_user_connection: limits the maximum number of concurrent connections for a single database user.
    • max_connections: limits the total number of concurrent connections allowed for the entire DB instance (including all users).
    • wait_timeout: controls the maximum idle duration of a client connection. When a non-interactive client connection (for example, a connection established through application code) is idle (with no active queries) for a specified number of seconds, the database automatically disconnects it.
    • interactive_timeout: controls the maximum idle duration of a client connection. When an interactive client connection (for example, a connection established through the MySQL command line tool such as mysql -u root -p) is idle (with no active queries) for a specified number of seconds, the database automatically disconnects it.
  • Set alarm rules: Set threshold alarms for key metrics. For example, you can set an alarm to be triggered when the number of connections exceeds 80% of the maximum allowed connections, prompting timely intervention.
  • Periodically terminate idle sessions and long sessions.

Optimization Suggestions

  • Enable and properly configure a connection pool on the application side.
  • Temporarily increase the number of database connections.

    When there is a sudden traffic surge or short-term overload, you can temporarily increase the value of max_connections. For example, if the maximum number of connections is 6,500, you can double the value to 13000. This is only a temporary mitigation measure and may trigger OOM exceptions. Therefore, restore the value to the original one immediately after the service stabilizes. After the value is increased, new connections may still use up the maximum number of connections. Therefore, you also need to throttle abnormal SQL statements.

  • SQL throttling and abnormal SQL governance
    • Throttle abnormal SQL statements (such as slow queries and long-running transactions) to prioritize core services. Use TaurusDB SQL throttling to set a threshold to restrict the concurrency of resource-intensive SQL statements.
    • Use slow query logs to locate and optimize high-frequency slow SQL statements, reducing the connection duration of a single request.
  • If there are too many types of SQL statements and it is difficult to quickly throttle them, you can use the following methods:
    • Performing a primary/standby switchover: If the number of connections to the primary node reaches the upper limit but the number of connections to a read replica is not high, you can perform a primary/standby switchover to quickly restore services.
    • Rebooting the read replica: If the number of connections to a read replica reaches the upper limit, you can reboot the read replica to release the occupied connection resources.
    • Rebooting the DB instance: If the number of connections to all nodes reaches the upper limit and the service cannot be restored by other means, you can reboot the instance. However, exercise caution when performing this operation because the reboot will cause short-time service interruption and clear the hot data in the buffer pool. You are advised to perform this operation during off-peak hours.