There Is a Disconnection Every 45 Days Due to the istio-citadel Certificate System

Scenario

The number of connections of multiple DB instances decreased sharply at the same time every 45 days. The following figure shows the number of total connections on the Cloud Eye console.

A large number of errors were reported on the client, as shown in the following figure.

Troubleshooting

1. Check whether a scheduled task with an interval of 45 days exists on the service side.
2. If the client uses a certificate encryption system, such as istio, analyze certificate-related logs and check whether information similar to the following is displayed: If yes, the problem is caused by expired certificates.

   

The life of an istio-citadel certificate is 45 days on the client. When the certificate has expired, the client initiates a database disconnection request.

Solution

• Set a proper expiration time for the istio-citadel certificate on the client and take preventive measures when the certificate expires.
• Check whether any other certificates have expired on the client.