Help Center/ MapReduce Service/ User Guide/ MRS Cluster O&M/ MRS Cluster Alarm Handling Reference/ ALM-45654 Flink HA Certificate Is About to Expire
Updated on 2024-09-23 GMT+08:00
View PDF
Share

ALM-45654 Flink HA Certificate Is About to Expire

This section applies to MRS 3.3.0 or later.

Alarm Description

Flink checks whether the HA certificate file is about to expire in the first health check or at 01:00:00 every day. This alarm is generated when the remaining validity period is less than or equal to 30 days. This alarm is automatically cleared when the remaining validity period is greater than 30 days.

Alarm Attributes

Alarm ID

Alarm Severity

Auto Cleared

45654

Major

Yes

Alarm Parameters

Parameter

Description

Source

Specifies the cluster for which the alarm is generated.

ServiceName

Specifies the service for which the alarm is generated.

RoleName

Specifies the role for which the alarm is generated.

HostName

Specifies the host for which the alarm is generated.

Impact on the System

If the certificate expires, the HA function of the FlinkServer in active/standby mode is affected. Flink jobs cannot be submitted on the FlinkServer. For FlinkServers in dual-active mode, the HA function is not affected.

Possible Causes

The HA certificate is about to expire.

Handling Procedure

View alarm information.

  1. Log in to FusionInsight Manager, choose O&M > Alarm > Alarms > ALM-45654 Flink HA Certificate Is About to Expire, view Location, obtain the name of the host for which the alarm is generated, and click the host name to view its IP address.

Check whether the HA certificate file in the system is valid. If it is not, generate a new one.

  1. Log in to the host for which the alarm is generated as user omm.
  2. Run the cd ${BIGDATA_HOME}/FusionInsight_Flink_*/install/FusionInsight-Flink-*/ha/local/cert command to go to the directory where the HA certificate is stored.
  3. Run the openssl x509 -noout -text -in server.crt command to query the effective time and due time of the HA certificate.
  4. Perform 6 to 7 during off-peak hours to update the certificate file as needed.
  5. Run the cd ${BIGDATA_HOME}/FusionInsight_Flink_*/install/FusionInsight-Flink-*/flink/sbin command to go to the Flink script directory.
  6. Run the sh proceed_ha_ssl_cert.sh command to generate a new HA certificate. Then, check whether the alarm is cleared 1 minute later.

    • If yes, go to 9.
    • If no, go to 8.

  7. On the node where the standby FlinkServer instance is located, repeat 6 to 7. Then, check whether the alarm is cleared 1 minute later.

    • If yes, go to 9.
    • If no, go to 10.

  8. Check whether this alarm is generated again during periodic system check.

    • If yes, go to 10.
    • If no, no further action is required.

Collect fault information.

  1. On FusionInsight Manager, choose O&M. In the navigation pane on the left, choose Log > Download.
  2. Expand the Service drop-down list, and select Flink for the target cluster.
  3. Click in the upper right corner, and set Start Date and End Date for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click Download.
  4. Contact O&M personnel and provide the collected logs.

Alarm Clearance

This alarm is automatically cleared after the fault is rectified.

Related Information

None.