Updated on 2024-11-13 GMT+08:00

ALM-50229 Doris FE Failed to Connect to OBS

Alarm Description

The system checks whether the connection between the Doris FE nodes and OBS is available every 30 seconds. This alarm is generated when the connection status code is not 0.

This alarm is cleared when the connection status code is 0.

This alarm applies only to MRS 3.3.1 or later.

Alarm Attributes

Alarm ID

Alarm Severity

Auto Cleared

50229

Critical

Yes

Alarm Parameters

Type

Parameter

Description

Location Information

Source

Specifies the cluster or system for which the alarm was generated.

ServiceName

Specifies the service for which the alarm was generated.

RoleName

Specifies the role for which the alarm was generated.

HostName

Specifies the host for which the alarm was generated.

Additional Information

Detail

Specifies the alarm triggering condition.

Impact on the System

Some functions of Doris are unavailable, for example, cold-hot separation and Hive OBS Catalog.

Possible Causes

  • The obtained AK/SK is invalid.
  • This alarm is generated when OBS connection fails.

Handling Procedure

Determine the cause.

  1. Log in to FusionInsight Manager and choose O&M > Alarm > Alarms. In the alarm list, view the role name and obtain the IP address of the instance in Location of the alarm whose ID is 50229, and check the CurrentValue in Additional Information.

    • If CurrentValue is 2, the obtained AK/SK is invalid. Go to 2.
    • If CurrentValue is 3, OBS fails to be connected. Go to 7.

The obtained AK/SK is invalid.

  1. Log in to the MRS Service console, move the cursor on the username in the upper right corner, and choose My Credentials.
  2. Click Access Keys and check whether Status of the target key is Enabled.

    • If yes, go to 4.
    • If no, click Enable in the Operation column of the row containing the key.

  3. Click Delete in the row where the key is to delete the key. Click Create Access Key and click OK. Download the new access key and obtain the AK and SK.
  4. Set the obs.access_key and obs.secret_key parameters to the obtained AK/SK.
  5. Wait for about 1 minute, log in to FusionInsight Manager, choose O&M > Alarm > Alarms, and check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 7.

Failed to connect to OBS.

  1. Check whether the network connection between the cluster and OBS is normal. For details, see Performing the Initial Configuration.

    • If yes, go to 8.
    • If no, go to 12.

  2. Log in to the MRS management console. In the service list, choose Identity and Access Management. Click Agencies in the navigation pane. In the agency list, click the agency name configured for the MRS cluster.
  3. Click Permissions and click the name of each policy in the permission list.
  4. In the Content area, search for Action and check whether obs is contained.

    • If yes, go to 12.
    • If no, go to 11.

  5. Create an OBS permission policy by following the instructions provided in the guide for configuring doris cold and hot data separation. Wait for about 15 to 20 minutes and check whether the alarm is cleared.

    • If yes, no further action is required.
    • If no, go to 12.

  1. Contact O&M engineers for fault diagnosis and rectification.

Alarm Clearance

This alarm is automatically cleared after the fault is rectified.

Related Information

None.