Setting Alarm Alerting via SMN for an OpenSearch Cluster

Scenarios

In CSS, the open-source OpenSearch alerting plugin (opensearch-alerting) and OpenSearch notification plugin (opensearch-notifications and opensearch-notifications-core) are installed by default. They trigger alerts when data meets predefined conditions.

• By default, the open-source alert plugin opensearch-alerting is installed for OpenSearch clusters of version 1.3.6. This plugin consists of three components: Alerts, Monitors, and Destinations. CSS integrates the SMN service in the Destinations component, allowing it to send alarm notifications to the SMN service as a destination.
• For OpenSearch 2.19.0 clusters, the open-source OpenSearch alerting plugin (opensearch-alerting) and OpenSearch notification plugin (opensearch-notifications and opensearch-notifications-core) are installed by default. The Destinations component of the OpenSearch alerting plugin is now an independent notification plugin that manages notification channels. CSS integrates the SMN service in its Notifications component, so it can use SMN as a notification channel.

For more information about the OpenSearch alerting plugin, see Alerting - OpenSearch Documentation. For more information about the OpenSearch notifications plugin, see Notifications - OpenSearch Documentation.

Prerequisites

• You have created a topic on the SMN console. For details, see Creating a Topic.
• You have obtained the CSS administrator account. This account has the permission to access CSS, check the agency list, create agencies, and grant permissions to agencies.

Authorizing Users to Use SMN

1. Log in to the CSS management console.

   You must log in using a CSS administrator account.

2. In the navigation pane, choose Service Authorization.
3. On the Service Authorization page, click Create Agency. In the dialog box displayed, confirm that the agency is successfully created.
   • If an agency has been created, "css_smn_agency exist, no need to created." is displayed in the upper right corner.
   • If you do not have the permission to create an agency, an error message will be displayed in the upper right corner indicating "no permission", in which case, check that the administrator account has been assigned the IAM permission.

Setting Alarm Notifications via SMN (OpenSearch 2.19.0)

1. Log in to the CSS management console.

   Log in using an account with CSS permissions.

2. In the navigation pane on the left, choose Clusters > OpenSearch.
3. In the cluster list, find the target cluster, and click Dashboards in the Operation column to log in to OpenSearch Dashboards.
4. On the OpenSearch Dashboards console, expand the menu in the upper-left corner, and choose Notifications.
5. Create an SMN channel to send alert messages.
   1. On the Channels page, click Create channel and configure a notification channel.

      Table 1 Channel parameters

      Parameter	Description
      Name	Custom channel name.
      Description	Custom channel description.
      Type	Retain the default value SMN.
      Topic	Select the SMN topic you have created in Prerequisites for sending alert messages.

      Figure 1 Create channel
      
   2. Click Create.
   3. Return to the Channels page. If the new channel is displayed on the Channels page, it has been created successfully.
      Figure 2 Channels list
      
6. On the OpenSearch Dashboards console, expand the menu in the upper-left corner, and choose Alerting.
7. Create a monitor and configure alarm triggers and monitoring frequency.
   1. Click the Monitors tab on the Alerting page and click Create monitor to configure monitor information.

      Table 2 Monitor parameters

      Parameter	Description
      Monitor details
      Monitor name	User-defined monitor name
      Monitor type	Monitor type. The value can be Per query monitor (common monitoring), Per bucket monitor (aggregation bucket monitoring), and Per cluster metrics monitor (cluster metric monitoring).
      Monitor defining method	Monitor defining method. Extraction query editor is recommended. Visual editor Extraction query editor Anomaly detector The options of Monitor defining method are determined by the Monitor type you selected.
      Detector	If Monitor defining method is set to Anomaly detector, select an exception detection task.
      Frequency	Select the monitoring frequency and set the monitoring interval. The options include: By interval Daily Weekly Monthly Custom cron expression
      Select data
      Index	When Monitor defining method is set to Visual editor or Extraction query editor, you need to specify the index to be monitored.
      Time field	When Monitor defining method is set to Visual editor, you need to specify the time field to define counting parameters such as count.
      Query
      Metrics	When Monitor defining method is set to Visual editor, you need to set the metrics range for extracting statistics.
      Time range for the last	When Monitor defining method is set to Visual editor, you need to set the monitoring time range for plugins.
      Data filter	When Monitor defining method is set to Visual editor, you need to set filters for data search.
      Group by	When Monitor defining method is set to Visual editor, you need to specify a field so that each value of the field triggers an alarm.
      Define extraction query	When Monitor defining method is set to Extraction query editor, you need to enter the query statement to define the monitoring.
      Request type	When Monitor type is set to Per cluster metrics monitor, you need to specify the request type to monitor cluster metrics, such as the running status and CPU usage.
      Preview query and performance	Preview the query result and verify query performance under the current configuration.

   2. Click Add trigger to add triggers and specify the alarm triggering conditions and actions to be triggered when an alarm is reported.
   3. On the Triggers page, set the alarm triggering sensitivity and message release on the destination end.

      Table 3 Trigger parameters

      Parameter	Description
      Trigger name	User-defined trigger name.
      Severity level	Sensitivity of a trigger, that is, the number of alarms that need to be triggered before an alarm message is sent. 1 indicates the highest sensitivity.
      Trigger condition	Trigger condition. An alarm is triggered when the trigger condition is hit.
      Action name	Trigger action name.
      Channels	Select the SMN destination created in 5.
      Message subject	A description of the message.
      Message	Alarm message body. By default, the subject and body are defined when the destination is an email address. For details, see Message Publishing.
      Perform action	When Monitor type is set to Per bucket monitor, you need to set whether to send alarms in combination. The value can be: Per execution: A combination alarm is sent when multiple alarm triggering conditions are hit. Per alert: Alarms are sent separately when multiple alarm triggering conditions are hit.
      Actionable alerts	When Monitor type is set to Per bucket monitor, set this parameter to Per alert. You need to set the alarms that can be executed after alarm triggering conditions are hit. De-duplicated: Alarms that have been triggered. OpenSearch retains the existing alarms to prevent the plugin from creating duplicate alarms. New: Newly created alarms. Completed: Alarms that are no longer ongoing.
      Throttling	Message sending frequency. It limits the number of notification messages can be received in a specified period. For example, if this parameter is set to 10 minutes, SMN sends only one alarm notification in the next 10 minutes even if the trigger condition is hit for multiple times. After 10 minutes, SMN sends another alarm notification if the alarm condition is met.

      Figure 3 Setting the destination of a trigger action
      
   4. Click Send test message. If a subscriber receives an email, as shown in Figure 5, the trigger is configured successfully.
      Figure 4 Sending a test message
      
      Figure 5 Email notification
      
   5. Click Create to return to the monitor details page. The detector is successfully created.

Setting Alarm Notifications via SMN (OpenSearch 1.3.6)

1. Log in to the CSS management console.

   Log in using an account with CSS permissions.

2. In the navigation pane on the left, choose Clusters > OpenSearch.
3. In the cluster list, find the target cluster, and click Dashboards in the Operation column to log in to OpenSearch Dashboards.
4. On the OpenSearch Dashboards page, choose OpenSearch Plugins > Alerting in the navigation tree on the left.
5. Create an SMN destination to send alert messages.
   1. On the Alerting page, click the Destinations tab and click Add destination to configure destination information.

      Table 4 Destinations parameters

      Parameter	Description
      Name	User-defined destination name
      Type	Retain the default value SMN.
      Topic	Select the SMN topic you have created in Prerequisites for sending alert messages.

      Figure 6 Add destination
      
   2. Click Create to return to the destination list. The created SMN destination is displayed in the list.
      Figure 7 Destination list
      
6. Create a monitor and configure alarm triggers and monitoring frequency.
   1. Click the Monitors tab on the Alerting page and click Create monitor to configure monitor information.

      Table 5 Monitor parameters

      Parameter	Description
      Monitor details
      Monitor name	User-defined monitor name
      Monitor type	Monitor type. The value can be Per query monitor (common monitoring), Per bucket monitor (aggregation bucket monitoring), and Per cluster metrics monitor (cluster metric monitoring).
      Monitor defining method	Monitor defining method. Extraction query editor is recommended. Visual editor Extraction query editor Anomaly detector The options of Monitor defining method are determined by the Monitor type you selected.
      Detector	If Monitor defining method is set to Anomaly detector, select an exception detection task.
      Frequency	Select the monitoring frequency and set the monitoring interval. The options include: By interval Daily Weekly Monthly Custom cron expression
      Data source
      Index	When Monitor defining method is set to Visual editor or Extraction query editor, you need to specify the index to be monitored.
      Time field	When Monitor defining method is set to Visual editor, you need to specify the time field to define counting parameters such as count.
      Query
      Metrics	When Monitor defining method is set to Visual editor, you need to set the metrics range for extracting statistics.
      Time range for the last	When Monitor defining method is set to Visual editor, you need to set the monitoring time range for plugins.
      Data filter	When Monitor defining method is set to Visual editor, you need to set filters for data search.
      Group by	When Monitor defining method is set to Visual editor, you need to specify a field so that each value of the field triggers an alarm.
      Define extraction query	When Monitor defining method is set to Extraction query editor, you need to enter the query statement to define the monitoring.
      Request type	When Monitor type is set to Per cluster metrics monitor, you need to specify the request type to monitor cluster metrics, such as the running status and CPU usage.

   2. Click Add trigger to add triggers and specify the alarm triggering conditions and actions to be triggered when an alarm is reported.
   3. On the Triggers page, set the alarm triggering sensitivity and message release on the destination end.

      Table 6 Trigger parameters

      Parameter	Description
      Trigger name	User-defined trigger name
      Severity level	Sensitivity of a trigger, that is, the number of alarms that need to be triggered before an alarm message is sent. 1 indicates the highest sensitivity.
      Trigger condition	Trigger condition. An alarm is triggered when the trigger condition is hit.
      Action name	Trigger action name
      Destination	Select the SMN destination created in 5.
      Message	Alarm message body By default, the subject and body are defined when the destination is an email address. For details, see Message Publishing.
      Perform action	When Monitor type is set to Per bucket monitor, you need to set whether to send alarms in combination. The value can be: Per execution: A combination alarm is sent when multiple alarm triggering conditions are hit. Per alert: Alarms are sent separately when multiple alarm triggering conditions are hit.
      Actionable alerts	When Monitor type is set to Per bucket monitor, set this parameter to Per alert. You need to set the alarms that can be executed after alarm triggering conditions are hit. De-duplicated: Alarms that have been triggered. OpenSearch retains the existing alarms to prevent the plugin from creating duplicate alarms. New: Newly created alarms. Completed: Alarms that are no longer ongoing.
      Throttling	Message sending frequency. It limits the number of notification messages can be received in a specified period. For example, if this parameter is set to 10 minutes, SMN sends only one alarm notification in the next 10 minutes even if the trigger condition is hit for multiple times. After 10 minutes, SMN sends another alarm notification if the alarm condition is met.

      Figure 8 Setting the destination of a trigger action
      
   4. Click Send test message. If a subscriber receives an email, as shown in Figure 10, the trigger is configured successfully.
      Figure 9 Sending a test message
      
      Figure 10 Email notification
      
   5. Click Create to return to the monitor details page. The detector is successfully created.