Using an Alarm Template to Set Metric Alarm Rules in Batches

Scenario

During routine O&M, O&M personnel need to set threshold-crossing alarms for key metrics (such as the CPU usage, memory usage, and disk usage) of a large number of cloud resources. In this way, they can receive notifications and handle faults in a timely manner when a resource is abnormal. If there are a large number of cloud resources, creating alarm rules one by one is time-consuming and labor-consuming. In addition, configurations may be inconsistent and the alarm rules for some key metrics may be missing.

AOM 2.0 provides the alarm template function. A group of alarm rules are saved as a template. You can use the template to create threshold-crossing alarm rules, event alarm rules, and PromQL alarm rules for multiple metrics of the same cloud service in batches.

This practice describes how to set threshold-crossing alarm rules in batches based on alarm templates and set alarm notification policies to implement alarm closed-loop management.

Step 1: Create an Alarm Notification Rule

Step 2: Create a Custom Template Group

Step 3: Add Alarm Rule Templates to the Template Group

Step 4: Associate the Alarm Templates with Prometheus Instances

Step 5: Check Whether the Alarm Rules Take Effect

Prerequisites

• An ECS has been obtained and metrics have been ingested. For details, see Connecting Running Environments to AOM.
• You have created a topic and added a subscription in SMN. In this practice, the topic name is aom-alarm and Protocol is set to Email or SMS. For details, see Creating a Topic and Adding a Subscription.

Constraints

• A maximum of 150 alarm templates can be created.
• A maximum of 3,000 alarm rules (including metric alarm rules and event alarm rules) can be created.
• A maximum of 1,000 alarm notification rules can be created.

Step 1: Create an Alarm Notification Rule

1. Log in to the AOM 2.0 console.
2. In the navigation pane, choose Alarm Center > Alarm Notification.
3. On the Notification Rules tab, click Create and set parameters.

   Parameter	Example Value	Description
   Notification Rule Name	ecs-alarm-notify	Name of the notification rule. Enter up to 100 characters and do not start or end with an underscore (_), hyphen (-), or period (.). Only digits, letters, underscores, hyphens, and periods are allowed.
   Enterprise Project	default	Enterprise project.
   Rule Type	Prometheus monitoring	Prometheus monitoring: If a metric or an event meets the alarm condition, the system will send an alarm notification based on the associated SMN topic and message template.
   Rule Settings	Simple	In this example, select Simple to directly configure the notification scope.
   Recipient	Subscription Topic: aom-alarm	Select the created SMN topic aom-alarm.
   Days	Mon to Sun	On which days alarm notifications will be sent.
   Valid From	00:00-23:59	Cloud Eye sends notifications only within the notification window.
   Message Template	aom.built-in.template.en	Preset message template provided by AOM.

4. Click OK to complete the creation of the alarm notification rule.

Step 2: Create a Custom Template Group

1. In the navigation pane, choose Alarm Center > Alarm Templates.
2. Click the Custom tab and click Create Custom Template.
3. In the Select Alarm Source dialog box, select Prometheus monitoring and click Create Custom Template.
4. Set the parameters of the custom template group and click Confirm. After the template group is created, you can add alarm rule templates to it.

   Parameter	Example Value	Description
   Group Name	ecs-threshold-template-group	Name of an alarm template group. Enter a maximum of 100 characters. Only letters, digits, underscores (_), and hyphens (-) are allowed. Do not start or end with a special character.
   Enterprise Project	default	Enterprise project.

Step 3: Add Alarm Rule Templates to the Template Group

After the template group is created, add alarm rule templates to it.

1. On the Custom page, select the template group created in Step 2: Create a Custom Template Group, click Create Template, and set alarm template parameters.

   Parameter	Example Value	Description
   Template Name	ecs-cpu-usage-high	Alarm rule template name. Enter a maximum of 100 characters. Only letters, digits, underscores (_), and hyphens (-) are allowed. Do not start or end with an underscore or hyphen.
   Rule Type	Metric alarm rule	Type of the alarm rule.
   Configuration Mode	Interaction	Configuration mode of a metric alarm rule. Options: Interaction (through the GUI) and PromQL (using PromQL statements).
   Alarm Rule Details	Multiple Metrics	Calculation is performed based on the preset alarm conditions one by one. If one of the conditions is met, an alarm will be triggered.
   Metric	aom_node_cpu_usage (CPU usage)	Metric to be monitored.
   Statistical Period	1 minute	Interval at which metric data is aggregated.
   Rule	Avg > 80	Condition for triggering a metric alarm.
   Consecutive Periods	3	When the metric value meets the alarm condition for a specified number of consecutive periods, a metric alarm will be generated. For Consecutive Periods, the value ranges from 1 to 30.
   Alarm Severity	Minor	Severity of a metric alarm.
   Check Interval	Custom interval: 1 minute	Interval at which metric query and analysis results are checked.
   Alarm Clearance	1	The alarm will be cleared when the alarm condition is not met for a specified number of consecutive periods.
   Action Taken for Insufficient Data	Disable	Action to be taken when no metric data is generated or metric data is insufficient for a specified number of consecutive periods. You can set this option based on your requirements.

2. After the setting is complete, click OK.
3. Add threshold-crossing alarm rules for other key metrics of the ECS by referring to 1. The following table lists examples.

   Rule Name	Metric	Rule	Consecutive Periods	Alarm Severity
   ecs-cpu-usage-high	aom_node_cpu_usage (CPU usage)	Avg > 80	3	Minor
   ecs-cpu-usage-critical	aom_node_cpu_usage (CPU usage)	Avg > 90	3	Major
   ecs-memory-usage-high	aom_node_memory_usage (physical memory usage)	Avg > 80	3	Minor
   ecs-memory-usage-critical	aom_node_memory_usage (physical memory usage)	Avg > 90	3	Major
   ecs-disk-usage-high	aom_node_disk_usage (disk usage)	Avg > 80	3	Minor
   ecs-disk-usage-critical	aom_node_disk_usage (disk usage)	Avg > 90	3	Major
   ecs-disk-usage-full	aom_node_disk_usage (disk usage)	Avg > 95	1	Critical

4. After all alarm rule templates are created, the template group configuration is complete.

Step 4: Associate the Alarm Templates with Prometheus Instances

After the alarm templates are created, associate them with specific Prometheus instances for the alarm rules to take effect.

1. On the Custom page, select the alarm rule templates created in Step 3: Add Alarm Rule Templates to the Template Group and click Apply Template.

   Parameter	Example Value	Description
   Prometheus Instance	Prometheus_AOM_Default	The drop-down list displays the default Prometheus instance, common Prometheus instances, and Prometheus instances for cloud services under the current account. Select a desired instance.
   Enterprise Project	default	Select an enterprise project. Your settings here are applicable only to new rules.
   Update Alarm Rules	Enable	Whether to update the alarm rules associated with the template. Enable: The existing alarm rules associated with the alarm template will be updated. This function is enabled by default. Disable: The existing alarm rules associated with the alarm template will not be updated.
   Notify When	Select Alarm triggered and Alarm cleared.	Set the scenario for sending alarm notifications. Alarm triggered: An alarm notification is sent when the trigger condition is met. Alarm cleared: A notification is sent when the alarm clearance condition is met.
   Alarm Mode	Direct alarm reporting	Set the alarm mode. Direct alarm reporting: An alarm is directly sent when the alarm condition is met. You need to set Frequency and enable Notification Rule. Alarm noise reduction: Alarms are sent only after being processed based on noise reduction rules, preventing alarm storms.
   Frequency	Once	Select the frequency of sending alarm notifications.
   Notification Rule	Enable the option and select the ecs-alarm-notify created in Step 1: Create an Alarm Notification Rule.	Enable Notification Rule and select a notification rule. (If Notification Rule is disabled, the template cannot be used.) Then the system sends alarm notifications based on the associated SMN topic and message template.

2. Click Confirm.

   After the configuration is complete, all alarm rules in the alarm template are applied to the associated Prometheus instance in batches. You do not need to create them one by one. You can view the alarm rules created in batches using the alarm template on the Alarm Center > Alarm Rules > Prometheus Monitoring page.

Step 5: Check Whether the Alarm Rules Take Effect

1. In the navigation pane, choose Alarm Center > Alarm Rules.
2. On the Prometheus Monitoring tab page, view the alarm rules created in batches using the alarm template. Ensure that the rule status is normal.
3. If the metric data meets the specified threshold conditions, the system will automatically generate an alarm. In the navigation pane, choose Alarm Center > Alarm List to view the generated alarm.
4. If the metric data meets the configured notification policy, the system will send alarm notifications to specified personnel by email or SMS based on the associated SMN topic and message template.

Helpful Links

• Creating an AOM Alarm Notification Rule
• Managing Prometheus Monitoring Alarm Templates