Viewing Performance Metrics of a Real-Time Service on Cloud Eye

ModelArts Metrics

The cloud service platform provides Cloud Eye to help you better understand the statuses of ModelArts real-time services and model loads. You can use Cloud Eye to automatically monitor your ModelArts real-time services and model loads in real time and manage alarms and notifications so that you can obtain the performance metrics of ModelArts and models.

**Table 1** ModelArts metrics
ID	Name	Description	Value Range	Monitored Object	Monitoring Interval
cpu_usage	CPU Usage	CPU usage of ModelArts Unit: %	≥ 0%	ModelArts model loads	1 minute
mem_usage	Memory Usage	Memory usage of ModelArts Unit: %	≥ 0%	ModelArts model loads	1 minute
gpu_util	GPU Usage	GPU usage of ModelArts Unit: %	≥ 0%	ModelArts model loads	1 minute
gpu_mem_usage	GPU Memory Usage	GPU memory usage of ModelArts Unit: %	≥ 0%	ModelArts model loads	1 minute
npu_util	NPU Usage	NPU usage of ModelArts Unit: %	≥ 0%	ModelArts model loads	1 minute
npu_mem_usage	NPU Memory Usage	NPU memory usage of ModelArts Unit: %	≥ 0%	ModelArts model loads	1 minute
successfully_called_times	Number of Successful Calls	Times that ModelArts services have been successfully called Unit: counts/minute	≥ counts/minute	ModelArts model loads ModelArts real-time services	1 minute
failed_called_times	Number of Failed Calls	Times that ModelArts services failed to be called Unit: counts/minute	≥ counts/minute	ModelArts model loads ModelArts real-time services	1 minute
total_called_times	Total Calls	Times that ModelArts services are called Unit: counts/minute	≥ counts/minute	ModelArts model loads ModelArts real-time services	1 minute
disk_read_rate	Disk Read Rate	Disk read rate of ModelArts Unit: bit/minute	≥ bit/minute	ModelArts model loads	1 minute
disk_write_rate	Disk Write Rate	Disk write rate of ModelArts Unit: bit/minute	≥ bit/minute	ModelArts model loads	1 minute
send_bytes_rate	Uplink rate	Outbound network traffic rate of ModelArts. Unit: bit/minute	≥ bit/minute	ModelArts model loads	1 minute
recv_bytes_rate	Downlink rate	Inbound network traffic rate of ModelArts.	≥ bit/minute	ModelArts model loads	1 minute
req_count_2xx	2xx Responses	Number of times that the API returns a 2xx response	≥ counts/minute	ModelArts real-time services	1 minute
req_count_4xx	4xx Errors	Number of times that the API returns a 4xx error	≥ counts/minute	ModelArts real-time services	1 minute
req_count_5xx	5xx Errors	Number of times that the API returns a 5xx error	≥ counts/minute	ModelArts real-time services	1 minute
avg_latency	Average Latency	Average latency of the API	≥ ms	ModelArts real-time services	1 minute
If a monitored object has multiple dimensions, all dimensions are mandatory when you use APIs to query the metrics. The following provides an example of using the multi-dimensional dim to query a single monitoring metric: dim.0=service_id,530cd6b0-86d7-4818-837f-935f6a27414d&dim.1="model_id,3773b058-5b4f-4366-9035-9bbd9964714a The following provides an example of using the multi-dimensional dim to query monitoring metrics in batches: "dimensions": [ { "name": "service_id", "value": "530cd6b0-86d7-4818-837f-935f6a27414d" } { "name": "model_id", "value": "3773b058-5b4f-4366-9035-9bbd9964714a" } ]

**Table 2** Dimension description
Key	Value
service_id	Real-time service ID
model_id	Model ID

Setting Alarm Rules

Setting alarm rules allows you to customize the monitored objects and notification policies so that you can know the status of ModelArts real-time services and models in a timely manner.

An alarm rule includes the alarm rule name, monitored object, metric, threshold, monitoring interval, and whether to send a notification. This section describes how to set alarm rules for ModelArts services and models.

Only real-time services in the Running status can be interconnected with CES.

Prerequisites:

A ModelArts real-time service has been created.
ModelArts monitoring has been enabled on Cloud Eye. To do so, log in to the Cloud Eye console. On the Cloud Eye page, click Custom Monitoring. Then, enable ModelArts monitoring as prompted.

Set an alarm rule in any of the following ways:

Set an alarm rule for all ModelArts services.
Set an alarm rule for a ModelArts service.
Set an alarm rule for a model version.
Set an alarm rule for a metric of a service or model version.

Method 1: Setting an Alarm Rule for All ModelArts Services

Log in to the management console.
In the Service List, click Cloud Eye under Management & Governance.
In the navigation pane on the left, choose Alarm Management > Alarm Rules and click Create Alarm Rule.
On the Create Alarm Rule page, set Resource Type to ModelArts, Dimension to Service, and Method to Configure manually, and set alarm policies. Then, confirm settings and click Create.

Method 2: Setting an Alarm Rule for a Single Service

Log in to the management console.
In the Service List, click Cloud Eye under Management & Governance.
In the navigation pane, choose Cloud Service Monitoring > ModelArts.
Locate a real-time service for which you want to create an alarm rule and click Create Alarm Rule in the Operation column.
On the Create Alarm Rule page, create an alarm rule for ModelArts real-time services and models as prompted.

Method 3: Setting an Alarm Rule for a Model Version

Log in to the management console.
In the Service List, click Cloud Eye under Management & Governance.
In the navigation pane, choose Cloud Service Monitoring > ModelArts.
Click the down arrow next to the target real-time service name. Then, click Create Alarm Rule in the Operation column of the target version.
On the Create Alarm Rule page, create an alarm rule for model loads as prompted.

Method 4: Setting an Alarm Rule for a Metric of a Service or Model Version

Log in to the management console.
In the Service List, click Cloud Eye under Management & Governance.
In the navigation pane, choose Cloud Service Monitoring > ModelArts.
Click the down arrow next to the target real-time service name. Then, click the target version and view alarm rule details.
On the alarm rule details page, click the plus sign (+) in the upper right corner of a metric and set an alarm rule for the metric.

Viewing Monitoring Metrics

Cloud Eye on the cloud service platform monitors the statuses of ModelArts real-time services and model loads. You can obtain the monitoring metrics of each ModelArts real-time service and model on the management console. It takes a period of time to transmit and display monitored data. The statuses displayed on the Cloud Eye console are obtained 5 to 10 minutes before. You can view the monitoring data of a newly created real-time service 5 to 10 minutes later.

Prerequisites:

The ModelArts real-time service is running properly.

Alarm rules have been configured on the Cloud Eye page. For details, see Setting Alarm Rules.
The real-time service has been properly running for at least 10 minutes.
The monitored data and graphics are available for a new real-time service after the service runs for at least 10 minutes.

Cloud Eye does not display the metrics of a faulty or deleted real-time service. The monitoring metrics can be viewed after the real-time service starts or recovers.

The monitoring data is unavailable without alarm rules configured in Cloud Eye. For details, see Setting Alarm Rules.

Log in to the management console.
In the Service List, click Cloud Eye under Management & Governance.
In the navigation pane, choose Cloud Service Monitoring > ModelArts.
View monitoring graphs.
- Viewing monitoring graphs of a real-time service: Click View Metric in the Operation column.
- Viewing monitoring graphs of the model loads: Click next to the target real-time service, and click View Metric in the Operation column of the target model.
In the monitoring area, you can select a period to view the monitoring data.

You can view the monitoring data in the last 1 hour, 3 hours, or 12 hours. To view the monitoring curve of a longer time range, click to enlarge the graph.