Obtaining Time-based Service Metric Statistics
Function
This API is used to obtain service metric details. This allows you to view chronological trends for metrics such as call count, failure rate, token volume, input token size, output token size, E2E latency, TPM, RPM, QPS, and average generation time. Data is retained for 30 days only.
Constraints
- Region restrictions: Only the CN-Hong Kong region is supported.
- API rate limiting: The total number of requests for this API from all users cannot exceed 80 within 20 seconds.
- User request limit: The number of requests for this API from a single user cannot exceed 1 within 20 seconds.
- Rate-limiting response: When the rate limit is exceeded, the API returns the HTTP status code 429 "Too Many Requests".
- Retry suggestion: If the rate limit is exceeded, wait 20 seconds and try again.
URI
POST /v1/{project_id}/maas/monitoring/{service_id}/show-detail-chart
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| project_id | Yes | String | Definition: Project ID. For details about how to obtain the project ID, see Obtaining a Project ID and Name. Constraints: N/A Range: The value can contain 32 characters. Only lowercase letters and digits are allowed. Default Value: N/A |
| service_id | Yes | String | Definition: Service IDs to be queried. Services are filtered based on the input service ID list. If this parameter is not specified, all service names corresponding to the IDs are returned. You can obtain the service ID from the response body during service creation, or call the API for obtaining the service list. The service_id field indicates the service ID. Constraints: N/A Range: The value can contain 1 to 128 characters. Only letters, digits, underscores (_), and hyphens (-) are allowed. Default Value: N/A |
Request Parameters
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| X-Auth-Token | Yes | String | Definition: User token. It can be obtained by calling the IAM API that is used to obtain a user token. The value of X-Subject-Token in the response header is the user token. For details, see Authentication. Constraints: N/A Range: N/A Default Value: N/A |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| service_type | Yes | Integer | Definition: Service type. Constraints: N/A Range:
Default Value: N/A |
| api_keys | No | Array of strings | Definition: API key tag list, which is used for filtering. MaaS services support API key calls. Go to the API key management page to get the API key tag. The Tag field in the API key list shows the API key tag.
Constraints: N/A Range:
Default Value: N/A |
| version_id | No | String | Definition: Service version ID. If this parameter is not specified, all service versions are queried. To query the service version ID, you can call the API for querying the service version. Constraints: N/A Range: The value can contain 1 to 128 characters. Only letters, digits, underscores (_), and hyphens (-) are allowed. Default Value: N/A |
| ips | No | Array of strings | Definition: IP address list, which indicates the source IP addresses of the clients that have been called. If this parameter is not specified, all IP addresses are queried. To query the IP address, you can call the API for obtaining the IP address list. Constraints: The value must be in the IP address format. Range: N/A Default Value: N/A |
| start_time | Yes | Long | Definition: Timestamp of the start time, in milliseconds. Constraints: The end time must be within 30 days from the start time. Range: The value must be greater than 0 and no greater than the value of end_time. Default Value: N/A |
| end_time | Yes | Long | Definition: Timestamp of the end time, in milliseconds. Constraints: The end time must be within 30 days from the start time. Range: The value must be greater than 0. Default Value: N/A |
| time_granularity | Yes | Integer | Definition: Time granularity. Constraints: The time range (interval between the start time and end time) and time precision must meet the following rules:
Range: The value must be an integer ranging from 1 to 3.
Default Value: N/A |
| timezone | No | String | Definition: Time zone. Constraints: N/A Range: The value must comply with the IANA time zone specifications, such as Asia/Shanghai and UTC. Default Value: Asia/Shanghai (GMT+8) |
| infer_type | Yes | String | Definition: Service inference type. Range:
Constraints: N/A Default Value: N/A |
| metric | No | String | Definition: Metric name. Constraints: N/A Range:
Default Value: N/A |
Response Parameters
Status code: 200
| Parameter | Type | Description |
|---|---|---|
| total | Integer | Definition: Total number of returned items. Range: N/A |
| count | Integer | Definition: Total number of returned items. Range: N/A |
| items | Array of DetailStatistics objects | Definition: Detailed metric statistics, which are displayed by time segment. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| time | Long | Definition: Timestamp, in milliseconds. Range: N/A |
| request_count | Integer | Definition: Number of calls. Range: N/A |
| succ_count | Integer | Definition: Number of successful calls. If batch inference is used, this parameter indicates the number of successful inference times. However, batch inference is under restricted use. To use it, submit a service ticket. Range: N/A |
| error_count | Integer | Definition: Number of successful calls. If batch inference is used, this parameter indicates the number of successful inference times. However, batch inference is under restricted use. To use it, submit a service ticket. Range: N/A |
| error_rate | Double | Definition: Call failure rate. If batch inference is used, this parameter indicates the inference failure rate. However, batch inference is under restricted use. To use it, submit a service ticket. Range: 0 to 1 |
| total_token | Double | Definition: Total number of called tokens (in thousands) Range: N/A |
| avg_total_token | Double | Definition: Average number of tokens (in thousands) Range: N/A |
| max_total_token | Double | Definition: Maximum number of tokens (in thousands) Range: N/A |
| p50_total_token | Double | Definition: Median number of tokens (in thousands) Range: N/A |
| p80_total_token | Double | Definition: 80th percentile number of tokens (in thousands) Range: N/A |
| p90_total_token | Double | Definition: 90th percentile number of tokens (in thousands) Range: N/A |
| p99_total_token | Double | Definition: 99th percentile number of tokens (in thousands) Range: N/A |
| prompt_token | Double | Definition: Total number of input tokens (in thousands) Range: N/A |
| avg_prompt_token | Double | Definition: Average number of input tokens (in thousands) Range: N/A |
| max_prompt_token | Double | Definition: Maximum number of input tokens (in thousands) Range: N/A |
| p50_prompt_token | Double | Definition: Median number of input tokens (in thousands) Range: N/A |
| p80_prompt_token | Double | Definition: 80th percentile number of input tokens (in thousands) Range: N/A |
| p90_prompt_token | Double | Definition: 90th percentile number of input tokens (in thousands) Range: N/A |
| p99_prompt_token | Double | Definition: 99th percentile number of input tokens (in thousands) Range: N/A |
| completion_token | Double | Definition: Total number of output tokens (in thousands) Range: N/A |
| avg_completion_token | Double | Definition: Average number of output tokens (in thousands) Range: N/A |
| max_completion_token | Double | Definition: Maximum number of output tokens (in thousands) Range: N/A |
| p50_completion_token | Double | Definition: Median number of output tokens (in thousands) Range: N/A |
| p80_completion_token | Double | Definition: 80th percentile number of output tokens (in thousands) Range: N/A |
| p90_completion_token | Double | Definition: 90th percentile number of output tokens (in thousands) Range: N/A |
| p99_completion_token | Double | Definition: 99th percentile number of output tokens (in thousands) Range: N/A |
| avg_latency | Double | Definition: Average E2E latency (ms) Range: N/A |
| max_latency | Double | Definition: Maximum E2E latency (ms) Range: N/A |
| p50_latency | Double | Definition: Median E2E latency (ms) Range: N/A |
| p80_latency | Double | Definition: 80th percentile E2E latency (ms) Range: N/A |
| p90_latency | Double | Definition: 90th percentile E2E latency (ms) Range: N/A |
| p99_latency | Double | Definition: 99th percentile E2E latency (ms) Range: N/A |
| avg_ttft | Double | Definition: Average TTFT (ms). Only streaming responses are counted. Range: N/A |
| max_ttft | Double | Definition: Maximum TTFT (ms). Only streaming responses are counted. Range: N/A |
| p50_ttft | Double | Definition: Median TTFT (ms). Only streaming responses are counted. Range: N/A |
| p80_ttft | Double | Definition: 80th percentile TTFT (ms). Only streaming responses are counted. Range: N/A |
| p90_ttft | Double | Definition: 90th percentile TTFT (ms). Only streaming responses are counted. Range: N/A |
| p99_ttft | Double | Definition: 99th percentile TTFT (ms). Only streaming responses are counted. Range: N/A |
| avg_tpot | Double | Definition: Average TPOT (ms). Only streaming responses are counted. Range: N/A |
| max_tpot | Double | Definition: Maximum TPOT (ms). Only streaming responses are counted. Range: N/A |
| p50_tpot | Double | Definition: Median TPOT (ms). Only streaming responses are counted. Range: N/A |
| p80_tpot | Double | Definition: 80th percentile TPOT (ms). Only streaming responses are counted. Range: N/A |
| p90_tpot | Double | Definition: 90th percentile TPOT (ms). Only streaming responses are counted. Range: N/A |
| p99_tpot | Double | Definition: 99th percentile TPOT (ms). Only streaming responses are counted. Range: N/A |
| rpm | Double | Definition: Number of requests processed per minute. Range: N/A |
| tpm | Double | Definition: Number of tokens processed per minute (thousand tokens/minute). Range: N/A |
| avg_generation_time | Double | Definition: Average generation duration (s), that is, the average time generating an image or video. Only video and image generation models are supported. Range: N/A |
| cache_token | Double | Definition: Number of cache hits, that is, the total number of tokens that hit the cache in the request. Range: N/A |
| cache_hit_ratio | Double | Definition: Cache hit rate, that is, the ratio of cache hit tokens in the request to the total input tokens. Range: 0 to 1 |
| total_token_list | Array of GradientIndicatorResult objects | Definition: Total token statistics, which includes detailed metric data in the specified period. If the request parameter metric is set to total_token, statistics about all tokens are returned. Range: N/A |
| prompt_token_list | Array of GradientIndicatorResult objects | Definition: Input token statistics, which includes detailed metric data in the specified period. If the request parameter metric is set to prompt_token, statistics about all input tokens are returned. Range: N/A |
| completion_token_list | Array of GradientIndicatorResult objects | Definition: Output token statistics, which includes detailed metric data in the specified period. If the request parameter metric is set to completion_token, statistics about all output tokens are returned. Range: N/A |
| rpm_list | Array of GradientIndicatorResult objects | Definition: RPM details, which includes detailed metric data in the specified period. If the request parameter metric is set to rpm, RPM details are returned. Range: N/A |
| infer_times | Integer | Definition: Total number of inference times. This parameter is related to batch inference. However, batch inference is under restricted use. To use it, submit a service ticket. Range: N/A |
| completion_tasks_count | Integer | Definition: Number of completed tasks. This parameter is related to batch inference. However, batch inference is under restricted use. To use it, submit a service ticket. Range: N/A |
| avg_consume_time | Double | Definition: Average task handling duration, in minutes. This parameter is related to batch inference. However, batch inference is under restricted use. To use it, submit a service ticket. Range: N/A |
| qps | Integer | Definition: Number of queries per second (QPS). The peak QPS in the minute is displayed. Range: N/A |
| video_generate_duration | Double | Definition: Total duration of the generated video. Range: N/A |
| image_generate_nums | Integer | Definition: Total number of generated images. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| name | String | Definition: Metric name. Range:
|
| value | Object | Definition: Metric value. The int and double types are supported. Range: N/A |
Status code: 400
| Parameter | Type | Description |
|---|---|---|
| error_code | String | Definition: Error code, which identifies the error type. For details, see MaaS Error Codes. Range: N/A |
| error_msg | String | Definition: Error description. For details, see MaaS Error Codes. Range: N/A |
Request Example
Query the metric data generated by real-time inference text of the preset service in the last 14 days. The service ID is 4f6d50ec-0e80-4ea0-983b-d0ad1ede7596 and the version ID is ac73463d-4453-4d62-a3d9-31b627a116b1.
/v1/{{project_id}}/maas/monitoring/4f6d50ec-0e80-4ea0-983b-d0ad1ede7596/show-detail-chart
{
"service_type" : 2,
"start_time" : 1768320000000,
"end_time" : 1769518975857,
"timezone" : "Asia/Shanghai",
"time_granularity" : 3,
"version_id" : "ac73463d-4453-4d62-a3d9-31b627a116b1",
"infer_type" : "real_time"
} Response Example
Status code: 200
Success response
{
"total" : 14,
"count" : 14,
"items" : [ {
"time" : 1768320000000,
"request_count" : 35,
"succ_count" : 13,
"error_count" : 22,
"error_rate" : 0.6286,
"total_token" : 13.149,
"avg_total_token" : 1.011,
"max_total_token" : 3.043,
"p50_total_token" : 0,
"p80_total_token" : 0.163,
"p90_total_token" : 1.647,
"p99_total_token" : 3.043,
"prompt_token" : 5.445,
"avg_prompt_token" : 0.419,
"max_prompt_token" : 2.747,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0.02,
"p90_prompt_token" : 0.03,
"p99_prompt_token" : 2.747,
"completion_token" : 7.704,
"avg_completion_token" : 0.593,
"max_completion_token" : 1.828,
"p50_completion_token" : 0,
"p80_completion_token" : 0.133,
"p90_completion_token" : 1.583,
"p99_completion_token" : 1.828,
"avg_latency" : 22811.23,
"max_latency" : 70615,
"p50_latency" : 0,
"p80_latency" : 5839,
"p90_latency" : 59330,
"p99_latency" : 70615,
"avg_ttft" : 522.79,
"max_ttft" : 1240.61,
"p50_ttft" : 373.97,
"p80_ttft" : 634.03,
"p90_ttft" : 1012.55,
"p99_ttft" : 1240.61,
"avg_tpot" : 36.12,
"max_tpot" : 43,
"p50_tpot" : 37.27,
"p80_tpot" : 38.3,
"p90_tpot" : 39.54,
"p99_tpot" : 43,
"rpm" : 0.02,
"tpm" : 0.009,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1768406400000,
"request_count" : 3,
"succ_count" : 1,
"error_count" : 2,
"error_rate" : 0.6667,
"total_token" : 1.533,
"avg_total_token" : 1.533,
"max_total_token" : 1.533,
"p50_total_token" : 0,
"p80_total_token" : 0,
"p90_total_token" : 1.533,
"p99_total_token" : 1.533,
"prompt_token" : 0.013,
"avg_prompt_token" : 0.013,
"max_prompt_token" : 0.013,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0,
"p90_prompt_token" : 0.013,
"p99_prompt_token" : 0.013,
"completion_token" : 1.52,
"avg_completion_token" : 1.52,
"max_completion_token" : 1.52,
"p50_completion_token" : 0,
"p80_completion_token" : 0,
"p90_completion_token" : 1.52,
"p99_completion_token" : 1.52,
"avg_latency" : 56872,
"max_latency" : 56872,
"p50_latency" : 0,
"p80_latency" : 0,
"p90_latency" : 56872,
"p99_latency" : 56872,
"avg_ttft" : 258.86,
"max_ttft" : 258.86,
"p50_ttft" : 258.86,
"p80_ttft" : 258.86,
"p90_ttft" : 258.86,
"p99_ttft" : 258.86,
"avg_tpot" : 37.27,
"max_tpot" : 37.27,
"p50_tpot" : 37.27,
"p80_tpot" : 37.27,
"p90_tpot" : 37.27,
"p99_tpot" : 37.27,
"rpm" : 0,
"tpm" : 0.001,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1768492800000,
"request_count" : 0,
"succ_count" : 0,
"error_count" : 0,
"error_rate" : 0,
"total_token" : 0,
"avg_total_token" : 0,
"max_total_token" : 0,
"p50_total_token" : 0,
"p80_total_token" : 0,
"p90_total_token" : 0,
"p99_total_token" : 0,
"prompt_token" : 0,
"avg_prompt_token" : 0,
"max_prompt_token" : 0,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0,
"p90_prompt_token" : 0,
"p99_prompt_token" : 0,
"completion_token" : 0,
"avg_completion_token" : 0,
"max_completion_token" : 0,
"p50_completion_token" : 0,
"p80_completion_token" : 0,
"p90_completion_token" : 0,
"p99_completion_token" : 0,
"avg_latency" : 0,
"max_latency" : 0,
"p50_latency" : 0,
"p80_latency" : 0,
"p90_latency" : 0,
"p99_latency" : 0,
"avg_ttft" : 0,
"max_ttft" : 0,
"p50_ttft" : 0,
"p80_ttft" : 0,
"p90_ttft" : 0,
"p99_ttft" : 0,
"avg_tpot" : 0,
"max_tpot" : 0,
"p50_tpot" : 0,
"p80_tpot" : 0,
"p90_tpot" : 0,
"p99_tpot" : 0,
"rpm" : 0,
"tpm" : 0,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1768579200000,
"request_count" : 0,
"succ_count" : 0,
"error_count" : 0,
"error_rate" : 0,
"total_token" : 0,
"avg_total_token" : 0,
"max_total_token" : 0,
"p50_total_token" : 0,
"p80_total_token" : 0,
"p90_total_token" : 0,
"p99_total_token" : 0,
"prompt_token" : 0,
"avg_prompt_token" : 0,
"max_prompt_token" : 0,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0,
"p90_prompt_token" : 0,
"p99_prompt_token" : 0,
"completion_token" : 0,
"avg_completion_token" : 0,
"max_completion_token" : 0,
"p50_completion_token" : 0,
"p80_completion_token" : 0,
"p90_completion_token" : 0,
"p99_completion_token" : 0,
"avg_latency" : 0,
"max_latency" : 0,
"p50_latency" : 0,
"p80_latency" : 0,
"p90_latency" : 0,
"p99_latency" : 0,
"avg_ttft" : 0,
"max_ttft" : 0,
"p50_ttft" : 0,
"p80_ttft" : 0,
"p90_ttft" : 0,
"p99_ttft" : 0,
"avg_tpot" : 0,
"max_tpot" : 0,
"p50_tpot" : 0,
"p80_tpot" : 0,
"p90_tpot" : 0,
"p99_tpot" : 0,
"rpm" : 0,
"tpm" : 0,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1768665600000,
"request_count" : 0,
"succ_count" : 0,
"error_count" : 0,
"error_rate" : 0,
"total_token" : 0,
"avg_total_token" : 0,
"max_total_token" : 0,
"p50_total_token" : 0,
"p80_total_token" : 0,
"p90_total_token" : 0,
"p99_total_token" : 0,
"prompt_token" : 0,
"avg_prompt_token" : 0,
"max_prompt_token" : 0,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0,
"p90_prompt_token" : 0,
"p99_prompt_token" : 0,
"completion_token" : 0,
"avg_completion_token" : 0,
"max_completion_token" : 0,
"p50_completion_token" : 0,
"p80_completion_token" : 0,
"p90_completion_token" : 0,
"p99_completion_token" : 0,
"avg_latency" : 0,
"max_latency" : 0,
"p50_latency" : 0,
"p80_latency" : 0,
"p90_latency" : 0,
"p99_latency" : 0,
"avg_ttft" : 0,
"max_ttft" : 0,
"p50_ttft" : 0,
"p80_ttft" : 0,
"p90_ttft" : 0,
"p99_ttft" : 0,
"avg_tpot" : 0,
"max_tpot" : 0,
"p50_tpot" : 0,
"p80_tpot" : 0,
"p90_tpot" : 0,
"p99_tpot" : 0,
"rpm" : 0,
"tpm" : 0,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1768752000000,
"request_count" : 3,
"succ_count" : 3,
"error_count" : 0,
"error_rate" : 0,
"total_token" : 0.533,
"avg_total_token" : 0.178,
"max_total_token" : 0.228,
"p50_total_token" : 0.199,
"p80_total_token" : 0.199,
"p90_total_token" : 0.228,
"p99_total_token" : 0.228,
"prompt_token" : 0.264,
"avg_prompt_token" : 0.088,
"max_prompt_token" : 0.139,
"p50_prompt_token" : 0.105,
"p80_prompt_token" : 0.105,
"p90_prompt_token" : 0.139,
"p99_prompt_token" : 0.139,
"completion_token" : 0.269,
"avg_completion_token" : 0.09,
"max_completion_token" : 0.123,
"p50_completion_token" : 0.086,
"p80_completion_token" : 0.086,
"p90_completion_token" : 0.123,
"p99_completion_token" : 0.123,
"avg_latency" : 2962.33,
"max_latency" : 5112,
"p50_latency" : 2129,
"p80_latency" : 2129,
"p90_latency" : 5112,
"p99_latency" : 5112,
"avg_ttft" : 349,
"max_ttft" : 424.79,
"p50_ttft" : 422.49,
"p80_ttft" : 422.49,
"p90_ttft" : 424.79,
"p99_ttft" : 424.79,
"avg_tpot" : 27.02,
"max_tpot" : 40.27,
"p50_tpot" : 20.7,
"p80_tpot" : 20.7,
"p90_tpot" : 40.27,
"p99_tpot" : 40.27,
"rpm" : 0,
"tpm" : 0,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1768838400000,
"request_count" : 0,
"succ_count" : 0,
"error_count" : 0,
"error_rate" : 0,
"total_token" : 0,
"avg_total_token" : 0,
"max_total_token" : 0,
"p50_total_token" : 0,
"p80_total_token" : 0,
"p90_total_token" : 0,
"p99_total_token" : 0,
"prompt_token" : 0,
"avg_prompt_token" : 0,
"max_prompt_token" : 0,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0,
"p90_prompt_token" : 0,
"p99_prompt_token" : 0,
"completion_token" : 0,
"avg_completion_token" : 0,
"max_completion_token" : 0,
"p50_completion_token" : 0,
"p80_completion_token" : 0,
"p90_completion_token" : 0,
"p99_completion_token" : 0,
"avg_latency" : 0,
"max_latency" : 0,
"p50_latency" : 0,
"p80_latency" : 0,
"p90_latency" : 0,
"p99_latency" : 0,
"avg_ttft" : 0,
"max_ttft" : 0,
"p50_ttft" : 0,
"p80_ttft" : 0,
"p90_ttft" : 0,
"p99_ttft" : 0,
"avg_tpot" : 0,
"max_tpot" : 0,
"p50_tpot" : 0,
"p80_tpot" : 0,
"p90_tpot" : 0,
"p99_tpot" : 0,
"rpm" : 0,
"tpm" : 0,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1768924800000,
"request_count" : 0,
"succ_count" : 0,
"error_count" : 0,
"error_rate" : 0,
"total_token" : 0,
"avg_total_token" : 0,
"max_total_token" : 0,
"p50_total_token" : 0,
"p80_total_token" : 0,
"p90_total_token" : 0,
"p99_total_token" : 0,
"prompt_token" : 0,
"avg_prompt_token" : 0,
"max_prompt_token" : 0,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0,
"p90_prompt_token" : 0,
"p99_prompt_token" : 0,
"completion_token" : 0,
"avg_completion_token" : 0,
"max_completion_token" : 0,
"p50_completion_token" : 0,
"p80_completion_token" : 0,
"p90_completion_token" : 0,
"p99_completion_token" : 0,
"avg_latency" : 0,
"max_latency" : 0,
"p50_latency" : 0,
"p80_latency" : 0,
"p90_latency" : 0,
"p99_latency" : 0,
"avg_ttft" : 0,
"max_ttft" : 0,
"p50_ttft" : 0,
"p80_ttft" : 0,
"p90_ttft" : 0,
"p99_ttft" : 0,
"avg_tpot" : 0,
"max_tpot" : 0,
"p50_tpot" : 0,
"p80_tpot" : 0,
"p90_tpot" : 0,
"p99_tpot" : 0,
"rpm" : 0,
"tpm" : 0,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1769011200000,
"request_count" : 0,
"succ_count" : 0,
"error_count" : 0,
"error_rate" : 0,
"total_token" : 0,
"avg_total_token" : 0,
"max_total_token" : 0,
"p50_total_token" : 0,
"p80_total_token" : 0,
"p90_total_token" : 0,
"p99_total_token" : 0,
"prompt_token" : 0,
"avg_prompt_token" : 0,
"max_prompt_token" : 0,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0,
"p90_prompt_token" : 0,
"p99_prompt_token" : 0,
"completion_token" : 0,
"avg_completion_token" : 0,
"max_completion_token" : 0,
"p50_completion_token" : 0,
"p80_completion_token" : 0,
"p90_completion_token" : 0,
"p99_completion_token" : 0,
"avg_latency" : 0,
"max_latency" : 0,
"p50_latency" : 0,
"p80_latency" : 0,
"p90_latency" : 0,
"p99_latency" : 0,
"avg_ttft" : 0,
"max_ttft" : 0,
"p50_ttft" : 0,
"p80_ttft" : 0,
"p90_ttft" : 0,
"p99_ttft" : 0,
"avg_tpot" : 0,
"max_tpot" : 0,
"p50_tpot" : 0,
"p80_tpot" : 0,
"p90_tpot" : 0,
"p99_tpot" : 0,
"rpm" : 0,
"tpm" : 0,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1769097600000,
"request_count" : 0,
"succ_count" : 0,
"error_count" : 0,
"error_rate" : 0,
"total_token" : 0,
"avg_total_token" : 0,
"max_total_token" : 0,
"p50_total_token" : 0,
"p80_total_token" : 0,
"p90_total_token" : 0,
"p99_total_token" : 0,
"prompt_token" : 0,
"avg_prompt_token" : 0,
"max_prompt_token" : 0,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0,
"p90_prompt_token" : 0,
"p99_prompt_token" : 0,
"completion_token" : 0,
"avg_completion_token" : 0,
"max_completion_token" : 0,
"p50_completion_token" : 0,
"p80_completion_token" : 0,
"p90_completion_token" : 0,
"p99_completion_token" : 0,
"avg_latency" : 0,
"max_latency" : 0,
"p50_latency" : 0,
"p80_latency" : 0,
"p90_latency" : 0,
"p99_latency" : 0,
"avg_ttft" : 0,
"max_ttft" : 0,
"p50_ttft" : 0,
"p80_ttft" : 0,
"p90_ttft" : 0,
"p99_ttft" : 0,
"avg_tpot" : 0,
"max_tpot" : 0,
"p50_tpot" : 0,
"p80_tpot" : 0,
"p90_tpot" : 0,
"p99_tpot" : 0,
"rpm" : 0,
"tpm" : 0,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1769184000000,
"request_count" : 0,
"succ_count" : 0,
"error_count" : 0,
"error_rate" : 0,
"total_token" : 0,
"avg_total_token" : 0,
"max_total_token" : 0,
"p50_total_token" : 0,
"p80_total_token" : 0,
"p90_total_token" : 0,
"p99_total_token" : 0,
"prompt_token" : 0,
"avg_prompt_token" : 0,
"max_prompt_token" : 0,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0,
"p90_prompt_token" : 0,
"p99_prompt_token" : 0,
"completion_token" : 0,
"avg_completion_token" : 0,
"max_completion_token" : 0,
"p50_completion_token" : 0,
"p80_completion_token" : 0,
"p90_completion_token" : 0,
"p99_completion_token" : 0,
"avg_latency" : 0,
"max_latency" : 0,
"p50_latency" : 0,
"p80_latency" : 0,
"p90_latency" : 0,
"p99_latency" : 0,
"avg_ttft" : 0,
"max_ttft" : 0,
"p50_ttft" : 0,
"p80_ttft" : 0,
"p90_ttft" : 0,
"p99_ttft" : 0,
"avg_tpot" : 0,
"max_tpot" : 0,
"p50_tpot" : 0,
"p80_tpot" : 0,
"p90_tpot" : 0,
"p99_tpot" : 0,
"rpm" : 0,
"tpm" : 0,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1769270400000,
"request_count" : 0,
"succ_count" : 0,
"error_count" : 0,
"error_rate" : 0,
"total_token" : 0,
"avg_total_token" : 0,
"max_total_token" : 0,
"p50_total_token" : 0,
"p80_total_token" : 0,
"p90_total_token" : 0,
"p99_total_token" : 0,
"prompt_token" : 0,
"avg_prompt_token" : 0,
"max_prompt_token" : 0,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0,
"p90_prompt_token" : 0,
"p99_prompt_token" : 0,
"completion_token" : 0,
"avg_completion_token" : 0,
"max_completion_token" : 0,
"p50_completion_token" : 0,
"p80_completion_token" : 0,
"p90_completion_token" : 0,
"p99_completion_token" : 0,
"avg_latency" : 0,
"max_latency" : 0,
"p50_latency" : 0,
"p80_latency" : 0,
"p90_latency" : 0,
"p99_latency" : 0,
"avg_ttft" : 0,
"max_ttft" : 0,
"p50_ttft" : 0,
"p80_ttft" : 0,
"p90_ttft" : 0,
"p99_ttft" : 0,
"avg_tpot" : 0,
"max_tpot" : 0,
"p50_tpot" : 0,
"p80_tpot" : 0,
"p90_tpot" : 0,
"p99_tpot" : 0,
"rpm" : 0,
"tpm" : 0,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1769356800000,
"request_count" : 0,
"succ_count" : 0,
"error_count" : 0,
"error_rate" : 0,
"total_token" : 0,
"avg_total_token" : 0,
"max_total_token" : 0,
"p50_total_token" : 0,
"p80_total_token" : 0,
"p90_total_token" : 0,
"p99_total_token" : 0,
"prompt_token" : 0,
"avg_prompt_token" : 0,
"max_prompt_token" : 0,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0,
"p90_prompt_token" : 0,
"p99_prompt_token" : 0,
"completion_token" : 0,
"avg_completion_token" : 0,
"max_completion_token" : 0,
"p50_completion_token" : 0,
"p80_completion_token" : 0,
"p90_completion_token" : 0,
"p99_completion_token" : 0,
"avg_latency" : 0,
"max_latency" : 0,
"p50_latency" : 0,
"p80_latency" : 0,
"p90_latency" : 0,
"p99_latency" : 0,
"avg_ttft" : 0,
"max_ttft" : 0,
"p50_ttft" : 0,
"p80_ttft" : 0,
"p90_ttft" : 0,
"p99_ttft" : 0,
"avg_tpot" : 0,
"max_tpot" : 0,
"p50_tpot" : 0,
"p80_tpot" : 0,
"p90_tpot" : 0,
"p99_tpot" : 0,
"rpm" : 0,
"tpm" : 0,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
}, {
"time" : 1769443200000,
"request_count" : 0,
"succ_count" : 0,
"error_count" : 0,
"error_rate" : 0,
"total_token" : 0,
"avg_total_token" : 0,
"max_total_token" : 0,
"p50_total_token" : 0,
"p80_total_token" : 0,
"p90_total_token" : 0,
"p99_total_token" : 0,
"prompt_token" : 0,
"avg_prompt_token" : 0,
"max_prompt_token" : 0,
"p50_prompt_token" : 0,
"p80_prompt_token" : 0,
"p90_prompt_token" : 0,
"p99_prompt_token" : 0,
"completion_token" : 0,
"avg_completion_token" : 0,
"max_completion_token" : 0,
"p50_completion_token" : 0,
"p80_completion_token" : 0,
"p90_completion_token" : 0,
"p99_completion_token" : 0,
"avg_latency" : 0,
"max_latency" : 0,
"p50_latency" : 0,
"p80_latency" : 0,
"p90_latency" : 0,
"p99_latency" : 0,
"avg_ttft" : 0,
"max_ttft" : 0,
"p50_ttft" : 0,
"p80_ttft" : 0,
"p90_ttft" : 0,
"p99_ttft" : 0,
"avg_tpot" : 0,
"max_tpot" : 0,
"p50_tpot" : 0,
"p80_tpot" : 0,
"p90_tpot" : 0,
"p99_tpot" : 0,
"rpm" : 0,
"tpm" : 0,
"avg_generation_time" : 0,
"cache_token" : 0,
"cache_hit_ratio" : 0,
"total_token_list" : null,
"prompt_token_list" : null,
"completion_token_list" : null,
"rpm_list" : null,
"infer_times" : 0,
"completion_tasks_count" : 0,
"avg_consume_time" : 0,
"qps" : 0,
"video_generate_duration": 0,
"image_generate_nums": 0
} ]
} Status code: 400
Failure response
{
"error_code" : "ModelArts.0104",
"error_msg" : "Inference type realtime is invalid. The inference type must be real_time or batch."
} Status Codes
| Status Code | Description |
|---|---|
| 200 | Success response |
| 400 | Failure response |
Error Codes
For details, see Error Codes.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot
