Creating an HRA Policy

Function

This API is used to create an HRA policy for a deployed service that supports HRA policies. It is used when you need to automatically adjust the number of service instances based on the service load or at a specified time. Before calling this API, ensure that the service has been deployed and a valid service ID has been obtained. In addition, detailed HRA policy parameters, such as the HRA time, instance quantity range, and conditional trigger, have been provided. After the creation, the system automatically adjusts the number of service instances based on the preset policy. This ensures the performance and availability of the service within the specified period. If the provided service ID is invalid, the parameter settings are incorrect, or the system resources are insufficient, an error message is displayed, prompting you to check the validity of the input data or contact technical support.

Debugging

You can debug this API through automatic authentication in API Explorer or use the SDK sample code generated by API Explorer.

Authorization Information

Each account has all the permissions required to call all APIs, but IAM users must be assigned the required permissions.

If you are using role/policy-based authorization, see Permissions Policies and Supported Actions for details on the required permissions.
If you are using identity policy-based authorization, the following identity policy-based permissions are required.

Action

Access Level

Resource Type (*: required)

Condition Key

Alias

Dependencies

modelarts:service:createHra

Read

service *

-

-

-

Action	Access Level	Resource Type (*: required)	Condition Key	Alias	Dependencies
modelarts:service:createHra	Read	service *	-	-	-

URI

POST /v2/{project_id}/services/{service_id}/deployments/{deployment_id}/hra

**Table 1** Path Parameters
Parameter	Mandatory	Type	Description
project_id	Yes	String	Definition Project ID. For details, see Obtaining a Project ID and Name. Constraints N/A Range Project ID. Default Value N/A
service_id	Yes	String	Definition: Service ID, which can be obtained from the response body during service creation or by querying the service list. The service_id field indicates the service ID. Constraints N/A Range Service ID. Default Value N/A
deployment_id	Yes	String	Definition Deployment ID, which you can obtain from the response body when adding the deployment or by obtaining service deployments. The deployment_id field indicates the deployment ID. Constraints N/A Range Deployment ID. Default Value N/A

Request Parameters

**Table 2** Request header parameters
Parameter	Mandatory	Type	Description
X-Auth-Token	Yes	String	Definition User token. It can be obtained by calling the IAM API used to obtain a user token. The value of X-Subject-Token in the response header is the user token. For details about how to obtain the token, see Obtaining a User Token Through Password Authentication. Constraints N/A Range N/A Default Value N/A
Content-Type	Yes	String	Definition Message body type or format. Constraints N/A Range application/json application/json;charset=utf-8 Default Value N/A

**Table 3** Request body parameters
Parameter	Mandatory	Type	Description
workspace_id	Yes	String	Definition Workspace ID. If no workspace is created, the default value is 0. If a workspace is created and used, use the actual value. Range N/A
hra_rules	Yes	Array of HraRuleCreateRequest objects	Definition HRA rule list. Constraints N/A
disable	No	Boolean	Definition User-controlled enable/disable switch. Range true indicates disabled and false indicates enabled. Constraints N/A Default Value N/A
min_replicas	No	Integer	Definition Minimum number of replicas. This parameter is invalid because the current version does not support actual scaling. Range 1 to 128
max_replicas	No	Integer	Definition Maximum number of replicas. This parameter is invalid because the current version does not support actual scaling. Range 1 to 128
scale_window	No	Integer	Definition Scaling time window. This parameter is invalid because the current version does not support actual scaling. Range N/A

**Table 4** HraRuleCreateRequest
Parameter	Mandatory	Type	Description
scaler_type	No	String	Definition Scaling type. Range SIMULATOR_ALGO: simulator algorithm scaling type. Constraints N/A Default Value N/A
slo_info	Yes	Array of SloInfo objects	Definition SLO configuration parameters. Range N/A
metrics	Yes	Array of Metrics objects	Definition Metric information. Range N/A
role_replica	No	Array of RoleReplica objects	Definition Role scaling policy (This parameter is invalid because actual scaling is not supported.) Range 1 to 128

**Table 5** SloInfo
Parameter	Mandatory	Type	Description
slo_objectives	Yes	Array of SloObjectives objects	Definition Expected simulation metric. Range N/A
predict_window_seconds	No	Integer	Definition Prediction time window. Constraints 60 to 600 Range N/A Default Value 60

**Table 6** SloObjectives
Parameter	Mandatory	Type	Description
metric_ttft	No	Integer	Definition Time to first token (TTFT), in milliseconds. Range 0 to 10000 Constraints N/A Default Value 100
metric_tpot	No	Integer	Definition Time per output token (TPOT), in milliseconds. Range 0 to 1000 Constraints N/A Default Value 50
percental	No	Integer	Definition SLO satisfaction percentage. Range 0 to 100

**Table 7** Metrics
Parameter	Mandatory	Type	Description
endpoint	Yes	String	Definition Metric collection address. The value can be an IP address, a domain name, or localhost. Range N/A
path	No	String	Definition Metric collection path. Range N/A
port	Yes	String	Definition Metric collection port. Range 1 to 65535
scheme	Yes	String	Definition Metric collection protocol. Range HTTP HTTPS
metrics_source	No	String	Definition Metric source type. Range CONTAINER: within a container OTHERS: other external addresses Constraints N/A Default Value CONTAINER

**Table 8** RoleReplica
Parameter	Mandatory	Type	Description
name	Yes	String	Definition Role name. Range N/A
max_replicas	No	Integer	Definition Maximum number of replicas. Range 1 to 128
min_replicas	No	Integer	Definition Minimum number of replicas. Range 1 to 128

Response Parameters

Status code: 200

**Table 9** Response body parameters
Parameter	Type	Description
id	String	Definition HRA policy ID. Range N/A
workspace_id	String	Definition Workspace ID. If no workspace is created, the default value is 0. If a workspace is created and used, use the actual value. Range N/A
hra_rules	Array of HraRuleResponse objects	Definition HRA rule list.
disable	Boolean	Definition User-controlled enable/disable switch. Range true indicates disabled and false indicates enabled. Constraints N/A Default Value N/A
policy_status	String	Definition Policy status. Range CREATING ACTIVE: available INACTIVE: unavailable DELETED Constraints N/A Default Value N/A
min_replicas	Integer	Definition Minimum number of replicas. This parameter is invalid because the current version does not support actual scaling. Range 1 to 128
max_replicas	Integer	Definition Maximum number of replicas. This parameter is invalid because the current version does not support actual scaling. Range 1 to 128
scale_window	Integer	Definition Scaling time window. This parameter is invalid because the current version does not support actual scaling. Range N/A

**Table 10** HraRuleResponse
Parameter	Type	Description
id	String	Definition Rule ID, which can be obtained from the response body during HRA policy creation. You can also query the ratio detection information of the inference unit to obtain the HRA policy of the current user. The id field indicates the rule ID. Constraints N/A Range Rule ID. Default Value N/A
name	String	Definition Rule name. Range N/A
disable	Boolean	Definition Specifies whether to disable the rule. Range true: disabled false: enabled Constraints N/A Default Value N/A
scaler_type	String	Definition Scaling type. Range SIMULATOR_ALGO: simulator algorithm scaling type. Constraints N/A Default Value N/A
rule_status	String	Definition HRA rule status. Range CREATING CONFIG_SUCCESS EXECUTE_SUCCESS DELETED FAILED Constraints N/A Default Value N/A
slo_info	Array of SloInfo objects	Definition SLO configuration parameters. Range N/A
metrics	Array of Metrics objects	Definition Metric information. Range N/A
role_replica	Array of RoleReplica objects	Definition Role scaling policy (This parameter is invalid because actual scaling is not supported.)

**Table 11** SloInfo
Parameter	Type	Description
slo_objectives	Array of SloObjectives objects	Definition Expected simulation metric. Range N/A
predict_window_seconds	Integer	Definition Prediction time window. Constraints 60 to 600 Range N/A Default Value 60

**Table 12** SloObjectives
Parameter	Type	Description
metric_ttft	Integer	Definition Time to first token (TTFT), in milliseconds. Range 0 to 10000 Constraints N/A Default Value 100
metric_tpot	Integer	Definition Time per output token (TPOT), in milliseconds. Range 0 to 1000 Constraints N/A Default Value 50
percental	Integer	Definition SLO satisfaction percentage. Range 0 to 100

**Table 13** Metrics
Parameter	Type	Description
endpoint	String	Definition Metric collection address. The value can be an IP address, a domain name, or localhost. Range N/A
path	String	Definition Metric collection path. Range N/A
port	String	Definition Metric collection port. Range 1 to 65535
scheme	String	Definition Metric collection protocol. Range HTTP HTTPS
metrics_source	String	Definition Metric source type. Range CONTAINER: within a container OTHERS: other external addresses Constraints N/A Default Value CONTAINER

**Table 14** RoleReplica
Parameter	Type	Description
name	String	Definition Role name. Range N/A
max_replicas	Integer	Definition Maximum number of replicas. Range 1 to 128
min_replicas	Integer	Definition Minimum number of replicas. Range 1 to 128

Example Requests

Create a rule. This API is used when no rule exists in the service.

POST https://{end_point}/v2/{project_id}/services/{service_id}/deployments/{deployment_id}/hra

{
  "workspace_id" : "0",
  "disable" : false,
  "hra_rules" : [ {
    "scaler_type" : "SIMULATOR_ALGO",
    "slo_info" : [ {
      "slo_objectives" : [ {
        "metric_ttft" : 100,
        "metric_tpot" : 50,
        "percental" : 90
      } ],
      "predict_window_seconds" : 60
    } ],
    "metrics" : [ {
      "endpoint" : "localhost",
      "path" : "/metrics",
      "port" : 9876,
      "scheme" : "http",
      "metrics_source" : "CONTAINER"
    } ],
    "role_replica" : [ {
      "max_replicas" : 3,
      "min_replicas" : 1,
      "name" : "prefill"
    }, {
      "max_replicas" : 3,
      "min_replicas" : 1,
      "name" : "decode"
    } ]
  } ],
  "min_replicas" : 1,
  "max_replicas" : 128,
  "scale_window" : 0
}

Example Responses

Status code: 200

Response body for creating an HRA policy.

{
  "disable" : false,
  "hra_rules" : [ {
    "disable" : false,
    "id" : "7a88d578-7373-4508-a47a-8e2dd3939ea4",
    "metrics" : [ {
      "endpoint" : "localhost",
      "metrics_source" : "CONTAINER",
      "path" : "/metrics",
      "port" : "9876",
      "scheme" : "http"
    } ],
    "name" : "rule-wzwq",
    "role_replica" : [ {
      "max_replicas" : 3,
      "min_replicas" : 1,
      "name" : "prefill"
    }, {
      "max_replicas" : 3,
      "min_replicas" : 1,
      "name" : "decode"
    } ],
    "rule_status" : "CREATING",
    "scaler_type" : "SIMULATOR_ALGO",
    "slo_info" : [ {
      "predict_window_seconds" : 60,
      "slo_objectives" : [ {
        "metric_tpot" : 50,
        "metric_ttft" : 100,
        "percental" : 90
      } ]
    } ]
  } ],
  "id" : "1a71d5a4-abc4-44b4-a145-3b59aa1a8951",
  "max_replicas" : 128,
  "min_replicas" : 1,
  "policy_status" : "INACTIVE",
  "scale_window" : 0,
  "workspace_id" : "0"
}