Adding a Deployment
Function
This API is used to deploy a model as a real-time service.
Debugging
You can debug this API through automatic authentication in API Explorer or use the SDK sample code generated by API Explorer.
Authorization Information
Each account has all the permissions required to call all APIs, but IAM users must be assigned the required permissions.
- If you are using role/policy-based authorization, see Permissions Policies and Supported Actions for details on the required permissions.
- If you are using identity policy-based authorization, the following identity policy-based permissions are required.
Action
Access Level
Resource Type (*: required)
Condition Key
Alias
Dependencies
modelarts:service:createDeployment
Write
service *
-
-
-
URI
POST /v2/{project_id}/services/{service_id}/deployments
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| project_id | Yes | String | Definition Project ID. For details, see Obtaining a Project ID and Name. Constraints N/A Range Project ID. Default Value N/A |
| service_id | Yes | String | Definition Service ID, which can be obtained from the response body during service creation or by obtaining services owned by the current user. The service_id field is the service ID. Constraints N/A Range Service ID. Default Value N/A |
Request Parameters
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| X-Auth-Token | Yes | String | Definition User token. It can be obtained by calling the IAM API used to obtain a user token. The value of X-Subject-Token in the response header is the user token. For details about how to obtain the token, see Obtaining a User Token Through Password Authentication. Constraints N/A Range N/A Default Value N/A |
| Content-Type | Yes | String | Definition Message body type or format. Constraints N/A Range Default Value N/A |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| id | No | String | Definition Deployment ID. Constraints N/A Range N/A Default Value N/A |
| name | Yes | String | Definition Deployment name. Constraints N/A Range N/A Default Value N/A |
| pool_id | No | String | Definition Resource pool ID. Services in a specified resource pool are queried. By default, no resource pool is filtered. You can obtain the value by querying the resource pool list. Constraints N/A Range N/A Default Value N/A |
| count | No | Integer | Definition Number of service instances in the deployment scenario. Constraints N/A Range [1, 128] Default Value N/A |
| system_log_dump_enable | No | Boolean | Definition Specifies whether to enable system log dump. Constraints This parameter is available only for NPU resource pools. Range N/A Default Value false |
| unit_configs | No | Array of UnitConfig objects | Definition Inference unit configuration. Constraints For single-node inference, the number of instance units is 1. For distributed inference, the number of instances can be configured based on the framework. |
| weight | Yes | Integer | Definition Weight percentage of the traffic allocated to the model. This parameter is mandatory only when the model is deployed as a real-time service. Constraints N/A Range [0,100] Default Value N/A |
| secret_type | No | String | Definition Secret type. You can select the secret type (dew or agency) to be used here. Constraints
Range Default Value N/A |
| secret_name | No | String | Definition Secret name. When using a DEW secret, you can select the secret to be used here. Constraints N/A |
| priority | No | Integer | Definition Service priority. Constraints
Range [1, 3] Default Value N/A |
| high_avail_switch | No | Boolean | Definition Specifies whether to enable HA. Once enabled, pods of different instances are evenly distributed to different nodes. (This parameter is about to be brought offline. Use the schedule_strategy field instead.) Constraints N/A Range Default Value true |
| schedule_strategy | No | String | Definition Scheduling policy. Constraints N/A Range Default Value HIGH_AVAILABILITY |
| version | No | String | Definition Service version, which consists of digits and periods (.), for example, 1.0.1. Constraints N/A Range The value ranges from 0.0.0 to 99.99.99 and can contain at most eight characters. Default Value N/A |
| version_id | No | String | Definition Version ID, which can be obtained by querying the version list. Constraints N/A Range N/A Default Value N/A |
| description | No | String | Definition Service description. Constraints N/A Range The value can contain a maximum of 512 characters and cannot contain greater-than signs (>) or less-than signs (<). Default Value This parameter is left blank by default. |
| framework | No | String | Definition Service framework. Constraints Only the following enumerated values are supported: COMMON | VLLM | MINDIE. Range Only the following enumerated values are supported: COMMON | VLLM | MINDIE. Default Value COMMON. |
| running_count | No | Integer | Definition Number of service running instances in the deployment scenario. Constraints N/A Range [1, 128] Default Value N/A |
| deploy_type | No | String | Definition Deployment type. Range |
| mirror_traffic_enable | No | Boolean | Definition Specifies whether to enable image traffic. Range N/A |
| mirror_traffic_weight | No | Integer | Definition Image traffic weight. Range 50 |
| version_count | No | Integer | Definition Number of service versions. Range N/A |
| update_at | No | Long | Definition Update time, which is automatically generated. Range Millisecond-level timestamp, which is a 13-digit number. |
| model | No | ModelResource object | Definition Model configuration shared by all inference units. You can select a model and weight file to use with the image. This configuration takes effect for all inference units. Constraints N/A |
| advanced_config | No | AdvancedConfig object | Definition Advanced service deployment settings. Constraints N/A Range N/A Default Value N/A |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| id | No | String | Definition Instance unit ID. Constraints N/A Range N/A Default Value N/A |
| name | No | String | Definition Role name of the instance unit. Constraints The value can contain a maximum of 16 characters and must meet the regular expression ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$. Default Value N/A |
| role | No | String | Definition Instance unit role. Constraints N/A Range Default Value N/A |
| custom_spec | No | CustomResourceSpec object | Definition Custom resource specifications. Constraints N/A Example "custom_spec": { "arch": "x86", "cpu": 0.51, "memory": 1024.0 }, |
| flavor | No | String | Definition Resource specifications. Select specifications based on service requirements. When specification is set to custom, the custom specifications are used. custom_spec specifies deployment specifications. Constraints N/A Range N/A Default Value N/A |
| flavor_display_name | No | String | Definition Display name of a resource specification. Constraints N/A Range N/A Default Value N/A |
| image | Yes | ImageInfo object | Definition Image configuration information, including the image path and image source. Constraints N/A |
| models | No | Array of ModelResource objects | Definition This parameter is discarded. You are advised to use files to configure models. Model configuration. You can select a model and weight file to use with the image. Constraints N/A |
| codes | No | Array of Code objects | Definition This parameter is discarded. You are advised to use files to configure code. Code configuration. You can select the OBS path where the code is stored. Constraints N/A |
| files | No | Array of FileInfo objects | Definition Model and code configuration. You can select the model and weight file to work with the image and the OBS path where the code is stored. Constraints N/A |
| dumps | No | Array of Dump objects | Definition Dump configuration. You can select the OBS bucket where the data is to be dumped. Constraints A maximum of 20 groups can be configured. |
| count | No | Integer | Definition Number of configuration instances. Constraints N/A Range N/A Default Value N/A |
| cpu_affinity | No | Boolean | Definition Whether to enable CPU pinning. Range |
| cmd | No | String | Definition Startup command. Constraints N/A Range N/A Default Value N/A |
| termination_grace | No | TerminationGrace object | Definition Graceful shutdown configuration. Constraints N/A |
| envs | No | Map<String,String> | Definition Environment variable. Constraints The length of a variable key cannot exceed 64 characters. The key can only contain letters, digits, underscores (_), hyphens (-), and periods (.), and cannot start with a digit. The value cannot contain HTML tags, including <^>. |
| readiness_health | No | Health object | Definition Readiness health check. Constraints N/A |
| startup_health | No | Health object | Definition Health checks are enabled. Constraints N/A |
| liveness_health | No | Health object | Definition Liveness health check. Range N/A |
| port | No | Integer | Definition Port. Constraints N/A Range [1, 65535] Default Value N/A |
| recovery | No | String | Definition Auto-rebuild policy. When enabled, the platform will automatically rebuild pods according to the policy if they restart due to deployment configuration changes or hardware failures. If disabled, the platform will not intervene. Constraints N/A Range
Default Value N/A |
| npu_reset_enable | No | Boolean | Definition Specifies whether to enable the recovery policy. Constraints N/A Range Default Value N/A |
| group_count | No | Integer | Definition Number of replicas in a unit. This parameter is invalid when deploy_type is set to SINGLE or workload_type is set to DEPLOYMENT. Constraints N/A Range [1, 100] or left empty. Default Value 1 |
| affinity | No | Affinity object | Definition Node affinity scheduling. Constraints N/A |
| security_config | No | ServiceSecurityConfig object | Definition Container user ID/user group ID. Constraints N/A |
| pool_resource_flavor | No | String | Definition Node pool resource specifications. Constraints The value can contain only letters, digits, periods (.), hyphens (-), and underscores (_). Range The value can contain a maximum of 128 characters. Default Value N/A |
| group_policy | No | UnitGroupPolicy object | Definition Scheduling constraints for single-replica resource instances. This parameter is valid only for A5 supernodes. Constraints
Range N/A Default Value N/A |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| gpu | No | Float | Definition Number of GPUs. Constraints N/A Range The value can be a decimal. It cannot be smaller than 0, with the third decimal place is rounded off. Default Value N/A |
| memory | No | Integer | Definition Memory size, in MB. Constraints N/A Range Only integers are supported. Default Value N/A |
| cpu | No | Float | Definition CPU cores. Constraints N/A Range The value can be a decimal. It cannot be smaller than 0.01, with the third decimal place is rounded off. Default Value N/A |
| ascend | No | Integer | Definition Number of Ascend chips. Constraints N/A Range N/A Default Value N/A |
| arch | No | String | Definition Architecture type. Constraints N/A Range x86/arm64. Default Value N/A |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| id | No | String | Definition Image ID. Constraints N/A Range N/A Default Value N/A |
| source | Yes | String | Definition Image type, which indicates the image source. Constraints N/A Range Default Value N/A |
| swr_path | Yes | String | Definition Image path. The address is different when the source value is different. Constraints N/A Range N/A Default Value N/A |
| category | No | String | Definition Flavors supported by the image. Constraints N/A Range Default Value CPU |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| source | Yes | String | Definition Code source type. Constraints N/A Range The value is case-insensitive. Default Value N/A |
| address | No | String | Definition Code source address, which complies with the format of different storage systems. Constraints N/A Range N/A Default Value N/A |
| mount_path | Yes | String | Definition Path mounted to the container. The value must start with a slash (/) and can contain hyphens (-), backslashes (), underscores (_), periods (.), letters, and digits. Constraints N/A Range The value must start and end with a slash (/) and can contain at most 255 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. Default Value N/A |
| host_cache | No | Boolean | Definition Specifies whether the model can be cached locally. This function is disabled by default. Constraints N/A Range Default Value false |
| efs_sub_path | No | String | Definition When the storage type is EFS, subdirectories can be configured. Constraints N/A Range N/A Default Value N/A |
| read_only | No | Boolean | Definition Specifies whether the mount permission is read-only. Constraints N/A Range Default Value N/A |
| os_warm_up | No | Boolean | Definition OS warmup. Constraints N/A Range Default Value N/A |
| source_name | No | String | Definition Warmup name. Constraints This parameter is mandatory when os_warm_up is set to true. Range The value can contain at most 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. Default Value N/A |
| asset_id | No | String | Definition Preset asset ID. Constraints This parameter is mandatory when preset assets are used. Range The value can contain at most 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. Default Value N/A |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| source | Yes | String | Definition Code source type. Constraints N/A Range Default Value N/A |
| address | No | String | Definition Code source address, which complies with the format of different storage systems. Constraints N/A Range N/A Default Value N/A |
| source_id | No | String | Definition Code source ID. Either this parameter or address must be specified. The SFS Turbo ID can be input only when source is set to EFS. Constraints N/A Range N/A Default Value N/A |
| mount_path | Yes | String | Definition Path mounted to the container. Constraints N/A Range The value must start and end with a slash (/) and can contain at most 255 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. Default Value N/A |
| efs_sub_path | No | String | Definition EFS subpath. Constraints N/A Range N/A Default Value N/A |
| read_only | No | Boolean | Definition Specifies whether the mount permission is read-only. Constraints N/A Range Default Value N/A |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| source | Yes | String | Definition Code source type. Constraints N/A Range The value is case-insensitive. Default Value N/A |
| address | No | String | Definition Code source address, which complies with the format of different storage systems. Constraints N/A Range N/A Default Value N/A |
| mount_path | Yes | String | Definition Path mounted to the container. Constraints N/A Range The value must start and end with a slash (/) and can contain at most 255 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. Default Value N/A |
| host_cache | No | Boolean | Definition Specifies whether the model can be cached locally. This function is disabled by default. Constraints N/A Range Default Value false |
| efs_sub_path | No | String | Definition When the storage type is EFS, subdirectories can be configured. Constraints N/A Range N/A Default Value N/A |
| read_only | No | Boolean | Definition Specifies whether the mount permission is read-only. Constraints N/A Range Default Value N/A |
| os_warm_up | No | Boolean | Definition OS warmup. Constraints N/A Range Default Value N/A |
| source_name | No | String | Definition Warmup name. Constraints This parameter is mandatory when os_warm_up is set to true. Range The value can contain at most 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. Default Value N/A |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| source | Yes | String | Definition Source type of the dump mount directory. Constraints N/A Range Default Value N/A |
| address | No | String | Definition Source address of the dump mount directory. The value can be an HTTPS or OBS address. Constraints N/A Range N/A Default Value N/A |
| mount_path | Yes | String | Definition Path mounted to the container. The value must start with a slash (/) and can contain hyphens (-), backslashes (), underscores (_), periods (.), letters, and digits. Constraints N/A Range N/A Default Value N/A |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| check_method | No | String | Definition: Health check method, which can be HTTP or EXEC (commands). Constraints: N/A Range: Default Value: During creation, if check_method is not specified, the default value HTTP is used. During update, if check_method is not specified, an error will be reported. |
| cmd | No | String | Definition Command line mandatory when the health check method is EXEC. Constraints The value can contain 0 to 1,024 characters and cannot contain the following characters: #~^$|%&*<>()'"[]{} Range N/A Default Value N/A |
| url | No | String | Definition Request address mandatory when the health check mode is HTTP. Constraints The value can contain 0 to 1,024 characters. The first character must be a slash (/), and the subsequent characters can be letters, digits, hyphens (-), underscores (_), slashes (/), and colons (:). Range N/A Default Value N/A |
| protocol | No | String | Definition Connection protocol. The default value is HTTP. Constraints N/A Range Default Value N/A |
| initial_delay_seconds | Yes | Integer | Definition Waiting time before the first detection. The default value is 30 and the minimum value is 1. Constraints N/A Range N/A Default Value 30 |
| timeout_seconds | Yes | Integer | Definition Timeout interval of the detection. The default value is 30 seconds, and the minimum value is 1 second. Constraints N/A Range The minimum value is 1 second. Default Value 30s |
| period_seconds | Yes | Integer | Definition Interval at which the health check is performed. The default value is 10 seconds, and the minimum value is 1 second. Constraints N/A Range The minimum value is 1 second. Default Value 10s |
| failure_threshold | Yes | Integer | Definition The minimum number of consecutive detection failures required after a successful detection. Detection is considered failed only when this threshold is reached. The default value is 3. The minimum value is 1. Constraints N/A Range The minimum value is 1. Default Value 3 |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| affinity_type | Yes | String | Definition Node affinity type. Constraints N/A Range Default Value N/A |
| required | Yes | Boolean | Definition Specifies whether to set strong affinity. Constraints N/A Range Default Value N/A |
| selection_mode | Yes | String | Definition Node selection mode. Constraints N/A Range IP Default Value N/A |
| targets | Yes | Map<String,String> | Definition List selected using the preceding method, whose length cannot exceed 20. Constraints N/A |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| user_id | No | Long | Definition user_id set during service container startup. This parameter can be left empty by default. Range [1, 60000] |
| group_id | No | Long | Definition group_id set during service container startup. This parameter can be left empty by default. Range [1, 60000] |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| deploy_timeout_minutes | No | Integer | Definition Deployment timeout. |
| upgrade_config | Yes | UpgradeConfig object | Definition Rolling upgrade configuration. |
| service_secret | No | ServiceSecret object | Definition Service secret. |
| dynamic_routing_enable | No | Boolean | Definition Intelligent routing switch. |
| strategy | No | String | Definition Intelligent routing policy. |
| ems_enable | No | Boolean | Definition EMS acceleration switch. |
| metric_api_scheme | No | String | Definition Intelligent routing metric collection scheme. |
| metric_api_port | No | String | Definition Intelligent routing metric collection port. |
| metric_api_path | No | String | Definition Intelligent routing metric collection URL. |
| custom_metrics_path | No | String | Definition Custom metric collection URL. |
| mooncake_enable | No | Boolean | Definition Mooncake cache switch. Constraints N/A Range Default Value false |
| port | Yes | Integer | Definition Container port. |
| protocol | Yes | String | Definition Container request protocol. When WSS or WS is selected, the service API is upgraded to WebSocket. WebSocket cannot be enabled together with Traffic Limit. Range |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| type | No | String | Definition Upgrade type. Constraints N/A Range Default Value N/A |
| rolling_update | No | RollingUpdate object | Definition Rolling update configuration. Constraints N/A |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| max_surge | No | String | Definition Maximum number of pods that can be started during rolling update. Constraints A percentage-type string. Range 1% to 100% Default Value 1%. |
| max_unavailable | No | String | Definition Maximum number of pods that can be deleted during rolling update. Constraints A percentage-type string. Range 1% to 100% Default Value 1%. |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| secret_enable | No | Boolean | Definition Specifies whether to enable the key. Constraints N/A Range Default Value false |
| secret_type | No | String | Definition Secret type. Constraints N/A Range Default Value N/A |
| secret_volumes | No | Array of SecretVolume objects | Definition Secret volume mounting. Constraints Maximum: 30 |
| group_enable | No | Boolean | Definition Specifies whether to enable the image user group. Constraints N/A Range Default Value false |
| group_id | No | Long | Definition User group ID of the image. Constraints N/A Range 1000 to 4294967294 Default Value N/A |
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| secret_name | No | String | Definition Secret name. Constraints N/A Range N/A Default Value N/A |
| secret_key | No | String | Definition Secret key. Constraints A string of 1 to 63 characters is matched. Only letters, digits, dots (.), underscores (_), and hyphens (-) are allowed. The string cannot start with two consecutive dots (..). Range N/A Default Value N/A |
| secret_value | No | String | Definition Secret value. Constraints The value is a Base64-encoded secret value and contains 1 to 32,768 characters. Range N/A Default Value N/A |
| mount_path | Yes | String | Definition Mount path. Constraints N/A Range The value must start and end with a slash (/) and can contain at most 255 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. Default Value N/A |
Response Parameters
Status code: 200
| Parameter | Type | Description |
|---|---|---|
| id | String | Definition Deployment ID, which you can obtain from the response body when adding the deployment or by obtaining service deployments. The deployment_id field indicates the deployment ID. Range Deployment ID. |
| name | String | Definition Service deployment name. |
| deployment_name | String | Definition Service deployment name. |
| infer_name | String | Definition Deployment ID (deprecated field). Range N/A |
| description | String | Definition Description. Range N/A |
| pool_id | String | Definition Dedicated resource pool ID. Range N/A |
| framework | String | Definition Algorithm framework. Range |
| version | String | Definition Current deployment version. |
| version_id | String | Definition Version ID, which can be obtained by querying the version list. Range N/A |
| version_count | Integer | Definition Number of service versions. Range N/A |
| count | Integer | Definition Number of service deployment instances. |
| weight | Integer | Definition Weight percentage of the traffic allocated to the model. This parameter is mandatory only when the model is deployed as a real-time service. Range [0,100] |
| priority | Integer | Definition Service priority. Range [1, 3] |
| secret_type | String | Definition Secret type. You can select the secret type (dew or agency) to be used here. Range |
| secret_name | String | Definition Secret name. When using a DEW secret, you can select the secret to be used here. Range The value can contain 1 to 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
| status | String | Definition Service deployment status |
| deploy_type | String | Definition Deployment type. Range |
| workload_type | String | Definition Workload type. Range |
| high_avail_switch | Boolean | Definition HA switch. Once enabled, pods of different instances are evenly distributed to different nodes. Range |
| system_log_dump_enable | Boolean | Definition Specifies whether to enable system log dump. Range N/A |
| schedule_strategy | String | Definition Scheduling policy. Range N/A |
| unit_configs | Array of UnitConfig objects | Definition Instance unit configuration. |
| model | GroupModel object | Definition Model configuration. You can select a model and weight file to use with the image. |
| advanced_config | AdvancedConfig object | Definition Advanced service deployment settings. |
| mirror_traffic_enable | Boolean | Definition Whether mirrored traffic is enabled. Range N/A |
| mirror_traffic_weight | Integer | Definition Mirrored traffic weight. Range N/A |
| user_name | String | Definition Username. Range N/A |
| agency_name | String | Definition Agency name. Range N/A |
| last_used_time | String | Definition Last use time. Range N/A |
| deployment_task_limit | DeploymentTaskLimit object | Definition Deployment task constraints. |
| frozen_infos | Array of FrozenInfo objects | Definition Freezing type returned when a service or deployment is frozen. |
| create_at | Long | Definition Creation time, which is automatically generated. Range Millisecond-level timestamp, which is a 13-digit number. |
| update_at | Long | Definition Update time, which is automatically generated. Range Millisecond-level timestamp, which is a 13-digit number. |
| Parameter | Type | Description |
|---|---|---|
| id | String | Definition Instance unit ID. Range N/A |
| name | String | Definition Role name of the instance unit. |
| role | String | Definition Instance unit role. Range |
| custom_spec | CustomResourceSpec object | Definition Custom resource specifications. |
| flavor | String | Definition Resource specifications. Select specifications based on service requirements. When specification is set to custom, the custom specifications are used. custom_spec specifies deployment specifications. Range N/A |
| flavor_display_name | String | Definition Display name of a resource specification. Range N/A |
| image | ImageInfo object | Definition Image configuration information, including the image path and image source. |
| models | Array of ModelResource objects | Definition This parameter is discarded. You are advised to use files to configure models. Model configuration. You can select a model and weight file to use with the image. |
| codes | Array of Code objects | Definition This parameter is discarded. You are advised to use files to configure code. Code configuration. You can select the OBS path where the code is stored. |
| files | Array of FileInfo objects | Definition Model and code configuration. You can select the model and weight file to work with the image and the OBS path where the code is stored. |
| dumps | Array of Dump objects | Definition Dump configuration. You can select the OBS bucket where the data is to be dumped. |
| count | Integer | Definition Number of configuration instances. Range N/A |
| cpu_affinity | Boolean | Definition Whether to enable CPU pinning. Range |
| cmd | String | Definition Startup command. Range N/A |
| termination_grace | TerminationGrace object | Definition Graceful shutdown configuration. |
| envs | Map<String,String> | Definition Environment variable. |
| readiness_health | Health object | Definition Readiness health check. |
| startup_health | Health object | Definition Health checks are enabled. |
| liveness_health | Health object | Definition Liveness health check. Range N/A |
| port | Integer | Definition Port. Range [1, 65535] |
| recovery | String | Definition Auto-rebuild policy. When enabled, the platform will automatically rebuild pods according to the policy if they restart due to deployment configuration changes or hardware failures. If disabled, the platform will not intervene. Range |
| npu_reset_enable | Boolean | Definition Specifies whether to enable the recovery policy. Range |
| group_count | Integer | Definition Number of replicas in a unit. This parameter is invalid when deploy_type is set to SINGLE or workload_type is set to DEPLOYMENT. Range [1, 100] or left empty. |
| affinity | Affinity object | Definition Node affinity scheduling. |
| security_config | ServiceSecurityConfig object | Definition Container user ID/user group ID. |
| pool_resource_flavor | String | Definition Node pool resource specifications. Range The value can contain a maximum of 128 characters. |
| group_policy | UnitGroupPolicy object | Definition Scheduling constraints for single-replica resource instances. This parameter is valid only for A5 supernodes. Range N/A |
| Parameter | Type | Description |
|---|---|---|
| gpu | Float | Definition Number of GPUs. Range The value can be a decimal. It cannot be smaller than 0, with the third decimal place is rounded off. |
| memory | Integer | Definition Memory size, in MB. Range Only integers are supported. |
| cpu | Float | Definition CPU cores. Range The value can be a decimal. It cannot be smaller than 0.01, with the third decimal place is rounded off. |
| ascend | Integer | Definition Number of Ascend chips. Range N/A |
| arch | String | Definition Architecture type. Range x86/arm64. |
| Parameter | Type | Description |
|---|---|---|
| id | String | Definition Image ID. Range N/A |
| source | String | Definition Image type, which indicates the image source. Range |
| swr_path | String | Definition Image path. The address is different when the source value is different. Range N/A |
| category | String | Definition Flavors supported by the image. Range |
| Parameter | Type | Description |
|---|---|---|
| source | String | Definition Code source type. Range The value is case-insensitive. |
| address | String | Definition Code source address, which complies with the format of different storage systems. Range N/A |
| mount_path | String | Definition Path mounted to the container. The value must start with a slash (/) and can contain hyphens (-), backslashes (), underscores (_), periods (.), letters, and digits. Range The value must start and end with a slash (/) and can contain at most 255 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
| host_cache | Boolean | Definition Specifies whether the model can be cached locally. This function is disabled by default. Range |
| efs_sub_path | String | Definition When the storage type is EFS, subdirectories can be configured. Range N/A |
| read_only | Boolean | Definition Specifies whether the mount permission is read-only. Range |
| os_warm_up | Boolean | Definition OS warmup. Range |
| source_name | String | Definition Warmup name. Range The value can contain at most 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
| asset_id | String | Definition Preset asset ID. Range The value can contain at most 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
| Parameter | Type | Description |
|---|---|---|
| source | String | Definition Code source type. Range |
| address | String | Definition Code source address, which complies with the format of different storage systems. Range N/A |
| source_id | String | Definition Code source ID. Either this parameter or address must be specified. The SFS Turbo ID can be input only when source is set to EFS. Range N/A |
| mount_path | String | Definition Path mounted to the container. Range The value must start and end with a slash (/) and can contain at most 255 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
| efs_sub_path | String | Definition EFS subpath. Range N/A |
| read_only | Boolean | Definition Specifies whether the mount permission is read-only. Range |
| Parameter | Type | Description |
|---|---|---|
| source | String | Definition Code source type. Range The value is case-insensitive. |
| address | String | Definition Code source address, which complies with the format of different storage systems. Range N/A |
| mount_path | String | Definition Path mounted to the container. Range The value must start and end with a slash (/) and can contain at most 255 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
| host_cache | Boolean | Definition Specifies whether the model can be cached locally. This function is disabled by default. Range |
| efs_sub_path | String | Definition When the storage type is EFS, subdirectories can be configured. Range N/A |
| read_only | Boolean | Definition Specifies whether the mount permission is read-only. Range |
| os_warm_up | Boolean | Definition OS warmup. Range |
| source_name | String | Definition Warmup name. Range The value can contain at most 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
| Parameter | Type | Description |
|---|---|---|
| source | String | Definition Source type of the dump mount directory. Range |
| address | String | Definition Source address of the dump mount directory. The value can be an HTTPS or OBS address. Range N/A |
| mount_path | String | Definition Path mounted to the container. The value must start with a slash (/) and can contain hyphens (-), backslashes (), underscores (_), periods (.), letters, and digits. Range N/A |
| Parameter | Type | Description |
|---|---|---|
| pre_stop_cmd | String | Definition: The shutdown command is triggered when the container receives a stop signal, but it must be completed within the grace period or the shutdown times out. If this happens, the container will be forcefully stopped. You can use this command to perform operations such as closing database connections, releasing file handles, and stopping child processes. Range |
| termination_grace_period_seconds | Integer | Definition This parameter indicates the maximum length of time that can pass between when a Pod receives a stop signal to when it is forcefully stopped. It is used for the Pod to perform cleanup operations (such as closing connections, releasing resources, and saving states). Range |
| Parameter | Type | Description |
|---|---|---|
| check_method | String | Definition: Health check method, which can be HTTP or EXEC (commands). Range: |
| cmd | String | Definition Command line mandatory when the health check method is EXEC. Range N/A |
| url | String | Definition Request address mandatory when the health check mode is HTTP. Range N/A |
| protocol | String | Definition Connection protocol. The default value is HTTP. Range |
| initial_delay_seconds | Integer | Definition Waiting time before the first detection. The default value is 30 and the minimum value is 1. Range N/A |
| timeout_seconds | Integer | Definition Timeout interval of the detection. The default value is 30 seconds, and the minimum value is 1 second. Range The minimum value is 1 second. |
| period_seconds | Integer | Definition Interval at which the health check is performed. The default value is 10 seconds, and the minimum value is 1 second. Range The minimum value is 1 second. |
| failure_threshold | Integer | Definition The minimum number of consecutive detection failures required after a successful detection. Detection is considered failed only when this threshold is reached. The default value is 3. The minimum value is 1. Range The minimum value is 1. |
| Parameter | Type | Description |
|---|---|---|
| affinity_type | String | Definition Node affinity type. Range |
| required | Boolean | Definition Specifies whether to set strong affinity. Range |
| selection_mode | String | Definition Node selection mode. Range IP |
| targets | Map<String,String> | Definition List selected using the preceding method, whose length cannot exceed 20. |
| Parameter | Type | Description |
|---|---|---|
| user_id | Long | Definition user_id set during service container startup. This parameter can be left empty by default. Range [1, 60000] |
| group_id | Long | Definition group_id set during service container startup. This parameter can be left empty by default. Range [1, 60000] |
| Parameter | Type | Description |
|---|---|---|
| schedule_scope | String | Definition Scheduling scope. This parameter is valid only for A5 supernodes. Range |
| Parameter | Type | Description |
|---|---|---|
| source | String | Definition Code source type. Range The value is case-insensitive. |
| address | String | Definition Code source address, which complies with the format of different storage systems. Range N/A |
| mount_path | String | Definition Path mounted to the container. The value must start with a slash (/) and can contain hyphens (-), backslashes (), underscores (_), periods (.), letters, and digits. Range N/A |
| host_cache | Boolean | Definition Specifies whether the model can be cached locally. This function is disabled by default. Range |
| efs_sub_path | String | Definition When the storage type is EFS, subdirectories can be configured. Range N/A |
| read_only | Boolean | Definition Specifies whether the mount permission is read-only. Range |
| os_warm_up | Boolean | Definition OS warmup. Range |
| source_name | String | Definition Warmup name. Range The value can contain at most 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
| asset_id | String | Definition Preset asset ID. Range The value can contain at most 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
| Parameter | Type | Description |
|---|---|---|
| deploy_timeout_minutes | Integer | Definition Deployment timeout. |
| upgrade_config | UpgradeConfig object | Definition Rolling upgrade configuration. |
| service_secret | ServiceSecret object | Definition Service secret. |
| dynamic_routing_enable | Boolean | Definition Intelligent routing switch. |
| strategy | String | Definition Intelligent routing policy. |
| ems_enable | Boolean | Definition EMS acceleration switch. |
| metric_api_scheme | String | Definition Intelligent routing metric collection scheme. |
| metric_api_port | String | Definition Intelligent routing metric collection port. |
| metric_api_path | String | Definition Intelligent routing metric collection URL. |
| custom_metrics_path | String | Definition Custom metric collection URL. |
| mooncake_enable | Boolean | Definition Mooncake cache switch. Range |
| port | Integer | Definition Container port. |
| protocol | String | Definition Container request protocol. When WSS or WS is selected, the service API is upgraded to WebSocket. WebSocket cannot be enabled together with Traffic Limit. Range |
| Parameter | Type | Description |
|---|---|---|
| type | String | Definition Upgrade type. Range |
| rolling_update | RollingUpdate object | Definition Rolling update configuration. |
| Parameter | Type | Description |
|---|---|---|
| max_surge | String | Definition Maximum number of pods that can be started during rolling update. Range 1% to 100% |
| max_unavailable | String | Definition Maximum number of pods that can be deleted during rolling update. Range 1% to 100% |
| Parameter | Type | Description |
|---|---|---|
| secret_enable | Boolean | Definition Specifies whether to enable the key. Range |
| secret_type | String | Definition Secret type. Range |
| secret_volumes | Array of SecretVolume objects | Definition Secret volume mounting. |
| group_enable | Boolean | Definition Specifies whether to enable the image user group. Range |
| group_id | Long | Definition User group ID of the image. Range 1000 to 4294967294 |
| Parameter | Type | Description |
|---|---|---|
| secret_name | String | Definition Secret name. Range N/A |
| secret_key | String | Definition Secret key. Range N/A |
| secret_value | String | Definition Secret value. Range N/A |
| mount_path | String | Definition Mount path. Range The value must start and end with a slash (/) and can contain at most 255 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
| Parameter | Type | Description |
|---|---|---|
| max_concurrent_task | Integer | Definition Maximum number of concurrent tasks Range [1, 100] |
Example Requests
Create a real-time inference service.
POST https://{endpoint}/v2/{project_id}/services/{service_id}/deployments
{
"description" : null,
"name" : "deployment-example",
"weight" : 100,
"pool_id" : "pool-example",
"framework" : "COMMON",
"count" : 1,
"advanced_config" : {
"upgrade_config" : {
"rolling_update" : {
"max_surge" : "1%",
"max_unavailable" : "1%"
},
"type" : "ROLLING"
},
"port" : 0,
"protocol" : "HTTPS"
}
} Example Responses
Status code: 200
{
"count" : 1,
"framework" : "COMMON",
"id" : "8a927efb-43f6-4a0b-8222-a1b87d2d2fa8",
"name" : "deployment-example",
"description" : "infer-description",
"pool_id" : "pool-xinfer-1216-dev-04f258c84780d5a52f3bc00dc15aa5e7",
"priority" : 1,
"unit_configs" : [ {
"codes" : [ ],
"count" : 1,
"cpu_affinity" : false,
"custom_spec" : {
"cpu" : 0.1,
"memory" : 50
},
"envs" : {
"additionalProp1" : "string",
"additionalProp2" : "string",
"a" : "1"
},
"id" : "8a0e4ab7970bc24d01970bcd7f460003",
"image" : {
"category" : "CPU",
"id" : "bc84b0e49e910cd7918b1dfaeaa478f9c7e5242c6733525ca88304f026fa24e4",
"source" : "SWR",
"swr_path" : "swr.cn-north-7.myhuaweicloud.com/example/helloworld-example:0.0.1"
},
"npu_reset_enable" : false,
"port" : 65535,
"role" : "COMMON"
} ],
"weight" : 100,
"version" : "0.0.0",
"status" : "DEPLOYING"
} Status Codes
| Status Code | Description |
|---|---|
| 200 | Creating a real-time inference service succeeded. |
Error Codes
See Error Codes.
What is your overall rating for this page?
Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot