Obtaining Fine-tuning Training Job Details
Function
This API is used to obtain details about a specified fine-tuning training job on ModelArts.
It is used when you need to view the details about a specified training job, such as the running status and configuration information. Before using this API, ensure that you have obtained the training job ID and have the permission to view job details. After the query is complete, the platform returns the details about the training job, including the job status, configuration, and logs. If the training job ID does not exist or you do not have the operation permission, the API will return an error message.
Debugging
You can debug this API through automatic authentication in API Explorer or use the SDK sample code generated by API Explorer.
Authorization Information
Each account has all the permissions required to call all APIs, but IAM users must be assigned the required permissions.
- If you are using role/policy-based authorization, see Permissions Policies and Supported Actions for details on the required permissions.
- If you are using identity policy-based authorization, no identity policy-based permission required for calling this API.
URI
GET /v2/{project_id}/training-jobs/{training_job_id}/ft-detail
| Parameter | Mandatory | Type | Description |
|---|---|---|---|
| project_id | Yes | String | Definition: Project ID. For details, see Obtaining a Project ID and Name. Constraints: N/A Range: The value can contain 1 to 64 characters. Only letters, digits, and hyphens (-) are allowed. Default Value: N/A |
| training_job_id | Yes | String | Definition: Training job ID. For details, see Obtaining Training Jobs. Constraints: N/A Range: N/A Default Value: N/A |
Request Parameters
None
Response Parameters
Status code: 200
| Parameter | Type | Description |
|---|---|---|
| project_id | String | Project ID. |
| task_id | String | Training task ID. |
| task_name | String | Training task name. |
| task_desc | String | Training job description. |
| metadata | JobMetadataResponse object | Definition: Training job metadata. |
| spec | SpecResponse object | Definition: Training job specifications. |
| model_asset_id | String | Model ID. |
| model_type | String | Definition Model type. The value can be TextGeneration (text generation) or ImageUnderstanding (image understanding). Range TextGeneration or ImageUnderstanding |
| model_source | String | Model source |
| train_type | String | Definition Training type. The value can be SFT (full fine-tuning), PRETRAIN (pre-training), LORA (LoRA fine-tuning), DPO (DPO reinforcement learning), or RFT (RFT reinforcement learning). Range SFT (full fine-tuning), PRETRAIN (pre-training), LORA (LoRA fine-tuning), DPO (DPO reinforcement learning), or RFT (RFT reinforcement learning) SFT |
| checkpoint_config | String | Resumable training configuration. |
| task_parameters | String | Training task parameters. |
| create_time | Long | Creation time. |
| update_time | Long | Time when a training task is updated. The value is updated when the task status is updated or the description is modified. |
| train_process | Double | Training task progress. |
| datasets_config | Array of DatasetConfig objects | Dataset configuration of the training task. |
| status | Status object | Definition: Training job status. |
| auto_publish_config | String | Automatic publish configuration |
| asset_code | String | Model asset name |
| asset_name | String | Asset name |
| asset_desc | String | Model asset description |
| asset_series | String | Model series |
| asset_version | String | Asset version |
| asset_type | String | Asset type |
| asset_source | String | Asset source |
| asset_group_id | String | Asset group ID |
| sub_asset_type | String | Asset subtype |
| category | String | Asset category |
| api_version | String | Asset API version |
| root_asset_id | String | Root asset ID. |
| train_cost_time | Long | Training task duration |
| workspace_id | String | Name of the workspace to which the task belongs |
| user_id | String | User ID |
| user_name | String | Username |
| pool_type | String | Resource pool type |
| pool_id | String | Resource pool ID |
| pool_node_count | String | Number of used resource pool instances |
| flavor_id | String | Used resource pool PUs |
| priority | Integer | Priority |
| training_info | String | Estimated training duration |
| train_output_path | String | Definition: output path of training artifacts, for example, obs://yyy/test/. Range: N/A |
| asset_capabilities | Array of strings | Training model type |
| continue_task | ContinueTask object | Resumable training task information |
| Parameter | Type | Description |
|---|---|---|
| id | String | Definition: Training job ID, which is generated and returned by ModelArts after a training job is created. Range: N/A |
| name | String | Definition: Name of a training job. Range: The value must contain 1 to 64 characters consisting of only digits, letters, underscores (_), and hyphens (-). |
| workspace_id | String | Definition: Workspace where a specified job is located. Range: N/A |
| description | String | Definition: Definition of a training job. Range: N/A |
| create_time | Long | Definition: Time when a training job was created, in milliseconds. The value is generated and returned by ModelArts after a training job is created. Range: N/A |
| user_name | String | Definition: Username for creating a training job. The username is generated and returned by ModelArts after a training job is created. Range: N/A |
| annotations | Map<String,String> | Definition: advanced function configuration of a training job. The key is the function switch/configuration name, and the value is a string. (Even if the value is a number or Boolean value, it is represented by a string, for example, "true" or "3".) |
| training_experiment_reference | TrainingExperimentResp object | Definition: Training experiment parameters. |
| Parameter | Type | Description |
|---|---|---|
| name | String | Definition: Experiment name. Range: The value can contain a maximum of 128 characters. Special characters are not allowed. |
| id | String | Parameter Explanation: Experiment ID. Value Range: N/A. |
| serial_number | String | Definition: sequence number of the current training job in the training experiment to which the job belongs. The default value is 0. |
| Parameter | Type | Description |
|---|---|---|
| resource | Resource object | Definition: Resource flavor of a training job. Select either flavor_id or pool_id and flavor_id. |
| volumes | Array of JobVolumeResp objects | Definition: Mounting volume information of a training job. |
| log_export_path | LogExportPathResp object | Definition: Log output of a training job. |
| schedule_policy | SchedulePolicyResp object | Definition: Scheduling policy of a training job. |
| custom_metrics | Array of CustomMetrics objects | Definition: Metric collection configuration. |
| output_model | OutputModelResp object | Definition: Output information of a custom training job. |
| asset_model | AssetModelResp object | Definition: Information about the model created using a custom training job. |
| Parameter | Type | Description |
|---|---|---|
| policy | String | Definition: Resource flavor mode of a training job. Range: |
| flavor_id | String | Definition: Resource flavor ID of a training job. flavor_id cannot be specified for CPU-based dedicated resource pools. Range: The options for GPU- or Ascend-based dedicated resource pools are as follows: |
| flavor_name | String | Definition: Read-only flavor name returned by ModelArts when flavor_id is used. Range: N/A |
| node_count | Integer | Definition: Number of resource replicas selected for a training job. Range: greater than or equal to 1 |
| pool_id | String | Definition: ID of the resource pool selected for a training job. Range: N/A |
| pool_group_id | String | Definition: ID of the resource pool federation selected for a training job. Range: N/A |
| flavor_detail | FlavorDetail object | Definition: Flavor details of a training job or algorithm. This parameter is available only for public resource pools. |
| main_container_allocated_resources | Definition: Resource specifications actually obtained by the training container of a training job. | |
| main_container_customized_flavor | Definition: Custom flavor of a training job. |
| Parameter | Type | Description |
|---|---|---|
| flavor_type | String | Definition: Resource flavor type. Range: |
| billing | BillingInfo object | Definition: Billing information of a resource flavor. |
| flavor_info | FlavorInfo object | Definition: Resource flavor details. |
| Parameter | Type | Description |
|---|---|---|
| code | String | Definition: Billing code. Range: N/A |
| unit_num | Integer | Definition: Billing unit. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| max_num | Integer | Definition: Maximum number of nodes that can be selected. The value 1 indicates that the distributed mode is not supported. Range: N/A |
| cpu | Cpu object | Definition: CPU specifications. |
| gpu | Gpu object | Definition: GPU specifications. |
| npu | Npu object | Definition: Ascend specifications. |
| memory | Memory object | Definition: Memory information. |
| disk | Disk object | Definition: Disk information. |
| Parameter | Type | Description |
|---|---|---|
| arch | String | Definition: CPU architecture. Range: N/A |
| core_num | Integer | Definition: Number of cores. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| unit_num | Integer | Definition: Number of GPUs. Range: N/A |
| product_name | String | Definition: Product name. Range: N/A |
| memory | String | Definition: Memory size, in GB. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| unit_num | String | Definition: Number of NPUs. Range: N/A |
| product_name | String | Definition: Product name. Range: N/A |
| memory | String | Definition: Memory. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| size | Integer | Definition: Memory size. Range: N/A |
| unit | String | Definition: Number of memory units. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| size | String | Definition: Disk size. Range: N/A |
| unit | String | Definition: Unit of the disk size. Generally, the unit is GB. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| cpu_arch | String | Definition: CPU architecture. Range: N/A |
| cpu_core_num | Float | Definition: Number of cores. Range: N/A |
| mem_size | Float | Definition: Memory information. Range: N/A |
| accelerator_num | Float | Definition: Number of accelerator cards. Range: N/A |
| accelerator_type | String | Definition: Type of accelerator cards. For example, ascend-d910b and ascend-snt9c. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| cpu_core_num | Float | Definition: Number of CPU cores. Range: greater than 0 |
| mem_size | Float | Definition: Memory size, in GB. Range: greater than 0 |
| accelerator_num | Float | Definition: Number of accelerator cards. Range: greater than or equal to 0 |
| Parameter | Type | Description |
|---|---|---|
| nfs | NfsResp object | Definition: Volumes attached in NFS mode. |
| Parameter | Type | Description |
|---|---|---|
| nfs_server_path | String | Definition: NFS server path, for example, 10.10.10.10:/example/path. Range: N/A |
| local_path | String | Definition: Path for attaching volumes to the training container, for example, /example/path. Range: N/A |
| read_only | Boolean | Definition: Specifies whether the disks attached to the container in NFS mode are read-only. Range: |
| Parameter | Type | Description |
|---|---|---|
| obs_url | String | Definition: OBS path for storing training job logs, for example, obs://example/path. Range: N/A |
| host_path | String | Definition: Path of the host where training job logs are stored, for example, /example/path. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| required_affinity | RequiredAffinityResp object | Definition: Affinity requirements of a training job. |
| priority | Integer | Definition: Priority of a training job. Range: 0 to 3 |
| preemptible | Boolean | Definition: Whether the resource can be preempted. Range: |
| Parameter | Type | Description |
|---|---|---|
| affinity_type | String | Definition: Affinity scheduling policy. Range: |
| job_level | String | Definition: Overall network topology constraint of a job. This parameter is only valid when affinity_type is set to networkTopology. The system schedules all tasks of the job to the node group at the level specified by job_level or lower. When you deliver a training job to a supernode resource pool, if the overall network topology constraint of the job is not set, the system assigns the value cluster by default. Range |
| affinity_group_size | Integer | Definition: Size of an affinity group. Range: N/A |
| affinity_group_level | String | Definition: Network topology constraint of an affinity group. This parameter is only valid when affinity_type is set to networkTopology. The system schedules the affinity group consisting of affinity_group_size tasks to a node group whose level is not higher than affinity_group_level. When you deliver a training job to the supernode resource pool, if the network topology constraints of the affinity group are not set, the system sets the value to hyperinstanceGroup by default. Range |
| Parameter | Type | Description |
|---|---|---|
| exec | Exec object | Definition: Metrics are collected in CLI mode. |
| http_get | HttpGet object | Definition: Metrics are collected in HTTP mode. |
| Parameter | Type | Description |
|---|---|---|
| command | Array of strings | Definition: Metrics are collected in CLI mode. |
| Parameter | Type | Description |
|---|---|---|
| path | String | Definition: URL for obtaining metrics through HTTP. Range: N/A |
| port | Integer | Definition: Port for obtaining metrics through HTTP. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| obs | ObsModelResp object | Definition: OBS output information stored by a custom training job. |
| Parameter | Type | Description |
|---|---|---|
| obs_path | String | Definition: OBS path for storing custom training jobs, for example, obs://example/path. Range: N/A |
| local_path | String | Definition: Path of the host where custom training jobs are stored, for example, /example/path. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| id | String | Definition: Model ID. Range: N/A |
| name | String | Definition: Model name. Range: N/A |
| code | String | Definition: model code. Range: N/A |
| version | String | Definition: Model version. Range: N/A |
| location | String | Definition: Model address. Range: N/A |
| desc | String | Definition: Model description. Range: N/A |
| series | String | Definition: Model brand. Range: N/A |
| type | String | Definition: Model type. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| dataset_name | String | Training dataset name, which can be obtained from the name response body in the dataset list API. |
| dataset_source | String | Source of the used dataset. The value can be datamng (data engineering), OBS, or DB (database). |
| dataset_id | String | Training dataset ID, which can be obtained from the dataset_id response body in the dataset list API. |
| split_ratio | Integer | Splitting ratio of the training and validation datasets. This parameter is used when the model supports the validation dataset and the validation dataset is from the selected training dataset. The value is greater than or equal to 1 and less than or equal to 50. |
| used_step | String | Phase in which the dataset is used. The value can be train (training), eval (verification), or test. |
| dataset_proportion | Integer | Dataset combination, indicating the ratio of the dataset used for training. |
| Parameter | Type | Description |
|---|---|---|
| phase | String | Definition: Level-1 status of a training job. Range: |
| secondary_phase | String | Definition: Level-2 status of a training job. The values are internal detailed statuses and may be added, changed, or deleted. Dependency on the status is not recommended. Range: |
| duration | Long | Definition: Running duration of a training job, in ms. Range: N/A |
| node_count_metrics | Array<Array<Integer>> | Definition: number of nodes during training job running. Each inner array indicates a [time point, number of running nodes] 2-tuple, which records the number of nodes running at a specific time point. |
| tasks | Array of strings | Definition: Training job subtask name. |
| start_time | Long | Definition: training job start time. The value is a Unix timestamp, in milliseconds. Range: N/A |
| task_statuses | Array of TaskStatuses objects | Definition: Status of the first failed subtask of a training job. |
| running_records | Array of RunningRecord objects | Definition: Running and fault recovery records of a training job. |
| retention_time | Integer | Definition: Retention period of a job. reserved_time is set during training job creation. Range: N/A Default value: N/A |
| task_ips | Array of TaskIP objects | Definition: IP addresses of tasks in a training job. host_ips, and only the records that match the specified IP addresses are returned. Range: N/A host_ips is not passed, no value is returned. |
| Parameter | Type | Description |
|---|---|---|
| task | String | Definition: Training job subtask name. Range: N/A |
| exit_code | Integer | Definition: Exit code of a training job subtask. Range: N/A |
| message | String | Definition: Error message of a training job subtask. Range: N/A |
| Parameter | Type | Description |
|---|---|---|
| start_at | Long | Definition: Unix timestamp of the start time in the current running record, in seconds. Range: N/A |
| end_at | Long | Definition: Unix timestamp of the end time in the current running record, in seconds. Range: N/A |
| xpu_start_at | Long | Definition: Unix timestamp of the accelerator card startup time in the current running record, in seconds. Range: N/A |
| start_type | String | Definition: Startup mode of the current execution. Range |
| end_reason | String | Definition: Reason why the running ends. Range: N/A |
| end_related_task | String | Definition: ID of the task worker (for example, worker-0) that ends the running. Range: N/A |
| end_recover | String | Definition: Fault tolerance policy adopted when the execution ends abnormally. Range |
| end_recover_before_downgrade | String | Definition: There is a downgrade relationship between policies. If a policy fails to be executed, it will be downgraded to another specified policy. end_recover_before_downgrade indicates the tolerance policy used before end_recover is downgraded. Range: same as that of end_recover. |
| recover_records | Array of RecoverRecord objects | Definition: details about all fault tolerance policies adopted when the execution ends abnormally. |
| Parameter | Type | Description |
|---|---|---|
| recover_start_at | Long | Definition: Unix timestamp of the start time of the fault tolerance policy, in seconds. The timestamp is also the fault occurrence time. Range: N/A |
| recover_end_at | Long | Definition: Unix timestamp of the end time of the fault tolerance policy, in seconds. Range: N/A |
| recover | String | Definition: Fault tolerance policy. Range |
| fault_scenario | String | Definition: Fault scenario. Range |
| reason | String | Definition: Fault cause. Range: N/A |
| related_task | String | Definition: ID of the task worker (for example, worker-0) that ends the running. Range: N/A |
| recover_result | String | Definition: Fault recovery result. Range |
| Parameter | Type | Description |
|---|---|---|
| task | String | Task name, for example, worker-0. |
| ip | String | Task or pod IP address. |
| host_ip | String | Host IP address. |
| schedule_count | Integer | Number of times the current task is scheduled. The default value is 1. The value increases in scenarios such as rescheduling and preemption. |
| Parameter | Type | Description |
|---|---|---|
| checkpoint_id | String | Breakpoint ID |
| source_model_id | String | ID of the model of a resumable training task |
| source_model_name | String | Name of the model of a resumable training task |
| epoch | Integer | Number of rounds. |
| steps | Integer | Number of steps. |
| is_best | Boolean | Whether the value is optimal |
| skipped_steps | Integer | Number of steps to be skipped. 0 indicates that no step is skipped. |
Status code: 400
| Parameter | Type | Description |
|---|---|---|
| error_msg | String | Error message |
| error_code | String | Error code |
| error_solution | String | Solution |
Example Requests
The following shows how to query a training job whose UUID is 3faf5c03-aaa1-4cbe-879d-24b05d997347.
GET https://{endpoint}/v2/{project_id}/training-jobs/3faf5c03-aaa1-4cbe-879d-24b05d997347/ft-detail Example Responses
Status code: 200
ok
{
"metadata" : {
"id" : "3faf5c03-aaa1-4cbe-879d-24b05d997347",
"name" : "trainjob--py14_mem06-108",
"description" : "",
"create_time" : 1636447346315,
"workspace_id" : "0",
"user_name" : ""
},
"status" : {
"phase" : "Abnormal",
"secondary_phase" : "CreateFailed",
"duration" : 0,
"start_time" : 0,
"node_count_metrics" : [ [ 1636447746000, 0 ], [ 1636447755000, 0 ], [ 1636447756000, 0 ] ],
"tasks" : [ "worker-0" ],
"running_records" : [ {
"start_at" : 1701327093,
"end_at" : 1701322341,
"start_type" : "init_or_rescheduled",
"end_recover" : "job_reschedule",
"end_reason" : "exit with 127",
"end_related_task" : "worker-2",
"end_recover_before_downgrade" : "npu_proc_restart"
}, {
"start_at" : 1701323345,
"end_at" : 1701325432,
"start_type" : "init_or_rescheduled",
"end_reason" : "job completed"
} ]
},
"spec" : {
"resource" : {
"flavor_id" : "modelarts.vm.pnt1.large.eco",
"node_count" : 1,
"flavor_detail" : {
"flavor_type" : "GPU",
"billing" : {
"code" : "modelarts.vm.gpu.pnt1.eco",
"unit_num" : 1
},
"flavor_info" : {
"cpu" : {
"arch" : "x86",
"core_num" : 8
},
"gpu" : {
"unit_num" : 1,
"memory" : "8GB"
},
"memory" : {
"size" : 64,
"unit" : "GB"
}
}
},
"main_container_allocated_resources" : {
"cpu_arch" : "x86",
"cpu_core_num" : 5,
"mem_size" : 44,
"accelerator_num" : 1,
"accelerator_type" : "nvidia-v100-pcie32"
}
},
"custom_metrics" : [ {
"exec" : {
"command" : [ "cat", "/a/b/c.prom" ]
}
}, {
"http_get" : {
"path" : "/raw_text",
"port" : 10001
}
} ]
},
"model_asset_id" : "14f39822-d31a-4ef1-b990-b606fe99496b"
} Status code: 400
Format of the body for a common error response. The following shows the returned information when a training job with ID 3f5d6706-7b67-408d-8ba0-ec08048c45ee is not found.
{
"error_msg" : "Job not found.",
"error_code" : "ModelArts.2755",
"error_solution" : "Check whether the training job in the request is valid."
} Status Codes
| Status Code | Description |
|---|---|
| 200 | ok |
| 400 | Format of the body for a common error response. The following shows the returned information when a training job with ID 3f5d6706-7b67-408d-8ba0-ec08048c45ee is not found. |
Error Codes
See Error Codes.
What is your overall rating for this page?
Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot