Help Center/ Object Storage Migration Service/ API Reference/ APIs V2 (Recommended)/ Migration Task Group Management/ Creating a Migration Task Group

Updated on 2024-12-30 GMT+08:00

View PDF

Creating a Migration Task Group

Function

This API is used to create a migration task group. After a migration task group is created, migration tasks are automatically created in the task group. You do not need to manually call APIs to create migration tasks.

Calling Method

For details, see Calling APIs.

URI

POST /v2/{project_id}/taskgroups

**Table 1** Path parameters
Parameter	Mandatory	Type	Description
project_id	Yes	String	The project ID. Minimum length: 1 character Maximum length: 1,024 characters

Request

**Table 2** Request header parameters
Parameter	Mandatory	Type	Description
X-Auth-Token	Yes	String	The token used for IAM authentication. Minimum length: 1 character Maximum length: 16,384 characters

**Table 3** Request body parameters
Parameter	Mandatory	Type	Description
src_node	Yes	TaskGroupSrcNode object	The source information.
description	No	String	The task group description, which cannot exceed 255 characters. The following special characters are not allowed: ^<>&"' Minimum length: 0 characters Maximum length: 255 characters
dst_node	Yes	TaskGroupDstNode object	The destination information.
enable_metadata_migration	No	Boolean	Indicates whether metadata migration is enabled. The default value is false. Even if this function is disabled, the ContentType metadata will still be migrated to ensure a successful migration. Default value: false
enable_failed_object_recording	No	Boolean	Indicates whether to record the objects that failed to be migrated. Default value: true
enable_restore	No	Boolean	Indicates whether archive data restoration is enabled. If this function is enabled, archive data is automatically restored and migrated.
enable_kms	Yes	Boolean	Indicates whether KMS encryption is enabled. This function is disabled by default. Default value: false
task_type	No	String	The task type. The default value is PREFIX. LIST indicates that the system will migrate the objects specified in the object list. URL _LIST indicates that the system will migrate the objects specified in the URL list. PREFIX indicates that the system will migrate the objects with a specific prefix. The value can be: LIST URL_LIST PREFIX
bandwidth_policy	No	Array of BandwidthPolicyDto objects	The traffic limiting rules. Each element in the array corresponds to the maximum bandwidth in a time period. A maximum of five periods are allowed, and the periods cannot overlap. Array length: 0 to 5
smn_config	No	SmnConfig object	The SMN message sending configuration.
source_cdn	No	SourceCdnReq object	Indicates whether migration from CDN is enabled. If this field is contained, migration from CDN is supported. Otherwise, it is not supported. If migration from CDN is enabled, the source objects to be migrated are obtained from the specified CDN domain name during migration.
migrate_since	No	Long	A timestamp in second. The system migrates only the objects that are modified after the specified time. No time is specified by default. Minimum value: 0 Maximum value: 9223372036854775807 Default value: 0
object_overwrite_mode	No	String	Indicates whether to skip a source object or allow the source object to overwrite its paired destination object. The default value is SIZE_LAST_MODIFIED_COMPARISON_OVERWRITE. NO_OVERWRITE indicates the system never allows overwrite. The system always skips source objects and keeps their paired destination objects. SIZE_LAST_MODIFIED_COMPARISON_OVERWRITE indicates the system allows overwrite based on the results of size or modification time checks. It is the default setting. If a source object is not as large as or was last modified more recently than its paired destination object, the source object will overwrite the destination object. Otherwise, the source object will be skipped. CRC64_COMPARISON_OVERWRITE indicates the system allows overwrite if the source and destination objects have different CRC64 checksums. This option is only available for migration on Huawei Cloud or from Alibaba Cloud or Tencent Cloud. If a source object has a CRC64 checksum different from the paired destination object, the source object will overwrite the destination object. Otherwise, the source object will be skipped. If one of them does not have a CRC64 checksum, their sizes and last modification times are checked. FULL_OVERWRITE indicates the system always allows overwrite. The system always allows source objects to overwrite their paired destination objects. The value can be: NO_OVERWRITE SIZE_LAST_MODIFIED_COMPARISON_OVERWRITE CRC64_COMPARISON_OVERWRITE FULL_OVERWRITE
dst_storage_policy	No	String	The destination storage class. STANDARD indicates the Huawei Cloud OBS Standard. IA indicates the Huawei Cloud OBS Infrequent Access. ARCHIVE indicates the Huawei Cloud OBS Archive. DEEP_ARCHIVE indicates the Huawei Cloud OBS Deep Archive. SRC_STORAGE_MAPPING converts the source storage class into an OBS storage class based on the predefined rules. Default value: STANDARD Minimum length: 0 characters Maximum length: 128 characters The value can be: STANDARD IA ARCHIVE DEEP_ARCHIVE SRC_STORAGE_MAPPING
consistency_check	No	String	The consistency check method, which is used to check whether objects are consistent before and after migration. The selected check method and results will be recorded in the object list. The default value is size_last_modified. size_last_modified is the default setting. The system checks object consistency with object size and last modification time. If a source object is as large as but was last modified earlier than its paired destination object, the system considers the source object does not need to be migrated or has been already migrated successfully. crc64 is only available for migration on Huawei Cloud or from Alibaba Cloud or Tencent Cloud. If a source object and its paired destination object have CRC64 checksums, the checksums are checked. Otherwise, their sizes and last modification times are checked. no_check is only available for migration of HTTP/HTTPS data. This option takes effect for source objects whose sizes cannot be obtained using the content-length field in the standard HTTP protocol. These source objects will overwrite their paired destination objects directly. If the size of a source object can be obtained from the Content-Length field in the standard HTTP protocol, its size and last modification time will be checked. Default value: size_last_modified The value can be: size_last_modified crc64 no_check
enable_requester_pays	No	Boolean	Indicates whether to let the requester make payment. After this function is enabled, the requester pays for the request and data transmission. Default value: false

**Table 4** **TaskGroupSrcNode** field description
Parameter	Mandatory	Type	Description
ak	No	String	The AK for accessing the source bucket. This parameter is mandatory when task_type is not url_list. Minimum length: 0 characters Maximum length: 100 characters
sk	No	String	The SK for accessing the destination bucket. This parameter is mandatory when task_type is not url_list. Minimum length: 0 characters Maximum length: 100 characters
json_auth_file	No	String	The file used for Google Cloud Storage authentication. Minimum length: 0 characters Maximum length: 65,535 characters
app_id	No	String	This parameter is mandatory when cloud_type is Tencent. Minimum length: 0 characters Maximum length: 255 characters
region	No	String	The region where the source bucket is located. region is mandatory when task_type is not URL_LIST. Minimum length: 0 characters Maximum length: 100 characters
object_key	No	Array of strings	The name prefixes of objects to be migrated if task_type is set to prefix. If you want to migrate the entire bucket, set this parameter to [""]. Minimum length: 0 characters Maximum length: 1,024 characters Array length: 0 to 500
bucket	No	String	The source bucket. Minimum length: 0 characters Maximum length: 1,024 characters
cloud_type	No	String	The source cloud service provider. If task_type is URL_LIST, this parameter is mandatory and must be URLSource. The value can be AWS, Azure, Aliyun, Tencent, HuaweiCloud, QingCloud, KingsoftCloud, Baidu, Qiniu, URLSource, Google, or UCloud. The default value is Aliyun. Default value: Aliyun Minimum length: 0 characters Maximum length: 100 characters
list_file	No	ListFile object	The list file configuration. list_file is mandatory when task_type is set to LIST or URL_LIST.

**Table 5** **ListFile** field description
Parameter	Mandatory	Type	Description
list_file_key	Yes	String	The object names in the object list file or URL list file. Minimum length: 1 character Maximum length: 1,024 characters
obs_bucket	Yes	String	The name of the OBS bucket for storing the object list files. Ensure that the OBS bucket is in the same region as the destination bucket, or the task will fail to be created. Minimum length: 0 characters Maximum length: 1,024 characters
list_file_num	No	String	The number of stored object list files. Minimum length: 0 characters Maximum length: 65,535 characters

**Table 6** **TaskGroupDstNode** field description
Parameter	Mandatory	Type	Description
ak	Yes	String	The AK for accessing the destination bucket. Minimum length: 0 characters Maximum length: 100 characters
sk	Yes	String	The SK for accessing the destination bucket. Minimum length: 0 characters Maximum length: 100 characters
region	Yes	String	The region where the destination bucket is located. The value must be the same as that of the service endpoint. Minimum length: 0 characters Maximum length: 100 characters
bucket	Yes	String	The destination bucket. Minimum length: 0 characters Maximum length: 1,024 characters
cloud_type	No	String	The destination cloud. Huawei Cloud is the destination by default. The default value is HEC. Default value: HEC Minimum length: 0 characters Maximum length: 100 characters
save_prefix	No	String	The path prefix in the destination bucket. The prefix is added before an object key to form a new key. The length of the new key cannot exceed 1,024 characters. Minimum length: 0 characters Maximum length: 1,024 characters

**Table 7** **BandwidthPolicyDto** field description
Parameter	Mandatory	Type	Description
end	Yes	String	The end time of the traffic limiting rule. The format is hh:mm, for example, 12:03.
max_bandwidth	Yes	Long	The maximum traffic bandwidth allowed in the specified time period. The unit is byte/s. The value ranges from 1,048,576 bytes/s (equivalent to 1 MB/s) to 209,715,200 bytes/s (equivalent to 200 MB/s). Minimum value: 1048576 Maximum value: 209715200
start	Yes	String	The start time of the traffic limiting rule. The format is hh:mm, for example, 12:03.

**Table 8** **SmnConfig** field description
Parameter	Mandatory	Type	Description
language	No	String	The management console language chosen by you. The value can be zh-cn or en-us. Default value: en-us The value can be: zh-cn en-us
topic_urn	Yes	String	The SMN message topic URN bound to a migration task. Minimum length: 0 characters Maximum length: 1,024 characters
trigger_conditions	Yes	Array of strings	The trigger conditions of sending messages using SMN. The message varies depending on the status of the migration task. The migration task state can be SUCCESS or FAILURE. FAILURE: indicates that an SMN message will be sent after the migration task fails. SUCCESS: indicates that an SMN message will be sent after the migration task succeeds. Array length: 1 to 2
message_template_name	No	String	If this parameter is specified, SMN messages are sent using the specified template. Template example: {"Task_Status": "", "Task_Name": "", "Start_Time": "", "Total_Time_Used": "", "Transferred_Data": "", "Average_Speed": "", "Source_Bucket": "", "Destination_Bucket": "", "List_File_Bucket": "", "List_File_Key": "", "Success_object_list_path": "", "Skip_object_list_path": "", "Failed_object_list_path": "" }

**Table 9** **SourceCdnReq** field description
Parameter	Mandatory	Type	Description
authentication_key	No	String	The CDN authentication key. If the CDN authentication is required, this parameter is mandatory. If no authorization is required, do not set this parameter. Qiniu: You do not need to set this parameter. Aliyun: Set this parameter based on the value of authentication_type. KingsoftCloud: You do not need to set this parameter. Minimum length: 0 characters Maximum length: 255 characters
authentication_type	Yes	String	The authentication type. Default value: NONE The value can be: NONE QINIU_PRIVATE_AUTHENTICATION ALIYUN_OSS_A ALIYUN_OSS_B ALIYUN_OSS_C KSYUN_PRIVATE_AUTHENTICATION AZURE_SAS_TOKEN TENCENT_COS_A TENCENT_COS_B TENCENT_COS_C TENCENT_COS_D
domain	Yes	String	The domain name from which to obtain objects to be migrated. Minimum length: 0 characters Maximum length: 1,024 characters
protocol	Yes	String	The protocol type. The value can be: HTTP HTTPS

Response

Status code: 200

**Table 10** Response body parameters
Parameter	Type	Description
group_id	String	The ID of the created migration task group. Minimum length: 1 character Maximum length: 100 characters

Status code: 400

**Table 11** Response body parameters
Parameter	Type	Description
error_msg	String	The error message.
error_code	String	The error code.

Status code: 403

**Table 12** Response body parameters
Parameter	Type	Description
error_msg	String	The error message.
error_code	String	The error code.

Status code: 500

**Table 13** Response body parameters
Parameter	Type	Description
error_msg	String	The error message.
error_code	String	The error code.

Example Request

This example creates a migration task group to migrate objects listed in TXT object list files stored in the hw/ directory to the destination bucket zzww.

POST https://{endpoint}/v2/{project_id}/taskgroups

{
  "enable_failed_object_recording" : true,
  "src_node" : {
    "bucket" : "hw-src",
    "cloud_type" : "HuaweiCloud",
    "ak" : "******",
    "sk" : "******",
    "region" : "cn-north-7",
    "list_file" : {
      "obs_bucket" : "list-file-bucket",
      "list_file_key" : "hw/"
    }
  },
  "enable_kms" : false,
  "task_type" : "LIST",
  "enable_restore" : false,
  "dst_node" : {
    "bucket" : "zzww",
    "cloud_type" : "HEC",
    "ak" : "******",
    "sk" : "******",
    "region" : "cn-north-7"
  },
  "enable_metadata_migration" : true
}

This example creates a migration task group to migrate objects prefixed with Objectlist from the source bucket src-bucket to the destination bucket dst-bucket.

POST https://{endpoint}/v2/{project_id}/taskgroups

{
  "src_node" : {
    "region" : "cn-north-4",
    "ak" : "******",
    "sk" : "******.",
    "bucket" : "src-bucket",
    "cloud_type" : "HuaweiCloud",
    "object_key" : [ "Objectlist" ]
  },
  "dst_node" : {
    "region" : "cn-north-4",
    "ak" : "******",
    "sk" : "******",
    "bucket" : "dst-bucket"
  },
  "task_type" : "PREFIX",
  "enable_failed_object_recording" : true,
  "enable_kms" : false,
  "bandwidth_policy" : [ {
    "max_bandwidth" : 1048576,
    "start" : "00:01",
    "end" : "23:59"
  } ]
}

This example creates a migration task group to migrate objects listed in the TXT URL list files stored in the test01/ directory to the destination bucket jjyy.

POST https://{endpoint}/v2/{project_id}/taskgroups

{
  "src_node" : {
    "cloud_type" : "URLSource",
    "list_file" : {
      "obs_bucket" : "url-file-bucket",
      "list_file_key" : "test01/"
    }
  },
  "description" : "",
  "dst_node" : {
    "region" : "cn-north-4",
    "ak" : "******",
    "sk" : "******.",
    "bucket" : "jjyy",
    "cloud_type" : "HEC"
  },
  "enable_failed_object_recording" : true,
  "enable_restore" : false,
  "enable_metadata_migration" : false,
  "enable_kms" : false,
  "task_type" : "URL_LIST"
}

Example Response

Status code: 200

Created

{
  "group_id" : "9ad15f51-778f-4684-b47b-4c5109c0e105"
}

Status Codes

Status Code	Description
200	Created
400	Bad Request
403	Forbidden
500	Server Error

Error Codes

For details, see Error Codes.

Parent topic: Migration Task Group Management

Previous topic: Listing Migration Task Groups

Next topic: Querying a Migration Task Group

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel