Creating a Batch Object Storage Migration Workflow

Prerequisites

You have created a migration plan that includes source buckets to be migrated together.

Procedure

1. Log in to the MgC console.
2. In the navigation pane on the left, choose Migrate > Workflows. Select a migration project in the upper left corner of the page.
3. Click Create Workflow in the upper right corner of the page.
4. In the Batch Object Storage Migration card, click Preview Steps to view the stages and steps predefined in the template and the detailed description of each stage and step. Steps of the Automated type will be automatically performed by MgC. Click Configure Workflow in the lower right corner.
5. In the Basic Information area, enter a name and description for the workflow.
6. In the Migration Plan area, select the created object migration plan. Then you should view the overview of the migration plan. Click View Details to view more information about the plan.

   

7. In the Migration Cluster area, select the cluster used for the migration. The cluster selected in the migration plan is used by default, but you can select another one if needed. The modification is applied to the current workflow but not to the migration plan.
8. Configure the migration task based on Table 1.

   Table 1 Parameters for configuring a migration task

   Parameter	Value	Description
   Concurrent Subtasks	-	Set the maximum number of concurrent subtasks. The value cannot exceed the number of online migration nodes multiplied by 20. For example, if there are 2 online migration nodes, the maximum number of subtasks can be 40 or any number below.
   Overwrite Existing	Never	Files existing at the migration target are never overwritten.
   	Always	Files existing at the migration target are always overwritten.
   	If older or different size	Files existing at the migration target are overwritten if they are older than or have different sizes from files at the migration source.
   Migrate Metadata	-	Decide whether to migrate metadata. If you select this option, object metadata will be migrated. If you do not select this option, only the ContentType metadata will be migrated.
   Clear Cluster	-	Determine whether to clear the migration cluster after the migration is complete. If you select this option, a step for clearing the migration cluster will be created in the workflow. You can also choose whether to clear resources used by the cluster, such as NAT gateways, security groups, and VPCEP resources. If you do not select this option, a step for clearing the migration cluster will not be created in the workflow.

9. (Optional) Configure advanced options based on Table 2.

   Table 2 Advanced options

   Parameter	Description
   Enable KMS Encryption	If you do not select this option, objects are in the same encryption status before and after the migration. If you select this option, all migrated objects will be encrypted before they are stored in the target bucket. NOTE: Using KMS to encrypt migrated data may slow down the migration speed by about 10%. This option is only available when KMS is supported in the region you are migrating to.
   Restore Archive Data	Only restored data can be migrated. You can select this option if the source cloud platform supports automatic restoration of archive data. Currently, the following cloud platforms can automatically restore archive objects: Huawei Cloud, Alibaba Cloud, Kingsoft Cloud, and Tencent Cloud. If you do not select this option, the system records archive objects in the list of objects that failed to be migrated and continues to migrate other objects in the migration task. If you select this option, the system automatically restores and migrates archive objects in the migration task. If an archive object fails to be restored, the system skips it and records it in the list of objects that failed to be migrated and continues to migrate other objects in the migration task. NOTE: The system will restore archive data before migrating it, and you pay the source cloud platform for the API requests and storage space generated accordingly.
   Filter Source Data	Filter files to be migrated by applying filters. For details about the filters, see Source Data Filters.
   Download Data from CDN	If the default domain name cannot meet your migration requirements, then as long as the source cloud service provider supports custom domain names, you can bind a custom domain name to the source bucket, and enable the CDN service on the source platform to reduce data download expenses. Enter a custom domain name in the Domain Name text box and select a transmission protocol. HTTPS is more secure than HTTP and is recommended. If the migration source is the Alibaba Cloud OSS or Tencent Cloud COS, you also need to select an authentication type and enter an authentication key.
   Send SMN Notification	Determine whether to use SMN to get notifications about migration results. If you do not select this option, no SMN messages are sent after the migration. If you select this option, after the migration, SMN messages are sent to the subscribers of the selected topic. You can select the language and trigger conditions for sending messages.
   Limit Traffic	Allocate the maximum bandwidth to be used by the workflow during a specified period. If you do not select this option, migration traffic is not limited. If you select this option, limit the migration traffic by setting Start Time, End Time, and Bandwidth Limit. For example, if you set Start Time to 08:00, End Time to 12:00, and Bandwidth Limit to 20 MB/s, the maximum migration speed is limited to 20 MB/s when the migration task is running from 08:00 to 12:00. The migration speed is not limited beyond this period. NOTE: The rate limit ranges from 0 MB/s to 1,048,576 MB/s. A maximum of five rules can be added. The time is the local standard time of the region you are migrating to.

10. Click Next: Confirm.
11. Confirm the workflow settings, and click Confirm. The Run Workflow dialog box is displayed, which indicates that the workflow has been created.
    • If you want to start the migration immediately, click Confirm to run the workflow.
    • If you want to add a stage or step to the workflow, click Cancel. The workflow enters a Waiting state, and the migration is not started. To start the migration, click Run in the Operation column.

12. On the migration workflow details page, view the workflow settings and the migration progress. You can also perform the following operations:
    • Move the cursor to the migration progress bar of a resource. In the displayed window, view the migration details about the resource.
    • When a migration reaches a step that requires manual confirmation, place the cursor on the progress bar and click Confirm next to the step status in the displayed window. The migration can continue only after you confirm.
    • In the Basic Information area, click Manage next to the cluster name. The cluster details page is displayed on the right. On the displayed page, you can:
      • Add, edit, or delete traffic limiting rules to control cluster traffic based on your requirements.
      • Add or delete migration nodes, list nodes, or upgrade plug-ins for existing nodes as required.