Creating a Batch File Storage Migration Workflow

Notes and Constraints

For more information, see What Are the Restrictions on Using MgC for Storage Migration?

Warnings

When you create a workflow, there are three options for Overwrite Existing: Never, Always, and If older or different size. If you choose Never, restarting the migration task after interruptions or pauses may cause incomplete data migration, but the task may still be displayed as successful, which affects data integrity. Use the Never option with caution.

Prerequisites

You have created a batch file storage migration plan.

Procedure

1. Sign in to the MgC console.
2. In the navigation pane on the left, choose Migrate > Workflows. Select a migration project in the upper left corner of the page.
3. Click Create Workflow in the upper right corner of the page.
4. In the Batch File Storage Migration card, click Preview Steps to view the stages and steps predefined in the template and the detailed description of each stage and step. Steps of the Automated type will be automatically performed by MgC. Click Configure Workflow in the lower right corner.

   

5. In the Basic Information area, enter a name and description for the workflow.
6. In the Migration Plan area, select the created migration plan. Then you should view the overview of the migration plan. Click View Details to view more information about the plan.

   

7. In the Migration Cluster area, select the cluster used for the migration. The cluster specified in the migration plan is preselected by default, but you can select another one if needed. The modification is applied to the current workflow but not to the migration plan.
8. Configure the migration task based on Table 1.

   Table 1 Parameters for configuring a migration task

   Parameter	Value	Description
   Concurrent Subtasks	-	Set the maximum number of concurrent subtasks. There cannot be more than 10 concurrent subtasks for each online migration node. For example, if there are 2 online migration nodes, the maximum number of subtasks can be 20 or any number below.
   Overwrite Existing	Never	Files existing at the migration target are never overwritten. WARNING: If you choose Never for the initial migration, the attributes of involved parent folders at the source will not be migrated to the target. As a result, the folder attributes may be incomplete at the target. To avoid this issue, use the Never option with caution for the initial migration. If a migration task is paused or interrupted and then restarted or resumed, the Never option will cause the system to skip files that were not completely migrated earlier, but the task may still be marked as successful. This affects data integrity. To avoid this issue, use the Never option with caution.
   	Always	Files existing at the migration target are always overwritten.
   	If older or different size	Files that already exist at the target will be overwritten if they are older than or have different sizes from the paired files at the source. Verification will be performed for folders after their contents are migrated. Folders that already exist at the target will be overwritten if they have different last modification times, sizes, or permissions from the paired folders at the source. NOTE: For empty folders, the overwrite policy is the same as that for files.
   Migrate Metadata	-	Decide whether to migrate metadata. If you select this option, object metadata will be migrated. If you do not select this option, only the Content-Type and Content-Encoding metadata will be migrated.

9. (Optional) Configure advanced options based on Table 2.

   Table 2 Advanced options

   Parameter	Description
   Sync Incremental Before Cutover	If you select Yes, an incremental migration step will be automatically added after the migration verification step in the migration phase. If you select No, the service cutover phase starts after the migration verification step in the migration phase is complete.
   Filter Source Data	Filter files and directories to be migrated by applying filters. For details about the filters, see Source Data Filters.
   Send SMN Notification	Determine whether to use SMN to get notifications about migration results. If you do not select this option, no SMN messages will be sent after the migration is complete. If you select this option, after the migration is complete, SMN messages are sent to the subscribers of the selected topic. You can select the language and trigger conditions for sending messages.
   Limit Traffic	Allocate the maximum bandwidth to be used by the workflow during a specified period. If you do not select this option, migration traffic is not limited. If you select this option, limit the migration traffic by setting Start Time, End Time, and Bandwidth Limit. For example, if you set Start Time to 08:00, End Time to 12:00, and Bandwidth Limit to 20 MB/s, the maximum migration speed is limited to 20 MB/s when the migration runs from 08:00 to 12:00. The migration speed is not limited beyond this period. NOTE: The bandwidth limit ranges from 0 MB/s to 1,048,576 MB/s. A maximum of five rules can be added. The time is the local standard time of the region you are migrating to.

10. Click Next: Confirm.
11. Confirm the workflow settings, and click Confirm. The Run Workflow dialog box is displayed, which indicates that the workflow has been created.
    • If you want to start the migration immediately, click Confirm to run the workflow.
    • If you want to add a stage or step to the workflow, click Cancel. The workflow enters a Waiting state, and the migration is not started. To start the migration, click Run in the Operation column.

12. In the workflow list, click the workflow name to go to its details page. You can view the configuration information and migration progress of the workflow. You can also perform the following operations:
    • Move the cursor to the migration progress bar of a resource. In the displayed window, view the migration details about the resource.
    • When a migration reaches a step that requires manual confirmation, place the cursor on the progress bar and click Confirm next to the step status in the displayed window. The migration can continue only after you confirm.
    • In the Basic Information area, click Manage next to the migration cluster name. The cluster details page is displayed on the right. On the displayed page, you can:
      • Add, edit, or delete traffic limiting rules to control cluster traffic based on your requirements.
      • Add or delete migration nodes or list nodes, or upgrade plug-ins for existing nodes as required.