Help Center/ Migration Center/ User Guide/ New Edition/ Migration Plans (New)/ Creating a Batch Object Storage Migration Plan
Updated on 2025-09-12 GMT+08:00

Creating a Batch Object Storage Migration Plan

You can configure migration plans using templates designed for different migration scenarios. Then you can create migration workflows on the top of these plans. Follow this section to configure a migration plan for batch migrating object storage buckets.

The Batch Object Storage Migration template is designed to migrate buckets in batches. If only one bucket needs to be migrated, create a migration workflow directly.

Prerequisites

  • Source buckets have been discovered over the Internet and their details have been collected. For details, see Cloud Discovery.
  • (Optional) A deep collection has been performed for the buckets to be migrated.

Warnings

When creating a workflow, there are four overwriting policies for objects with the same name at the source and target: If older or different size, Never, Always, and If different CRC64 checksum. If you choose Never, restarting a migration after an interruption or pause may lead to incomplete data migration, even though the task may appear successful. This could impact data integrity, so use the Never option with caution.

Constraints

  • Migration plans can be created only in the AP-Singapore and TR-Istanbul regions.
  • A migration plan can be used to migrate buckets from the same source platform and to the same target region.
  • A source bucket can be included in multiple migration plans.
  • You can define the migration scope for a source bucket using multiple prefixes, but only one prefix can be specified to control data replacement in the target bucket.
  • Each user is allowed to migrate a maximum of 20 buckets at the same time in a single migration cluster.

Step 1: Create a Migration Plan

  1. Sign in to the MgC console. In the navigation pane, under Project, select an application migration project from the drop-down list.
  2. In the navigation pane, choose Migration Plans (NEW).
  3. Click Create Migration Plan.
  4. Set migration plan parameters based on Table 1.

    Table 1 Parameters for creating a migration plan

    Parameter

    Configuration

    Migration Plan Template

    Select Batch Object Storage Migration.

    Plan Name

    The system assigns a default plan name. You can also specify a plan name.

    Description (Optional)

    Enter a description.

  5. Click Confirm. The migration plan for batch bucket migration is created, and the page for configuring source and target resources is displayed.

Step 2: Configure Migration Resources

  1. On the Configure Source/Target page, select the source cloud platform from the Source Platform drop-down list. A migration plan can be used to migrate buckets from the same source platform and to the same target region.
  2. Select the region where the target buckets are located from the Target Region drop-down list.
  3. Above the source bucket list, click Add.
  4. Select the buckets to be migrated, click Modify in the Operation column, configure endpoints, and click Confirm. A maximum of 100 buckets can be added.
  5. In the Buckets area, set the bucket type, source credential, target bucket, and target credential. After the settings are complete, click Save in the Operation column. You can also click Save All after setting multiple buckets. You can modify the settings after saving them.
  6. After you configure all the buckets, click Next.

Step 3: Configure Migration Settings

  1. In the Migration Settings area, set the migration method, migration scope, and (optional) target prefix. Refer to Table 2 to learn how to configure Migration Method and Migration Scope.

    Table 2 Migration methods and scope

    Migration Method

    Description

    Migration Scope

    Full migration

    Migrates all data in a source bucket or in specific paths.

    N/A

    Prefix migration

    Migrates objects matched with the specified names or prefixes.

    There are two methods to specify source prefixes. If no source prefix is specified, all objects are migrated.

    • Import Source Prefixes: Import a TXT file that lists prefixes.
    • Add Source Prefixes: Enter prefixes.

    List migration

    Migrates objects recorded in the list files. The list files must be stored in the target bucket.

    (Mandatory) Enter the path for storing the list files in the Migration Scope box and end the path with a slash (/).

  2. (Optional) In the Source Settings area, select Migrate Metadata and Restore Archive Data.

    If the source is Alibaba Cloud OSS or Tencent Cloud COS, you can enable Download Data from CDN. If CDN authentication is enabled, you also need to select an authentication type and enter an authentication key.

  3. In the Target Settings area, set parameters according to Table 3.

    Table 3 Target settings

    Parameter

    Sub-parameter

    Description

    Overwrite Existing

    If older or different size

    • The system replaces existing target files if they are older than or differ in size from their source counterparts. Files with matching modification times and sizes remain unchanged and are skipped from migration.
    • The system verifies folders after their content is migrated. Folders that already exist at the target will be overwritten if they have different last modification times, sizes, or permissions from the paired folders at the source.

    Never

    Files existing at the target will never be overwritten.

    WARNING:

    If you choose Never, restarting a migration after an interruption or pause may lead to incomplete data migration, even though the task may appear successful. This could impact data integrity, so use the Never option with caution.

    Always

    Files existing at the target will always be overwritten.

    If different CRC 64 checksum

    If a source object has a CRC64 checksum different from the paired target object, the source object will overwrite the target one. Otherwise, the source object will be skipped during the migration. If either of them does not have a CRC64 checksum, their sizes and last modification times are checked.

    NOTE:
    • This option is only available for migration within Huawei Cloud or from Alibaba Cloud or Tencent Cloud.
    • Using this option requires that target OBS buckets be whitelisted for the CRC64 feature.

    Target Storage Class

    Standard

    For details about storage classes, see How Do I Choose Storage Classes?

    NOTE:

    CRC64-based consistency verification is not available for Archive and Deep Archive storage classes. Even if you choose the CRC64-based verification method, the system automatically uses the object size and last modification time to verify data consistency.

    Infrequent Access

    Archive

    Deep Archive

    Same as source

  4. In the Consistency Verification area, select a consistency check method.

    • Size and last modified: With this default setting, the system verifies object consistency with object size and last modification time.
    • CRC64 checksum: The system checks data consistency by comparing the CRC64 values in the metadata of source and target objects. If a source object or the paired target object does not have a CRC64 checksum, the OMS-calculated CRC64 value is used for verification. CRC64 verification may generate extra public traffic and request costs. For details, see Consistency Verification.
      • This option is only available for migration within Huawei Cloud or from AWS, Alibaba Cloud, or Tencent Cloud.
      • This option is only available for migration from NAS_NFS_V3_MOUNT and NAS_NFS_V3_PROTOCOL.
      • Using this option requires that target OBS buckets be whitelisted for the CRC64 feature.

  5. In the Migration Settings area, set parameters according to Table 4.

    Table 4 Migration settings

    Parameter

    Description

    Concurrent Subtasks

    User-defined. There cannot be more than 10 concurrent subtasks for each online migration node. For example, if there are 2 online migration nodes, the maximum number of subtasks can be 20 or any number below.

    Limit Traffic

    Set the maximum bandwidth to be used by the migration workflow during different periods.

    • If you do not select this option, migration traffic is not limited.
    • If you select this option, limit the migration traffic by setting Start Time, End Time, and Bandwidth Limit.

      For example, if you set Start Time to 08:00, End Time to 12:00, and Bandwidth Limit to 20 MB/s, the maximum migration speed is limited to 20 MB/s from 08:00 to 12:00. The migration speed is not limited outside this period.

      NOTE:
      • The bandwidth limit ranges from 1 MB/s to 1,048,576 MB/s.
      • A maximum of five rules can be added.
      • The time is the local standard time of the region you are migrating to.

  6. (Optional) Configure advanced options based on Table 5.

    Table 5 Advanced options

    Parameter

    Description

    Enable KMS Encryption

    • If you do not select this option, whether migrated data will be encrypted in the target buckets depends on the server-side encryption setting of the buckets.
    • If you select this option, all migrated objects will be encrypted before they are stored to the target bucket.
    NOTE:
    • Using KMS to encrypt migrated data may slow down the migration speed by about 10%.
    • This option is only available when KMS is supported in the region you are migrating to.

    Filter Source Data

    Filter files to be migrated by applying filters. For details about the filters, see Source Data Filters.

    Send SMN Notifications

    Determine whether to use SMN to get notifications about migration results.

    • If you do not select this option, no SMN messages will be sent after the migration is complete.
    • If you select this option, after the migration is complete, SMN messages will be sent to the subscribers of the selected topic. You can select the language and trigger conditions for sending messages.

    Migrating Incremental Data

    • If this option is not selected, incremental migration is not performed.
    • If this option is selected, configure the overwriting policy and the execution mode for incremental migration. For details, see Configuring Incremental Migration Settings.

  7. After the migration settings are configured, click Next to open the Configure Migration Cluster page.

Step 4: Configure a Migration Cluster

  1. In the Configuration Method area, select a method of configuring a migration cluster. If a cluster is available, choose Use existing cluster. If no migration clusters are available, choose Explore recommendations and create new cluster to assess the appropriate cluster size. You can assess how large of a migration cluster is required for the migration and then create a migration cluster with the recommended specifications. Alternatively, you can skip this step and use an existing migration cluster. For details about how to create a cluster, see Creating a Migration Cluster.
  2. In the Migration Cluster area, select the cluster to be used and click OK. If you want to modify an existing cluster to meet your requirements, see Managing a Migration Cluster.
  3. After the migration cluster is configured, click Next to open the confirmation page.

Step 5: Confirm the Configurations

After performing steps 1 to 4, check the resource and migration settings on the Confirm page. To update the settings, click Previous to return to the corresponding stage for modification.

Step 6: Perform Follow-Up Operations

After confirming that the configurations are correct, perform follow-up operations as required.

Click Complete to save the migration plan. Then you can:
On the Confirm page, click Create Workflow. The Start Workflow dialog box is displayed, indicating that the workflow has been created.
  • If you want to start the migration immediately, click OK to start the workflow and view the workflow details.
  • If you want to save the migration workflow without starting the migration, click Cancel. The migration workflow will move to a Waiting status, and the migration has not started yet. To run the workflow, click Run in the Operation column to manually start the migration.

For details about migration workflows, see Creating a Batch Object Storage Migration Workflow.

Configuring Incremental Migration Settings

The following table describes the parameters about incremental migration.

Table 6 Configuring incremental migration settings

Parameter

Description

Overwrite Existing During Incremental Migration

See Table 3.

NOTE:

This setting only applies to incremental migration.

Incremental Migration Method

  • Manual: You can manually initiate an incremental migration after the first incremental migration defined in the workflow is complete.
  • Automated: You can define the interval, maximum attempts, and time zone to trigger incremental migrations.

    For example, if you configure the incremental migration settings as shown in the following figure, the system will initiate incremental migration at 8:00 in the morning on Mondays (GMT+08:00) for the next two weeks.

  • If you select Manual, incremental migration will be executed immediately when the workflow reaches the BatchCreateIncrementalMigrationTasks step.
  • If you select Automated, when the workflow reaches the BatchCreateIncrementalMigrationTasks step, you can:
    • Manually trigger incremental migration immediately when you are waiting for automatic incremental migration.
    • Skip incremental migration and verify services.