Help Center/ Migration Center/ User Guide/ Application Migration Projects/ Storage Migration/ File Storage Migration (Exclusive Cluster)
Updated on 2026-05-22 GMT+08:00
View PDF
Share

File Storage Migration (Exclusive Cluster)

What Is File Storage Migration (Exclusive Cluster)?

This solution provides isolated clusters and dedicated lines for customized configurations. Supporting NAS migration from various cloud providers, it ensures a highly efficient and comprehensive migration experience. Using exclusive clusters will incur fees. For details, see Billing.

Notes and Constraints

For the constraints, see What Are the Restrictions on Using MgC for Storage Migration?

Warnings

When creating a task, there are four overwriting policies for objects with the same name at the source and target: If older or different size, Never, Always, and If different CRC64 checksum. If you choose Never, restarting a migration after an interruption or pause may lead to incomplete data migration, even though the task may appear successful. This could impact data integrity, so use the Never option with caution.

Procedure

Table 1 introduces the complete process of file storage migration.

Table 1 File storage migration process

Step

Description

Preparations
  • Before using MgC, prepare a HUAWEI ID or an IAM user that can access MgC, and grant them relevant permissions.
  • Create a file storage migration project on the MgC console.
  • Create a migration cluster to improve migration efficiency.

Step 1: Configure the Migration Task

Configure the basic information, source and destination information, migration parameters, and advanced options.

Step 2: Confirm the Information

Confirm the information entered for configuring the file storage migration task.

Step 3: Track the Migration Task

Start the migration task and check the migration status, phases, and steps.

Preparations

Prepare resources by referring to Table 2 before creating a cross-region server migration task.

Table 2 Preparations

Item

Requirement

Preparing accounts and permissions
  1. Sign up for a HUAWEI ID, enable Huawei Cloud services, and complete real-name authentication. If you have already enabled Huawei Cloud services and completed real-name authentication, skip this step.
    1. Visit Huawei Cloud and click Sign Up.
    2. Sign up for a HUAWEI ID and enable Huawei Cloud services.
    3. (Optional) Complete real-name authentication. Real-name authentication is the process of submitting valid identity information (such as a passport or business license) for verification when you use cloud services in the Chinese mainland. There are individual real-name authentication and enterprise real-name authentication.

      According to relevant laws and regulations, unauthenticated accounts cannot purchase cloud services in the Chinese mainland and your account and rights may not be protected. For more information, see Real-Name Authentication Overview.

      To complete real-name authentication, refer to:
  2. Create an IAM user and grant permissions to the user. Skip this step if you use a HUAWEI ID (administrator account).
    1. Create an IAM user. For details, see Creating an IAM User or Creating an IAM User (New Version).
    2. Create a custom policy and grant permissions to the IAM user. For details, see Using Roles or Policies to Assign Custom Permissions to IAM Users or Using Identity Policies to Assign Custom Permissions to IAM Users (New Version). For details about how to grant permissions, see Assigning Permissions to an IAM User or Assigning Permissions to an IAM User (New Version).
  3. (Optional) Obtain the AK/SK of the target account.

    The AK/SK of the target account is required for authentication, especially when a destination OBS bucket is used. For details about how to obtain the AK/SK of the target account, see Creating an Access Key or Access Keys (New Version).

Creating an application migration project

Create a project to separate and manage migration resources for different scenarios, to streamline organization management and resource allocation. For details, see Creating an Application Migration Project.

Creating a migration cluster

A migration cluster is a core component for storage workflows. It allows you to create and manage migration nodes, list nodes, and deploy and upgrade migration plugins, ensuring smooth operations of storage workflows. For details, see Creating a Migration Cluster.

Step 1: Configure the Migration Task

  1. In the navigation pane on the left, choose Storage Migration > File Storage.
  2. Click Create Migration Task.
  3. Set basic information based on Table 3.

    Table 3 Basic parameters

    Parameter

    Description

    Name

    User-defined. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. The name must start with a letter or digit, and it must not start or end with a period.

    Region

    Select the region you are migrating to.

    Description

    User-defined. It can contain a maximum of 255 characters. Allowed characters include letters, digits, hyphens (-), underscores (_), commas (,), periods (.), spaces, and decimal points (.). It cannot start with a decimal point (.), hyphen (-), or underscore (_).

    Cluster

    Select a migration cluster. The cluster consists of a master node and several migration and list nodes. If there is no available cluster, create one by referring to Creating a Migration Cluster.

    NOTE:

    A master node is created by the system in a migration cluster by default. You do not need to configure it.

  4. Configure the migration source and target based on Table 4 and Table 5.

    Table 4 Parameters for configuring a migration source

    Parameter

    Description

    Remarks

    Location Type

    Supported migration sources include:

    • NAS_GFS (Google Cloud file systems)
    • NAS_SMB (SMB/CIFS file systems)
    • NAS_NFS_V3_MOUNT (non-standard NFSv3 file systems)
    • NAS_NFS_V3_PROTOCOL (standard NFSv3 file systems)

    -

    File System Address

    Enter the mount address of the source file system. The format is IP address:/ or IP address:/xxx, for example, 192.1.1.1:/0001.

    -

    Path

    Enter the directory that contains the files to be migrated. The format is /<folder-name>.

    Username

    Enter the username of the account that can access all files in the source file system, for example, administrator.

    These parameters are available when Location Type is set to NAS_SMB.

    Password

    Enter the password corresponding to the username.

    Domain on Windows

    Enter the corresponding domain name.

    NOTE:

    You only need to enter the content before .com. For example, if the domain is test.com, enter test.
    Table 5 Parameters for configuring a migration target

    Parameter

    Description

    Remarks

    Location Type

    Select a Huawei Cloud storage type based on the source storage type.

    -

    Credential

    Select the credential from the drop-down list. If the credential has not been added, click Add Credential, enter a credential name, select an authentication method, enter the AK/SK pair, and click Verify and Save.

    These parameters are available when Location Type is set to Huawei Cloud OBS.

    Bucket

    Select the OBS bucket you are migrating your data to.

    NOTE:

    Only Standard and Infrequent Access buckets are supported.

    Type

    Set this parameter based on the source bucket type. Check it in the basic information.

    Endpoint

    Enter the endpoint of the region where the target OBS bucket is located. For example, if the target bucket is located in the CN North-Beijing4 region of Huawei Cloud, enter obs.cn-north-4.myhuaweicloud.com.

    NOTE:

    You can view the endpoint in the OBS bucket overview.

    Specify Prefix

    Specify a prefix to rename or relocate objects migrated to the target bucket. For example, if you specify the prefix /D, source file /A/B/C.txt will be relocated to /D/A/B/C.txt after being migrated to the target bucket. For details, see:

    Adding a Name Prefix or Path Prefix to Migrated Objects

    File System Address

    Enter the mount address of the target file system. To obtain the mount address, go to the file system list and click the icon next to the address in the Mount Point column.

    These parameters are available when Location Type is set to NAS_SMB or NAS_NFS_V3_MOUNT.

    Path

    Enter the directory for storing migrated files. The format is /<folder-name>.

    Username

    Enter the username of the account that can access all files in the target file system, for example, administrator.

    These parameters are available when Location Type is set to NAS_SMB.

    Password

    Enter the password corresponding to the username.

    Domain on Windows

    Enter the corresponding domain name.

    NOTE:

    You only need to enter the content before .com. For example, if the domain is test.com, enter test.

  5. Configure migration settings based on Table 6.

    Table 6 Migration settings

    Parameter

    Sub-parameter

    Description

    Migration Method

    Full migration

    This method enables migration of all data in a source bucket or folder.

    List migration

    This method enables the migration of specific files and folders recorded in predefined lists.

    In List Path box, enter the path of the object lists stored in the target bucket. Restrictions on an object list file vary with the target location.

    • Target location: Huawei Cloud OBS
      • An object list file cannot exceed 30 MB.
      • An object list file must be a .txt file, and the Content-Type metadata must be text/plain.
      • An object list file must be in UTF-8 without BOM.
      • Each line in an object list file can contain only one object name, and the object name must be URL encoded.
      • Each line in an object list file cannot exceed 16 KB, or the migration will fail.
      • The Content-Encoding metadata of an object list file must be left empty, or the migration will fail.
      • An object list file can contain a maximum of 10,000 lines.
    • Target location: NAS
      • An object list file cannot exceed 30 MB.
      • An object list file must be a .txt file.
      • An object list file must be in UTF-8 without BOM.
      • Each line in an object list file can contain only one object name, and the object name must be URL encoded.
      • Each line in an object list file cannot exceed 16 KB, or the migration will fail.
      • An object list file can contain a maximum of 10,000 lines.
    CAUTION:

    Precautions for list-based migrations from NAS:

    • If the file system address ends with :/, enter absolute paths relative to the mount point in the list file.

      For example, if the file system address is 192.1.1.1:/ and the mount point is /mnt/turbo, enter absolute paths relative to /mnt/turbo.

      Consider that you want to migrate the test2.log file using the list-based migration method, enter /mgc/filter_test/test2.log in the list.

    • If the file system address contains additional content after :/, combine the part after :/ in the file system address with the absolute paths relative to the mount point.

      For example, if the file system address is 192.1.1.1:/mgc-test and the mount point is /mnt/turbo, combine /mgc-test with absolute paths relative to /mnt/turbo.

      Consider that you want to migrate the test2 file using the list-based migration method, enter /mgc-test/autotest/filter_test/test2 in the list.

    Prefix migration

    This option is only available for cloud storage migration.

    If you enter a file name or name prefix in the Prefix box, only the objects that exactly match the specified name or prefix are migrated.

    • If the files to be migrated are stored in the root directory of the source bucket, add their name prefixes directly. If the files are stored in a non-root directory, add their directories and name prefixes in the format of <folder-name>/<prefix>.
    • Use commas (,) to separate multiple prefixes.

    Concurrent Subtasks

    -

    The default value is 5, but the value can be changed. There cannot be more than 10 concurrent subtasks for each online migration node. For example, if there are 2 online migration nodes, there can be up to 20 subtasks.

    Overwrite Existing

    Never

    Files existing at the target will never be overwritten.

    WARNING:
    • If you choose Never for the initial migration, the attributes of involved parent folders at the source will not be migrated to the target. As a result, the folder attributes may be incomplete at the target. This could impact data integrity, so use the Never option with caution.
    • If you choose Never, restarting a migration after an interruption or pause may lead to incomplete data migration, even though the task may appear successful. This could impact data integrity, so use the Never option with caution.

    Always

    Files existing at the target will always be overwritten.

    If older or different size
    • The system replaces existing target files if they are older than or differ in size from their source counterparts. Files with matching modification times and sizes remain unchanged and are skipped during migration.
    • The system verifies folders after their content is migrated. Folders that already exist at the target will be overwritten if they have different last modification times, sizes, or permissions from the paired folders at the source.
      NOTE:

      The same overwriting policy is applied to empty folders as files.

    If different CRC 64 checksum
    If a source object has a CRC64 checksum different from the paired target object, the source object will overwrite the target one. Otherwise, the source object will be skipped during the migration. If either of them does not have a CRC64 checksum, their sizes and last modification times are checked.
    NOTE:
    • This option is only available for migration within Huawei Cloud or from Alibaba Cloud or Tencent Cloud.
    • Using this option requires that target OBS buckets be whitelisted for the CRC64 feature.

    Consistency Verification

    Size and last modified

    With this default method, the system checks data consistency by comparing object size and last modification time.

    CRC64 checksum
    The system checks data consistency by comparing the CRC64 values in the metadata of source and target objects. If a source object or the paired target object does not have a CRC64 checksum, the OMS-calculated CRC64 value is used for verification. CRC64 verification may generate extra public traffic and request costs. For details, see Consistency Verification.
    NOTE:
    • NAS_NFS_V3_MOUNT and NAS_NFS_V3_PROTOCOL are supported.
    • Using this option requires that target OBS buckets be whitelisted for the CRC64 feature.

    Migrate Metadata

    -

    Determine whether to migrate metadata.

    • If you select this option, object metadata will be migrated.
    • If you do not select this option, only the Content-Type and Content-Encoding metadata will be migrated. If the source storage service provider does not provide the metadata (for example, UCloud does not provide Content-Encoding), the metadata will not be migrated.

    Clear Cluster

    -

    Determine whether to clear the migration cluster after the migration is complete.

    • If you select this option, a step for clearing the migration cluster will be created in the workflow. You can also choose whether to clear resources used by the cluster, such as NAT gateways, security groups, and VPCEP resources.
    • If you do not select this option, a step for clearing the migration cluster will not be created in the workflow, but the migration cluster and its resources will automatically be deleted 30 days after the workflow is created.

  6. (Optional) Configure advanced options based on Table 7.

    Table 7 Advanced options

    Parameter

    Description

    Target Storage Class

    Choose the storage class that your data will be migrated to. For details about storage classes, see How Do I Choose Storage Classes?

    NOTE:

    CRC64-based consistency verification is not available for Archive and Deep Archive storage classes. Even if you choose the CRC64-based verification method, the system automatically uses the object size and last modification time to verify data consistency.

    Enable KMS Encryption

    This parameter is only available for migrations to Huawei Cloud OBS.

    • If you do not select this option, whether migrated data will be encrypted in the target buckets depends on the server-side encryption setting of the buckets.
    • If you select this option, all migrated objects will be encrypted before they are stored to the target bucket.
    NOTE:
    • Using KMS to encrypt migrated data may slow down the migration speed by about 10%.
    • This option is only available when KMS is supported in the region you are migrating to.

    Restore Archive Data
    • If you do not select this option, the system records archived objects in the list of objects that failed to be migrated and continues to migrate other objects in the migration task.
    • If you select this option, the system automatically restores and migrates archived objects in the migration task. If an archived object fails to be restored, the system skips it and records it in the list of objects that failed to be migrated and continues to migrate other objects in the migration task.
    NOTE:

    The system will restore all archived data found during the listing phase before migration. You need to pay the source cloud platform for the API requests and storage space generated accordingly. To avoid extra restoration fees, you are advised to manually restore the archived data on the source platform before starting the migration.

    Filter Source Data

    Use exclude patterns, include patterns, and time ranges to filter files to be migrated. For details about the filters, see Source Data Filters.

    Download Data from CDN

    If the default domain name cannot meet your migration requirements, then as long as the source cloud service provider supports custom domain names, you can bind a custom domain name to the source bucket, and enable the CDN service on the source platform to reduce data download expenses. Enter a custom domain name in the Domain Name text box and select a transmission protocol.

    These parameters are available when the source is Alibaba Cloud OSS or Tencent Cloud COS. If CDN authentication is enabled, you also need to select an authentication type and enter an authentication key.

    Send SMN Notifications

    Determine whether to use SMN to get notifications about migration results.

    • If you do not select this option, no SMN messages will be sent after the migration is complete.
    • If you select this option, after the migration is complete, SMN messages will be sent to the subscribers of the selected topic. You can select the language and trigger conditions for sending messages.

    Limit Traffic

    Set the maximum bandwidth for the migration task during different periods.

    • If you do not select this option, migration traffic will not be limited.
    • If you select this option, you can limit the migration traffic by setting Start Time, End Time, and Bandwidth Limit.
      For example, if you set Start Time to 08:00, End Time to 12:00, and Bandwidth Limit to 20 MB/s, the maximum migration speed is limited to 20 MB/s from 08:00 to 12:00. The migration speed is not limited beyond this period.
      NOTE:
      • The bandwidth limit ranges from 1 MB/s to 1,048,576 MB/s.
      • Time periods in different rules cannot overlap.

        For example, if there is a rule configured for the period from 8:00 to 12:00, you cannot configure rules for any overlapped periods, such as from 7:00 to 13:00, 7:00 to 8:00, and 9:00 to 12:00.

      • Start and end times must be on the same day, with the start time preceding the end time.

        For example, the time period from 23:00 to 01:00 of the next day is not allowed.

      • A maximum of five rules can be added.
      • The time is the local standard time of the region you are migrating to.

    Schedule Migration

    Schedule the migration to automatically run during a period.

    • If you do not select this option, you need to manually start or stop the migration.
    • If you select this option, the migration runs during the specified period and stops outside that period.

      Examples:

      • If you set Start Time to 08:00 and End Time to 12:00, the migration task runs from 08:00 to 12:00 every day. The migration stops outside that period.
      • If you set Start Time to 12:00 and End Time to 08:00, the migration runs from 12:00 of the current day to 08:00 of the next day. The migration stops outside that period.

Step 2: Confirm the Information

  1. After the migration task is configured, click Next to confirm the information.
  2. Confirm the settings and click Confirm. The Start Migration Task dialog box is displayed, indicating that the workflow has been created.

    • If you want to start the migration immediately, click Confirm to run the workflow.
    • If you want to add a stage or step to the workflow, click Cancel. The workflow changes to the Waiting state, and the migration has not started yet. To start the migration, click Run in the Operation column.

Step 3: Track the Migration Task

In the migration task list, click the task name to go to the Task Details page. Check the task progress and perform the following operations on it.
  • Checking the resource status: Click a resource name. On the Resource Details page displayed on the right, you can review the basic information, migration status, and migration progress of the resource. You can also perform the operations below.

    Operation

    Step

    Replacing the migration cluster

    You can replace the migration cluster only when the migration task (workflow) is Paused. Only healthy clusters can be used.

    1. In the Basic Information area, click Replace next to the migration cluster name.
    2. In the displayed dialog box, select a new cluster from the drop-down list and click Confirm. After the cluster is replaced, the workflow starts to run automatically.

    Modifying the migration schedule
    1. In the Basic Information area, click Modify under Schedule Migration.
    2. Set Start Time and End Time, and click Confirm.

    Modifying the number of concurrent subtasks
    1. In the Migration Progress area, click Modify above Expected Concurrent Subtasks to change the expected number of concurrent subtasks. There cannot be more than 10 concurrent subtasks for each online migration node. For example, if there are 2 online migration nodes, there can be up to 20 subtasks.
    2. Click Confirm.

    Adding traffic limiting rules
    1. In the Migration Speed area, click Add to add a rule to limit the bandwidth the migration can use in a specified period.
      NOTE:
      • The bandwidth limit ranges from 1 MB/s to 1,048,576 MB/s.
      • Time periods in different rules cannot overlap.

        For example, if there is a rule configured for the period from 8:00 to 12:00, you cannot configure rules for any overlapped periods, such as from 7:00 to 13:00, 7:00 to 8:00, and 9:00 to 12:00.

      • Start and end times must be on the same day, with the start time preceding the end time.

        For example, the time period from 23:00 to 01:00 of the next day is not allowed.

      • A maximum of five rules can be added.
      • The time is the local standard time of the region you are migrating to.
    2. Click Save.

    Obtaining the lists of migrated, skipped, or failed files

    In the File Statistics area, view the paths of the lists of migrated, skipped, or failed files. Click a file path, and it will take you to the OBS bucket where the list is stored. You can download the list from the bucket.

    Viewing traffic statistics

    In the Traffic Statistics area, view the migration traffic in the last 1 hour, last 6 hours, last 24 hours, or the entire migration period.
  • Confirming steps: Some resources require manual confirmation. In this case, click Confirm in the Migration Status column or in the Operation column to continue the migration.

Checking Task Configurations

In the migration task list, click Check Configuration in the Operation column. On the displayed page, click Manage next to the cluster name. The cluster details page is displayed on the right. On the displayed page, you can:
  • Add, edit, or delete traffic limiting rules to control cluster traffic based on your requirements.
  • Add or delete migration nodes or list nodes, or upgrade plug-ins for existing nodes as required.

Deleting Resources from a Migration Task

Resources in the Failed, Paused, or Completed state can be deleted. If all resources in a migration task are deleted, the migration task will also be deleted.

  1. In the migration task list, click a migration task name to go to the task details page.
  2. In the resource list, select the resources to be deleted and click Delete above the list.
  3. Confirm the selected resources, enter DELETE, and click Confirm.

    To prevent resource deletion failures caused by disconnection of the task from related services (such as SMS, OMS, and OBS), you can select Force Delete to forcibly delete the resources. However, this operation only deletes the resource data from MgC. It may not delete the related tasks and resources in the related cloud services like SMS, OMS, and OBS. You must delete any remaining resources manually.

Source Data Filters

Table 8 describes the rules and restrictions for setting source data filters.

Table 8 Filter options

Parameter

Description

Pattern Rule

Constraint

Exclude Patterns

If a file matches any excluded pattern, the file will not be migrated or compared for consistency. Both exact match and fuzzy match are supported.
  • Exact match
    You need to specify absolute paths and use backslashes (\) to escape special characters in the paths.
    CAUTION:

    Precautions for configuring exclude and include patterns:

    • OBS uses slashes ( / ) to define directory structures. When you set patterns to include or exclude folders, use their complete paths with slashes included.

      For example, folder A is represented as A/ in OBS.

    • If the file system address ends with :/, when you configure the paths to be excluded or included, enter their absolute paths relative to the mount point.

      For example, if the file system address is 192.1.1.1:/ and the mount point is /mnt/turbo, enter absolute paths relative to /mnt/turbo.

      For example: 
      [root@oms-cluster-ecs filter_test]# pwd
/mnt/sts_turbo/mgc/filter_test
[root@oms-cluster-ecs- filter_test]# ll
drwxr-xr-x 2 root root 0 Aug 16 15:27 test2
-rw-r--r-- 1 root root 5 Aug 16 15:27 test2.log

      To exclude the test2.log file from the migration, you can enter its absolute path /mgc/filter_test/test2.log in the Exclude Patterns box.

    • If the file system address contains additional content after :/, when you configure paths to be excluded or included, combine the part after :/ in file system address with the absolute paths relative to the mount point.

      For example, if the file system address is 192.1.1.1:/mgc-test and the mount point is /mnt/turbo, combine /mgc-test with absolute paths relative to /mnt/turbo.

      For example: 
      [root@oms-cluster-ecs execution-service]# cd /mnt/turbo/autotest/filter_test/
[root@oms-cluster-ecs- filter_test]# ll
-rw-r--r-- 1 root root   14 Aug  8 09:22 test1.log
drwxr-xr-x 1 root root 4096 Aug  8 09:22 test2
-rw-r--r-- 1 root root   14 Aug  8 09:22 test2.log

      To exclude the test2.log file from the migration, enter /mgc-test/autotest/filter_test/test2 in the Exclude Patterns box.

  • Fuzzy match
    • An asterisk (*) matches zero or more characters except for slashes (/).
    • A pair of asterisks (**) matches zero or more characters including slashes (/).
    • A question mark (?) matches exactly one character, but not slashes (/).
    • Commas (,) are used to separate patterns in {}. Patterns in {} are in an OR relationship.
    • Wildcard characters asterisk (*) and question mark (?) are escaped by backslashes (\). In other cases, a backslash (\) means itself.
  • Except for {}, consecutive characters specified in pattern rules are not allowed, for example, ***, *?, **?, ?*, ?**, *{*, *}*, *}?, ?{*, {*}, {,}, {*,, ,*}, and ,*,
  • Only asterisks (*) can be used as wildcard characters in {}.
  • Nesting {} inside another set of {} is not allowed.
  • Excluded patterns take precedence over included patterns.
  • Semicolons (;) are used to separate patterns outside {}.

Include Patterns
  • If no included patterns are specified, all files in the source will be migrated.
  • If included patterns are specified, only the files whose absolute paths match the specified patterns will be migrated or compared for consistency.

Time Range

Filters files and directories to be migrated based on when they were last modified. Only files and directories whose last modification times fall in the configured time range will be migrated.

The start time and end time can be left empty. If they are left empty, the system will not filter out source files by time. The time can be precise to the minute.

The following table lists example pattern rules for different scenarios.

Assume that you want to migrate the directory test in the source storage system.

  • If the source storage system is a NAS device, enter /test as an "include" pattern.
  • If the source storage system is an object storage system, enter test as an "include" pattern.

Scenario

Example Pattern for NAS

Example Pattern for Object Storage

Description

File paths that end with xx

/xx/**xx

xx/**xx

xx can be an expression containing asterisks (*) and question marks (?).

Files whose names start with xx in the root directory

/xx*

  • /testssss matches the pattern.
  • /test/xx does not match the pattern.

/xx*

  • testssss matches the pattern.
  • test/xx does not match the pattern.

File paths that start with xx

/xx**

/xx/**

xx**

xx/**

Files whose names contain xx

**xx*

**xx*

File paths that contain xx

**xx**

**xx**

File paths that start with xx and end with yy.

/xx**yy

xx**yy

xx and yy can be expressions containing asterisks (*) and question marks (?).

File paths that end with xx or yy

**{xx,yy}

**{xx,yy}

Files whose names contain xx or yy

**{xx,yy}*

**{xx,yy}*

File paths that contain xx or yy

**{xx,yy}**

**{xx,yy}**

Consistency Verification

MgC uses OMS's consistency verification feature to validate data consistency by comparing object sizes, last modified timestamps, and CRC64 checksums. The following describes these verification methods in detail.

Verification Method

Description

Scenario

Verification Process

Impact

Object size and last modification time

Only object sizes and last modified timestamps are verified. The data content is not verified.

All scenarios

After the migration is complete, the system compares the sizes and last modified timestamps of source and target objects.

None

CRC64 checksum

The CRC64 checksums, calculated using the ECMA-182 algorithm, are used to verify data consistency.

Scenarios where both source and target objects have CRC64 checksums

After the migration, the system invokes APIs to retrieve the CRC64 checksums of source and target objects and then compares them to verify consistency.

Retrieving the CRC64 checksum of an object may generate API request costs as well as public traffic costs.

Scenarios where source objects lack CRC64 checksums but target objects have CRC64 checksums

During migration, the system instructs OMS to compute CRC64 checksums for source objects. After migration is complete, the system invokes APIs to retrieve CRC64 checksums of target objects. These values are then compared to ensure data consistency.

Retrieving the CRC64 checksum of an object may generate API request costs as well as public traffic costs.

Scenarios where source objects have CRC64 checksums but target objects lack CRC64 checksums

After migration is complete, the system invokes APIs to retrieve CRC64 checksums of source objects and instructs OMS to compute CRC64 checksums for target objects. These values are then compared to ensure data consistency.
  • Retrieving the CRC64 checksum of an object may generate API request costs as well as public traffic costs.
  • To compute checksums, OMS retrieves data streams from the target. The time required for verifying these objects may be comparable to the time spent on migrating them.

Scenarios where both source and target objects lack CRC64 checksums

The system instructs OMS to compute CRC64 checksums for source objects during the migration and to compute CRC64 checksums for target objects after the migration. These values are then compared to ensure data consistency.

To compute checksums, OMS retrieves data streams from the target. The time required for verifying these objects may be comparable to the time spent on migrating them.
Parent Topic: Storage Migration