Help Center/ Migration Center/ Best Practices/ Storage Migration/ Migrating Archive (Cold) Data
Updated on 2025-10-11 GMT+08:00
View PDF
Share

Migrating Archive (Cold) Data

Background

Cold storage is a long-term backup solution for infrequently accessed data. For example, data accessed annually can be stored in archive storage, while data accessed every few years can be stored in deep archive storage. This storage mode has advantages such as high security, durability, and low costs.

When migrating cold storage data, you need to consider data access latency because data is usually in a "frozen" state. Data must be restored or unfrozen before it can be accessed. Typically, there are two restoration options: standard and expedited retrieval. Their differences are as follows:

  • Standard: Archive objects can be restored within 3 to 5 hours. This option is ideal for scenarios where immediate access is not required.
  • Expedited: Archive objects can be restored within 1 to 5 minutes, and additional fees may be incurred. This option is ideal for scenarios where immediate access is required.

MgC storage migration workflows enable you to restore archive source data expeditiously.

Supported Regions

The following regions are supported:

  • LA-Santiago
  • LA-Sao Paulo
  • TR-Istanbul
  • AP-Bangkok
  • AP-Singapore
  • AP-Jakarta
  • ME-Riyadh
  • CN North-Beijing4
  • CN East-Shanghai1

Constraints

  • See What Are the Restrictions on Using MgC for Storage Migration?
  • Data in the Deep Cold Archive storage of Alibaba Cloud OSS cannot be restored using the restoration function of MgC. You need to manually restore the data before migration.
  • Microsoft Azure Blob does not support automatic archiving of restored data. If needed, you can manually archive the data after the migration is complete.

Preparations

  • Prepare a Huawei account.

    Before using MgC, prepare a HUAWEI ID or an IAM user that can access MgC and obtain an AK/SK pair for the account or IAM user. For details, see Preparations.

  • Create an application migration project.

    Create a migration project on the MgC console. For details, see Managing Migration Projects. Set Project Type to Application migration.

Creating a Migration Workflow

  1. Sign in to the MgC console. In the navigation pane, under Project, select your application migration project from the drop-down list.
  2. In the navigation pane, choose Migration Workflows (NEW).
  3. On the Workflows tab, click Create Workflow to open the Select Creation Method page.
  4. Select General Storage Migration and click OK to open the page for creating a workflow.
  5. Set workflow basics based on Table 1.

    Table 1 Basic settings

    Parameter

    Description

    Name

    User-defined.

    Target Region

    Select the region where the target bucket is located from the drop-down list.

    Description

    User-defined.

    Cluster

    Select the created cluster.

  6. Configure the migration source and target based on Table 2 and Table 3.

    Table 2 Parameters for configuring a migration source

    Parameter

    Description

    Location Type

    Select the source cloud platform and data source.

    Credential

    Select the credential from the drop-down list. If the credential has not been added, click Add Credential, enter a credential name, select an authentication method, enter the AK/SK pair, and click Verify and Save.

    Bucket

    Enter the name of the source bucket to be migrated.

    NOTE:
    Symbolic links cannot be used for specifying migration paths which define the migration scope. To migrate a path pointed to by a symbolic link, you need to:
    • Enter the actual path when specifying the migration path.
    • After the migration is complete, manually create a symbolic link to the path at the target.

    Endpoint

    Enter the endpoint of the region where the source bucket is located.

    For example, if Location Type is set to Alibaba Cloud OSS and the source bucket is located in CN East 1 (Hangzhou), enter oss-cn-hangzhou.aliyuncs.com.

    Type

    This parameter is available when Location Type is set to Huawei Cloud OBS. Set this parameter based on the source bucket type. You can view the bucket type in the basic information.

    APPID

    This parameter is available when Tencent Cloud COS is selected for Location Type. Enter the APPID of your Tencent Cloud account.

    NOTE:

    You can view the APPID on the account information page of the Tencent Cloud console.
    Table 3 Parameters for configuring a migration target

    Parameter

    Description

    Location Type

    Select Huawei Cloud OBS.

    Credential

    Select the credential from the drop-down list. If the credential has not been added, click Add Credential, enter a credential name, select an authentication method, enter the AK/SK pair, and click Verify and Save.

    Bucket

    Select the created OBS bucket.

    NOTE:

    Only Standard and Infrequent Access buckets are supported.

    Type

    Set this parameter based on the source bucket type. You can view the bucket type in the basic information.

    Endpoint

    Enter the endpoint of the region where the target bucket is located.

    For example, if the target bucket is located in the CN North-Beijing4 region of Huawei Cloud, enter obs.cn-north-4.myhuaweicloud.com.

    NOTE:

    You can view the endpoint in the OBS bucket overview.

    Specify Prefix

    This parameter is optional. Specify a prefix to rename or relocate objects migrated to the target bucket. For example, if you specify the prefix /D, source file /A/B/C.txt will be relocated to /D/A/B/C.txt after being migrated to the target bucket. For details, see:

    Adding a Name Prefix or Path Prefix to Migrated Objects

  7. Configure migration settings based on Table 4.

    Table 4 Migration settings

    Parameter

    Sub-parameter

    Description

    Task Type

    Full migration

    Migrates all data in a source bucket or in specific paths.

    List migration

    Migrates objects recorded in the list files.

    In List Path box, enter the path of the object lists stored in the target bucket. Restrictions on an object list file vary with the target location.

    • Target location: Huawei Cloud OBS
      • An object list file cannot exceed 30 MB.
      • An object list file must be a .txt file, and the Content-Type metadata must be text/plain.
      • An object list file must be in UTF-8 without BOM.
      • Each line in an object list file can contain only one object name, and the object name must be URL encoded.
      • Each line in an object list file cannot exceed 16 KB, or the migration will fail.
      • The Content-Encoding metadata of an object list file must be left empty, or the migration will fail.
      • An object list file can contain a maximum of 10,000 lines.
    • Target location: NAS
      • An object list file cannot exceed 30 MB.
      • An object list file must be a .txt file.
      • An object list file must be in UTF-8 without BOM.
      • Each line in an object list file can contain only one object name, and the object name must be URL encoded.
      • Each line in an object list file cannot exceed 16 KB, or the migration will fail.
      • An object list file can contain a maximum of 10,000 lines.
    CAUTION:

    Precautions for list-based migrations from NAS:

    • If the file system address ends with :/, enter absolute paths relative to the mount point in the list file.

      For example, if the file system address is 192.1.1.1:/ and the mount point is /mnt/turbo, enter absolute paths relative to /mnt/turbo.

      Consider that you want to migrate the test2.log file using the list-based migration method, enter /mgc/filter_test/test2.log in the list.

    • If the file system address contains additional content after :/, combine the part after :/ in file system address with the absolute paths relative to the mount point.

      For example, if the file system address is 192.1.1.1:/mgc-test and the mount point is /mnt/turbo, combine /mgc-test with absolute paths relative to /mnt/turbo.

      Consider that you want to migrate the test2 file using the list-based migration method, enter /mgc-test/autotest/filter_test/test2 in the list.

    Prefix migration

    This option is only available for cloud storage migration.

    If you enter a file name or name prefix in the Prefix box, only the objects that exactly match the specified name or prefix are migrated.

    NOTICE:
    • If the files to be migrated are stored in the root directory of the source bucket, add their name prefixes directly. If the files are stored in a non-root directory, add their directories and name prefixes in the format of <folder-name>/<prefix>.
    • Use commas (,) to separate multiple prefixes.

    Listing Mode

    NOTE:

    This parameter is available only when Task Type is set to Full migration or Prefix migration

    Serial

    This is the default listing mode if the source is a bucket.

    Parallel

    This is the default listing mode if the source is a parallel file system (PFS). If this mode is selected when the source is a bucket, the listing operation may take a long time.

    Concurrent Subtasks

    -

    User-defined. There cannot be more than 10 concurrent subtasks for each online migration node. For example, if there are 2 online migration nodes, the maximum number of subtasks can be 20 or any number below.

    Overwrite Existing

    Never

    Files existing at the target will never be overwritten.

    WARNING:
    • If you choose Never for the initial migration, the attributes of involved parent folders at the source will not be migrated to the target. As a result, the folder attributes may be incomplete at the target. To avoid this issue, use the Never option with caution for the initial migration.
    • If you choose Never, restarting a migration after an interruption or pause may lead to incomplete data migration, even though the task may appear successful. This could impact data integrity, so use the Never option with caution.

    Always

    Files existing at the target will always be overwritten.

    If older or different size
    • The system replaces existing target files if they are older than or differ in size from their source counterparts. Files with matching modification times and sizes remain unchanged and are skipped during migration.
    • The system verifies folders after their content is migrated. Folders that already exist at the target will be overwritten if they have different last modification times, sizes, or permissions from the paired folders at the source.
      NOTE:

      The same overwriting policy is applied to empty folders as files.

    If different CRC 64 checksum
    If a source object has a CRC64 checksum different from the paired target object, the source object will overwrite the target one. Otherwise, the source object will be skipped during the migration. If either of them does not have a CRC64 checksum, their sizes and last modification times are checked.
    NOTE:
    • This option is only available for migration within Huawei Cloud or from Alibaba Cloud or Tencent Cloud.
    • Using this option requires that target OBS buckets be whitelisted for the CRC64 feature.

    Consistency Verification

    Size and last modified

    With this default method, the system checks data consistency by comparing object size and last modification time.

    CRC64 checksum
    The system verifies data consistency by comparing CRC64 values in the metadata. If a source object and the paired target object have CRC64 checksums, the checksums are checked. Otherwise, their sizes and last modification times are checked.
    NOTE:
    • This option is only available for migration within Huawei Cloud or from Alibaba Cloud or Tencent Cloud.
    • Using this option requires that target OBS buckets be whitelisted for the CRC64 feature.

    Migrate Metadata

    -

    Determine whether to migrate metadata.

    • If you select this option, object metadata will be migrated.
    • If you do not select this option, only the Content-Type and Content-Encoding metadata will be migrated.

    Clear Cluster

    -

    Determine whether to clear the migration cluster after the migration is complete.

    • If you select this option, a step for clearing the migration cluster will be created in the workflow. You can also choose whether to clear resources used by the cluster, such as NAT gateways, security groups, and VPCEP resources.
    • If you do not select this option, a step for clearing the migration cluster will not be created in the workflow, but the migration cluster and its resources will automatically be deleted 30 days after the workflow is created.

  8. Click before (Optional) Advanced Settings to expand advanced options and select Restore Archive Data. Configure the other advanced options as needed. For details, see Table 5.

    Table 5 Advanced options

    Function

    Description

    Target Storage Class

    Choose the storage class that your data will be migrated to. For details about storage classes, see How Do I Choose Storage Classes?

    NOTE:

    CRC64-based consistency verification is not available for Archive and Deep Archive storage classes. Even if you choose the CRC64-based verification method, the system automatically uses the object size and last modification time to verify data consistency.

    Enable KMS Encryption
    • If you do not select this option, whether migrated data will be encrypted in the target buckets depends on the server-side encryption setting of the buckets.
    • If you select this option, all migrated objects will be encrypted before they are stored to the target bucket.
    NOTE:
    • Using KMS to encrypt migrated data may slow down the migration speed by about 10%.
    • This option is only available when KMS is supported in the region you are migrating to.

    Restore Archive Data
    • If you do not select this option, the system records archived objects in the list of objects that failed to be migrated and continues to migrate other objects in the migration task.
    • If you select this option, the system automatically restores and migrates archived objects in the migration task. If an archived object fails to be restored, the system skips it and records it in the list of objects that failed to be migrated and continues to migrate other objects in the migration task.
      NOTE:
      • The system will restore all listed archived data before migrating it, and you pay the source cloud platform for the API requests and storage space generated accordingly.
      • Data in the Deep Cold Archive storage of Alibaba Cloud OSS cannot be restored using the restoration function of MgC. You need to manually restore the data before migration.
      • Microsoft Azure Blob does not support automatic archiving of restored data. If needed, you can manually archive the data after the migration is complete.

    Filter Source Data

    Use exclude patterns, include patterns, and time ranges to filter files to be migrated. For details about filters, see Source Data Filters.

    Send SMN Notifications

    Determine whether to use SMN to get notifications about migration results.

    • If you do not select this option, no SMN messages will be sent after the migration is complete.
    • If you select this option, after the migration is complete, SMN messages will be sent to the subscribers of the selected topic. You can select the language and trigger conditions for sending messages.

    Limit Traffic

    Set the maximum bandwidth to be used by the migration workflow during different periods.

    • If you do not select this option, migration traffic will not be limited.
    • If you select this option, you can limit the migration traffic by setting Start Time, End Time, and Bandwidth Limit.
      For example, if you set Start Time to 08:00, End Time to 12:00, and Bandwidth Limit to 20 MB/s, the maximum migration speed is limited to 20 MB/s from 08:00 to 12:00. The migration speed is not limited outside this period.
      NOTE:
      • The bandwidth limit ranges from 1 MB/s to 1,048,576 MB/s.
      • A maximum of five rules can be added.
      • The time is the local standard time of the region you are migrating to.

    Schedule Migration

    Schedule the migration to run during a period.

    • If you do not select this option, you need to manually start or stop the migration.
    • If you select this option, the migration runs during the specified period and stops outside that period.

      For example:

      • If you set Start Time to 08:00 and End Time to 12:00, the migration task runs from 08:00 to 12:00 every day. The migration stops outside that period.
      • If you set Start Time to 12:00 and End Time to 08:00, the migration runs from 12:00 of the current day to 08:00 of the next day. The migration stops outside that period.

  9. Click Next: Confirm.
  10. Confirm the workflow settings and click Confirm. The Run Workflow dialog box is displayed, which indicates that the workflow has been created.

    • If you want to start the migration immediately, click Confirm to run the workflow.
    • If you want to add a stage or step to the workflow, click Cancel. The workflow enters a Waiting state, and the migration has not started yet. To start the migration, click Run in the Operation column.

  11. On the migration workflow details page, view the workflow settings and the migration progress. You can also perform the operations below.

    • Move the cursor to the migration progress bar of a resource. In the displayed window, view the migration details about the resource.
    • When a migration reaches a step that requires manual confirmation, place the cursor on the progress bar and click Confirm next to the step status in the displayed window. The migration can continue only after you confirm.
    • In the Basic Information area, click Manage next to the cluster name. The cluster details page is displayed on the right. On the displayed page, you can:
      • Add, edit, or delete traffic limiting rules to control cluster traffic based on your requirements.
      • Add or delete migration nodes or list nodes, or upgrade plug-ins for existing nodes as required.

Parent Topic: Storage Migration