Migrating Archive (Cold) Data

Background

Cold storage is a long-term backup solution for infrequently accessed data. For example, data accessed annually can be stored in archive storage, while data accessed every few years can be stored in deep archive storage. This storage mode has advantages such as high security, durability, and low costs.

When migrating cold storage data, you need to consider data access latency because data is usually in a "frozen" state. Data must be restored or unfrozen before it can be accessed. Typically, there are two restoration options: standard and expedited retrieval. Their differences are as follows:

Standard: Archive objects can be restored within 3 to 5 hours. This option is ideal for scenarios where immediate access is not required.
Expedited: Archive objects can be restored within 1 to 5 minutes, and additional fees may be incurred. This option is ideal for scenarios where immediate access is required.

MgC storage migration workflows enable you to restore archive source data expeditiously.

Supported Regions

The following regions are supported:

LA-Santiago
LA-Sao Paulo
TR-Istanbul
AP-Bangkok
AP-Singapore
AP-Jakarta
ME-Riyadh

Notes and Constraints

For more information, see What Are the Restrictions on Using MgC for Storage Migration?

Preparations

Preparing a Huawei account
Before using MgC, prepare a HUAWEI ID or an IAM user that can access MgC and obtain an AK/SK pair for the account or IAM user. For details about how to obtain an access key, see Making Preparations.
Creating a migration project
On the MgC console, create a migration project. For details, see Managing Migration Projects.

Creating an OBS bucket
Create an OBS bucket on Huawei Cloud in the target region for receiving source data. For details, see Creating a Bucket.
Configuring permissions
Ensure that the source and target accounts have the permissions required for the migration. For details, see How Do I Obtain Required Permissions for the Source and Target Accounts?
Creating a migration cluster
You can create a dedicated migration cluster for this task. A cluster consists of a master node, and several list and migration nodes. For details about how to create a cluster, see Creating a Migration Cluster.

Creating a Migration Workflow

Sign in to the MgC console.
In the navigation pane on the left, choose Migrate > Workflows. In the upper left corner of the page, select the migration project you created.
Click Create Workflow in the upper right corner of the page.
Select Storage Migration and click Configure Workflow.

Set workflow basics based on Table 1.

**Table 1** Basic parameters
Parameter	Description
Name	User-defined
Region	Select the region where the target bucket is located from the drop-down list.
Description	User-defined
Cluster	Select the cluster you created.

Configure the migration source and target based on Table 2 and Table 3.

**Table 2** Parameters for configuring a migration source
Parameter	Description
Location Type	Select the source cloud platform and data source.
AK	Enter the AK of the source cloud account.
SK	Enter the SK of the source cloud account.
Bucket	Enter the name of the source bucket to be migrated.
Endpoint	Enter the endpoint of the region where the source bucket is located. For example, if Location Type is set to Alibaba Cloud OSS and the source bucket is located in CN East 1 (Hangzhou), enter oss-cn-hangzhou.aliyuncs.com.
Type	This parameter is available when Huawei Cloud OBS is selected for Location Type. Set this parameter based on the source bucket type. You can view the bucket type in the basic information.
AppID	This parameter is available when Tencent Cloud COS is selected for Location Type. Enter the AppID of your Tencent Cloud account. NOTE: You can view the AppID on the account information page of the Tencent Cloud console.

**Table 3** Parameters for configuring a migration target
Parameter	Description
Location Type	Select Huawei Cloud OBS.
AK	Enter the AK/SK pair of the target Huawei Cloud account.
SK	Enter the AK/SK pair of the target Huawei Cloud account.
Bucket	Select the created OBS bucket.
Endpoint	Enter the endpoint of the region where the target bucket is located. For example, if the target bucket is located in the CN North-Beijing4 region of Huawei Cloud, enter obs.cn-north-4.myhuaweicloud.com. NOTE: You can view the endpoint in the basic information about the bucket.
Specify Prefix	This parameter is optional. Specify a prefix to rename or relocate objects migrated to the target bucket. For example, if you specify the prefix /D, source file /A/B/C.txt will be relocated to /D/A/B/C.txt after being migrated to the target bucket. For details, see: Adding a Name Prefix or Path Prefix to Migrated Objects

Configure the migration settings based on Table 4.

**Table 4** Migration settings
Parameter	Option	Description
Task Type	List migration	Migrates objects recorded in the list files.
Concurrent Subtasks	-	Specify the maximum number of concurrent subtasks. There cannot be more than 10 concurrent subtasks for each online migration node. For example, if there are 2 online migration nodes, the maximum number of subtasks can be 20 or any number below.
Overwrite Existing	Never	Files existing at the migration target will never be overwritten. WARNING: If you choose Never for the initial migration, the attributes of involved parent folders at the source will not be migrated to the target. As a result, the folder attributes may be incomplete at the target. To avoid this issue, use the Never option with caution for the initial migration. If a migration task is paused or interrupted and then restarted or resumed, the Never option will cause the system to skip files that were not completely migrated earlier, but the task may still be marked as successful. This affects data integrity. To avoid this issue, use the Never option with caution.
	Always	Files existing at the migration target will always be overwritten.
	If older or different size	Files that already exist at the target will be overwritten if they are older than or have different sizes from the paired files at the source. Verification will be performed for folders after their contents are migrated. Folders that already exist at the target will be overwritten if they have different last modification times, sizes,or permissions from the paired folders at the source. NOTE: For empty folders, the overwrite policy is the same as that for files.
Clear Cluster	-	Determine whether to clear the migration cluster after the migration is complete. If you select this option, a step for clearing the migration cluster will be created in the workflow. You can also choose whether to clear resources used by the cluster, such as NAT gateways, security groups, and VPCEP resources. If you do not select this option, a step for clearing the migration cluster will not be created in the workflow.

Click

before (Optional) Advanced Settings to expand advanced options and select Restore Archive Data. Configure the other advanced options as needed. For details, see Table 5.

**Table 5** Advanced options
Function	Description
Target Storage Class	Choose the storage class that your data will be migrated to in the target bucket. For details about storage classes, see Introduction to Storage Classes.
Enable KMS Encryption	If you do not select this option, objects are in the same encryption status before and after the migration. If you select this option, all migrated objects will be encrypted before they are stored in the target bucket. NOTE: Using KMS to encrypt migrated data may slow down the migration speed by about 10%. This option is only available when KMS is supported in the region you are migrating to.
Restore Archive Data	If you do not select this option, the system records archived objects in the list of objects that failed to be migrated and continues to migrate other objects in the migration task. If you select this option, the system automatically restores and migrates archived objects in the migration task. If an archive object fails to be restored, the system skips it and records it in the list of objects that failed to be migrated and continues to migrate other objects in the migration task. NOTE: The system will restore archive data before migrating it, and you pay the source cloud platform for the API requests and storage space generated accordingly.
Filter Source Data	Filter files to be migrated by applying filters. For details about filters, see Source Data Filters.
Send SMN Notification	Determine whether to use SMN to get notifications about migration results. If you do not select this option, no SMN messages are sent after the migration is complete. If you select this option, after the migration is complete, SMN messages are sent to the subscribers of the selected topic. You can select the language and trigger conditions for sending messages.
Limit Traffic	Allocate the maximum bandwidth to be used by the workflow during a specified period. If you do not select this option, migration traffic is not limited. If you select this option, limit the migration traffic by setting Start Time, End Time, and Bandwidth Limit. For example, if you set Start Time to 08:00, End Time to 12:00, and Bandwidth Limit to 20 MB/s, the maximum migration speed is limited to 20 MB/s when the migration task runs from 08:00 to 12:00. The migration speed is not limited beyond this period. NOTE: The rate limit ranges from 0 MB/s to 1,048,576 MB/s. A maximum of five rules can be added. The time is the local standard time of the region you are migrating to.
Schedule Migration	Schedule the migration to run during a period. If you do not select this option, you need to manually start or stop the migration. If you select this option, the migration runs during the specified period and stops beyond that period. For example: If you set Start Time to 08:00 and End Time to 12:00, the migration task runs from 08:00 to 12:00 every day. The migration stops beyond that period. If you set Start Time to 12:00 and End Time to 08:00, the migration runs from 12:00 of the current day to 08:00 of the next day. The migration stops beyond that period.

Click Next: Confirm.
Confirm the workflow settings, and click Confirm. The Run Workflow dialog box is displayed, which indicates that the workflow has been created.
- If you want to start the migration immediately, click Confirm to run the workflow.
- If you want to add a stage or step to the workflow, click Cancel. The workflow enters a Waiting state, and the migration has not started. To start the migration, click Run in the Operation column.
On the migration workflow details page, view the workflow settings and the migration progress. You can also perform the following operations:
- Move the cursor to the migration progress bar of a resource. In the displayed window, view the migration details about the resource.
- When a migration reaches a step that requires manual confirmation, place the cursor on the progress bar and click Confirm next to the step status in the displayed window. The migration can continue only after you confirm.
- In the Basic Information area, click Manage next to the migration cluster name. The cluster details page is displayed on the right. On the displayed page, you can:
  - Add, edit, or delete traffic limiting rules to control cluster traffic based on your requirements.
  - Add or delete migration nodes or list nodes, or upgrade plug-ins for existing nodes as required.