Creating a Task to Migrate a File or Folder

Scenarios

Object Storage Migration Service (OMS) is an online data migration service that helps you quickly, easily, and securely move data from object storage on other clouds to Object Storage Service (OBS) on Huawei Cloud. This section describes how to use OMS to migrate specific files and folders.

The following configurations are used in this example:

• Source: Huawei Cloud
• Destination: Huawei Cloud
• Migration method: File/Folder
• Bucket name: 1216testobsbucket
• Folder: BigData
• Destination configuration: Retain default settings.

Precautions

• By default, if a task stays in the waiting state for more than 30 days, OMS automatically puts the task into the Failed state and clears the AK/SK pair used by the task. To prevent task failures caused by blocking, do not create tasks too frequently.
• If a single file is larger than 500 GB, you are advised to use MgC dedicated clusters to improve the migration speed and stability.
• During the migration, the system automatically creates a temporary folder named OMS in the destination bucket. Do not perform any operations on this folder, including but not limited to modifying, deleting, or adding data in the folder. Otherwise, the migration will be interrupted or fail.
• You have created fewer than 1,000,000 migration tasks within the past 24 hours.
• Migration tasks in the Waiting state under your account are fewer than 1,000,000.

Procedure

Preparations

Before creating an object storage migration task, configure the account, permissions, and destination OBS bucket by referring to Preparing for Migration.

Step 1: Performing Pre-Migration Evaluation

1. Log in to the OMS console.
2. Click in the upper left corner and select a region from the drop-down list.
3. On the Migration Tasks page, click Create Migration Task in the upper right corner.
4. Read the service disclaimer, select I have read and agree to the service disclaimer, and click OK. The Create Migration Task page is displayed.
5. Determine whether to enable pre-migration evaluation.
   • If you disable it, pre-migration evaluation will not be performed.
   • If you enable it, specify the number of files to be migrated, the amount of data to be migrated, and the desired migration duration. Then, click Start Evaluation. After the evaluation is complete, the result is displayed under Start Evaluation.

Step 2: Setting Source and Destination Parameters

1. Select the source parameters, as shown in Figure 1. Then, set the parameters based on Table 1.
   Figure 1 Creating a task
   

   Table 1 Source parameters

   Parameter	Example Value	Description
   Data Source	Huawei Cloud	Select the source cloud service provider.
   Access Key ID	A valid AK of Huawei Cloud.	Enter the AK for accessing the source platform. It can contain a maximum of 100 characters.
   Secret Access Key	A valid SK of Huawei Cloud.	Enter the SK for accessing the source platform. The SK matches the AK, and they form a pair. It can contain a maximum of 100 characters.
   KMS Key	Disabled	Choose whether to use a KMS key to encrypt your source key.
   Specify Source Bucket	No	Enable this option if your source account does not have the permissions required to list buckets but has the permissions to list objects in buckets, obtain object metadata, and obtain object content. Then specify the bucket or path to be migrated. For details, see Creating a Migration Task.
   Bucket	1216testobsbucket	Click the Bucket drop-down list or the refresh icon and select the bucket where data needs to be migrated. OMS will check whether the source platform can be connected using the information you provided. Once the source platform is connected, as long as your account has the necessary permissions to list buckets, all buckets in your account will appear in the Bucket drop-down list.

2. Configure destination parameters by referring to Table 2.

   Table 2 Destination parameters

   Parameter	Example Value	Description
   Access Key ID	Enter the AK for accessing Huawei Cloud.	Enter the AK for accessing Huawei Cloud. It can contain a maximum of 100 characters.
   Secret Access Key	Enter the SK for accessing Huawei Cloud.	Enter the SK for accessing Huawei Cloud. The SK matches the AK, and they form a pair. It can contain a maximum of 100 characters.
   KMS Key	Disabled	Choose whether to use a KMS key to encrypt your destination key.
   Bucket	1216testobsbucket	Click the Bucket drop-down list or click Refresh and select the Huawei Cloud OBS bucket for storing the migrated data. The bucket must be located in the region where you are using OMS.

Step 3: Setting Task Parameters

1. After the source/destination is selected, click Next in the lower right corner to access the Set Task Parameters page.
2. Set source parameters as prompted. Set parameters for other migration methods by referring to Creating a Migration Task.
   Figure 2 Source configurations
   

   Table 3 Source parameters

   Parameter	Example Value	Description
   Migration Method	File/Folder	Set this based on your requirements.
   File/Folder	BigData	Select a file or folder to be migrated. If it is not specified, all objects in the bucket will be migrated by default. If only a prefix is entered, all matched objects will be migrated.

3. Set destination parameters as prompted.

   Table 4 Destination parameters

   Parameter	Example Value	Description
   Specify Prefix	Disabled	Enabled: You need to enter a prefix. After the migration, the prefix will be added to the object names in the destination bucket. For details, see Adding a Name Prefix or Path Prefix to Migrated Objects.
   Object Storage Class	Retain the default settings.	Choose the storage class that your objects will be migrated to. For details, see How Do I Choose Storage Classes?

4. Click next to Advanced Options to expand the advanced options. Set the parameters based on Table 5.

   Table 5 Advanced parameters

   Parameter	Example Value	Description
   Object Metadata	Do not migrate	Choose a metadata migration method.
   Selective Migration	Disabled	If you enable this option, you can specify a time so that only source objects modified after that time are migrated.
   Failed Object Record	Enabled	Determine whether to record the objects that fail to be migrated.
   Archive Data Restoration	Disabled	Determine whether to automatically restore archived data and migrate it. Enabling this option will incur additional costs. NOTE: This function is applicable to the following clouds: Huawei Cloud (COLD), Alibaba Cloud (Archive), Kingsoft Cloud (ARCHIVE), Tencent Cloud (ARCHIVE), and UCloud (ARCHIVE).
   SMN Notification	No	If you enable this option, you will be notified of the migration result. Ensure that you have subscribed to the selected topic. Otherwise, you will not receive SMN notifications at the desired address.
   Traffic Limit	Disabled	Set the maximum bandwidth for the migration task during different periods.
   Object Overwrite	Overwrite if source newer or different size	How to overwrite objects with the same name at the source and destination.
   Consistency Check	By size and last modification time	How to check the data consistency after the migration.
   Priority	Medium	If there are a large number of tasks, you can set task priorities to control which tasks take precedence over other tasks.
   Description	Customizable	Enter a brief description about the migration task. A maximum of 255 characters is allowed.

5. Click Next in the lower right corner of the page.
6. Confirm the migration information and click Migrate Now.

   Read the prompted information. After you confirm, click OK.

7. Go back to the task list and check the created task.
   Figure 3 Task list
   

8. Click the task name to check the number of migrated objects, total number of objects, task progress, migration speed, source and destination information, and migration task settings.