Help Center > > Getting Started

Creating a Migration Task

Updated at: Apr 08, 2021 GMT+08:00

Scenarios

This section describes how to create a task to migrate your source objects.

The following uses the File/Folder migration method as an example.

Background

The following migration sources are supported:

  • HUAWEI CLOUD
  • AWS
  • Alibaba Cloud
  • Microsoft Azure
  • Tencent Cloud
  • Qiniu Cloud
  • QingCloud
  • Baidu Cloud
  • Kingsoft Cloud
  • HTTP/HTTPS data source

OMS makes API calls to the object storage services on source platforms during migration. These API calls may generate fees on the source platforms and you will be responsible for those fees based on those source platform billing structures.

Prerequisites

  • You have obtained the AK/SK pairs for the source and destination platform accounts.
    • The source platform account needs the following permissions:
      • Listing all buckets
      • Obtaining bucket location
      • Listing objects
      • Obtaining object metadata
      • Obtaining object content

      For details about how to obtain these permissions, see Source Permissions.

    • The destination platform account needs the following permissions:
      • Listing all buckets
      • Obtaining bucket location
      • Listing objects
      • Obtaining object metadata
      • Obtaining object content
      • Listing uploaded parts
      • Restoring archive objects

      For details about how to obtain these permissions, see Destination Permissions.

  • You have created at least one bucket on OBS.
  • You have created less than 500 migration tasks during the past 24 hours.
  • You have less than 500 migration tasks in the Waiting to migrate state.
  • You have registered an account with HUAWEI CLOUD and completed real-name authentication.

Constraints

OMS only supports the migration of the ContentType fields of object metadata.

Procedure

  1. Log in to the management console.
  2. Click in the upper left corner and select the desired region.
  3. Click Service List. Under Migration, click Object Storage Migration Service.

    The Object Storage Migration Service page is displayed.

  4. In the upper right corner, click Create Migration Task.

    Read the privacy statement, select Agree to the above privacy agreements, and click OK.
    Figure 1 Creating a task

  5. For step Select Source/Destination, you need to:

    1. Configure source parameters. For details, see Table 1.
      Table 1 Source parameters

      Parameter

      Description

      Data Source

      Specifies the source cloud service provider.

      Access Key

      Specifies the AK of the source platform account. Enter a maximum of 100 characters.

      Secret Access Key

      Specifies the SK of the source platform account. The SK matches with the AK. Enter a maximum of 100 characters.

      Specify Bucket Name

      If you do not have the permissions required to list buckets but have the permissions to list objects in buckets, obtain object metadata, and obtain objects, enable this parameter to specify the path of the source bucket for the migration.

      For details, see Table 2.

      Table 2 Description of Specify Bucket Name

      If You Need to...

      Then...

      Enable Specify Bucket Name

      1. Select the region where the source bucket resides from the Select Region drop-down list.
      2. In the Bucket Name or Path text box, enter the source bucket name or the source bucket path with the bucket name.
      3. Click Connect to Source Bucket.

        OMS will check whether the source platform information is correct. If the entered information is correct, a message is displayed indicating that the source bucket is connected.

      Disable Specify Bucket Name (This feature is disabled by default.)

      1. Click Connect to Source Bucket.

        OMS will check whether the source platform information is correct. If the entered information is correct and your account has the permissions needed to list bucket names, all buckets are listed in the Bucket drop-down list.

      2. In the Bucket drop-down list, select the bucket where the data to be migrated resides.
    2. Configure destination parameters. For details, see Table 3.
      Table 3 Destination parameters

      Parameter

      Description

      Access Key

      Specifies the AK for accessing HUAWEI CLOUD. Enter a maximum of 100 characters.

      Secret Access Key

      Specifies the SK for accessing HUAWEI CLOUD. The SK matches with the AK. Enter a maximum of 100 characters.

      After entering the AK and SK, click Connect to Destination Bucket.

      Bucket

      Specifies the destination OBS bucket for storing the migrated data.

  6. Click Next.

    The Select Migration Method page is displayed, as shown in Figure 2.
    Figure 2 Selecting a migration method

  7. Click Select next to the Bucket box. The Select File/Folder dialog box is displayed, as shown in Figure 3. Select the files or folders to be migrated and click OK.

    Figure 3 Selecting files or folders
    • If you select Qiniu Cloud for Data Source, the File/Folder migration method is unavailable.
    • If you do not select specific files or folders, all objects in the source bucket will be migrated.
    • For each migration task, you can select a maximum of 500 files or folders in a given directory.
    • The name of any individual object on OBS cannot exceed 1,024 characters. If the name of a migrated object on OBS exceeds this limit, the migration task will fail.

  8. (Optional) Click Show next to Advanced Settings to expand the advanced parameters.

    Figure 4 Setting advanced options

  9. (Optional) Set Advanced Settings parameters. For details, see Table 4.

    Table 4 Advanced Settings parameters

    Parameter

    Description

    Data Encryption

    If you select KMS, migrated data is encrypted before being transferred to the destination OBS buckets.
    NOTE:

    Using KMS to encrypt migrated data may decrease the migration speed by about 10%.

    This option is only available if KMS encryption is supported in the current region.

    Failed Object Record

    Specifies whether to record the objects that fail to be migrated.

    • Set to No

      The system does not record the objects that fail to be migrated. If the migration fails and needs to be retried, all objects in the source bucket need to be transmitted again.

    • Set to Yes

      The system will record the objects that failed to be migrated. If the migration fails and needs to be retried, only the failed objects need to be transmitted again.

    NOTE:

    The objects that fail to be migrated are recorded in an object list file named by the task name and uploaded to the oms/failed_object_lists directory of the destination bucket.

    This file contains information about the failed migration tasks, including the failure causes, occurrence time (local UTC), URL-encoded names of failed objects, non-URL-encoded names of failed objects, and error codes returned by source SDKs.

    This file is stored in the destination bucket after the objects in the file are successfully retransmitted. If you do not need the file, delete it manually.

    Incremental Data

    Specifies whether to filter source objects to be migrated by time.

    • If this option is set to No, source objects are not filtered by time.
    • If this option is set to Yes, only the source objects that are modified or added after the specified time are migrated. You need to click to specify the time.

    SMN Notification

    Specifies whether to use SMN to notify you of migration results.
    • Set to No

      No SMN message is sent after the migration.

    • Set to Yes

      You will receive an SMN message informing you of the migration results.

      Perform the following steps to set SMN parameters:

      1. Click Add next to the Select SMN Topic area.

        The Select SMN Topic dialog box is displayed.

      2. Select a topic for receiving SMN messages and click OK.

        If OMS is not authorized to publish messages to the selected SMN topic, a Warning dialog box will display after you click OK. If you need the authorization, click OK to authorize OMS. Otherwise, click Cancel to return to the Select SMN Topic page and select another SMN topic.

      1. Configure Trigger Condition. You can select one of the following options or both:
      • Migration succeeded

        An SMN message will be sent after the migration task succeeds.

      • Migration failed

        An SMN message will be sent after the migration task fails.

      NOTE:

      If you experience trouble in using SMN, see What Can I Do If the SMN Function Works Improperly?

    Traffic Limit

    Set the maximum bandwidth for the migration task during a specified period of time.

    • If you do not need to limit migration traffic, select No.
    • If you need to limit migration traffic, select Yes.

      Perform the following steps to set traffic limits:

      1. Click Add Rule.

        A traffic limiting rule is displayed.

      2. Specify Start Time, End Time, and Maximum Bandwidth.

        The value range of the maximum bandwidth is 1 Mbyte/s to 200 Mbyte/s.

      3. Repeat 1 and 2 to add another traffic limiting rule.

        You can click in the Operation column to delete a rule.

        NOTE:

        You can create a maximum of five rules.

        You can set traffic limiting rules in the task details even after the task is created.

    Description

    In the Description area, enter a maximum of 255 characters to briefly describe the migration task.

  10. Click Next to enter the confirmation page.
  11. Confirm the settings of the migration task and start the migration.

    1. Check whether the settings of migration parameters are correct.

      If the settings are incorrect, click Previous to return to the previous page for modification.

    2. Click Migrate Now.

      A message is displayed, indicating that the migration task is successfully created. The migration task is displayed in the migration task list.

      For details about migration task statuses, see Viewing a Migration Task.

      After the task is created, a task ID is generated for you for quick identification. It is displayed in the upper left corner of the task list. It is composed of the source bucket name, destination bucket name, and time sequence ID.

Did you find this page helpful?

Submit successfully!

Thank you for your feedback. Your feedback helps make our documentation better.

Failed to submit the feedback. Please try again later.

Which of the following issues have you encountered?







Please complete at least one feedback item.

Content most length 200 character

Content is empty.

OK Cancel