Scenario

CDM allows you to directly migrate object storage data from a third-party cloud to OBS without forwarding or writing code.

This section describes how to use CDM to migrate files from OSS on Alibaba Cloud to OBS. The procedure is as follows:

1. Creating a CDM Cluster and Binding an EIP to the Cluster
2. Creating an OBS Link
3. Creating an OSS Link
4. Creating a Migration Job

Preparing Data

• Endpoint for accessing OSS, for example, oss-cn-hangzhou.aliyuncs.com
• AK, temporary credential, or security token for accessing OSS
• Domain name, port number, AK, and SK for accessing OBS

Creating a CDM Cluster and Binding an EIP to the Cluster

1. Log in to the CDM management console and create a CDM cluster. For details about how to create a CDM cluster, see Creating a CDM Cluster. The key configurations are as follows:
   • Select the cdm.medium instance, which is applicable to most migration scenarios.
   • If the cluster is used only to migrate data from third-party data sources to OBS on HUAWEI CLOUD, there is no special requirements on the VPC, subnet, and security group of the CDM cluster. You can specify them based on your needs.

2. After the CDM cluster is created, on the Cluster Management page, click Bind EIP in the Operation column to bind an EIP to the cluster. The CDM cluster accesses Alibaba Cloud OSS through the public network.

   Because data is imported to HUAWEI CLOUD, 5 Mbit/s bandwidth for the EIP is enough.

   

   If SSL encryption is configured for the access channel of a local data source, CDM cannot connect to the data source using the EIP.

Creating an OBS Link

1. Click Job Management in the Operation column of the CDM cluster. On the page that is displayed, choose Link Management > Create Link. The page for selecting a connector is displayed. See Figure 1.
   Figure 1 Selecting a connector
   

2. Select Object Storage Service and click Next. On the page that is displayed, configure the OBS link parameters.
   • Name: Enter a custom link name, for example, obslink.
   • OBS Server and Port: Enter the actual OBS address information.
   • AK and SK: Enter the AK and SK used for logging in to OBS.
   Figure 2 Creating an OBS link
   

3. Click Save. The Link Management page is displayed.

Creating an OSS Link

1. On the Link Management tab page, click Create Link. On the page that is displayed, select Alibaba Cloud OSS, click Next, and configure the required link parameters. See Figure 3.
   • Name: Enter a custom link name.
   • OSS Endpoint: Configure the endpoint of the data to be migrated. Select the endpoint based on the region where the bucket resides.
   • Authentication Method: Select an authentication method based on your needs, for example, Access key.
   • AK and SK: Enter the AK and SK used for logging in to OSS.
   • IP and domain Name Mapping: Configure the mapping between IP addresses and domain names.
   Figure 3 Creating an OSS link
   

2. Click Save. The Link Management page is displayed.

Creating a Migration Job

1. Choose Table/File Migration > Create Job to create a job for migrating data from OSS to OBS. See Figure 4.
   Figure 4 Creating a job for migrating data from OSS to OBS
   
   • Job Name: Enter a custom job name.
   • Source Job Configuration
     • Source Link Name: Select the osslink link created in Creating an OSS Link.
     • Bucket Name: Select the bucket from which the data is to be migrated.
     • Source Directory/File: Set this parameter to the path of the data to be migrated. You can migrate all files in the bucket.
     • File Format: Select Binary. It is applicable to file copy. To write files to databases, select CSV or JSON.
     • Retain the default values of the optional parameters in Show Advanced Attributes. For details, see From OBS/OSS/KODO/COS/S3.
   • Destination Job Configuration
     • Destination Link Name: Select the obslink link created in Creating an OBS Link.
     • Bucket Name: Select the bucket to which data is to be written.
     • Write Directory: Select the path for storing data.
     • File Format: Select Binary. The value must be the same as that on the migration source.
     • Retain the default values of other optional parameters. For details, see To OBS.

2. Click Next and set task parameters. Generally, retain the default values of all parameters.
   In this step, you can configure the following optional functions:

   • Retry Upon Failure: If the job fails to be executed, you can determine whether to automatically retry. Retain the default value Never.
   • Group: Select the group to which the job belongs. The default group is DEFAULT. On the Job Management page, jobs can be displayed, started, or exported by group.
   • Schedule Execution: To configure scheduled jobs, see Scheduling Job Execution. Retain the default value No.
   • Concurrent Extractors: Enter the number of extractors to be concurrently executed. Retain the default value 1.
   • Write Dirty Data: Specify this parameter if data that fails to be processed or filtered out during job execution needs to be written to OBS for future viewing. Before writing dirty data, create an OBS link. Retain the default value No so that dirty data is not recorded.
   • Delete Job After Completion: Retain the default value Do not delete.

3. Click Save and Run. The Job Management page is displayed, on which you can view the job execution progress and result.
4. After the job is successfully executed, in the Operation column of the job, click Historical Record to view the job's historical execution records and read/write statistics.

   On the Historical Record page, click Log to view the job logs.