Scenario

CDM can periodically upload new files to OBS. You do not need to compile code or manually upload the files, but directly use the massive storage capabilities of OBS to back up files.

This section describes how to periodically back up FTP files to OBS with CDM.

For example, the to_obs_test directory on the FTP server contains one subdirectory another_dir and two files file1 and file2. file2 is in the another_dir directory. Figure 1 shows the files. Configure a scheduled job of CDM to transfer these files to OBS and add file3 and file4 to the directory to verify that CDM can periodically transfer new files to OBS.

Figure 1 Files on the FTP server

Prerequisites

• You have sufficient EIP quota.
• You have created an OBS bucket and obtained the access key (AK and SK).
• You have obtained the IP address, username, and password of the FTP server.
• If the FTP server is in the on-premises environment, ensure that the FTP server is accessible through the public network, or the VPN or Direct Connect between the on-premises data center and HUAWEI CLOUD has been established.

Creating a CDM Cluster and Binding an EIP to the Cluster

1. Log in to the CDM management console and click Buy CDM Cluster to create a CDM cluster. The key configurations are as follows:
   • Select the cdm.medium instance, which is applicable to most migration scenarios.
   • If the cluster is used only to migrate data from third-party data sources to OBS on HUAWEI CLOUD, there is no special requirements on the VPC, subnet, and security group of the CDM cluster. You can specify them based on your needs.

2. After the CDM cluster is created, on the Cluster Management page, click Bind EIP in the Operation column to bind an EIP to the cluster. The CDM cluster uses the EIP to access the on-premises FTP server.
   

   If SSL encryption is configured for the access channel of a local data source, CDM cannot connect to the data source using the EIP.

Creating an OBS Link

1. Click Job Management in the Operation column of the CDM cluster. On the page that is displayed, choose Link Management > Create Link. The page for selecting a connector is displayed. See Figure 2.
   Figure 2 Selecting a connector
   

2. Select Object Storage Service and click Next. On the page that is displayed, configure the OBS link parameters.
   • Name: Enter a custom link name, for example, obslink.
   • OBS Server and Port: Enter the actual OBS address information.
   • AK and SK: Enter the AK and SK used for logging in to OBS.
   Figure 3 Creating an OBS link
   

3. Click Save. The Link Management page is displayed.

Creating an FTP Link

1. On the Link Management tab page, click Create Link. On the page that is displayed, select FTP, click Next, and configure the FTP link parameters.
   • Name: Enter a custom link name, for example, ftplink.
   • Host Name/IP Address and Port: Enter the address information about the FTP server.
   • Username and Password: Enter the username and password used for logging in to the FTP server.

2. Click Save. The Link Management page is displayed.

Creating a Scheduled Migration Job

1. Choose Table/File Migration > Create Job to create a data migration job.
   Figure 4 Creating a job for migrating data from FTP to OBS
   
   • Job Name: Enter a custom job name.
   • Source Link Name: Select the ftplink link created in Creating an FTP Link.
     • Source Directory/File: Select the path where to_obs_test is located.
     • File Format: Select Binary. It is applicable to file copy. To write files to databases, select CSV or JSON.
   • Destination Link Name: Select the obslink link created in Creating an OBS Link.
     • Bucket Name: Select the OBS bucket for storing FTP files.
     • Write Directory: Select an existing directory or manually enter one. If the entered directory does not exist, CDM automatically creates one, for example, /to/ftp2obs/.
     • File Format: Select Binary. The value must be the same as that on the migration source.
     • Duplicate File Processing Method: Select Skip to avoid transferring duplicate files.

2. Click Next and configure the scheduled task. In this example, the scheduled task is executed every 10 minutes. Retain the default values for other parameters.
   Figure 5 Scheduling job execution
   

3. Click Save and Run.

Verifying Backup

1. After the job is executed successfully, log in to the OBS client. You can see that the corresponding files exist on OBS. Figure 6 shows the files on OBS.
   Figure 6 Files on the OBS client
   

2. In the FTP server directories, add files file3 and file4. file3 and file1 are in the same directory, and file2 and file4 are in the same directory. See Figure 7.
   Figure 7 New files on the FTP server
   

3. Wait 10 minutes and CDM automatically triggers the scheduled job. Then you can view the new files file3 and file4 after logging in to OBS. Figure 8 shows the new files on OBS.
   Figure 8 New files on OBS
   

4. On the Job Management page, click Historical Record in the Operation column to view the job's historical execution records and read/write statistics.
5. Click Log to view the job logs.