Creating a Synchronization Task

Scenario

A synchronization task allows you to proactively synchronize new or modified objects from the source bucket to the destination bucket. You need to create a function on the source platform and create an event trigger for the function using the source object storage service. When a source object is added or modified, the source object storage service calls the synchronization API of OMS through the function, and the object is synchronized to the destination OBS bucket on Huawei Cloud.

Synchronization tasks depend on source event triggers, source function services, and public network transmission. They may fail in some cases. You are not advised to use this feature for data synchronization in DR scenarios.

Scenarios

Enter the AKs and SKs for accessing the source and destination buckets, and configure the synchronization task parameters for migrating source objects contained in the synchronization request and viewing the task details.

You can only create synchronization tasks in CN North-Beijing4 and CN East-Shanghai1.

Prerequisites

You have registered a HUAWEI ID and enabled Huawei Cloud services.

You have obtained the AK/SK pairs and required permissions for the source and destination platform accounts.
- The source platform account needs the permissions for:
  - Listing buckets
  - Obtaining bucket locations
  - Listing objects
  - Obtaining object metadata
  - Obtaining object content
  For details about how to obtain these permissions, see Source Permissions.
- The destination platform account needs the permissions for:
  - Listing buckets
  - Obtaining bucket locations
  - Listing objects
  - Obtaining object metadata
  - Obtaining object content
  - Listing uploaded parts
  - Restoring archived objects
  For details about how to obtain these permissions, see Destination Permissions.
You have created at least one bucket on OBS.
You have created less than 1,000,000 migration tasks within the past 24 hours.
You have less than 1,000,000 migration tasks in Waiting state.
You have less than 5 synchronization tasks in the Synchronizing state.

Procedure

Sign in to the console and go to the OMS page.
In the navigation pane on the left, choose Synchronization Tasks.
In the upper right corner, click Create Synchronization Task.
Read the service disclaimer, select I have read and agree to the service disclaimer, and click OK. The page for creating a synchronization task is displayed.

On the Select Source/Destination page, perform the following operations:

1. Configure source parameters by referring to Table 1.

**Table 1** Source parameters
Parameter	Description
Data Source	Select the cloud platform from which you are migrating data. Huawei Cloud AWS Alibaba Cloud Microsoft Azure Tencent Cloud Google Cloud Qiniu Cloud QingCloud Baidu Cloud Kingsoft Cloud UCloud
appID	Enter the APPID, an identifier, of the account for logging in to the Tencent Cloud console. This parameter is only available if the source cloud service provider is Tencent Cloud.
Access Key ID	Enter the AK for accessing the source platform. It can contain a maximum of 100 characters.
Secret Access Key	Enter the SK for accessing the source platform. The SK matches the AK, and they group into a pair. It can contain a maximum of 100 characters.
Specify Source Bucket	Enable this option if your source account does not have the permissions required to list buckets but has the permissions to list objects in buckets, obtain object metadata, and obtain object content. Then specify the bucket or path to be migrated. For details, see Table 2.

**Table 2** Description of **Specify Source Bucket**
If You Need to...	Then...
Enable Specify Source Bucket.	Select the region where the source bucket is located from the Region drop-down list. In the Bucket text box, enter the bucket or the path to be migrated. The path must have the bucket name included. Click Connect. OMS will check whether the source platform can be connected using the information you entered. If the source platform can be connected, the system will notify you.
Disable Specify Source Bucket (This feature is disabled by default.)	Click List Buckets. OMS will check whether the source platform can be connected using the information you entered. After the source platform is connected, all buckets in your account will be listed in the Bucket drop-down list on the premises that your account has the permissions needed to list buckets. In the Bucket drop-down list, select the bucket where the data to be migrated is stored.

2. Configure destination parameters by referring to Table 3.

**Table 3** Destination parameters
Parameter	Description
Access Key ID	Enter the AK for accessing Huawei Cloud. It can contain a maximum of 100 characters.
Secret Access Key	Enter the SK for accessing Huawei Cloud. The SK matches the AK, and they group into a pair. It can contain a maximum of 100 characters.
Bucket	Select a destination OBS bucket for storing the migrated data.

Click Next to set task parameters. For details about the parameters, see Table 4.

**Table 4** Task parameters
Parameter	Description
Object Metadata	If you select Do not migrate, OMS will only migrate the Content-Type and Content-Encoding metadata. If you select Migrate, OMS will migrate supported metadata. Only Chinese characters, English characters, digits, and hyphens (-) can be migrated. Other characters cannot be migrated.
Data Encryption	If you choose to enable server-side encryption, data is encrypted before being transferred to the destination OBS buckets. NOTE: SSE-KMS is used by default. Other encryption methods or custom encryption is not supported. Using KMS to encrypt migrated data may decrease the migration speed by about 10%. This option is only available if KMS encryption is supported in the current region.
Object Storage Class	Choose the storage class that your data will be migrated to. For details about storage classes, see Storage Classes.
Archive Data Restoration	Archived data must be restored before being migrated. You can select this option if your source cloud service provider supports automatic restoration of archived data. If you choose to enable this option, the system automatically restores and migrates archived objects in the migration task. If an archived object fails to be restored, the system records it in the list of objects that failed to be migrated and continues to migrate other objects in the migration task. NOTICE: The following cloud service providers can automatically restore archived objects: Huawei Cloud, Alibaba Cloud, Kingsoft Cloud, and Tencent Cloud. It takes at least a minute to restore an archived object. The larger the archived object is, the longer it takes to restore it, but if an archived object cannot be restored within three minutes, the restoration fails. The time needed depends on the cloud service provider. For details, see the documentation of your source cloud service provider. Two types of fees are generated during archived data restoration: fees for API calls and fees for archived object restoration. These fees are billed by the source cloud service provider. After the restoration is complete, archived objects will be in restored state for a certain period of time. During this period, objects can be downloaded or read. After this period expires, archived objects need to be restored again for migration. For details about how long they stay in the restored state, see the documentation of the cloud service provider. The copies generated for restoring archived data will be stored for three days by default. If there is more than 3 TB of data or more than 3 million objects in a source bucket, create a migration task group to migrate data in case the migration fails because the restored data is archived again. Pausing or restarting a migration task will result in restoration of all archived data again, no matter it has been restored or not. After objects are migrated from the source platform to Huawei Cloud, they are in the same storage class as the destination bucket. If you choose to disable this option, the system directly records archived objects in the list of objects that failed to be migrated, skips them, and continues to migrate other objects in the migration task.
Object Overwrite	Overwrite if source newer or different size: If a source object is not as large as or was last modified more recently than its paired destination object, the source object will overwrite the destination object. Otherwise, the source object will be skipped. Never overwrite: The system always skips source objects and keeps their paired destination objects. Overwrite if different CRC64 checksum: If a source object has a CRC64 checksum different from the paired destination object, the source object will overwrite the destination object. Otherwise, the source object will be skipped. If either of them does not have a CRC64 checksum, their sizes and last modification times are checked. NOTE: This option is only available for migration on Huawei Cloud or from Alibaba Cloud or Tencent Cloud. This option is applied only when the source object has the same encryption status as the paired destination one. Always overwrite: The system always allows source objects to overwrite their paired destination objects.
Consistency Check	By size and last modification time: This is the default setting. The system checks object consistency with object size and last modification time. By CRC64 checksum: The system checks object consistency with CRC64 checksums. If a source object and the paired destination object have CRC64 checksums, the checksums are checked. Otherwise, their sizes and last modification times are checked.

Click Next to enter the confirmation page.

Confirm the settings of the synchronization task and start the synchronization.

1. Check that the migration settings are correct.

If the settings are incorrect, click Previous to return to the previous page for modification.

2. Click Synchronize Now.

A message is displayed indicating that the task has been submitted. Switch back to the synchronization task list to view the task.

For details about task statuses, see Table 5.

**Table 5** Synchronization task statuses
Status	Description
Synchronizing	Indicates that the objects in the source bucket are being synchronized. NOTE: After you create a synchronization task, the task status is Synchronizing. It means that the synchronization requests from the source bucket can be received and the source objects can be synchronized. You need to configure synchronization requests on the source platform to migrate newly added or modified objects in the source bucket. For more details, see Configuring the Source Storage Service to Send Synchronization Requests. Then OMS will synchronize your source objects.
Paused	Indicates that the synchronization of source objects is paused. NOTE: If you want to pause a synchronization task, you are advised to stop writing data to the source bucket and wait for 15 minutes before pausing the synchronization task. Otherwise, it may cause a migration failure for some objects covered by the received synchronization requests.

Manage the synchronization task.
- Starting the synchronization task
- Only synchronization tasks in the Paused state can be started.
- For security purposes, the system does not save your personal data. When you restart the synchronization task, you need to enter the AKs and SKs for accessing the source and destination buckets again to complete identity authentication.
- Pausing the synchronization task
- Only synchronization tasks in the Synchronizing state can be paused.
- If you want to pause a synchronization task, you are advised to stop writing data to the source bucket and wait for 15 minutes before clicking Pause on the synchronization task page. Otherwise, it may cause a migration failure for some objects covered by the received synchronization requests.
- Deleting the synchronization task
- Only synchronization tasks in the Paused state can be deleted.