Configuring the Source Storage Service to Send Synchronization Requests

Scenarios

After you create a synchronization task, you need to configure the source object storage service to request OMS to migrate newly added and modified source objects. To do so, you can use a message notification or function service on the source platform.

Desired Effects

• The names of newly added and modified objects in the source bucket can be obtained in real time.
• The OMS synchronization API can be invoked to migrate the newly added and modified objects to the destination bucket.

Configuration Methods

• Using the source object storage service and the function service
  

  On the source platform, you need to create a function and configure event triggers for synchronizing incremental source objects through OMS.

  When objects are added to or modified in the source bucket, the source object storage service will call the configured function to trigger migration of these objects to the destination bucket.

• Using a service system and a message notification service
  

  When objects are added or modified in the source bucket, the service system captures the information about these objects through the messages received from the source object storage service or by proactively analyzing database changes. Then the system invokes the synchronization API of OMS to migrate the newly added and modified objects from the source bucket to the destination bucket.

Procedure

Following operations use Huawei Cloud as the source platform to describe how to use FunctionGraph to configure synchronization requests:

The created function must be in the same region as the source bucket.

1. Log in to the management console.
2. Click Service List and choose Compute > FunctionGraph.

   The FunctionGraph console is displayed.

3. In the navigation pane on the left, choose Templates.
4. In the search box in the upper right corner of the displayed page, enter OMS and locate the Data synchronization template, as shown in Figure 1.
   Figure 1 OMS data synchronization template
   
   

   Click View details to view the template details, such as template type, input parameters, output parameters, and precautions.

5. Click Configure. The Select template tab page is displayed.
6. Set parameters in the Basic Information area by referring to Table 1.
   Figure 2 Setting basic parameters
   

   Table 1 Basic parameters for creating a function

   Parameter	Description
   Function Template	Retain the default value sync-oms-data-python.
   Region	Select the region where the source bucket is located.
   Function Name	Specify a function name. The name must: Consists of 1 character to 60 characters, allowing letters, digits, hyphens (-), and underscores (_). Starts with a letter and ends with a letter or digit.
   Agency	Select Use no agency.
   Enterprise Project	Set this parameter as needed. The default value is default.
   Runtime	The default value is Python 3.6 and cannot be modified.

7. Set parameters in the Environment Variables area by referring to Table 2.
   Figure 3 Setting environment variables
   

   Table 2 Environment variables

   Parameter	Description
   sync_url	Enter the address for receiving synchronization requests, which can be obtained from the synchronization task details.
   sync_task_id	Enter the synchronization task ID, which can be obtained from the synchronization task list page.
   log_bucket	Enter the name of the bucket used for storing the function invoking logs. The lists of requested or unrequested objects will also be stored into this bucket. CAUTION: The log bucket must be in the account used to create the synchronization task.
   obs_endpoint	Enter the OBS domain name in the region where the log bucket is located. For details about regions and endpoints, see Regions and Endpoints.
   hw_ak	Enter the AK of the account used to create the synchronization task.
   hw_sk	Enter the SK of the account used to create the synchronization task.

8. Click Create Function.

   The function details page is displayed.

9. Click Configuration tab, on the Basic Settings tab page, set Execution Timeout (s) to 60 and click Save.
   Figure 4 Setting the execution timeout interval
   

10. Choose Trigger and click Create Trigger. The Create Trigger dialog box is displayed, as shown in Figure 5. Set trigger parameters. For details about the parameters, see Table 3.
    Figure 5 Creating a trigger
    

    Table 3 Parameters required for creating a trigger

    Parameter	Description
    Trigger Type	Select Object Storage Service (OBS) from the drop-down list.
    Bucket Name	Select the source bucket that stores the data to be synchronized from the drop-down list.
    Events	Set this parameter based on your requirements.
    (Optional) Event Notification Name	An event name is automatically generated. You can specify a name as needed.
    (Optional) Prefix	Enter a prefix to limit notifications to objects whose names start with the matching characters.
    (Optional) Suffix	Enter a suffix to limit notifications to objects whose names end with the matching characters.

11. Click OK.
    

    You need to configure a test event to verify the function.

12. Click the Code tab and select Configure Test Event from the drop-down list in the Code Source area. The Configure Test Event dialog box is displayed, as shown in Figure 6. Set the parameters listed in Table 4.
    Figure 6 Configuring a test event
    

    Table 4 Parameters required for configuring a test event

    Parameter	Description
    Configure Test Event	Use the default option Create new test event.
    Event Template	Select obs-event-template in the event template list.
    Event Name	An event name is automatically generated. You can specify a name as needed.

13. Click Create.
14. Click Test. The test result is displayed on the right of the page.
    Figure 7 Test result
    
    

    After the test succeeds, upload an object to the source bucket. Wait for 10 minutes. Then the object will be synchronized to the destination bucket.

Obtaining Object Lists

The objects in the lists are those to be synchronized.