What Is Data+?

Data+ is a data processing service provided by OBS. It allows you to orchestrate workflows to automatically process (such as parse, transcode, and screenshot) data stored in OBS.

Data+ uses the capabilities of FunctionGraph to orchestrate complex service logic into workflows that can be triggered by an event or the calling of an API. Then the workflow will automatically complete the planned data processing tasks. Data+ provides a (GUI) that are easy and intuitive for you to work with. Data+ also provides pre-defined workflow templates that cover various scenarios, or you can customize functions on your terms. For details about workflow templates, see Built-in Template Parameters.

• Use an event trigger to start a workflow.

  You can configure an event trigger for a bucket to specify the conditions when a workflow is triggered. Particularly, you can specify which data is processed following what operation, or what complex tasks are executed asynchronously after being triggered by specified events. Most complex tasks have the same processing logic, so you can specify the same operation for the same type of objects. A typical scenario is that a workflow is triggered to parse or transcode a video once the video is uploaded to a bucket.

• Call an API to trigger a workflow.

  Occasionally, you may need to perform complex operations on a single object or a set of objects. In this case, you can call APIs to perform complex operations on individual objects, in other words, you can immediately start a specific workflow for an object.

Figure 1 OBS Data+

Service Advantages

• Easy to use: Data+ provides a GUI for you to easily orchestrate workflows.
• Powerful: Data+ enables you to orchestrate workflows that can leverage data processing services provided by HUAWEI CLOUD.
• Fault tolerant: Data+ has a built-in retry mechanism that enables it to retry failed or timeout tasks and respond properly to different errors. In addition, it provides an interface for resuming a workflow from where the workflow is suspended due to an exception or failure.

Application Scenarios

Templates are provided for video parsing, media transcoding, and frame capturing. For details, see Built-in Template Parameters. You can also customize functions for other scenarios.

Permissions

Before using Data+, you need to grant appropriate permissions to Data+ and other services, so that OBS Data+ can work with other services to process data as required. The authorization involves the following aspects:

1. Before using Data+, ensure that you account or IAM account has the Tenant Administrator permission specified in IAM. For details about this permission, see IAM Permissions.
2. When using Data+ for the first time, you need to grant permissions to Data+ as prompted on the GUI.

   Authorization content

   Authorize Data+ to use the FunctionGraph service and other data processing services, so that Data+ can create functions that are able to call APIs of other data processing services. Then data in your OBS buckets can be automatically processed based on workflows.

   With only a few clicks, the system will automatically create the required IAM agencies for you.

   • Agency 1: Authorize OBS to access FunctionGraph.
   • Agency 2: Authorize FunctionGraph to access OBS and other data processing services.
   

   • Before authorization, ensure that at least two IAM agencies can be created under your account.
   • The permissions to access a specific data processing service are authorized only when a workflow requires to access that data processing service.
   • If an IAM user under your account needs to use Data+, you need to authorize the user the permissions to access FunctionGraph, data processing services (such as MPC), and SMN. The authorized permissions should be consistent with the permissions authorized to the agency. For details about agency permissions, choose Identity and Access Management > Agencies on the management console to view agencies named with the obs_function_agency prefix.
3. Before using MPC capabilities (such as video parsing, frame capturing, media transcoding, and more), grant bucket access permissions to MPC. For details how to grant the permissions, see Authorizing Access to Cloud Resources.

   Access permissions to the following buckets must be authorized:

   • Frame capturing output bucket
   • Transcoding output bucket
   • Event source bucket corresponding to the event trigger associated with the workflow that uses video parsing, thumbnailing, or media asset transcoding
   

   You can also enable agency to authorize all buckets with one click. However, to comply with the principle of least privilege, you are advised to grant only required bucket permissions to other services.

Figure 2 Relationship between Data+, FunctionGraph, and other data processing services

Limitations and Constraints

• Data+ is currently available only in the CN North-Beijing1, CN North-Beijing4, CN East-Shanghai1, CN East-Shanghai2, and AP-Singapore regions.
• Only serial workflows are supported.
• An event trigger can be associated with only one workflow, but a workflow can have multiple event triggers.
• The name of each workflow should be unique. Even after a workflow is deleted, its name cannot be reused.
• The name of each trigger in a bucket should be unique.
• A Data+ workflow and the required FunctionGraph service, data processing services, SMN, as well as the associated OBS bucket should be in the default project of the same region.
  

  By default, a bucket and its Data+ workflows are in the same region. If workflow A is created for bucket A, workflow A is in the region of bucket A.