Data Ingestion Service (DIS) Trigger

This section describes how to create a DIS trigger through the FunctionGraph console. You can configure DIS events using the built-in event template to trigger the function.

Overview

DIS builds data streams for custom applications capable of processing or analyzing streaming data to address the challenge of transmitting data from outside the cloud to inside the cloud. It can continuously capture, transmit, and store terabytes of data from multiple sources.

You can create a function to automatically poll a DIS stream and process all new data records, such as, website click streams, financial transactions, social media streams, IT logs, and location-tracking events.

Example event of a DIS trigger

DIS triggers functions using the following event format. For details about the parameters, see Table 1.

{
    "ShardID": "shardId-0000000000",
    "Message": {
        "next_partition_cursor": "eyJnZXRJdGVyYXRvclBhcmFtIjp7InN0cmVhbS1uYW1lIjoiZGlzLXN3dGVzdCIsInBhcnRpdGlvbi1pZCI6InNoYXJkSWQtMDAwMDAwMDAwMCIsImN1cnNvci10eXBlIjoiVFJJTV9IT1JJWk9OIiwic3RhcnRpbmctc2VxdWVuY2UtbnVtYmVyIjoiNCJ9LCJnZW5lcmF0ZVRpbWVzdGFtcCI6MTUwOTYwNjM5MjE5MX0",
        "records": [
            {
                "partition_key": "shardId_0000000000",
                "data": "d2VsY29tZQ==",
                "sequence_number": "0"
            },
            {
                "partition_key": "shardId_0000000000",
                "data": "dXNpbmc=",
                "sequence_number": "1"
            },
            {
                "partition_key": "shardId_0000000000",
                "data": "RnVuY3Rpb25TdGFnZQ==",
                "sequence_number": "2"
            },
            {
                "partition_key": "shardId_0000000000",
                "data": "c2VydmljZQ==",
                "sequence_number": "3"
            }
        ],
        "millis_behind_latest": ""
    },
    "Tag": "latest",
    "StreamName": "dis-swtest"
}

**Table 1** Parameters of a DIS example event
Parameter	Category	Description
ShardID	String	Partition ID
next_partition_cursor	String	Next partition cursor
Records	Map	Data records stored in a DIS stream
partition_key	String	Partition key
data	String	Data blocks, which are added by the data producer to the stream
sequence_number	Int	Record ID, which is automatically allocated by DIS
Tag	String	Stream tag
StreamName	String	Stream name

Video Tutorial

This video shows how to use a DIS trigger to trigger a function and configure a test event to check whether the function can be executed properly.

Notes and Constraints

DIS triggers are available only in CN East-Shanghai1, CN-Hong Kong, and AP-Singapore. You can check whether the DIS trigger is supported on the console.
The valid payload size of a request body is 6 MB when a DIS trigger is used.

Prerequisites

Function and configuration:
- You have created a function. For details, see Creating a Function from Scratch
- You have configured the DIS agency permission for the function. For details, see Configuring Agency Permissions.
DIS:
You have created a DIS stream, for example, dis-function. For details, see Creating a DIS Stream.

Creating a DIS Trigger

Log in to the FunctionGraph console. In the navigation pane, choose Functions > Function List.
Click the function to be configured to go to the function details page.
Choose Configuration > Triggers and click Create Trigger.

Set DIS trigger parameters by referring to Table 2.

**Table 2** DIS trigger parameters
Parameter	Description	Example Value
Trigger Type	Mandatory. Choose Data Ingestion Service (DIS).	Data Ingestion Service (DIS)
Stream Name	Mandatory. Select an existing DIS stream. Common streams and advanced streams are supported. If no DIS stream is available, click Create Stream to create one. For details, see Creating a DIS Stream.	dis-function
Starting Position	Mandatory. Specify a position in the specified stream from which to start reading data. For details, see Obtaining Data Cursors. Options: TRIM_HORIZON: Data is read from the earliest valid records that are stored in the partition. For example, a tenant used a DIS stream to upload three pieces of data A1, A2, and A3. N days later, A1 has expired and A2 and A3 are still in the validity period. In this case, if the tenant uses TRIM_HORIZON to download the data, the system downloads data from A2. LATEST: Data is read just after the most recent record in the partition. This setting ensures that you always read the latest data.	latest
Max. Fetch Bytes	Mandatory. Maximum volume of data that can be fetched in each request. Only the records smaller than this value will be fetched. The unit can be KB or MB. For a common stream, the total size of records in a single request cannot exceed 1 MB (excluding partitionKey data). The value ranges from 0 KB to 1024 KB (0 MB to 1 MB). For an advanced stream, the total size of records in a single request cannot exceed 5 MB (excluding partitionKey data). The value ranges from 0 KB to 5,120 KB (0 MB to 5 MB).	1 MB
Pull Period	Mandatory. The period for pulling data from the stream. The unit can be millisecond or second. The value ranges from 1 ms to 60,000 ms (1s to 60s).	30s
Serial Data Processing	Mandatory. If this option is enabled, the system obtains data for the next processing only after the current data processing is complete. If this option is disabled, as shown in Figure 1, you can configure the concurrency (1–80) to limit the number of concurrent asynchronous invocation requests from a DIS trigger. This prevents a single DIS trigger with mass traffic from occupying the concurrency quota and affecting other DIS triggers. (Currently available only in CN North-Beijing4) Figure 1 Disabling serial data processing	Enabled

Click OK.

Modifying a DIS Trigger

Some parameters of DIS triggers can be modified.

Return to the FunctionGraph console. In the navigation pane, choose Functions > Function List.
Click the function to be configured to go to the function details page.
Choose Configuration > Triggers, and click Edit next to a DIS trigger.
Modify Max. Fetch Bytes, Pull Period, and Serial Data Processing as required, and click OK.

Configuring a DIS Test Event to Trigger a Function

Return to the FunctionGraph console. In the navigation pane, choose Functions > Function List.
Click the function to be configured to go to the function details page.
On the function details page, select a version.
On the Code tab page, click Test. The Configure Test Event dialog box is displayed.

Set the parameters described in Table 3 and click Save.

**Table 3** Test event information
Parameter	Description
Configure Test Event	You can choose to create a test event or edit an existing one. Use the default option Create new test event.
Event Templates	Select Data Ingestion Service (DIS) to use the built-in DIS event template.
Event Name	The event name can contain 1 to 25 characters and must start with a letter and end with a letter or digit. Only letters, digits, underscores (_), and hyphens (-) are allowed. For example, dis-123test.
Event data	The system automatically loads the built-in DIS event template, which is used in this example without modifications.