Updated on 2024-11-12 GMT+08:00

Using a Kafka Trigger

This section describes how to create a Kafka trigger and configure a Kafka event to trigger a function.

After a Kafka trigger is used, FunctionGraph periodically polls for new messages in a specific topic in a Kafka instance and passes the messages as input parameters to invoke functions. For details about the DMS for Kafka event source, see Supported Event Sources.

  • For details about the differences between DMS for Kafka and open-source Kafka, see Comparing DMS for Kafka and Open-Source Kafka.
  • In cases of Kafka data processing failure, the Kafka trigger will discard records that are larger than 6 MB.

Prerequisites

Before creating a trigger, ensure that you have prepared the following:

Creating a Kafka Trigger

  1. Log in to the FunctionGraph console. In the navigation pane, choose Functions > Function List.
  2. Click the function to be configured to go to the function details page.
  3. Choose Configuration > Triggers and click Create Trigger.

    Figure 1 Creating a trigger

  4. Set the following parameters:

    • Trigger Type: Select DMS (for Kafka).
    • Instance: Select a Kafka premium instance.
    • Topic: Select a topic of the Kafka premium instance.
    • Batch Size: Set the number of messages to be retrieved from a topic each time.
    • Username: Enter the username of the instance if SSL has been enabled for it.
    • Password: Enter the password of the instance if SSL has been enabled for it.

  5. Click OK.

    • After VPC access is enabled, you need to configure corresponding subnet permissions for the Kafka security group. For details about how to configure VPC access, see Configuring the Network.
    • You can create a Kafka trigger with multiple topics. You do not need to create one such trigger for each topic in the same instance.
      Figure 2 Selecting multiple topics

Configuring a Kafka Event to Trigger the Function

  1. Return to the FunctionGraph console. In the navigation pane, choose Functions > Function List.
  2. Click the function to be configured to go to the function details page.
  3. On the function details page, select a version.
  4. On the Code tab page, click Test. The Configure Test Event dialog box is displayed.
  5. Set the parameters described in Table 1 and click Save.

    Table 1 Test event information

    Parameter

    Description

    Configure Test Event

    You can choose to create a test event or edit an existing one.

    Use the default option Create new test event.

    Event Template

    Select DMS (for Kafka) to use the built-in Kafka event template.

    Event Name

    The event name can contain 1 to 25 characters and must start with a letter and end with a letter or digit. Only letters, digits, underscores (_), and hyphens (-) are allowed. For example, kafka-123test.

    Event data

    The system automatically loads the built-in Kafka event template, which is used in this example without modifications.

  6. Click Test. The function test result is displayed.