Forwarding Data to DMS for Kafka for Storage

Scenarios

If you want to store data reported by devices, you can either forward the data to application servers or to Distributed Message Service (DMS) for Kafka for storage.

In this example, data reported by all devices is forwarded to DMS for Kafka.

Buying a Kafka Instance

1. Access the Distributed Message Service for Kafka service page.
2. Click Access Console to go to the DMS for Kafka console.
3. Click Buy Instance in the upper right corner, select instance specifications and configure a security group (see How Do I Select and Configure a Security Group?) as required, and click Buy.
   Figure 1 Buying a Kafka Instance

   

Configuring IoTDA

Using IoTDA, you can create a product model, register a device, and set a data forwarding rule to forward data reported by the device to DMS for Kafka.

1. Access IoT Device Access and click Access Console. Click the target instance card.
2. In the navigation pane, choose Rules > Data Forwarding, and click Create Rule in the upper left corner.
3. Set the parameters based on the table below. The following parameter values are only examples. You can create your own rules by referring to Procedure. After setting the parameters, click Create Rule.

   Parameter	Description
   Rule Name	Description: Customize a name, for example, iotda-kafka.
   Description	Description: Customize a rule description, for example, forwarding data to DMS for Kafka for storage.
   Data Source	Description: Select Device property.
   Trigger	Description: Device property reported is automatically configured.
   Resource Space	Description: Select All resource spaces.

   Figure 2 Rules triggered by property reporting - Forwarding data to Kafka
   
4. Click the Set Forwarding Target tab, and then click Add to set a forwarding target.

   Parameter	Description
   Forwarding Target	Description: Select Distributed Message Service (DMS) for Kafka.
   Region	Description: Select the region where DMS for Kafka is located. If IoTDA is not authorized to access the service in this region, configure cloud service access authorization as prompted.
   Connection Address	Description: Obtain the connection address by referring to Connecting to a Kafka Instance with SASL Enabled. Standard edition instances support only access to Kafka over the public network. Enterprise edition instances support private network access to Kafka in the same subnet of the same VPC.
   Topic	Description: Customize a topic. For details, see Creating a Kafka topic.
   SASL	Description: If SASL authentication is enabled, enter the security protocol, SASL authentication mechanism, and SASL username and password you used when you buy a Kafka instance (see Buying a Kafka instance).
   Kafka Security Protocol	Description: Select the Kafka security protocol you used when you buy a Kafka instance (see Buying a Kafka instance).
   SASL Mechanism	Description: Select the SASL authentication mechanism you used when you buy a Kafka instance (see Buying a Kafka instance).
   SASL Username	Description: Enter the SASL username you used when you buy a Kafka instance (see Buying a Kafka instance).
   Password	Description: Enter the password you used when you buy a Kafka instance (see Buying a Kafka instance).

   Figure 3 Creating a forwarding target - to Kafka
   
5. Click Enable Rule to activate the configured data forwarding rule.
   Figure 4 Enabling a rule - Forwarding data to Kafka
   

Verifying the Configurations

• You can use a registered physical device to access the platform and enable the device to report data.
• You can also use a simulator to simulate a device to report data. For details, see Developing an MQTT-based Simulated Smart Street Light Online.

Expected result:

Log in to the Kafka console and click the Kafka instance name to go to the instance management page. On the Message Query page, you can view the data reported by the device.

Figure 5 Viewing Kafka messages - Kafka

You can also use the APIs to read files (see Querying Messages).