Forwarding Data to DMS for Kafka for Storage

Scenarios

If you want to store data reported by devices, you can either forward the data to application servers or to Distributed Message Service (DMS) for Kafka for storage.

In this example, data reported by all devices is forwarded to DMS for Kafka.

Buying a Kafka Instance

1. Log in to Huawei Cloud and visit DMS for Kafka.
2. Click Access Console to go to the DMS for Kafka console.
3. Click Buy Instance in the upper right corner, select instance specifications and configure a security group as required, and click Buy.
   Figure 1 Buying a Kafka Instance

   

Configuring IoTDA

Using IoTDA, you can create a product model, register a device, and set a data forwarding rule to forward data reported by the device to DMS for Kafka.

1. Access the IoTDA service page and click Access Console. Click the target instance card.
2. In the navigation pane, choose Rules > Data Forwarding, and click Create Rule in the upper left corner.
3. Set the parameters based on the table below. The following parameter values are only examples. You can create a rule by referring to Data Forwarding Overview and click Create Rule.

   Parameter	Description
   Rule Name	Customize a name, for example, iotda-kafka.
   Description	Customize a rule description, for example, forwarding data to DMS for Kafka for storage.
   Data Source	Select Device property.
   Trigger	Device property reported is automatically populated.
   Resource Space	Select All resource spaces.

   Figure 2 Rules triggered by property reporting - Forwarding data to Kafka
   
4. Click the Set Forwarding Target tab, and then click Add to set a forwarding target.

   Parameter	Description
   Forwarding Target	Select Distributed Message Service (DMS) for Kafka.
   Region	Select the region where DMS for Kafka is located. If IoTDA is not authorized to access the service in this region, configure cloud service access authorization as prompted.
   Connection Address	Obtain the connection address by following the instructions provided in Accessing a Kafka Instance with SASL. Basic and standard instances support only access to Kafka premium instances over the Internet. Enterprise instances support access to Kafka premium instances over a private network.
   Topic	Customize a topic. For details, see Creating a Topic.
   SASL	If SASL authentication is enabled, enter the security protocol, SASL authentication mechanism, and SASL username and password you used when you buy a Kafka instance.
   Kafka Security Protocol	Select the Kafka security protocol you used when you buy a Kafka instance.
   SASL Mechanism	Select the SASL authentication mechanism you enabled when you buy a Kafka instance.
   SASL Username	Enter the SASL username you entered when buying a Kafka instance.
   Password	Enter the password you entered when buying a Kafka instance.

   Figure 3 Creating a forwarding target - to Kafka with a custom certificate
   
5. Click Enable Rule to activate the configured data forwarding rule.
   Figure 4 Enabling a rule - Forwarding data to Kafka
   

Verifying the Configurations

• You can use a registered physical device to access the platform and enable the device to report data.
• You can also use a simulator to simulate a device to report data. For details, see Developing an MQTT-based Smart Street Light Online.

Expected result:

Log in to the DMS for Kafka management console and click the Kafka instance name to go to the instance management page. On the Message Query page, you can view the data reported by the device.

Figure 5 Viewing Kafka messages - Kafka

You can also use the DMS for Kafka API Querying Messages to read files.