Help Center> MapReduce Service> Developer Guide (3.x)> Kafka Development Guide (Security Mode)> Developing an Application> Typical Scenario Description
View PDF

Typical Scenario Description

Scenario

Kafka is a distributed message system, in which messages can be publicized or subscribed. A Producer is to be developed to send a message to a topic of a Kafka cluster every second, and a Consumer is to be implemented to ensure that the topic is subscribed and that messages of the topic are consumed in real time.

Development Idea

  1. Use a Linux client to create a topic.
  2. Develop a Producer to produce data to the topic.
  3. Develop a Consumer to consume the data of the topic.

Suggestions on Performance Tuning

  1. Create a topic and plan its partitions based on service requirements. The number of partitions limits the number of concurrent consumers.
  2. The key value of a message must be variable to ensure even message distribution.
  3. Consumers are advised to proactively submit the offset to avoid repeated consumption.