Kafka Application Development Approach

Scenario

Kafka is a distributed message system, in which messages can be publicized or subscribed. A Producer is to be developed to send a message to a topic of a Kafka cluster every second, and a Consumer is to be implemented to ensure that the topic is subscribed and that messages of the topic are consumed in real time.

Development Guidelines

1. Use a Linux client to create a topic. For details, see Kafka Shell Commands.
2. Develop a Producer to produce data to the topic.
3. Develop a Consumer to consume the data of the topic.

Performance Tuning Suggestions

1. Create a topic and plan its partitions based on service requirements. The number of partitions limits the number of concurrent consumers.
2. The key value of a message must be variable to ensure even message distribution.
3. Consumers are advised to proactively submit the offset to avoid repeated consumption.