Help Center/ Distributed Message Service for Kafka/ FAQs/ Connections/ Why Is It Not Recommended to Use a Sarama Client for Messaging?
Updated on 2024-11-27 GMT+08:00
View PDF

Why Is It Not Recommended to Use a Sarama Client for Messaging?

Symptom

If a Sarama client is used to send and receive messages, the following issues may occur:

  • Sarama cannot detect partition changes. Adding topic partitions requires client restart to enable consumption.
  • Sarama's default MaxProcessingTime is 100 ms. When this limit is reached, consumers can no longer consume messages.
  • If consumer offsets reset from the oldest (earliest) position, all messages starting from the earliest position may be repeatedly consumed after the client restarts.
  • A consumer that subscribes to multiple topics may not be able to consume any message from specific partitions.

Solution

Use confluent-kafka-go as the Kafka client library.

For details, see Table 1.

Table 1 Comparing common Go clients

Client

Pros

Cons

confluent-kafka-go
  • An official Kafka client by Confluent that supports full Kafka compatibility and all Kafka features
  • High stability and performance, and low latency based on librdkafka

High compiling complexity because Go compilers need extra resources to configure the imported C++ libraries

kafka-go
  • A simple and lite Kafka client easy for learning and usage
  • Reduced application size and complexity with limited library and fewer dependencies
  • Fewer advanced functions and configurations than confluent-kafka-go
  • Applicable only to simple scenarios that require low performance and throughput

Sarama

Better asynchronization and higher concurrency (written in the original Go language)
  • Many bugs, limited documentation
  • Deteriorates application performance when processing a large number of messages due to large memory consumption
Parent topic: Connections