Why Are Messages Stacked and Unconsumed in a New Partition of a Topic?

Possible Cause

If the librdkafka client is used as the consumer, this problem is a known bug. Specifically, if the leader consumer in a consumer group had not subscribed to the topic, the consumer group cannot subscribe to the new partition.

Solution

• Subscribe the consumer group to only one topic.
• Restart any of the consumer processes in the consumer group to trigger reassignment of the new partition.
• Add consumers to the group to trigger reassignment of the new partition.