Consumer Fails to Consume Data in a Newly Created Cluster, and the Message " GROUP_COORDINATOR_NOT_AVAILABLE" Is Displayed

Symptom

A Kafka cluster is created, and two Broker nodes are deployed. The Kafka client can be used for production but cannot be used for consumption. The Consumer fails to consume data, and the message "GROUP_COORDINATOR_NOT_AVAILABLE" is displayed. The key log is as follows:

2018-05-12 10:58:42,561 | INFO | [kafka-request-handler-3] | [GroupCoordinator 2]: Preparing to restabilize group DemoConsumer with old generation 118 | kafka.coordinator.GroupCoordinator (Logging.scala:68)
2018-05-12 10:59:13,562 | INFO | [executor-Heartbeat] | [GroupCoordinator 2]: Preparing to restabilize group DemoConsumer with old generation 119 | kafka.coordinator.GroupCoordinator (Logging.scala:68)

Possible Causes

The __consumer_offsets cannot be created.

Cause Analysis

1. As indicated by the log, a large number of _consumer_offset creation operations failed.
2. The number of Brokers for the cluster is 2.

3. However, the number of replicas for the _consumer_offset topic is 3. Therefore, the creation fails.

   

Solution

Expand the cluster to at least three streaming core nodes or perform the following steps to modify service configuration parameters:

1. Go to the service configuration page.
   • MRS Manager: Log in to MRS Manager, choose Services > Kafka > Service Configuration, and select All from Type.
   • FusionInsight Manager: Log in to FusionInsight Manager. Choose Cluster > Services > Kafka. Click Configurations and select All Configurations.

2. Search for offsets.topic.replication.factor and transaction.state.log.replication.factor and change their values to 2.
3. Click Save Configuration and select Restart the affected services or instances. Click OK to restart the services.