Help Center> MapReduce Service> Developer Guide (2.x and Earlier)> Kafka Application Development> Development Specifications> Rules
View PDF

Rules

When calling the Kafka API (AdminUtils.createTopic) to create a topic, you need to configure the ZkStringSerializer parameter.

  • Example in Java: 
    import org.I0Itec.zkclient.ZkClient;
import kafka.utils.ZKStringSerializer$;
...
ZkClient zkClient = new ZkClient(zkconnectstring, zkSessionTimeout, zkConnectionTimeout, ZKStringSerializer$.MODULE$);
AdminUtils.createTopic(zkClient, topic, partitions, replicationFactor, new Properties());
...
  • Example in Scala: 
    import org.I0Itec.zkclient.ZkClient;
import kafka.utils.ZKStringSerializer;
...
var zkclient: ZkClient = new ZkClient(zkconnectstring, zkSessionTimeout, zkConnectionTimeout, ZKStringSerializer)
AdminUtils.createTopic(zkClient, topic, partitions, replicationFactor, new Properties())

The number of partition copies cannot exceed the number of nodes.

The copy of a topic partition in Kafka is used to improve data reliability. Copies of the same partition are distributed on different nodes. Therefore, the number of copies cannot exceed the number of nodes.

Size of the fetch.message.max.bytes parameter of the Consumer client

The value of fetch.message.max.bytes on the consumer client must be greater than or equal to the maximum number of bytes in each message generated by the Producer client. If the parameter value is too small, messages generated by the Producer may fail to be consumed by the Consumer.