Kafka Common Configuration Parameters

log.dirs

List of Kafka data storage directories. Use commas (,) to separate multiple directories.

%{@auto.detect.datapart.bk.log.logs}

KAFKA_HEAP_OPTS

Specifies the JVM option used for Kafka to start broker. It is recommended that you set this parameter based on service requirements.

-Xmx6G -Xms6G

auto.create.topics.enable

Indicates whether a topic is automatically created. If this parameter is set to false, you need to run a command to create a topic before sending a message.

true

default.replication.factor

Default number of replicas of a topic is automatically created.

2

monitor.preInitDelay

Delay of the first health check after the server is started. If the startup takes a long time, increase the value of the parameter. Unit: millisecond

600,000

controller.socket.timeout.ms

Specifies the timeout for connecting controller to broker. Unit: millisecond

30,000

Generally, retain the default value of this parameter.

group.max.session.timeout.ms

Specifies the maximum session timeout during the consumer registration. Unit: millisecond

1800000

The configured value must be less than the value of this parameter.

group.min.session.timeout.ms

Specifies the minimum session timeout during the consumer registration. Unit: millisecond

6,000

The configured value must be greater than the value of this parameter.

offsets.commit.timeout.ms

Specifies the timeout for the Offset to submit requests. Unit: millisecond

5,000

Maximum latency for processing an Offset request.

replica.socket.timeout.ms

Specifies the timeout of the request for synchronizing replica data. Its value must be greater than or equal to that of the replica.fetch.wait.max.ms parameter. Unit: millisecond

30,000

Maximum timeout for establishing a channel before the synchronization thread sends a synchronization request. The value must be greater than that of the replica.fetch.wait.max.ms parameter.

request.timeout.ms

Specifies the timeout for waiting for a response after the client sends a connection request. Unit: millisecond

30,000

Timeout for transferring networkclient connection requests to the Controller and Replica threads on the Broker node. If no response is received within the timeout, the client resends the requests and returns an error message after the maximum number of retries is reached.

transaction.max.timeout.ms

Specifies the maximum timeout allowed by the transaction. Unit: millisecond

900,000

Maximum timeout of a transaction. If a request is not responded within the specified timeout, Broker returns an error in InitProducerIdRequest. This prevents a long client request timeout, ensuring that consumer can receive topics.

user.group.cache.timeout.sec

Specifies the time when the user group information is stored in the cache. Unit: second

300

Specifies the time for caching the mapping between users and user groups. If time exceeds the threshold, the system automatically runs the id -Gn command to query the user information. During this period, the mapping in the cache is used.

zookeeper.connection.timeout.ms

Specifies the timeout for connecting to ZooKeeper. Unit: millisecond

45,000

This parameter specifies the duration for connecting the ZooKeeper and zkclient for the first time. If the duration exceeds the value of this parameter, the zkclient automatically disconnects the connection.

zookeeper.session.timeout.ms

Specifies the ZooKeeper session timeout duration. During this period, ZooKeeper disconnects the connection if broker does not report its heartbeats to ZooKeeper. Unit: millisecond

45,000

ZooKeeper session timeout has the following functions:

1) Based on value of this parameter and the number of ZooKeeper URLs in ZKURL, if the connection duration exceeds the node timeout value (sessionTimeout/Number of transferred ZooKeeper URLs), the connection fails and the system attempts to connect to the next node.

2) After the connection is established, a session (for example, the temporary BrokerId node registered on the ZooKeeper) is cleared by the ZooKeeper a session timeout later if the broker is stopped.

request.timeout.ms

Specifies the timeout of a message request. Unit: millisecond

30,000

If a network fault occurs, increase the value of this parameter. If the value is too small, the Batch Expire occurs.

connections.max.idle.ms

Specifies the maximum retention period for idle connections. Unit: millisecond

600,000

If the idle connection time is greater than this parameter value, this connection is disconnected. If necessary, a new connection is created.

request.timeout.ms

Specifies the timeout for consumer requests. Unit: millisecond

30,000

If the request times out, the request will fail and be sent again.