Updated on 2023-04-28 GMT+08:00

Kafka Performance Tuning

Scenario

You can modify Kafka server parameters to improve Kafka processing capabilities in specific service scenarios.

Parameter Tuning

Modify the service configuration parameters. For details, see Modifying Cluster Service Configuration Parameters. For details about the tuning parameters, see Table 1.

Table 1 Tuning parameters

Parameter

Default Value

Scenario

num.recovery.threads.per.data.dir

10

During the Kafka startup process, if a large volume of data exists, you can increase the value of this parameter to accelerate the startup.

background.threads

10

Specifies the number of threads processed by a broker background task. If a large volume of data exists, you can increase the value of this parameter to improve broker processing capabilities.

num.replica.fetchers

1

Specifies the number of threads used when a replica requests to the Leader for data synchronization. If the value of this parameter is increased, the replica I/O concurrency increases.

num.io.threads

8

Specifies the number of threads used by the broker to process disk I/O. It is recommended that the number of threads be greater than or equal to the number of disks.

KAFKA_HEAP_OPTS

-Xmx6G -Xms6G

Specifies the Kafka JVM heap memory setting. If the data volume on the broker is large, adjust the heap memory size.