Help Center/ Data Lake Insight/ FAQs/ Flink Jobs/ Flink SQL Jobs/ How Do I Change the Number of Kafka Partitions in a Flink SQL Job Without Stopping It?
Updated on 2024-11-15 GMT+08:00
View PDF
Share

How Do I Change the Number of Kafka Partitions in a Flink SQL Job Without Stopping It?

Symptom

You used Flink 1.10 to run a Flink Opensource SQL job. You set the number of Kafka partitions for the job a small value at the beginning and need to increase the number now.

Solution

Add the following parameters to the SQL statement:

connector.properties.flink.partition-discovery.interval-millis="3000"

This statement allows you to increase or decrease the number of Kafka partitions without stopping the Flink job.

Parent topic: Flink SQL Jobs