Help Center/ Distributed Message Service for Kafka/ User Guide (ME-Abu Dhabi Region)/ FAQs/ Topics and Partitions/ Why Is Partition Quantity Limited?
Updated on 2023-05-06 GMT+08:00
View PDF
Share

Why Is Partition Quantity Limited?

Kafka manages messages by partition. If there are too many partitions, message creation, storage, and retrieval will be fragmented, affecting the performance and stability. If the total number of partitions of topics reaches the upper limit, you cannot create more topics.

The partition limit varies depending on the flavor, as shown in the following table.

Table 1 Kafka instance specifications

Flavor

Brokers

Maximum TPS per Broker

Maximum Partitions per Broker

Maximum Consumer Groups per Broker

Maximum Client Connections per Broker

Storage Space

kafka.2u4g.cluster

3–30

30,000

250

20

2000

300 GB–300,000 GB

kafka.4u8g.cluster

3–30

100,000

500

100

4000

300 GB–600,000 GB

kafka.8u16g.cluster

3–30

150,000

1000

150

4000

300 GB–900,000 GB

kafka.12u24g.cluster

3–30

200,000

1500

200

4000

300 GB–900,000 GB

kafka.16u32g.cluster

3–30

250,000

2000

200

4000

300 GB–900,000 GB
Parent topic: Topics and Partitions