Specifications
Kafka Instance Specifications
Kafka instances are compatible with open-source Kafka v1.1.0, v2.3.0, and v2.7. The instance specifications are represented by the ECS flavor and the number of brokers. Available options are kafka.2u4g.cluster, kafka.4u8g.cluster, kafka.8u16g.cluster, kafka.12u24g.cluster, and kafka.16u32g.cluster.
In the following table, transactions per second (TPS) are calculated assuming that the size of a message is 1 KB.
Flavor |
Brokers |
Maximum TPS per Broker |
Maximum Partitions per Broker |
Maximum Consumer Groups per Broker |
Maximum Client Connections per Broker |
Storage Space |
---|---|---|---|---|---|---|
kafka.2u4g.cluster |
3–30 |
30,000 |
250 |
20 |
2000 |
300 GB–300,000 GB |
kafka.4u8g.cluster |
3–30 |
100,000 |
500 |
100 |
4000 |
300 GB–600,000 GB |
kafka.8u16g.cluster |
3–30 |
150,000 |
1000 |
150 |
4000 |
300 GB–900,000 GB |
kafka.12u24g.cluster |
3–30 |
200,000 |
1500 |
200 |
4000 |
300 GB–900,000 GB |
kafka.16u32g.cluster |
3–30 |
250,000 |
2000 |
200 |
4000 |
300 GB–900,000 GB |
Flavor Selection
- kafka.2u4g.cluster with 3 brokers
Recommended for up to 3000 client connections, 60 consumer groups, and 100,000 TPS
- kafka.4u8g.cluster with 3 brokers
Recommended for up to 10,000 client connections, 300 consumer groups, and 300,000 TPS
- kafka.8u16g.cluster with 3 brokers
Recommended for up to 20,000 client connections, 600 consumer groups, and 600,000 TPS
- kafka.12u24g.cluster with 3 brokers
Recommended for up to 20,000 client connections, 600 consumer groups, and 900,000 TPS
- kafka.16u32g.cluster with 3 brokers
Recommended for up to 20,000 client connections, 600 consumer groups, and 1,200,000 TPS
Storage Space Selection
Kafka instances support multi-replica storage. The storage space is consumed by all replicas. When creating an instance, specify its storage space based on the expected service message size and the number of replicas.
For example, if the estimated message size is 100 GB, the disk capacity must be at least: 100 GB x Number of replicas + 100 GB (reserved space).
The storage space can be expanded as your service grows.
Topic Quantity
There are limits on the topic quantity and the aggregate number of partitions in the topics. When the partition quantity limit is reached, you can no longer create topics.
The number of topics is related to the maximum number of partitions allowed (see Figure 1) and the specified number of partitions in each topic (see Table 1).
The maximum number of partitions allowed for an instance with kafka.2u4g.cluster and 3 brokers is 750.
- If the number of partitions of each topic in the instance is 3, the maximum number of topics is 750/3 = 250.
- If the number of partitions of each topic in the instance is 1, the maximum number of topics is 750/1 = 750.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot