Updated on 2024-04-07 GMT+08:00

Does Specification Modification Affect Services?

Table 1 describes the impact of increasing or decreasing instance specification . It takes 5 to 10 minutes to modify specifications on one broker. The more brokers, the longer time the modification takes.

Table 1 Impact of specification modification

Modified Object

Impact

Broker quantity or bandwidth

  • Adding brokers or increasing the bandwidth does not affect the original brokers or services.
  • When you increase the bandwidth or add brokers, the storage space is proportionally expanded based on the current disk space. For example, assume that the original number of brokers of an instance is 3 and the disk size of each broker is 200 GB. If the broker quantity changes to 10 and the disk size of each broker is still 200 GB, the total disk size becomes 2000 GB.
  • New topics are created on new brokers, and the original topics are still on the original brokers, resulting in unbalanced partitions. You can reassign partitions to migrate the replicas of the original topic partitions to the new brokers.

Storage space

  • You can expand the storage space 20 times.
  • Storage space expansion does not affect services.

Broker flavor

  • Single-replica topics do not support message creation and retrieval during this period. Services will be interrupted.
  • If a topic has multiple replicas, scaling up or down the broker flavor does not interrupt services, but may cause disorder of partition messages. Evaluate this impact and avoid peak hours.
  • Broker rolling restarts will cause partition leader changes, interrupting connections for less than a minute when the network is stable. For multi-replica topics, configure the retry mechanism on the producer client. To do so:
    • If you use an open-source Kafka client, configure the retries parameter to a value in the range from 3 to 5.
    • If you use Flink, configure the retry policy by referring to the following code:
      StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
      env.setRestartStrategy(RestartStrategies.fixedDelayRestart(3, Time.seconds(20)));
  • If the total number of partitions created for an instance is greater than the upper limit allowed by a new flavor, scale-down cannot be performed. The maximum number of partitions varies with instance specifications. For details, see Specifications.

    For example, if 800 partitions have been created for a kafka.4u8g.cluster*3 instance, you can no longer scale down the instance to kafka.2u4g.cluster*3 because this flavor allows only 750 partitions.