Modifying Instance Specifications

Scenario

After creating a Kafka instance, you can increase or decrease its specifications. Table 1 lists available modification options.

**Table 1** Specification modification options
Old/New Flavor	Modified Object	Increase	Decrease
New flavor	Broker quantity	√	×
	Storage space	√	×
	Broker flavor	√	√
Old flavor	Bandwidth	√	×
	Storage space	√	×
	Broker flavor	×	×

Distinguishing between old and new specifications:

Old specifications: In the instance list, the instance specification is displayed as bandwidth (for example, 100 MB/s).
New specifications: In the instance list, the instance specification is displayed as the ECS flavor multiplied by the number of brokers (for example, kafka.2u4g.cluster*3 brokers).

Figure 1 Instance list
Click to enlarge

Impact of Specification Modification

It takes 5 to 10 minutes to modify specifications on one broker. The more brokers, the longer time the modification takes.

**Table 2** Impact of specification modification
Modified Object	Impact
Broker quantity or bandwidth	Adding brokers or increasing the bandwidth does not affect the original brokers or services. When you increase the bandwidth or add brokers, the storage space is proportionally expanded based on the current disk space. For example, assume that the original number of brokers of an instance is 3 and the disk size of each broker is 200 GB. If the broker quantity changes to 10 and the disk size of each broker is still 200 GB, the total disk size becomes 2000 GB. New topics are created on new brokers, and the original topics are still on the original brokers, resulting in unbalanced partitions. You can reassign partitions to migrate the replicas of the original topic partitions to the new brokers.
Storage space	You can expand the storage space 20 times. Storage space expansion does not affect services.
Broker flavor	Single-replica topics do not support message creation and retrieval during this period. Services will be interrupted. If a topic has multiple replicas, scaling up or down the broker flavor does not interrupt services, but may cause disorder of partition messages. Evaluate this impact and avoid peak hours. Broker rolling restarts will cause partition leader changes, interrupting connections for less than a minute when the network is stable. For multi-replica topics, configure the retry mechanism on the producer client. To do so: If you use an open-source Kafka client, configure the retries parameter to a value in the range from 3 to 5. If you use Flink, configure the retry policy by referring to the following code: StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); env.setRestartStrategy(RestartStrategies.fixedDelayRestart(3, Time.seconds(20))); If the total number of partitions created for an instance is greater than the upper limit allowed by a new flavor, scale-down cannot be performed. The maximum number of partitions varies with instance specifications. For details, see Specifications. For example, if 800 partitions have been created for a *kafka.4u8g.cluster3 instance, you can no longer scale down the instance to kafka.2u4g.cluster3* because this flavor allows only 750 partitions.

Process of Increasing or Decreasing Broker Flavors

When you scale up or down the broker flavor, a rolling restart is performed on brokers. The following process takes three brokers as an example:

Stop the Kafka process on Broker 0.
Scale up or down the flavor of Broker 0.
Restart the Kafka process on Broker 0.
Repeat 1 to 3 to scale up or down the flavor of Broker 1.
Repeat 1 to 3 to scale up or down the flavor of Broker 2.

Figure 2 Process of increasing or decreasing broker flavors
Click to enlarge

Procedure

Log in to the management console.
Click in the upper left corner to select a region.

Select the region where your Kafka instance is located.
Click and choose Application > Distributed Message Service for Kafka to open the console of DMS for Kafka.
In the row containing the instance for which you want to modify the specifications, choose More > Modify Specifications in the Operation column.
Specify the required storage space, broker quantity, broker flavor, or bandwidth.
To modify old specifications, perform the following steps:
- Increase the bandwidth.
  Specify a new bandwidth and click Next. Confirm the configurations and click Submit.
  
  View the new bandwidth of the instance in the Specifications column in the instance list.
  
  After increasing the bandwidth, add the IP address of the new broker to the client connection configuration to improve reliability.
- Expand the storage space.
  Specify a new storage space and click Next. Confirm the configurations and click Submit.
  
  View the new storage space in the Used/Available Storage Space (GB) column in the instance list.
To modify new specifications, perform the following steps:
- Expand the storage space.
  For Change By, select Storage. For Storage Space per Broker, specify a new storage space, and click Next. Confirm the configurations and click Submit.
  
  View the new storage space (Storage space per broker x Number of brokers) in the Used/Available Storage Space (GB) column in the instance list.
- Add brokers.
  For Change By, select Brokers and enter the number of brokers. If public access is enabled, configure EIPs for the new brokers. Then click Next. Confirm the configurations and click Submit.
  
  View the number of brokers in the Specifications column in the instance list.
  
  After adding brokers, add the IP addresses of the new brokers to the client connection configuration to improve reliability.
- Increase or decrease the broker flavor.
  For Change By, select Broker Flavor. Then, select a new broker flavor and click Next. Confirm the configurations and click Submit.
  
  View the broker flavor in the Flavor column in the instance list.