Modifying Kafka Topic Replicas

Operation Impact

• Partition reassignment on topics with a large amount of data consumes a large amount of network and storage bandwidth. As a result, service requests may time out or the latency may increase. Therefore, you are advised to perform reassignment during off-peak hours. Compare the current instance load based on the instance specifications to decide whether the remaining instance capacity can support partition reassignment. Do not reassign partitions when there is insufficient bandwidth or when the CPU usage is greater than 90%. To view data volume and CPU usage of a topic, see Message Size and CPU Usage on the monitoring page. For details, see Viewing Kafka Metrics.
• A throttle refers to the upper limit of the bandwidth for replication of a topic, to ensure that other topics on the instance are not affected. Note that throttles apply to replication triggered by both normal message production and partition reassignment. If the throttle is too small, normal message production may be affected, and partition reassignment may never complete. If partitions are continuously reassigned, contact customer service.
• You cannot delete topics whose reassignment tasks have started. Otherwise, the tasks will never complete.
• You cannot modify the partition quantity of topics whose reassignment tasks have started.
• Reassignment tasks cannot be manually stopped. Please wait until they complete.
• If partition reassignment has been scheduled, reassignment cannot be scheduled again for any topic in this instance until this reassignment is executed.
• After partition reassignment, the metadata of the topic changes. If the producer does not support the retry mechanism, a few requests will fail, causing some messages to fail to be produced.
• Reassignment takes a long time if the topic has a large amount of data. You are advised to decrease the topic retention period based on the topic consumption so that historical data of the topic can be deleted in a timely manner to accelerate the migration. To view data volume of a topic, see Message Size on the monitoring page. For details, see Viewing Kafka Metrics.

Preparing for Partition Reassignment

• To reduce the amount of data to be migrated, decrease the topic retention period without affecting services and wait for messages to age. After the reassignment is complete, you can restore the retention period. To change the topic retention period, see Changing Kafka Message Retention Period.
• The target broker should have sufficient disk space. To check available disk space of each broker, see Viewing Kafka Disk Usage. If the remaining disk capacity of the target broker is close to the amount of data to be migrated to the broker, expand the disk capacity before the reassignment.

Modifying Replicas by Automatic Reassignment

1. Log in to the console.
2. Click in the upper left corner to select a region.
   

   Select the region where your Kafka instance is located.

3. Click and choose Middleware > Distributed Message Service (for Kafka) to open the console of DMS for Kafka.
4. Click the desired Kafka instance to view the instance details.
5. In the navigation pane, choose Topics.
6. Go to the Auto page in either of the following ways:
   • Select one or more topics and choose Reassign > Auto above the topic list.
   • In the row containing the desired topic, choose More > Reassign > Auto.

7. Modify the replicas.

   Table 1 Parameters of automatic reassignment

   Parameter	Description
   Brokers	Select the brokers to assign the topic's partition replicas to.
   Topics	Enter the number of replicas. This number must be less than or equal to the number of brokers.
   Max. Bandwidth	Specify throttle. The default value is -1, indicating that there is no throttle. If the instance has low workload (for example, only 30/300 MB/s is used), you are not advised to limit the bandwidth. Otherwise, you are advised to set it to a value greater than or equal to the total production bandwidth of the to-be-reassigned topic multiplied by the maximum number of replicas of the to-be-reassigned topic. For details, see Calculating a Throttle.
   Execute	Specify when to execute the reassignment. Now means to execute it immediately. As scheduled means to execute it at the scheduled time.

8. (Optional) Click Calculate. Time Required indicates how long automatic balancing will take.

   The one-click calculation function does not affect the performance of Kafka instances.

9. Click OK.

   The following table lists how to check whether reassignment is complete (scheduled and non-scheduled tasks):

   Table 2 Checking the reassignment result

   Task Type	Reassignment Result
   Background tasks	In the upper left corner of the topic list, click View details and the Background Tasks > Background tasks page is displayed. The reassignment task is complete when it is in the Successful state, which means that the replicas are modified.
   Scheduled tasks	The Background Tasks > Scheduled tasks page is displayed. This page only shows whether scheduled tasks start to execute instead of whether they are successful. When the task status is Pending, reassignment has not been executed. When the task status is Successful, reassignment has started. When the task status is Cancel, reassignment has been canceled. Click Background tasks tab page. When the task status is Successful, reassignment has completed, which means that the replicas are modified.

Modifying Replicas by Manual Reassignment

1. Log in to the console.
2. Click in the upper left corner to select a region.
   

   Select the region where your Kafka instance is located.

3. Click and choose Middleware > Distributed Message Service (for Kafka) to open the console of DMS for Kafka.
4. Click the desired Kafka instance to view the instance details.
5. In the navigation pane, choose Topics.
6. Go to the Manual page in either of the following ways:
   • Select a topic and choose Reassign > Manual above the topic list. Manual reassignment does not support batch operations.
   • In the row containing the desired topic, choose More > Reassign > Manual.

7. Modify the replicas.
   • In the upper right corner of the Manual dialog box, click Delete Replica or Add Replica to reduce or increase the number of replicas for each partition of the topic.
   • Under the name of the replica to be reassigned, click the broker name or and select the target broker to migrate the replica to. Assign replicas of the same partition to different brokers.
   • Specify throttle. The default value is -1, indicating that there is no throttle. If the instance has low workload (for example, only 30/300 MB/s is used), you are not advised to limit the bandwidth. Otherwise, you are advised to set it to a value greater than or equal to the total production bandwidth of the to-be-reassigned topic multiplied by the maximum number of replicas of the to-be-reassigned topic. For details, see Calculating a Throttle.
   • For Execute, specify when to execute the reassignment. Now means to execute it immediately. As scheduled means to execute it at the scheduled time.

8. (Optional) Click Calculate. Time Required indicates how long manual balancing will take.

   The one-click calculation function does not affect the performance of Kafka instances.

9. Click OK.

   The following table lists how to check whether reassignment is complete (scheduled and non-scheduled tasks):

   Table 3 Checking the reassignment result

   Task Type	Reassignment Result
   Background tasks	In the upper left corner of the topic list, click View details and the Background Tasks > Background tasks page is displayed. The reassignment task is complete when it is in the Successful state, which means that the replicas are modified.
   Scheduled tasks	The Background Tasks > Scheduled tasks page is displayed. This page only shows whether scheduled tasks start to execute instead of whether they are successful. When the task status is Pending, reassignment has not been executed. When the task status is Successful, reassignment has started. When the task status is Cancel, reassignment has been canceled. Click Background tasks tab page. When the task status is Successful, reassignment has completed, which means that the replicas are modified.