Changing Kafka Partition Quantity

Notes and Constraints

• The number of partitions can only be increased.
• Instances created since May 17, 2023 do not have Kafka Manager. You cannot modify topic partitions for these instances using Kafka Manager.
• The partition quantity of topics of a single-node or cluster Kafka instance is limited. When the partition quantity limit is reached, you can no longer create topics. The quantity varies with instance specifications. For details, see Cluster Kafka Instances and Single-node Kafka Instances.
• For an instance with ciphertext access enabled, if allow.everyone.if.no.acl.found is set to false, the topic partition quantity can be modified on the client only by the initial user (set in first ciphertext access enablement).

Modifying Topic Partitions on the Console

1. Log in to the console.
2. Click in the upper left corner to select the region where your instance is located.
3. Click and choose Middleware > Distributed Message Service (for Kafka) to open the console of DMS for Kafka.
4. In the navigation pane, choose Kafka Instances.
5. Click the desired instance to go to the instance details page.
6. In the navigation pane, choose Topics.
7. Modify the number of partitions using either of the following methods:
   • Select one or more topics and click Edit Topic in the upper left corner.
   • In the row containing the desired topic, click Edit.

8. In the Edit Topic dialog box, enter the number of partitions and click OK.

   To ensure performance, a maximum of 200 partitions is allowed for each topic on the Kafka console.

Modifying Topic Partitions on Kafka Manager

1. Log in to Kafka Manager.
2. Choose Topic > List to view the list of topics.
3. Click a topic to view its details.
4. Click Add Partitions.
   Figure 1 Topic details page
   

5. Enter the number of partitions and click Add Partitions.
   To ensure performance, 200 or less partitions are recommended for each topic.

   Figure 2 Adding partitions
   

   If "Done" is displayed, the partitions are added successfully.

   Figure 3 Partitions added
   

Modifying Topic Partitions on the Client

If your Kafka client version is later than 2.2, you can use kafka-topics.sh to change the partition quantity.

• For a Kafka instance with ciphertext access disabled, run the following command in the /bin directory of the Kafka client:

  ./kafka-topics.sh --bootstrap-server ${connection-address} --topic ${topic-name} --alter --partitions ${number-of-partitions}

  Parameter description:

  • connection-address: can be obtained from the Connection area on the Basic Information page on the Kafka console.
  • topic-name: topic name.
  • number-of-partitions: number of partitions in a topic. To ensure performance, a partition number within 200 is recommended for each topic.

  Example:

  [root@ecs-kafka bin]# ./kafka-topics.sh --bootstrap-server 192.168.xx.xx:9092,192.168.xx.xx:9092,192.168.xx.xx:9092 --topic topic-01 --alter --partitions 6
  [root@ecs-kafka bin]#

• For a Kafka instance with ciphertext access enabled, do as follows:
  1. (Optional) Modify the client configuration file.
     View Security Protocol in the Connection area on the Basic Information page on the Kafka console. The configuration settings vary depending on the protocol.

     • SASL_PLAINTEXT: Skip this step and go to 2 if the username and password are already set. Otherwise, create the ssl-user-config.properties file in the /config directory on the Kafka client and add the following content to the file:

       security.protocol=SASL_PLAINTEXT
       # If the SASL mechanism is SCRAM-SHA-512, configure as follows:
       sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
       username="**********" \
       password="**********";        
       sasl.mechanism=SCRAM-SHA-512
       # If the SASL mechanism is PLAIN, configure as follows:
       sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
       username="**********" \
       password="**********";        
       sasl.mechanism=PLAIN

       Parameter description: username and password are the ones you set when enabling ciphertext access for the first time or when creating a user.

     • SASL_SSL: Skip this step and go to 2 if the username, password, and SSL certificate are already set. Otherwise, create the ssl-user-config.properties file in the /config directory on the Kafka client and add the following content to the file:

       security.protocol=SASL_SSL
       ssl.truststore.location={ssl_truststore_path}
       ssl.truststore.password=dms@kafka
       ssl.endpoint.identification.algorithm=
       # If the SASL mechanism is SCRAM-SHA-512, configure as follows:
       sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
       username="**********" \
       password="**********";        
       sasl.mechanism=SCRAM-SHA-512
       # If the SASL mechanism is PLAIN, configure as follows:
       sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
       username="**********" \
       password="**********";        
       sasl.mechanism=PLAIN

       Parameter description:

       • ssl.truststore.location: path for storing the client.jks certificate. Even in Windows, you need to use slashes (/) for the certificate path. Do not use backslashes (\), which are used by default for paths in Windows. Otherwise, the client will fail to obtain the certificate.
       • ssl.truststore.password is certified by the server, which must be set to dms@kafka and cannot be changed.
       • ssl.endpoint.identification.algorithm: whether to verify the certificate domain name. This parameter must be left blank, which indicates disabling domain name verification.
       • username and password: username and password you set when enabling ciphertext access for the first time or when creating a user.
  2. Run the following command in the /bin directory of the Kafka client:

     ./kafka-topics.sh --bootstrap-server ${connection-address} --topic ${topic-name} --alter --partitions ${number-of-partitions} --command-config ../config/ssl-user-config.properties 

     Parameter description:

     • connection-address: can be obtained from the Connection area on the Basic Information page on the Kafka console.
     • topic-name: topic name.
     • number-of-partitions: number of partitions in a topic. To ensure performance, a partition number within 200 is recommended for each topic.

     Example:

     [root@ecs-kafka bin]# ./kafka-topics.sh --bootstrap-server 192.168.xx.xx:9093,192.168.xx.xx:9093,192.168.xx.xx:9093 --topic topic-01 --alter --partitions 6 --command-config ../config/ssl-user-config.properties
     [root@ecs-kafka bin]#