このページは、お客様の言語ではご利用いただけません。Huawei Cloudは、より多くの言語バージョンを追加するために懸命に取り組んでいます。ご協力ありがとうございました。

Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive

Modifying Cluster Kafka Instance Specifications

Updated on 2025-02-14 GMT+08:00

After creating a Kafka instance, you can increase or decrease its specifications. Table 1 lists available modification options. Only one object can be modified per operation: broker quantity, bandwidth, storage space, or broker flavor.

Table 1 Specification modification options

Old/New Flavor

Modified Object

Increase

Decrease

New flavor

Broker quantity

×

Storage space

×

Broker flavor

Old flavor

Bandwidth

×

Storage space

×

Broker flavor

×

×

Distinguishing Between Old and New Specifications

  • Old specifications: In the instance list, the instance specification is displayed as bandwidth (for example, 100 MB/s).
  • New specifications: In the instance list, the instance specification is displayed as the ECS flavor multiplied by the number of brokers (for example, kafka.2u4g.cluster*3 brokers).
Figure 1 Instance list

Notes and Constraints

Unavailable for single-node instances.

Impact of Specification Modification

Table 2 Impact of specification modification

Modified Object

Impact

Bandwidth or broker quantity

  • Increasing the bandwidth or adding brokers does not affect the original brokers or services.
  • When you increase the bandwidth or change the broker quantity, the storage space is proportionally expanded based on the current disk space. For example, assume that the original number of brokers of an instance is 3 and the disk size of each broker is 200 GB. If the broker quantity changes to 10 and the disk size of each broker is still 200 GB, the total disk size becomes 2,000 GB.
  • New topics are created on new brokers, and the original topics are still on the original brokers, resulting in unbalanced partitions. You can reassign partitions to migrate the replicas of the original topic partitions to the new brokers.

Storage space

  • You can expand the storage space 20 times.
  • Storage space expansion does not affect services.

Broker flavor

  • Single-replica topics do not support message production and consumption during this period. Services will be interrupted.
  • If a topic has multiple replicas, modifying the broker flavor does not interrupt services, but may cause disorder of partition messages. Evaluate this impact and avoid peak hours.
  • Broker rolling restarts will cause partition leader changes, interrupting connections for less than a minute when the network is stable. For multi-replica topics, configure the retry mechanism on the producer client. To do so:
    • If you use an open-source Kafka client, configure the retries parameter to a value in the range from 3 to 5.
    • If you use Flink, configure the retry policy by referring to the following code:
      StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
      env.setRestartStrategy(RestartStrategies.fixedDelayRestart(3, Time.seconds(20)));
  • If the total number of partitions created for an instance is greater than the upper limit allowed by a new flavor, scale-down cannot be performed. The maximum number of partitions varies with instance specifications. For details, see Specifications.

    For example, if 800 partitions have been created for a kafka.4u8g.cluster*3 instance, you can no longer scale down the instance to kafka.2u4g.cluster*3 because this flavor allows only 750 partitions.

  • It takes 5 to 10 minutes to modify specifications on one broker. The more brokers, the longer time the modification takes.

Process of Increasing or Decreasing Broker Flavors

When you scale up or down the broker flavor, a rolling restart is performed on brokers. The following process takes three brokers as an example:

  1. Stop the Kafka process on Broker 0.
  2. Scale up or down the flavor of Broker 0.
  3. Restart the Kafka process on Broker 0.
  4. Repeat 1 to 3 to scale up or down the flavor of Broker 1.
  5. Repeat 1 to 3 to scale up or down the flavor of Broker 2.
Figure 2 Process of modifying a broker flavor

Procedure

  1. Log in to the console.
  2. Click in the upper left corner to select the region where your instance is located.
  3. Click and choose Middleware > Distributed Message Service (for Kafka) to open the console of DMS for Kafka.
  4. In the navigation pane, choose Kafka Instances.
  5. In the row containing the desired instance, choose More > Modify Specifications in the Operation column.
  6. Specify the required storage space, broker flavor, or bandwidth.

    • Increase the bandwidth (for earlier instances).
      1. Specify a new bandwidth and click Next.
      2. Confirm the configurations and click Submit.
      3. Return to the instance list and check whether the change succeeded.
        • If the instance status has changed from Changing to Running, the change succeeded. You can check the new bandwidth in the Flavor column.
          Figure 3 Viewing the increased bandwidth
        • If the instance status has changed from Changing to Change failed, the change failed. Move the cursor over Change failed to check the failure cause.

          Instances in the Change failed state cannot be restarted, modified, or deleted. After the instance status automatically changes from Change failed to Running, you can continue to perform operations on the instance. If the status does not change to Running, contact customer service.

      4. After increasing the bandwidth, add the IP address of the new broker to the client connection configuration to improve reliability.
    • Expand the storage space (for earlier instances).
      1. Specify a new storage space and click Next.
      2. Confirm the configurations and click Submit.
      3. Return to the instance list and check whether the change succeeded.
        • If the instance status has changed from Changing to Running, the change succeeded. View the new storage space in the Used/Available Storage Space (GB) column in the instance list.
          Figure 4 Viewing the increased storage space
        • If the instance status has changed from Changing to Change failed, the change failed. Move the cursor over Change failed to check the failure cause.

          Instances in the Change failed state cannot be restarted, modified, or deleted. After the instance status automatically changes from Change failed to Running, you can continue to perform operations on the instance. If the status does not change to Running, contact customer service.

    • Expand the storage space (for later instances).
      1. Select Storage for Change By. For Storage space per broker, enter a value. Storage spaces vary by instance specifications. For details, see Specifications. Click Next.
      2. Confirm the configurations and click Submit.
      3. Check the modification progress and estimated remaining time.
        1. In the instance list, click the instance to go to the instance details page.
        2. In the navigation pane, choose Background Tasks.
        3. Click the task whose name is Modify Specifications. The Specification Modification Task Details dialog box is displayed.
        4. Check the progress and estimated remaining time. In Steps, check the steps, start time, and end time.
      4. Check whether the modification is successful.
        • If the task is in the Successful state, the modification is successful. View the new storage space (Storage space per broker × Number of brokers) in the Used/Available Storage Space (GB) column in the instance list.
        • If the task is in the Failed state, the modification is not successful. Move the cursor over Failed or check the cause in Steps.

          After the modification fails, the instance is in the Change failed state, and cannot be restarted, modified, or deleted. After the instance status automatically changes from Change failed to Running, you can continue to perform operations. If the status does not change to Running, contact customer service.

      Figure 5 Expanding the storage space
    • Increase the broker quantity (for later instances).
      1. For Change By, select Brokers.
      2. For Brokers, specify the broker quantity. The broker quantity range varies by instance specifications. For details, see Specifications.
      3. If public access has been enabled, configure EIPs for the new brokers.
      4. For Subnet, retain the default settings.
      5. For Private IP Addresses, select Auto or Manual.
        • Auto: The system assigns an IP address from the subnet automatically.
        • Manual: Select the IP addresses for the new brokers from the drop-down list. If the number of selected IP addresses is less than the number of brokers, the remaining IP addresses will be automatically assigned.
      6. Click Next.
      7. Confirm the configurations and click Submit.
      8. Check the modification progress and estimated remaining time.
        1. In the instance list, click the instance to go to the instance details page.
        2. In the navigation pane, choose Background Tasks.
        3. Click the task whose name is Modify Specifications. The Specification Modification Task Details dialog box is displayed.
        4. Check the progress and estimated remaining time. In Steps, check the steps, start time, and end time.
      9. Check whether the modification is successful.
        • If the task is in the Successful state, the modification is successful. View the number of brokers in the Flavor column in the instance list.
        • If the task is in the Failed state, the modification is not successful. Move the cursor over Failed or check the cause in Steps.

          After the modification fails, the instance is in the Change failed state, and cannot be restarted, modified, or deleted. After the instance status automatically changes from Change failed to Running, you can continue to perform operations. If the status does not change to Running, contact customer service.

      10. After adding brokers, add the IP addresses of the new brokers to the client connection configuration to improve reliability.
      Figure 6 Adding brokers
    • Modifying broker flavors (for later instances).
      1. For Change By, select Broker Flavor.
      2. Specify a new broker flavor.
      3. In the Risk Check area, check for risks.

        If any risk is found, handle it as prompted and click Recheck. If the risks do not need to be handled, select I understand the risks.

      4. Click Next, confirm the information, and click Submit.
      5. Check the scaling progress and estimated remaining time.
        1. In the instance list, click the instance to go to the instance details page.
        2. In the navigation pane, choose Background Tasks.
        3. Click the task whose name is Modify Specifications. The Specification Modification Task Details dialog box is displayed.
        4. Check the progress and estimated remaining time. In Steps, check the steps, start time, and end time.
      6. Check whether the modification is successful.
        • If the task is in the Successful state, the modification is successful. View the broker flavor in the Flavor column in the instance list.
        • If the task is in the Failed state, the modification is not successful. Move the cursor over Failed or check the cause in Steps.

          After the modification fails, the instance is in the Change failed state, and cannot be restarted, modified, or deleted. After the instance status automatically changes from Change failed to Running, you can continue to perform operations. If the status does not change to Running, contact customer service.

      Figure 7 Increasing or decreasing a broker flavor

We use cookies to improve our site and your experience. By continuing to browse our site you accept our cookie policy. Find out more

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback