- What's New
- Function Overview
- Product Bulletin
-
Service Overview
- Infographics
- What Is DMS for Kafka?
- Product Advantages
- Application Scenarios
- Kafka Instance Specifications
- Comparing Single-node and Cluster Kafka Instances
- Comparing Kafka, RabbitMQ, and RocketMQ
- Comparing DMS for Kafka and Open-Source Kafka
- Security
- Notes and Constraints
- Related Services
- Basic Concepts
- Permissions
- Billing
- Getting Started
-
User Guide
- Process of Using Kafka
- Permissions Management
- Buying a Kafka Instance
- Configuring Topics
- Connecting to an Instance
- Managing Messages
- Managing Consumer Groups
- Managing Quotas
-
Managing Instances
- Viewing and Modifying Basic Information of a Kafka Instance
- Viewing Kafka Disk Usage
- Viewing Kafka Background Tasks
- Viewing Sample Code of Kafka Production and Consumption
- Modifying Kafka Instance Configuration Parameters
- Configuring Kafka Instance Tags
- Configuring Kafka Recycling Policies
- Upgrading the Kafka Instance Kernel
- Exporting the Kafka Instance List
- Restarting a Kafka Instance
- Deleting Kafka Instances
- Using Kafka Manager
- Modifying Instance Specifications
- Migrating Data
- Testing Instance Performance
- Applying for Increasing Kafka Quotas
- Monitoring and Alarms
- Viewing Kafka Audit Logs
-
Best Practices
- Kafka Best Practices
- Improving Kafka Message Processing Efficiency
- Optimizing Consumer Polling
- Interconnecting Logstash to Kafka to Produce and Consume Messages
- Using MirrorMaker to Synchronize Data Across Clusters
- Handling Message Accumulation
- Handling Service Overload
- Handling Uneven Service Data
- Configuring Message Accumulation Monitoring
- Suggestions on Using DMS for Kafka Securely
- Developer Guide
-
API Reference
- Before You Start
- API Overview
- Calling APIs
- Getting Started
-
APIs V2 (Recommended)
- Managing Lifecycle
-
Instance Management
- Resetting the Password
- Resetting Kafka Manager Password
- Restarting Kafka Manager
- Configuring Automatic Topic Creation
- Modifying the Private IP Address for Cross-VPC Access
- Querying Kafka Cluster Metadata
- Querying Coordinator Details of a Kafka Instance
- Reassigning Replicas of a Topic for a Kafka Instance
- Modifying Kafka Access Modes
- Querying the Disk Usage Status of Topics
- Initiating Partition Reassigning for a Kafka Instance
- Disabling Kafka Manager
- Deleting a User or Client Quota
- Querying User or Client Quotas
- Creating User or Client Quotas
- Modifying User or Client Quotas
-
Smart Connect
- Enabling Smart Connect (Pay-per-Use Instance)
- Disabling Smart Connect (Pay-per-Use Instance)
- Creating a Smart Connect Task
- Querying Smart Connect Tasks
- Querying Smart Connect Task Details
- Deleting a Smart Connect Task
- Pausing a Smart Connect Task
- Restarting a Smart Connect Task
- Starting a Smart Connect Task or Restarting a Paused or Running Smart Connect Task
- Specification Modification Management
- Topic Management
-
Managing Consumer Groups
- Querying Consumer Group Details
- Querying All Consumer Groups
- Deleting Consumer Groups of a Kafka Instance in Batches
- Creating a Consumer Group
- Resetting Consumer Group Offset to the Specified Position
- Querying the Offset of a Consumer Group
- Modifying All Consumer Groups
- Querying a Specified Consumer Group
- Deleting a Specified Consumer Group
- Modifying a Specified Consumer Group
- Querying Topics of a Specified Consumer Group
- Querying Consumers in a Specified Consumer Group
- User Management
- Managing Messages
- Background Task Management
- Tag Management
- Diagnosis Management
- Other APIs
- Permissions and Supported Actions
- Out-of-Date APIs
- Appendix
- Change History
- SDK Reference
-
FAQs
-
Instances
- Why Can't I Select Two AZs?
- Why Can't I View the Subnet and Security Group Information When Creating a DMS Instance?
- How Do I Select Storage Space for a Kafka Instance?
- How Do I Choose Between High I/O and Ultra-high I/O?
- Which Capacity Threshold Policy Should I Use?
- Which Kafka Versions Are Supported?
- What Is the ZooKeeper Address of a Kafka Instance?
- Are Kafka Instances in Cluster Mode?
- Can I Modify the Port for Accessing a Kafka Instance?
- How Long Are Kafka SSL Certificates Valid for?
- How Do I Synchronize Data from One Kafka Instance to Another?
- How Do I Change the SASL_SSL Setting of a Kafka Instance?
- How Do I Modify the SASL Mechanism?
- How Do I Change the Security Protocol?
- Will a Kafka Instance Be Restarted After Its Enterprise Project Is Modified?
- Are Kafka Brokers and ZooKeeper Deployed on the Same VM or on Different VMs?
- Can I Delete a Kafka Instance Billed in Yearly/Monthly Mode?
- Which Cipher Suites Are Supported by Kafka?
- Can I Change Single-AZ Deployment to Multi-AZ Deployment for an Instance?
- Does DMS for Kafka Support Cross-AZ Disaster Recovery? Where Can I Check Whether an Existing Instance is Across-AZs?
- Do Kafka Instances Support Disk Encryption?
- Can I Change the VPC and Subnet After a Kafka Instance Is Created?
- Where Can I Find Kafka Streams Use Cases?
- Can I Upgrade Kafka Instances?
- How Do I Bind an EIP Again?
- Specification Modification
-
Connections
- How Do I Select and Configure a Security Group?
- Can I Access a Kafka Instance Over a Public Network?
- How Many Connection Addresses Does a Kafka Instance Have by Default?
- Do Kafka Instances Support Cross-Region Access?
- Do Kafka Instances Support Cross-VPC Access?
- Do Kafka Instances Support Cross-Subnet Access?
- Does DMS for Kafka Support Authentication with Kerberos?
- Does DMS for Kafka Support Password-Free Access?
- How Do I Obtain the Public Access Address After Public Access Is Enabled?
- Does DMS for Kafka Support Authentication on Clients by the Server?
- Can I Use PEM SSL Truststore When Connecting to a Kafka Instance with SASL_SSL Enabled?
- What Are the Differences Between JKS and CRT Certificates?
- Which TLS Version Does DMS for Kafka Support?
- Is There a Limit on the Number of Client Connections to a Kafka Instance?
- How Many Connections Are Allowed from Each IP Address?
- Can I Change the Private Network Addresses of a Kafka Instance?
- Is the Same SSL Certificate Used for Different Instances?
- Why Is It Not Recommended to Use a Sarama Client for Messaging?
-
Topics and Partitions
- Is There a Limit on the Number of Topics in a Kafka Instance?
- Why Is Partition Quantity Limited?
- Can I Reduce the Partition Quantity?
- Why Do I Fail to Create Topics?
- Do Kafka Instances Support Batch Importing Topics or Automatic Topic Creation?
- Why Do Deleted Topics Still Exist?
- Can I View the Disk Space Used by a Topic?
- Can I Add ACL Permissions for Topics?
- What Should I Do If Kafka Storage Space Is Used Up Because Retrieved Messages Are Not Deleted?
- How Do I Increase the Partition Quantity?
- Will a Kafka Instance Be Restarted After Its Automatic Topic Creation Setting Is Modified?
- What Can I Do If a Consumer Fails to Retrieve Messages from a Topic Due to Insufficient Permissions?
-
Consumer Groups
- Do I Need to Create Consumer Groups, Producers, and Consumers for Kafka Instances?
- Will a Consumer Group Without Active Consumers Be Automatically Deleted in 14 Days?
- Why Does a Deleted Consumer Group Still Exist?
- Why Can't I View Consumers When Instance Consumption Is Normal?
- Can I Delete Unnecessary Topics in a Consumer Group?
-
Messages
- What Is the Maximum Size of a Message that Can be Created?
- Why Does Message Poll Often Fail During Rebalancing?
- Why Can't I Query Messages on the Console?
- What Can I Do If Kafka Messages Are Accumulated?
- Why Do Messages Still Exist After the Retention Period Elapses?
- Do Kafka Instances Support Delayed Message Delivery?
- How Do I View the Number of Accumulated Messages?
- Why Is the Message Creation Time Displayed as Year 1970?
- How Do I Modify message.max.bytes?
- Why Are Offsets Not Continuous?
-
Kafka Manager
- Can I Configure a Kafka Manager Account to Be Read-Only?
- Why Can't I See Broker Information After Logging In to Kafka Manager?
- Yikes! Insufficient partition balance when creating topic : projectman_project_enterprise_project Try again.
- Can I Query the Body of a Message by Using Kafka Manager?
- Can I Change the Port of the Kafka Manager Web UI?
- Which Topic Configurations Can Be Modified on Kafka Manager?
- Why Is Information Displayed on Kafka Manager Inconsistent with Cloud Eye Monitoring Data?
- How Do I Change a Partition Leader for a Topic in Kafka Manager?
- Why Is the Version on the Console Different from That in Kafka Manager?
- Why Does an Instance Contain Default Topics __trace and __consumer_offsets?
- Why Do I See a Deleted Consumer Group on Kafka Manager?
-
Monitoring & Alarm
- Why Can't I View the Monitoring Data?
- Why Is the Monitored Number of Accumulated Messages Inconsistent with the Message Quantity Displayed on the Kafka Console?
- Why Is a Consumer Group Still on the Monitoring Page After Being Deleted?
- Why Do Metrics Fluctuate Significantly (Disk Read/Write Speed, Average Disk Read/Write Time, and CPU Usage)?
- Why Does JVM Heap Memory Usage of JVM Fluctuate Significantly?
- Why Is Production Rate Still 0 When There Are Produced Messages?
-
Instances
-
Troubleshooting
- Troubleshooting Kafka Connection Exceptions
- Troubleshooting 6-Min Latency Between Message Creation and Retrieval
- Troubleshooting Message Creation Failures
- Troubleshooting Topic Deletion Failures
- Troubleshooting Failure to Log In to Kafka Manager in Windows
- Troubleshooting Error "Topic {{topic_name}} not present in metadata after 60000 ms" During Message Production or Consumption
- Flink 1.15 Consumption Progress Submission Failure
- Videos
-
More Documents
-
User Guide (ME-Abu Dhabi Region)
- Service Overview
- Getting Started
- Permissions Management
- Preparing Required Resources
- Buying an Instance
- Accessing a Kafka Instance
- Managing Instances
- Managing Topics
- Managing Messages
- Managing Users
- Managing Consumer Groups
- Managing Kafka Quotas
- Modifying Kafka Parameters
- Quotas
- Monitoring
- Auditing
-
FAQs
-
Instances
- Why Can't I Select Two AZs?
- Why Can't I View the Subnet and Security Group Information When Creating a DMS Instance?
- How Do I Select Storage Space for a Kafka Instance?
- How Do I Choose Between High I/O and Ultra-high I/O?
- Which Capacity Threshold Policy Should I Use?
- Which Kafka Versions Are Supported?
- What Is the ZooKeeper Address of a Kafka Instance?
- Are Kafka Instances in Cluster Mode?
- Can I Modify the Port for Accessing a Kafka Instance?
- How Long Are Kafka SSL Certificates Valid for?
- How Do I Synchronize Data from One Kafka Instance to Another?
- How Do I Change the SASL_SSL Setting of a Kafka Instance?
- How Do I Modify the SASL Mechanism?
- Will a Kafka Instance Be Restarted After Its Enterprise Project Is Modified?
- Are Kafka Brokers and ZooKeeper Deployed on the Same VM or on Different VMs?
- Which Cipher Suites Are Supported by Kafka?
- Can I Change an Instance from Single-AZ Deployment to Multi-AZ Deployment?
- Does DMS for Kafka Support Cross-AZ Disaster Recovery? Where Can I View the AZs Configured for an Existing Instance?
- Do Kafka Instances Support Disk Encryption?
- Does Specification Modification Affect Services?
- Can I Change the VPC and Subnet After a Kafka Instance Is Created?
- Where Can I Find Kafka Streams Use Cases?
- Can I Upgrade Kafka Instances?
- Why Is the Version on the Console Different from That in Kafka Manager?
- How Do I Bind an EIP Again?
-
Connections
- How Do I Select and Configure a Security Group?
- Can I Access a Kafka Instance Over a Public Network?
- How Many Connection Addresses Does a Kafka Instance Have by Default?
- Do Kafka Instances Support Cross-Region Access?
- Do Kafka Instances Support Cross-VPC Access?
- Do Kafka Instances Support Cross-Subnet Access?
- Does DMS for Kafka Support Authentication with Kerberos?
- Does DMS for Kafka Support Password-Free Access?
- How Do I Obtain the Public Access Address After Public Access Is Enabled?
- Does DMS for Kafka Support Authentication on Clients by the Server?
- Can I Use PEM SSL Truststore When Connecting to a Kafka Instance with SASL_SSL Enabled?
- What Are the Differences Between JKS and CRT Certificates?
- Which TLS Version Does DMS for Kafka Support?
- Is There a Limit on the Number of Client Connections to a Kafka Instance?
- How Many Connections Are Allowed from Each IP Address?
- Can I Change the Private Network Addresses of a Kafka Instance?
- Is the Same SSL Certificate Used for Different Instances?
-
Topics and Partitions
- Is There a Limit on the Number of Topics in a Kafka Instance?
- Why Is Partition Quantity Limited?
- Can I Change the Partition Quantity?
- Why Do I Fail to Create Topics?
- Do Kafka Instances Support Batch Importing Topics or Automatic Topic Creation?
- Why Do Deleted Topics Still Exist?
- Can I View the Disk Space Used by a Topic?
- Can I Add ACL Permissions for Topics?
- What Should I Do If Kafka Storage Space Is Used Up Because Retrieved Messages Are Not Deleted?
- How Do I Increase the Partition Quantity?
- Will a Kafka Instance Be Restarted After Its Automatic Topic Creation Setting Is Modified?
- How Do I Disable Automatic Topic Creation?
- Can I Delete Unnecessary Topics in a Consumer Group?
- What Can I Do If a Consumer Fails to Retrieve Messages from a Topic Due to Insufficient Permissions?
- Why Does an Instance Contain Default Topics __trace and __consumer_offsets?
- Consumer Groups
-
Messages
- What Is the Maximum Size of a Message that Can be Created?
- Why Does Message Poll Often Fail During Rebalancing?
- Why Can't I Query Messages on the Console?
- What Can I Do If Kafka Messages Are Accumulated?
- Why Do Messages Still Exist After the Retention Period Elapses?
- Do Kafka Instances Support Delayed Message Delivery?
- How Do I View the Number of Accumulated Messages?
- Why Is the Message Creation Time Displayed as Year 1970?
-
Kafka Manager
- Can I Configure a Kafka Manager Account to Be Read-Only?
- Why Can't I See Broker Information After Logging In to Kafka Manager?
- Yikes! Insufficient partition balance when creating topic : projectman_project_enterprise_project Try again.
- Can I Query the Body of a Message by Using Kafka Manager?
- Can I Change the Port of the Kafka Manager Web UI?
- Which Topic Configurations Can Be Modified on Kafka Manager?
- Why Is Information Displayed on Kafka Manager Inconsistent with Cloud Eye Monitoring Data?
- How Do I Change a Partition Leader for a Topic in Kafka Manager?
- Monitoring & Alarm
-
Instances
-
Troubleshooting
- Troubleshooting Kafka Connection Exceptions
- Troubleshooting 6-Min Latency Between Message Creation and Retrieval
- Troubleshooting Message Creation Failures
- Troubleshooting Topic Deletion Failures
- Troubleshooting Failure to Log In to Kafka Manager in Windows
- Troubleshooting Error "Topic {{topic_name}} not present in metadata after 60000 ms" During Message Production or Consumption
- Change History
- API Reference (ME-Abu Dhabi Region)
-
User Guide (Paris Region)
- Service Overview
- Getting Started
- Permissions Management
- Preparing Required Resources
- Creating an Instance
- Accessing a Kafka Instance
- Managing Instances
- Managing Topics
- Managing Messages
- Managing Users
- Managing Consumer Groups
- Managing Kafka Quotas
- Modifying Kafka Parameters
- Quotas
- Monitoring
- Auditing
-
FAQs
-
Instances
- Why Can't I Select Two AZs?
- Why Can't I View the Subnet and Security Group Information When Creating a DMS Instance?
- How Do I Select Storage Space for a Kafka Instance?
- How Do I Choose Between High I/O and Ultra-high I/O?
- Which Capacity Threshold Policy Should I Use?
- Which Kafka Versions Are Supported?
- What Is the ZooKeeper Address of a Kafka Instance?
- Are Kafka Instances in Cluster Mode?
- Can I Modify the Port for Accessing a Kafka Instance?
- How Long Are Kafka SSL Certificates Valid for?
- How Do I Synchronize Data from One Kafka Instance to Another?
- How Do I Change the SASL_SSL Setting of a Kafka Instance?
- How Do I Modify the SASL Mechanism?
- Will a Kafka Instance Be Restarted After Its Enterprise Project Is Modified?
- Are Kafka Brokers and ZooKeeper Deployed on the Same VM or on Different VMs?
- Which Cipher Suites Are Supported by Kafka?
- Can I Change an Instance from Single-AZ Deployment to Multi-AZ Deployment?
- Does DMS for Kafka Support Cross-AZ Disaster Recovery? Where Can I View the AZs Configured for an Existing Instance?
- Do Kafka Instances Support Disk Encryption?
- Can I Change the VPC and Subnet After a Kafka Instance Is Created?
- Where Can I Find Kafka Streams Use Cases?
- Can I Upgrade Kafka Instances?
- Why Is the Version on the Console Different from That in Kafka Manager?
- How Do I Bind an EIP Again?
- Specification Modification
-
Connections
- How Do I Select and Configure a Security Group?
- Can I Access a Kafka Instance Over a Public Network?
- How Many Connection Addresses Does a Kafka Instance Have by Default?
- Do Kafka Instances Support Cross-Region Access?
- Do Kafka Instances Support Cross-VPC Access?
- Do Kafka Instances Support Cross-Subnet Access?
- Does DMS for Kafka Support Authentication with Kerberos?
- Does DMS for Kafka Support Password-Free Access?
- How Do I Obtain the Public Access Address After Public Access Is Enabled?
- Does DMS for Kafka Support Authentication on Clients by the Server?
- Can I Use PEM SSL Truststore When Connecting to a Kafka Instance with SASL_SSL Enabled?
- What Are the Differences Between JKS and CRT Certificates?
- Which TLS Version Does DMS for Kafka Support?
- Is There a Limit on the Number of Client Connections to a Kafka Instance?
- How Many Connections Are Allowed from Each IP Address?
- Can I Change the Private Network Addresses of a Kafka Instance?
- Is the Same SSL Certificate Used for Different Instances?
- Why Is It Not Recommended to Use a Sarama Client for Messaging?
-
Topics and Partitions
- Is There a Limit on the Number of Topics in a Kafka Instance?
- Why Is Partition Quantity Limited?
- Can I Reduce the Partition Quantity?
- Why Do I Fail to Create Topics?
- Do Kafka Instances Support Batch Importing Topics or Automatic Topic Creation?
- Why Do Deleted Topics Still Exist?
- Can I View the Disk Space Used by a Topic?
- Can I Add ACL Permissions for Topics?
- What Should I Do If Kafka Storage Space Is Used Up Because Retrieved Messages Are Not Deleted?
- How Do I Increase the Partition Quantity?
- Will a Kafka Instance Be Restarted After Its Automatic Topic Creation Setting Is Modified?
- How Do I Disable Automatic Topic Creation?
- Can I Delete Unnecessary Topics in a Consumer Group?
- What Can I Do If a Consumer Fails to Retrieve Messages from a Topic Due to Insufficient Permissions?
- Why Does an Instance Contain Default Topics __trace and __consumer_offsets?
- Consumer Groups
-
Messages
- What Is the Maximum Size of a Message that Can be Created?
- Why Does Message Poll Often Fail During Rebalancing?
- Why Can't I Query Messages on the Console?
- What Can I Do If Kafka Messages Are Accumulated?
- Why Do Messages Still Exist After the Retention Period Elapses?
- Do Kafka Instances Support Delayed Message Delivery?
- How Do I View the Number of Accumulated Messages?
- Why Is the Message Creation Time Displayed as Year 1970?
-
Kafka Manager
- Can I Configure a Kafka Manager Account to Be Read-Only?
- Why Can't I See Broker Information After Logging In to Kafka Manager?
- Yikes! Insufficient partition balance when creating topic : projectman_project_enterprise_project Try again.
- Can I Query the Body of a Message by Using Kafka Manager?
- Can I Change the Port of the Kafka Manager Web UI?
- Which Topic Configurations Can Be Modified on Kafka Manager?
- How Do I Change a Partition Leader for a Topic in Kafka Manager?
- Monitoring & Alarm
-
Instances
-
Troubleshooting
- Troubleshooting Kafka Connection Exceptions
- Troubleshooting 6-Min Latency Between Message Creation and Retrieval
- Troubleshooting Message Creation Failures
- Troubleshooting Topic Deletion Failures
- Troubleshooting Failure to Log In to Kafka Manager in Windows
- Troubleshooting Error "Topic {{topic_name}} not present in metadata after 60000 ms" During Message Production or Consumption
- Change History
-
API Reference (Paris Region)
- Before You Start
- API Overview
- Calling APIs
- Getting Started
-
APIs V2 (Recommended)
- Lifecycle Management
-
Instance Management
- Resetting the Password
- Resetting Kafka Manager Password
- Configuring Automatic Topic Creation
- Modifying the Private IP Address for Cross-VPC Access
- Querying Kafka Cluster Metadata
- Querying Consumer Group Details
- Resetting Consumer Group Offset to the Specified Position
- Querying Coordinator Details of a Kafka Instance
- Adding Partitions to a Topic for a Kafka Instance
- Reassigning Replicas of a Topic for a Kafka Instance
- Querying the Disk Usage Status of Topics
- Querying All Consumer Groups
- Querying a Specific Consumer Group
- Deleting a Consumer Group from a Kafka Instance
- Batch Deleting Consumer Groups of a Kafka Instance
- Initiating Partition Reassigning for a Kafka Instance
- Specification Modification Management
- Topic Management
- User Management
- Message Query
- Background Task Management
- Tag Management
- Other APIs
- Permissions and Supported Actions
- Out-of-Date APIs
- Appendix
- Change History
- Developer Guide (Paris Region)
-
User Guide (Kuala Lumpur Region)
- Service Overview
- Getting Started
- Process of Using Kafka
- Permissions Management
- Buying a Kafka Instance
- Configuring Topics
- Connecting to an Instance
- Managing Messages
- Managing Consumer Groups
- Managing Quotas
-
Managing Instances
- Viewing and Modifying Basic Information of a Kafka Instance
- Viewing Kafka Disk Usage
- Viewing Kafka Background Tasks
- Viewing Sample Code of Kafka Production and Consumption
- Modifying Kafka Instance Configuration Parameters
- Configuring Kafka Instance Tags
- Exporting the Kafka Instance List
- Restarting a Kafka Instance
- Deleting Kafka Instances
- Using Kafka Manager
- Modifying Kafka Instance Specifications
- Migrating Data
- Applying for Increasing Kafka Quotas
- Monitoring and Alarms
- Viewing Kafka Audit Logs
-
FAQs
-
Instances
- Why Can't I Select Two AZs?
- Why Can't I View the Subnet and Security Group Information When Creating a DMS Instance?
- How Do I Select Storage Space for a Kafka Instance?
- How Do I Choose Between High I/O and Ultra-high I/O?
- Which Capacity Threshold Policy Should I Use?
- Which Kafka Versions Are Supported?
- What Is the ZooKeeper Address of a Kafka Instance?
- Are Kafka Instances in Cluster Mode?
- Can I Modify the Port for Accessing a Kafka Instance?
- How Long Are Kafka SSL Certificates Valid for?
- How Do I Synchronize Data from One Kafka Instance to Another?
- How Do I Change the SASL_SSL Setting of a Kafka Instance?
- How Do I Modify the SASL Mechanism?
- Will a Kafka Instance Be Restarted After Its Enterprise Project Is Modified?
- Are Kafka Brokers and ZooKeeper Deployed on the Same VM or on Different VMs?
- Which Cipher Suites Are Supported by Kafka?
- Can I Change an Instance from Single-AZ Deployment to Multi-AZ Deployment?
- Does DMS for Kafka Support Cross-AZ Disaster Recovery? Where Can I View the AZs Configured for an Existing Instance?
- Do Kafka Instances Support Disk Encryption?
- Can I Change the VPC and Subnet After a Kafka Instance Is Created?
- Where Can I Find Kafka Streams Use Cases?
- Can I Upgrade Kafka Instances?
- Why Is the Version on the Console Different from That in Kafka Manager?
- Specification Modification
-
Connections
- How Do I Select and Configure a Security Group?
- Can I Access a Kafka Instance Over a Public Network?
- How Many Connection Addresses Does a Kafka Instance Have by Default?
- Do Kafka Instances Support Cross-Region Access?
- Do Kafka Instances Support Cross-VPC Access?
- Do Kafka Instances Support Cross-Subnet Access?
- Does DMS for Kafka Support Authentication with Kerberos?
- Does DMS for Kafka Support Password-Free Access?
- Does DMS for Kafka Support Authentication on Clients by the Server?
- Can I Use PEM SSL Truststore When Connecting to a Kafka Instance with SASL_SSL Enabled?
- What Are the Differences Between JKS and CRT Certificates?
- Which TLS Version Does DMS for Kafka Support?
- Is There a Limit on the Number of Client Connections to a Kafka Instance?
- How Many Connections Are Allowed from Each IP Address?
- Can I Change the Private Network Addresses of a Kafka Instance?
- Is the Same SSL Certificate Used for Different Instances?
- Why Is It Not Recommended to Use a Sarama Client for Messaging?
-
Topics and Partitions
- Is There a Limit on the Number of Topics in a Kafka Instance?
- Why Is Partition Quantity Limited?
- Can I Reduce the Partition Quantity?
- Why Do I Fail to Create Topics?
- Do Kafka Instances Support Batch Importing Topics or Automatic Topic Creation?
- Why Do Deleted Topics Still Exist?
- Can I View the Disk Space Used by a Topic?
- Can I Add ACL Permissions for Topics?
- What Should I Do If Kafka Storage Space Is Used Up Because Retrieved Messages Are Not Deleted?
- How Do I Increase the Partition Quantity?
- Will a Kafka Instance Be Restarted After Its Automatic Topic Creation Setting Is Modified?
- Can I Delete Unnecessary Topics in a Consumer Group?
- What Can I Do If a Consumer Fails to Retrieve Messages from a Topic Due to Insufficient Permissions?
- Why Does an Instance Contain Default Topics __trace and __consumer_offsets?
- Consumer Groups
-
Messages
- What Is the Maximum Size of a Message that Can be Created?
- Why Does Message Poll Often Fail During Rebalancing?
- Why Can't I Query Messages on the Console?
- What Can I Do If Kafka Messages Are Accumulated?
- Why Do Messages Still Exist After the Retention Period Elapses?
- Do Kafka Instances Support Delayed Message Delivery?
- How Do I View the Number of Accumulated Messages?
- Why Is the Message Creation Time Displayed as Year 1970?
-
Kafka Manager
- Can I Configure a Kafka Manager Account to Be Read-Only?
- Why Can't I See Broker Information After Logging In to Kafka Manager?
- Yikes! Insufficient partition balance when creating topic : projectman_project_enterprise_project Try again.
- Can I Query the Body of a Message by Using Kafka Manager?
- Can I Change the Port of the Kafka Manager Web UI?
- Which Topic Configurations Can Be Modified on Kafka Manager?
- Why Is Information Displayed on Kafka Manager Inconsistent with Cloud Eye Monitoring Data?
- How Do I Change a Partition Leader for a Topic in Kafka Manager?
- Monitoring & Alarm
-
Instances
-
Troubleshooting
- Troubleshooting Kafka Connection Exceptions
- Troubleshooting 6-Min Latency Between Message Creation and Retrieval
- Troubleshooting Message Creation Failures
- Troubleshooting Topic Deletion Failures
- Troubleshooting Failure to Log In to Kafka Manager in Windows
- Troubleshooting Error "Topic {{topic_name}} not present in metadata after 60000 ms" During Message Production or Consumption
- Change History
- API Reference (Kuala Lumpur Region)
-
User Guide (ME-Abu Dhabi Region)
- General Reference
Copied.
Basic Concepts
DMS for Kafka of the cloud service platform uses Kafka as the message engine. This chapter presents explanations of basic concepts of Kafka.
Topic
A topic is a category for messages. Messages are created, retrieved, and managed in the form of topics.
Topics adopt the publish-subscribe pattern. Producers publish messages into topics. One or more consumers subscribe to the messages in the topics. The producers and consumers are not directly linked to each other.
Producer
A producer publishes messages into topics. The messages are then delivered to other systems or modules for processing as agreed.
Consumer
A consumer subscribes to messages in topics and processes the messages. For example, a monitoring and alarm platform (a consumer) subscribing to log messages in certain topics can identify alarm logs and then send SMS or email alarm notifications.
Broker
A broker is a Kafka process in a Kafka cluster. Each process runs on a server, so a broker includes the storage, bandwidth, and other server resources.
Partition
A topic is divided into partitions. Messages are distributed to multiple partitions to achieve scalability and fault tolerance.
Replica
A replica is a redundant copy of a partition in a topic. Each partition can have one or more replicas, enabling message reliability.
Messages in each partition are fully replicated and synchronized, preventing data loss if one replica fails.
Each partition has one replica as the leader which handles the creation and retrievals of all messages. The rest replicas are followers which replicate the leader.
Topics and partitions are logical concepts, while replicas and brokers are physical concepts. The following diagram shows the relationships between partitions, brokers, and topics in messages streaming.

Aging Time
The period that messages are retained for. Consumers must retrieve messages before this period ends. Otherwise, the messages will be deleted and can no longer be retrieved.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot