Distributed Message Service for Kafka
Distributed Message Service for Kafka
All results for "
" in this service
All results for "
" in this service
What's New
Function Overview
Product Bulletin
Vulnerability Notices
Vulnerability Fixing Policies
Service Overview
What Is DMS for Kafka?
Product Advantages
Application Scenarios
Kafka Instance Specifications
Single-node Kafka Instances
Cluster Kafka Instances
Comparing Single-node and Cluster Kafka Instances
Comparing Kafka, RabbitMQ, and RocketMQ
Comparing DMS for Kafka and Open-Source Kafka
Notes and Constraints
Related Services
Basic Concepts
Permissions
Billing
Billing
Billing Overview of DMS for Kafka
Billing Modes
Billing Modes of DMS for Kafka
Yearly/Monthly
Pay-per-Use
Billing Items
Billing Examples
Billing Mode Changes
Overview
Pay-per-Use to Yearly/Monthly
Yearly/Monthly to Pay-per-Use
Renewal
Overview
Manual Renewal
Auto Renewal
Bills
Arrears
Stopping Billing
Cost Management
Getting Started
Getting Started with Kafka for Message Production and Consumption
Common Practices
User Guide
Process of Using Kafka
Permissions Management
Creating an IAM User and Granting DMS for Kafka Permissions
Buying a Kafka Instance
Configuring Topics
Creating a Kafka Topic
Configuring Kafka Topic Permissions
Managing Topics
Viewing Kafka Topic Details
Modifying Kafka Topic Configurations
Changing Kafka Partition Quantity
Modifying Kafka Topic Replicas
Exporting the Kafka Topic List
Reassigning Kafka Partitions
Configuring Automatic Topic Creation
Deleting a Kafka Topic
Connecting to an Instance
Configuring Kafka Network Connections
Kafka Network Connection Conditions
Configuring Kafka Public Access
Accessing Kafka Using a VPC Endpoint Across VPCs
Accessing Kafka in a Public Network Using DNAT
Configuring Kafka Access Control
Configuring Plaintext or Ciphertext Access to Kafka Instances
Configuring Kafka ACL Users
Configuring the Kafka Client
Setting Parameters for Kafka Clients
Suggestions on Using the Kafka Client
Connecting to Kafka Using the Client (SASL Disabled)
Connecting to Kafka Using the Client (SASL Enabled)
Connecting to Kafka on the Console
Managing Messages
Viewing Kafka Messages
Changing Kafka Message Retention Period
Deleting Kafka Messages
Diagnosing Kafka Message Accumulation
Managing Consumer Groups
Creating a Kafka Consumer Group
Querying the Kafka Consumer Group List
Viewing Kafka Consumer Information
Viewing and Resetting Kafka Consumption Offsets
Viewing Kafka Rebalancing Logs
Modifying Kafka Consumer Group Description
Exporting Kafka Consumer Groups
Deleting a Kafka Consumer Group
Managing Quotas
Configuring Kafka Quotas
Monitoring Kafka Quotas
Managing Instances
Viewing and Modifying Basic Information of a Kafka Instance
Viewing Kafka Disk Usage
Viewing Kafka Background Tasks
Viewing Sample Code of Kafka Production and Consumption
Modifying Kafka Instance Configuration Parameters
Configuring Kafka Instance Tags
Exporting the Kafka Instance List
Restarting a Kafka Instance
Deleting Kafka Instances
Using Kafka Manager
Accessing Kafka Manager
Resetting Kafka Manager Password
Restarting Kafka Manager
Disabling Kafka Manager
Modifying Instance Specifications
Modifying Cluster Kafka Instance Specifications
Migrating Data
Kafka Data Migration Overview
Migrating Data Using Smart Connect
Enabling Smart Connect
Replicating Kafka Instance Data
Dumping Kafka Data to Object Storage Service (OBS)
Managing Smart Connect Tasks
Disabling Smart Connect
Testing Instance Performance
Kafka Production Rate and CPU Usage
Kafka Instance TPS
Applying for Increasing Kafka Quotas
Monitoring and Alarms
Viewing Kafka Metrics
Kafka Metrics
Configuring a Kafka Alarm Rule
Viewing Kafka Audit Logs
Best Practices
Overview
Improving Kafka Message Processing Efficiency
Interconnecting Logstash to Kafka to Produce and Consume Messages
Using MirrorMaker to Synchronize Data Across Clusters
Handling Message Accumulation
Handling Service Overload
Handling Uneven Service Data
Setting the Kafka Topic Partition Quantity
Configuring Message Accumulation Monitoring
Optimizing Consumer Polling
Developer Guide
Overview
Collecting Connection Information
Java
Configuring Kafka Clients in Java
Setting Up the Java Development Environment
Python
Go
Obtaining Kafka Clients
Using spring-kafka
Performance Whitepaper
Test Scenarios
Test Environment
Test Procedure
Test Results
API Reference
Before You Start
API Overview
Calling APIs
Making an API Request
Authentication
Returned Values
Getting Started
APIs V2 (Recommended)
Managing Lifecycle
Creating a Kafka Instance
Listing All Instances
Querying an Instance
Deleting an Instance
Modifying Instance Information
Batch Restarting or Deleting Instances
Obtaining Instance Configurations
Modifying Instance Configurations
Instance Management
Resetting the Password
Resetting Kafka Manager Password
Restarting Kafka Manager
Configuring Automatic Topic Creation
Modifying the Private IP Address for Cross-VPC Access
Querying Kafka Cluster Metadata
Querying Coordinator Details of a Kafka Instance
Reassigning Replicas of a Topic for a Kafka Instance
Modifying Kafka Access Modes
Querying the Disk Usage Status of Topics
Disabling Kafka Manager
Deleting a User or Client Quota
Querying User or Client Quotas
Creating User or Client Quotas
Modifying User or Client Quotas
Initiating Partition Reassignment for a Kafka Instance
Smart Connect
Enabling Smart Connect (Pay-per-Use Instance)
Disabling Smart Connect (Pay-per-Use Instance)
Creating a Smart Connect Task
Querying Smart Connect Tasks
Querying Smart Connect Task Details
Deleting a Smart Connect Task
Pausing a Smart Connect Task
Restarting a Smart Connect Task
Starting a Smart Connect Task or Restarting a Paused or Running Smart Connect Task
Specification Modification Management
Increasing Instance Specifications
Querying Product Information for Instance Specification Modification
Topic Management
Producing Messages to Kafka
Creating a Topic for a Kafka Instance
Listing Topics of a Kafka Instance
Modifying Topics of a Kafka Instance
Batch Deleting Topics of a Kafka Instance
Querying the Partition List of a Topic
Querying the Current Producer List of a Topic
Querying Topic Details
Deleting Topic Quotas
Creating a Topic Quota
Modifying Topic Quotas
Querying Topic Quotas
Managing Consumer Groups
Querying Consumer Group Details
Querying All Consumer Groups
Deleting Consumer Groups of a Kafka Instance in Batches
Creating a Consumer Group
Resetting Consumer Group Offset to the Specified Position
Querying the Offset of a Consumer Group
Modifying All Consumer Groups
Querying a Specified Consumer Group
Deleting a Specified Consumer Group
Modifying a Specified Consumer Group
Querying Topics of a Specified Consumer Group
Querying Consumers in a Specified Consumer Group
User Management
Querying the User List
Creating a User
Deleting Users in Batches
Resetting a User Password
Modifying User Parameters
Querying User Permissions
Granting User Permissions
Managing Messages
Querying Messages
Querying a Message with a Specified Offset
Querying a Message with a Specified Time Period
Querying Offset of the Earliest Message in a Partition
Querying Offset of the Latest Message in a Partition
Deleting a Kafka Message
Background Task Management
Listing Background Tasks
Querying a Background Task
Deleting a Background Task
Tag Management
Batch Adding or Deleting Tags
Listing Tags of an Instance
Listing Tags of a Project
Diagnosis Management
Pre-check Before Diagnosing Message Stack
Creating a Message Stack Diagnosis Task
Querying the Message Stack Diagnosis Report List
Batch Deleting Message Stack Diagnosis Reports
Querying Diagnosis Report Details
Other APIs
Listing Maintenance Time Windows
Listing AZ Information
Querying Product Specifications List
Querying Kafka Instance Monitoring Dimensions
Querying vCPUs of a Kafka Flavor
Permissions and Supported Actions
Out-of-Date APIs
API V1
APIs for Managing Instances
Creating an Instance
Querying an Instance
Modifying an Instance
Deleting an Instance
Restarting or Deleting Instances in Batches
Querying All Instances
Creating a Topic in a Kafka Instance
Querying a Topic in a Kafka Instance
Deleting Topics in a Kafka Instance in Batches
Other APIs
Querying AZ Information
Querying Product Specifications
Querying Maintenance Time Windows
API V2
Managing Lifecycle
Creating an Instance
Instance Management
Initiating Partition Reassignment for a Kafka Instance
Specification Modification Management
Querying Product Information for Instance Specification Modification
Increasing Instance Specifications
Appendix
Status Code
Error Codes
Instance Status
Obtaining a Project ID
Obtaining the Account Name and Account ID
Change History
SDK Reference
SDK Overview
FAQs
Instances
Why Can't I Select Two AZs?
Why Can't I View the Subnet and Security Group Information When Creating a DMS Instance?
How Do I Select Storage Space for a Kafka Instance?
How Do I Choose Between High I/O and Ultra-high I/O?
Which Capacity Threshold Policy Should I Use?
Which Kafka Versions Are Supported?
What Is the ZooKeeper Address of a Kafka Instance?
Are Kafka Instances in Cluster Mode?
Can I Modify the Port for Accessing a Kafka Instance?
How Long Are Kafka SSL Certificates Valid for?
How Do I Synchronize Data from One Kafka Instance to Another?
How Do I Change the SASL_SSL Setting of a Kafka Instance?
How Do I Modify the SASL Mechanism?
Will a Kafka Instance Be Restarted After Its Enterprise Project Is Modified?
Are Kafka Brokers and ZooKeeper Deployed on the Same VM or on Different VMs?
Which Cipher Suites Are Supported by Kafka?
Can I Change Single-AZ Deployment to Multi-AZ Deployment for an Instance?
Does DMS for Kafka Support Cross-AZ Disaster Recovery? Where Can I Check Whether an Existing Instance is Across-AZs?
Do Kafka Instances Support Disk Encryption?
Can I Change the VPC and Subnet After a Kafka Instance Is Created?
Where Can I Find Kafka Streams Use Cases?
Can I Upgrade Kafka Instances?
How Do I Bind an EIP Again?
Specification Modification
Does Specification Modification Affect Services?
Will Data Migration Be Involved When I Increase Specifications?
What Can I Do When I Fail to Increase Specifications Due to Insufficient Resources?
Connections
How Do I Select and Configure a Security Group?
Can I Access a Kafka Instance Over a Public Network?
How Many Connection Addresses Does a Kafka Instance Have by Default?
Do Kafka Instances Support Cross-Region Access?
Do Kafka Instances Support Cross-VPC Access?
Do Kafka Instances Support Cross-Subnet Access?
Does DMS for Kafka Support Authentication with Kerberos?
Does DMS for Kafka Support Password-Free Access?
How Do I Obtain the Public Access Address After Public Access Is Enabled?
Does DMS for Kafka Support Authentication on Clients by the Server?
Can I Use PEM SSL Truststore When Connecting to a Kafka Instance with SASL_SSL Enabled?
What Are the Differences Between JKS and CRT Certificates?
Which TLS Version Does DMS for Kafka Support?
Is There a Limit on the Number of Client Connections to a Kafka Instance?
How Many Connections Are Allowed from Each IP Address?
Can I Change the Private Network Addresses of a Kafka Instance?
Is the Same SSL Certificate Used for Different Instances?
Why Is It Not Recommended to Use a Sarama Client for Messaging?
Topics and Partitions
Is There a Limit on the Number of Topics in a Kafka Instance?
Why Is Partition Quantity Limited?
Can I Reduce the Partition Quantity?
Why Do I Fail to Create Topics?
Do Kafka Instances Support Batch Importing Topics or Automatic Topic Creation?
Why Do Deleted Topics Still Exist?
Can I View the Disk Space Used by a Topic?
Can I Add ACL Permissions for Topics?
What Should I Do If Kafka Storage Space Is Used Up Because Retrieved Messages Are Not Deleted?
How Do I Increase the Partition Quantity?
Will a Kafka Instance Be Restarted After Its Automatic Topic Creation Setting Is Modified?
What Can I Do If a Consumer Fails to Retrieve Messages from a Topic Due to Insufficient Permissions?
Why Does an Instance Contain Default Topics __trace and __consumer_offsets?
Consumer Groups
Do I Need to Create Consumer Groups, Producers, and Consumers for Kafka Instances?
Will a Consumer Group Without Active Consumers Be Automatically Deleted in 14 Days?
Why Do I See a Deleted Consumer Group on Kafka Manager?
Why Does a Deleted Consumer Group Still Exist?
Why Can't I View Consumers When Instance Consumption Is Normal?
Can I Delete Unnecessary Topics in a Consumer Group?
Messages
What Is the Maximum Size of a Message that Can be Created?
Why Does Message Poll Often Fail During Rebalancing?
Why Can't I Query Messages on the Console?
What Can I Do If Kafka Messages Are Accumulated?
Why Do Messages Still Exist After the Retention Period Elapses?
Do Kafka Instances Support Delayed Message Delivery?
How Do I View the Number of Accumulated Messages?
Why Is the Message Creation Time Displayed as Year 1970?
How Do I Modify message.max.bytes?
Why Are Offsets Not Continuous?
Kafka Manager
Can I Configure a Kafka Manager Account to Be Read-Only?
Why Can't I See Broker Information After Logging In to Kafka Manager?
Yikes! Insufficient partition balance when creating topic : projectman_project_enterprise_project Try again.
Can I Query the Body of a Message by Using Kafka Manager?
Can I Change the Port of the Kafka Manager Web UI?
Which Topic Configurations Can Be Modified on Kafka Manager?
Why Is Information Displayed on Kafka Manager Inconsistent with Cloud Eye Monitoring Data?
How Do I Change a Partition Leader for a Topic in Kafka Manager?
Why Is the Version on the Console Different from That in Kafka Manager?
Monitoring & Alarm
Why Can't I View the Monitoring Data?
Why Is the Monitored Number of Accumulated Messages Inconsistent with the Message Quantity Displayed on the Kafka Console?
Why Is a Consumer Group Still on the Monitoring Page After Being Deleted?
Why Do Metrics Fluctuate Significantly (Disk Read/Write Speed, Average Disk Read/Write Time, and CPU Usage)?
Why Does JVM Heap Memory Usage of JVM Fluctuate Significantly?
Troubleshooting
Troubleshooting Kafka Connection Exceptions
Troubleshooting 6-Min Latency Between Message Creation and Retrieval
Troubleshooting Message Creation Failures
Troubleshooting Topic Deletion Failures
Troubleshooting Failure to Log In to Kafka Manager in Windows
Troubleshooting Error "Topic {{topic_name}} not present in metadata after 60000 ms" During Message Production or Consumption
Videos