- What's New
- Function Overview
- Product Bulletin
-
Service Overview
- Infographics
- What Is DMS for RabbitMQ?
- Product Advantages
- Application Scenarios
- Specifications
- Comparing RabbitMQ, Kafka, and RocketMQ
- Comparing RabbitMQ AMQP-0-9-1 with Open-Source RabbitMQ
- Comparing RabbitMQ Versions
- Related Services
- Security
- Notes and Constraints
- Basic Concepts
- Exchanges
- Permissions Management
- Billing
- Getting Started
-
User Guide
- Process of Using RabbitMQ
- Permissions Management
- Buying a RabbitMQ Instance
- Configuring Virtual Hosts
- Accessing a RabbitMQ Instance
- Managing Messages
- Advanced Features
-
Managing Instances
- Viewing and Modifying Basic Information of a RabbitMQ Instance
- Viewing RabbitMQ Client Connection Addresses
- Managing RabbitMQ Instance Tags
- Configuring RabbitMQ Recycling Policies
- Resetting the RabbitMQ Instance Password
- Enabling RabbitMQ Plug-ins
- Exporting the RabbitMQ Instance List
- Deleting a RabbitMQ Instance
- Logging In to RabbitMQ Management UI
- Modifying RabbitMQ Instance Specifications
- Migrating RabbitMQ Services
- Testing Instance Performance
- Applying for Increasing RabbitMQ Quotas
- Viewing Metrics and Configuring Alarms
- Viewing RabbitMQ Audit Logs
-
Best Practices
- RabbitMQ Best Practices
- Automatic Recovery of a RabbitMQ Client from Network Exceptions
- Automatic Consumer Reconnection After a RabbitMQ Node Restart
- Improving RabbitMQ Performance
- Configuring Queue Load Balancing
- Deduplicating Messages Through Message Idempotence
- Suggestions on Using DMS for RabbitMQ Securely
- Developer Guide
-
API Reference
- Before You Start
- API Overview
- Calling APIs
- Getting Started
- APIs V2 (Recommended)
- Permissions and Supported Actions
- Out-of-Date APIs
- Appendix
- Change History
- SDK Reference
-
FAQs
-
Instances
- What RabbitMQ Version Does DMS for RabbitMQ Use?
- What SSL Version Does DMS for RabbitMQ Use?
- Why Can't I View the Subnet and Security Group Information During Instance Creation?
- How Are Requests Evenly Distributed to Each VM of a Cluster RabbitMQ Instance?
- Do Queues Inside a Cluster RabbitMQ Instance Have Any Redundancy Backup?
- Does DMS for RabbitMQ Support Data Persistence? How Do I Perform Scheduled Data Backups?
- How Do I Obtain the Certificate After SSL Has Been Enabled?
- Can I Change the SSL Setting of a RabbitMQ Instance?
- Can RabbitMQ Instances Be Scaled Up?
- Does RabbitMQ Support Two-Way Authentication?
- Does DMS for RabbitMQ Support CPU and Memory Upgrades?
- How Do I Disable the RabbitMQ Management UI?
- Can I Change the AZ for an Instance?
- How Do I Obtain the Region ID?
- Why Can't I Select Two AZs?
- How to Change Single-node RabbitMQ Instances to Cluster Ones?
- Can I Change the VPC and Subnet After a RabbitMQ Instance Is Created?
-
Connections
- How Do I Configure a Security Group?
- Why Does a Client Fail to Connect to a RabbitMQ Instance?
- Does DMS for RabbitMQ Support Public Access?
- Does DMS for RabbitMQ Support Cross-Region Deployment?
- Do RabbitMQ Instances Support Cross-VPC Access?
- Do RabbitMQ Instances Support Cross-Subnet Access?
- What Should I Do If I Fail to Access a RabbitMQ Instance with SSL Encryption?
- Can I Access a RabbitMQ Instance Using DNAT?
- Why Can't I Open the Management Web UI?
- Can a Client Connect to Multiple Virtual Hosts of a RabbitMQ Instance?
- Why Does a RabbitMQ Cluster Have Only One Connection Address?
- Do RabbitMQ Instances Support the Ping Command?
- Messages
- Monitoring & Alarm
-
Instances
- Videos
-
More Documents
-
User Guide (ME-Abu Dhabi Region)
- Service Overview
- Getting Started
- Process of Using RabbitMQ
- Permissions Management
- Buying a RabbitMQ Instance
- Configuring Virtual Hosts
- Accessing a RabbitMQ Instance
- Managing Messages
- Advanced Features
-
Managing Instances
- Viewing and Modifying Basic Information of a RabbitMQ Instance
- Viewing RabbitMQ Client Connection Addresses
- Managing RabbitMQ Instance Tags
- Resetting the RabbitMQ Instance Password
- Enabling RabbitMQ Plug-ins
- Using the rabbitmq_tracing Plug-in
- Exporting the RabbitMQ Instance List
- Deleting a RabbitMQ Instance
- Logging In to RabbitMQ Management UI
- Modifying RabbitMQ Instance Specifications
- Migrating RabbitMQ Services
- Applying for Increasing RabbitMQ Quotas
- Viewing Metrics and Configuring Alarms
- Viewing RabbitMQ Audit Logs
-
FAQs
-
Instances
- What RabbitMQ Version Does DMS for RabbitMQ Use?
- What SSL Version Does DMS for RabbitMQ Use?
- Why Can't I View the Subnet and Security Group Information During Instance Creation?
- How Are Requests Evenly Distributed to Each VM of a Cluster RabbitMQ Instance?
- Do Queues Inside a Cluster RabbitMQ Instance Have Any Redundancy Backup?
- Does DMS for RabbitMQ Support Data Persistence? How Do I Perform Scheduled Data Backups?
- How Do I Obtain the Certificate After SSL Has Been Enabled?
- Can I Change the SSL Setting of a RabbitMQ Instance?
- Can RabbitMQ Instances Be Scaled Up?
- Does RabbitMQ Support Two-Way Authentication?
- Does DMS for RabbitMQ Support CPU and Memory Upgrades?
- How Do I Disable the RabbitMQ Management UI?
- Can I Change the AZ for an Instance?
- How Do I Obtain the Region ID?
- Why Can't I Select Two AZs?
- How to Change Single-node RabbitMQ Instances to Cluster Ones?
- Can I Change the VPC and Subnet After a RabbitMQ Instance Is Created?
-
Connections
- How Do I Configure a Security Group?
- Why Does a Client Fail to Connect to a RabbitMQ Instance?
- Does DMS for RabbitMQ Support Public Access?
- Does DMS for RabbitMQ Support Cross-Region Deployment?
- Do RabbitMQ Instances Support Cross-VPC Access?
- Do RabbitMQ Instances Support Cross-Subnet Access?
- What Should I Do If I Fail to Access a RabbitMQ Instance with SSL Encryption?
- Can I Access a RabbitMQ Instance Using DNAT?
- Why Can't I Open the Management Web UI?
- Can a Client Connect to Multiple Virtual Hosts of a RabbitMQ Instance?
- Why Does a RabbitMQ Cluster Have Only One Connection Address?
- Messages
- Monitoring & Alarm
-
Instances
- Change History
- API Reference (ME-Abu Dhabi Region)
-
User Guide (Kuala Lumpur Region)
- Service Overview
- Permissions Management
- Preparing the Environment
- Buying an Instance
- Accessing a RabbitMQ Instance
- Operating RabbitMQ Instances
- Quotas
- Monitoring
- Auditing
-
FAQs
-
Instances
- What RabbitMQ Version Does DMS for RabbitMQ Use?
- What SSL Version Does DMS for RabbitMQ Use?
- Why Can't I View the Subnet and Security Group Information During Instance Creation?
- What If One RabbitMQ VM Fails to Be Restarted When a Cluster RabbitMQ Instance Is Being Restarted?
- How Are Requests Evenly Distributed to Each VM of a Cluster RabbitMQ Instance?
- Do Queues Inside a Cluster RabbitMQ Instance Have Any Redundancy Backup?
- Does DMS for RabbitMQ Support Data Persistence? How Do I Perform Scheduled Data Backups?
- How Do I Obtain the Certificate After SSL Has Been Enabled?
- Can I Change the SSL Setting of a RabbitMQ Instance?
- Can RabbitMQ Instances Be Scaled Up?
- Does DMS for RabbitMQ Support MQTT?
- How Do I Clear Queue Data?
- Does DMS for RabbitMQ Support CPU and Memory Upgrades?
- How Do I Disable the RabbitMQ Management UI?
- Can I Change the AZ for an Instance?
-
Connections
- How Do I Configure a Security Group?
- Why Does a Client Fail to Connect to a RabbitMQ Instance?
- Does DMS for RabbitMQ Support Public Access?
- Does DMS for RabbitMQ Support Cross-Region Deployment?
- Does DMS for RabbitMQ Support Cross-VPC Access?
- Does DMS for RabbitMQ Support Cross-Subnet Access?
- What Should I Do If I Fail to Access a RabbitMQ Instance with SSL Encryption?
- Can I Access a RabbitMQ Instance Using DNAT?
- Why Can't I Open the Management Web UI?
- Can a Client Connect to Multiple Virtual Hosts of a RabbitMQ Instance?
- Why Does a RabbitMQ Cluster Have Only One Connection Address?
- Plug-ins
- Messages
- Monitoring & Alarm
-
Instances
- Change History
- API Reference (Kuala Lumpur Region)
-
User Guide (ME-Abu Dhabi Region)
- General Reference
Show all
Copied.
Deduplicating Messages Through Message Idempotence
Overview
In RabbitMQ service processes, an idempotent message process refers to a situation where a message is re-sent and consumed for multiple times and each consumption result is the same, having no negative effects on services. Idempotent messages ensure consistency in the final processing results. Services are not affected no matter how many times a message is re-sent.
Take paying as an example. Assume that a user selects a product, makes payment, and receives multiple bills due to unstable Internet connection. The bills are all paid. However, the billing should take place only once and the merchant should generate only one order placement. In this case, idempotent messages can be used to avoid the repetition.
In actual applications, messages are re-sent because of intermittent network disconnections and client faults during message production or consumption. Message repetition can be classified into two scenarios.
- A producer repeatedly sends a message:
If a producer successfully sends a message to the server but does not receive a successful response due to an intermittent network disconnection, the producer determines that the message failed to be sent and tries resending the message. In this case, the server receives two messages of the same content. Consumers consume two messages of the same content.
- A consumer repeatedly consumes a message:
A message is successfully delivered to a consumer and processed. If the server does not receive a response from the consumer due to an intermittent network disconnection, the server determines that the message failed to be delivered. To ensure that the message is consumed at least once, the server retries delivering the message. As a result, the consumer consumes two messages of the same content.
Implementation
A globally unique ID can be used to determine whether a message is consumed repeatedly. If yes, the consumption result is returned. If no, consume the message and record the global ID.
- Producers set a unique ID for each message. Sample code is as follows:
// The message is persisted with a globally unique ID which is randomly generated. AMQP.BasicProperties.Builder builder = new AMQP.BasicProperties().builder(); builder.deliveryMode(2); builder.messageId(UUID.randomUUID().toString()); // The custom message. String message = "message content"; // Produce messages. Set exchangeName and routingKey to the actual values. channel.basicPublish("exchangeName", "routingKey", false, builder.build(), message.getBytes(StandardCharsets.UTF_8)); String messageId = builder.build().getMessageId(); System.out.println("messageID: " + messageId); System.out.println("Send message success!"); // Close the channel. channel.close(); // Close the connection. connection.close();
- Consumers deduplicate messages based on their IDs. Sample code is as follows:
// Create a table with message ID as the primary key. Unique primary keys of databases can be used to process RabbitMQ idempotence. // Before consumption, query the message from the database. If the message exists, it is consumed. Otherwise, consume it. //queueName: Use the actual queue name. channel.basicConsume("queueName", false, new DefaultConsumer(channel) { @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { //Obtain the message ID and check whether it is null. String messageId = properties.getMessageId(); if (StringUtils.isBlank(messageId)){ logger.info("messageId is null"); return; } // Query the database by primary key "message ID". If records exist, the message is consumed. Otherwise, consume the message and write it into the database. // Database querying logic ... //todo // If no record exists, consume the message. Or notify that the message is consumed. if (null == "{Found in the database}"){ // Obtain the message. String message = new String(body,StandardCharsets.UTF_8); // Manual response. channel.basicAck(envelope.getDeliveryTag(),false); logger.info("[x] received message: " + message + "," + "messageId:" + messageId); // Save the message to the database table, indicating that the message has been consumed. // The database input operation ... //todo } else { // If the message is consumed, skip it. logger.error("The message is already consumed."); } } });
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot