Updated on 2024-10-16 GMT+08:00

Kafka Best Practices

This section summarizes best practices of Distributed Message Service (DMS) for Kafka in common scenarios. Each practice is given a description and procedure.

Table 1 Kafka best practices

Best Practice

Description

Improving Kafka Message Processing Efficiency

This document provides producers and consumers with message suggestions, improving the efficiency and reliability of message sending and consumption.

Optimizing Consumer Polling

This document describes how to optimize consumer polling in scenarios where real-time message consumption is not required, saving resources when there are few or no messages.

Interconnecting Logstash to Kafka to Produce and Consume Messages

Kafka instances are available as the input and output sources of Logstash. This document describes how to connect Logstash to Kafka instances for message production and consumption.

Using MirrorMaker to Synchronize Data Across Clusters

MirrorMaker can mirror data from a source cluster to a target cluster. This document describes how to use MirrorMaker to synchronize data between two Kafka instances unidirectionally or bidirectionally.

Handling Message Accumulation

This document describes the causes of message stacking and the handling measures.

Handling Service Overload

This document describes the causes of high CPU usage and full disk space and the handling measures.

Handling Uneven Service Data

This document describes the causes of unbalanced service data and the handling measures.

Configuring Message Accumulation Monitoring

This document describes how to generate an alarm when the number of stacked messages exceeds a specified threshold. In this way, you can be aware of the service running status in time by SMS or email.

Suggestions on Using DMS for Kafka Securely

This document describes security best practices of using Kafka. It aims to provide a standard guide for overall security capabilities.