Help Center/ Distributed Message Service/ Service Overview/ Kafka Premium Instances and Advanced Queues/ Comparing Kafka Instances and DMS Advanced Queues
Updated on 2022-02-24 GMT+08:00
View PDF
Share

Comparing Kafka Instances and DMS Advanced Queues

Both Kafka premium instances and advanced queues are compatible with Apache Kafka. However, they differ in the following aspects.

Open-Source Compatibility

  • Advanced queues:

    Kafka 0.10.2.1

  • Kafka premium instances:

    Kafka 1.1.0

    With each version upgrade, Apache Kafka introduces new features, improves APIs, and updates producer and consumer configuration files. To check whether your application features and APIs are compatible with your Kafka clients, see the upgrade notes on the official Apache Kafka website.

Purchase

  • Advanced queues:

    An advanced queue (equivalent to a topic) is created on the DMS console. You do not need to configure the storage space or the bandwidth because these resources are allocated by the system.

  • Kafka premium instances:

    A Kafka premium instance is created on the DMS for Kafka console. Before creating a Kafka premium instance, determine the required bandwidth and storage space based on your service expectations for the next one or two years. You also need to prepare a VPC and security group for the instance.

    After the instance has been created, you must create topics in the instance and configure the number of partitions and replicas for the topics.

Usage

  • Advanced queues:

    Advanced queues are compatible with Kafka APIs. DMS provides SDKs in Java, Python, Lua, C, and Go languages. For details, see the DMS Kafka Developer Guide.

    To use open-source Kafka clients, see Using the Enhanced Java SDK. Add the enhanced Kafka Java SDK provided by DMS to the directory of the open-source client package and then pass the security authentication.

  • Kafka premium instances:

    Kafka premium instances are fully compatible with open-source Kafka. You can access Kafka premium instances and topics using open-source Kafka clients. If SASL access is enabled, you must use the SSL certificate provided by DMS.

Performance

  • Advanced queues:

    There are two queue modes: high-throughput and high-reliability. In the high-throughput mode, messages are flushed to disk asynchronously, ensuring high concurrency.

  • Kafka premium instances:

    Compute, bandwidth, and storage resources are physically isolated for each instance. Determine the required bandwidth and storage space when creating an instance. For storage space, you can choose Ultra-high I/O, which indicates that messages are stored on SSDs.

Other Dimensions

You can customize the number of partitions and replicas for a Kafka premium instance. Each topic can have 1 to 20 partitions and 1 to 3 replicas.

By default, an advanced queue has three partitions and three replicas. If this does not meet your requirement, submit a service ticket.

Divide a topic into a certain number of partitions so that messages can be evenly distributed to partitions, enabling load balancing and horizontal scalability. Different consumers can retrieve messages from one or more partitions, improving message processing performance.

With more replicas comes higher reliability. However, synchronizing messages between replicas consumes bandwidth and offsets compute performance.