Help Center/ Distributed Message Service for Kafka/ Developer Guide/ Overview
Updated on 2023-11-29 GMT+08:00
View PDF

Overview

Kafka instances are compatible with Apache Kafka and can be accessed using open-source Kafka clients. To access an instance in SASL mode, you will also need certificates.

This document describes how to collect instance connection information, such as the instance connection address, certificate required for SASL connection, and information required for public access. It also provides examples of accessing an instance in Java, Python, and Go.

The examples only demonstrate how to invoke Kafka APIs for producing and consuming messages. For more information about the APIs provided by Kafka, visit the Kafka official website.

Client Network Environment

A client can access a Kafka instance in any of the following modes:

  • If the client runs an Elastic Cloud Server (ECS) and is in the same region and VPC as the Kafka instance, the client can access the instance using a public network IP address.
  • If the client runs an ECS and is in the same region but not the same VPC as the Kafka instance, the client can access the instance using one of the following methods:
    • Establish a VPC peering connection to allow two VPCs to communicate with each other. For details, see VPC Peering Connection.
    • Use VPC Endpoint (VPCEP) to establish a cross-VPC connection.
  • If the client is not in the same network environment or region as the Kafka instance, the client can access the instance using a public network IP address.

    For public access, modify the inbound rules of the security group configured for the Kafka instance, allowing access over port 9094 (SASL disabled) or 9095 (SASL enabled).

The three modes differ only in the connection address for the client to access the instance. This document takes intra-VPC access as an example to describe how to set up the development environment.

If the connection times out or fails, check the network connectivity. You can use telnet to test the connection address and port of the instance.