Updated on 2024-05-24 GMT+08:00

Kafka Network Connection Conditions

A client can connect to a Kafka instance in public or private networks. Notes before using a private network:

  • By default, a client and a Kafka instance are interconnected when they are deployed in the same VPC.
  • If they are not, you need to interconnect them because of isolation among VPCs.

Table 1 lists how a client can connect to a Kafka instance.

Table 1 Connection modes

Mode

How To Do

Reference

Public access

Enable public access on the Kafka console and configure an elastic IP (EIP). The client can connect to the Kafka instance through the EIP.

Configuring Kafka Public Access

Configure port mapping using DNAT. The client can connect to the Kafka instance in a public network.

Accessing Kafka in Public Networks Using DNAT

Private access

A client and a Kafka instance are interconnected when they are deployed in the same VPC.

-

When a client and a Kafka instance are deployed in different VPCs of the same region, connect the client and the Kafka instance across VPCs using a VPC endpoint.

Accessing Kafka Across VPCs Using VPCEP

When a client and a Kafka instance are deployed in different VPCs of the same region, interconnect two VPCs using a VPC peering connection.

VPC Peering Connection

When a client and a Kafka instance are not deployed in the same region, create a cloud connection for loading VPCs to be interconnected.

Connecting VPCs in the Same Account

Before connecting a client to a Kafka instance, allow accesses for the following security groups.

After a security group is created, its default inbound rule allows communication among ECSs within the security group and its default outbound rule allows all outbound traffic. In this case, you can access a Kafka instance within a VPC, and do not need to add rules according to Table 2.

Table 2 Security group rules

Direction

Protocol

Port

Source

Description

Inbound

TCP

9094

0.0.0.0/0

Access a Kafka instance through the public network (without SSL encryption).

Inbound

TCP

9092

0.0.0.0/0

  • Access a Kafka instance within a VPC (without SSL encryption).
  • Access a Kafka instance across VPCs using a peering connection (without SSL encryption).

Inbound

TCP

9095

0.0.0.0/0

Access a Kafka instance through the public network (with SSL encryption).

Inbound

TCP

9093

0.0.0.0/0

  • Access a Kafka instance within a VPC (with SSL encryption).
  • Access a Kafka instance across VPCs using a peering connection (with SSL encryption).

Inbound

TCP

9011

198.19.128.0/17

Access a Kafka instance across VPCs using a VPC endpoint (with or without SSL encryption).

Inbound

TCP

9011

0.0.0.0/0

Access a Kafka instance using DNAT (with or without SSL encryption).