Updated on 2024-04-07 GMT+08:00

Step 1: Prepare the Environment

VPC

A VPC provides an isolated virtual network for your Kafka instances. You can configure and manage the network as required.

  1. Before creating a Kafka instance, ensure that a VPC and a subnet are available.

    For details, see Creating a VPC. If you already have an available VPC and subnet, you do not need to create new ones.

    Note the following when creating a VPC and subnet:

    • The VPC and the Kafka instance must be in the same region.
    • Use the default settings when creating a VPC and subnet.

  2. Before creating a Kafka instance, ensure that a security group is available.

    For details, see Creating a Security Group. If you already have an available security group, you do not need to create a new one.

    To use Kafka instances, add the security group rules described in Table 1. Other rules can be added based on site requirements.
    Table 1 Security group rules

    Direction

    Protocol

    Port

    Source address

    Description

    Inbound

    TCP

    9093

    0.0.0.0/0

    Access Kafka instance in the same VPC over private network (SSL enabled).

    After a security group is created, it has a default inbound rule that allows communication among ECSs within the security group and a default outbound rule that allows all outbound traffic. If you access your Kafka instance within a VPC, you do not need to add the rules described in Table 1.

ECS

Before connecting to a Kafka instance, ensure that you have created an ECS, installed the JDK, configured environment variables, and downloaded an open-source Kafka client. The following steps describe how to complete these preparations. A Linux ECS is taken as an example. For more information on how to install JDK and configure the environment variables for a Windows ECS, please search the Internet.

  1. Log in to the management console, click in the upper left corner, click Elastic Cloud Server under Computing, and then create an ECS.

    For details, see Creating an ECS. If you already have an available ECS, skip this step.

  2. Log in to the ECS.
  3. Install JDK or JRE, and add the following contents to .bash_profile in the home directory to configure the environment variables JAVA_HOME and PATH. In this command, /opt/java/jdk1.8.0_151 is the JDK installation path. Change it to the path where you install JDK or JRE.

    export JAVA_HOME=/opt/java/jdk1.8.0_151 
    export PATH=$JAVA_HOME/bin:$PATH

    Run the source .bash_profile command for the modification to take effect.

    Use Oracle JDK instead of ECS's default JDK (for example, OpenJDK), because ECS's default JDK may not be suitable. Obtain Oracle JDK 1.8.111 or later from Oracle's official website.

  4. Download an open-source Kafka client.

    If the version of the Kafka instance is 2.7, download the client at https://archive.apache.org/dist/kafka/2.7.2/kafka_2.12-2.7.2.tgz.

    wget https://archive.apache.org/dist/kafka/2.7.2/kafka_2.12-2.7.2.tgz

  5. Run the following command to decompress the package:

    tar -zxf kafka_2.12-2.7.2.tgz

Follow-Up Procedure

Step 2: Create a Kafka Instance