Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive

Step 1: Prepare the Environment

Updated on 2023-05-06 GMT+08:00

VPC

A VPC provides an isolated virtual network for your Kafka instances. You can configure and manage the network as required.

  1. Before creating a Kafka instance, ensure that a VPC and a subnet are available.

    For details, see Creating a VPC. If you already have an available VPC and subnet, you do not need to create new ones.

    Note the following when creating a VPC and subnet:

    • The VPC and the Kafka instance must be in the same region.
    • Use the default settings when creating a VPC and subnet.

  2. Before creating a Kafka instance, ensure that a security group is available.

    For details, see Creating a Security Group. If you already have an available security group, you do not need to create a new one.

    Note the following when creating a security group:

    • Set Template to Custom.
    • To use Kafka instances, add the security group rules described in Table 1. Other rules can be added based on site requirements.
      Table 1 Security group rules

      Direction

      Protocol

      Port

      Source

      Description

      Inbound

      TCP

      9094

      0.0.0.0/0

      Access a Kafka instance through the public network (without SSL encryption).

      Inbound

      TCP

      9092

      0.0.0.0/0

      Access a Kafka instance within a VPC (without SSL encryption).

      Inbound

      TCP

      9095

      0.0.0.0/0

      Access a Kafka instance through the public network (with SSL encryption).

      Inbound

      TCP

      9093

      0.0.0.0/0

      Access a Kafka instance within a VPC (with SSL encryption).

      Inbound

      TCP

      9999

      0.0.0.0/0

      Access Kafka Manager.

      NOTE:

      After a security group is created, it has a default inbound rule that allows communication among ECSs within the security group and a default outbound rule that allows all outbound traffic. If you access your Kafka instance within a VPC, you do not need to add the rules described in Table 1.

(Optional) EIP

To access a Kafka instance over a public network, prepare an elastic IP address (EIP) in advance.

For details, see Assigning an EIP.

Note the following when creating EIPs:

  • The EIPs must be created in the region the Kafka instance is in.
  • The number of EIPs must be the same as the number of Kafka instance brokers.
  • The Kafka console cannot identify IPv6 EIPs.

ECS

Before connecting to a Kafka instance, ensure that you have purchased an ECS, installed the JDK, configured environment variables, and downloaded an open-source Kafka client. The following steps describe how to complete these preparations. A Linux ECS is taken as an example. For more information on how to install JDK and configure the environment variables for a Windows ECS, please search the Internet.

  1. Log in to the management console. In the upper left corner, hover the mouse pointer over . Under Compute, click Elastic Cloud Server, and then create an ECS.

    For details, see Creating an ECS. If you already have an available ECS, skip this step.

  2. Log in to the ECS.
  3. Install JDK or JRE, and add the following contents to .bash_profile in the home directory to configure the environment variables JAVA_HOME and PATH. In this command, /opt/java/jdk1.8.0_151 is the JDK installation path. Change it to the path where you install JDK or JRE.

    export JAVA_HOME=/opt/java/jdk1.8.0_151 
    export PATH=$JAVA_HOME/bin:$PATH

    Run the source .bash_profile command for the modification to take effect.

    NOTE:

    Use Oracle JDK instead of ECS's default JDK (for example, OpenJDK), because ECS's default JDK may not be suitable. Obtain Oracle JDK 1.8.111 or later from Oracle's official website.

  4. Download an open-source Kafka client.

    If the version of the Kafka instance is 1.1.0, download the client at https://archive.apache.org/dist/kafka/1.1.0/kafka_2.11-1.1.0.tgz.

    wget https://archive.apache.org/dist/kafka/1.1.0/kafka_2.11-1.1.0.tgz

    If the version of the Kafka instance is 2.3.0, download the client at https://archive.apache.org/dist/kafka/2.3.0/kafka_2.11-2.3.0.tgz.

    wget https://archive.apache.org/dist/kafka/2.3.0/kafka_2.11-2.3.0.tgz

    If the version of the Kafka instance is 2.7, download the client at https://archive.apache.org/dist/kafka/2.7.2/kafka_2.12-2.7.2.tgz.

    wget https://archive.apache.org/dist/kafka/2.7.2/kafka_2.12-2.7.2.tgz

  5. Run the following command to decompress the package:

    tar -zxf ${kafka_tar}

    In the preceding command, kafka_tar indicates the name of the client package. For example:

    tar -zxf kafka_2.12-2.7.2.tgz

Follow-Up Procedure

Step 2: Create a Kafka Instance

We use cookies to improve our site and your experience. By continuing to browse our site you accept our cookie policy. Find out more

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback