Help Center/ Cloud Transformation/ Quickly Deploying a Highly Available Pulsar Cluster/ Solution Overview
Updated on 2024-06-27 GMT+08:00
View PDF
Share

Solution Overview

Application Scenarios

This solution helps you quickly deploy a highly available Pulsar cluster on Huawei Cloud based on Apache Pulsar. Apache Pulsar is the top-level, open-source project of the Apache Software Foundation. It is a next-generation cloud-native distributed message flow platform. It integrates messaging, storage, and lightweight functional computing. It uses decoupled storage and computing to support multi-tenancy, persistent storage, and multi-equipment room and cross-region data replication. Apache Pulsar features strong consistency, high throughput, low latency, and high scalability, and has other streaming data storage characteristics.

Solution Architecture

The following figure shows the solution architecture.

Figure 1 Solution architecture

To use this solution, you need to:

  • Create ten Elastic Cloud Servers (ECSs) and use three of them to deploy three ZooKeeper nodes, three to deploy three bookie nodes, three to deploy three broker nodes, and the remaining one to create a manager node.
  • Create an Elastic IP (EIP) and associate it with the manager node so that the manager node can be accessed from the public network.
  • Create a Virtual Private Cloud (VPC) and a subnet for configuring the network of nodes.
  • Create a security group to control network access to Pulsar nodes.

Advantages

  • High availability and reliability

The solution uses cluster deployment, decoupled storage and computing, data redundancy, and strong data consistency. These features all help improve system reliability and ensure good performance.

  • Open-source customizability

This solution and Pulsar are both open-source and free for commercial use. You can even use the source code for secondary development.

  • Easy deployment

This solution helps you easily deploy a Pulsar cluster using ECSs and install Pulsar Manager, a web-based GUI management and monitoring tool, to quickly use Distributed Message Service (DMS) on the cloud.

Constraints

  • Before deploying this solution, sign up for a HUAWEI ID, enable Huawei Cloud services, and complete real-name authentication. Ensure that your account is not in arrears or frozen. If you select yearly/monthly billing, ensure that your account has sufficient balance. If you do not have sufficient balance, you can go to Billing Center to pay for the order manually.
  • Before deploying this solution, ensure that your account has sufficient IAM permissions. For details, see Preparations.