Creating a Logstash Cluster (New Version)
Logstash clusters created in CSS can be used for efficient ingestion, parsing, processing, and transmission of vast quantities of data. They support real-time log analytics, data integration and transformation, and cross-system data transfers while providing capabilities for data monitoring, alarming, as well as in-depth analysis and visualization.
You can create a Logstash cluster in either of the following ways:
- Method 1: Create a Logstash cluster on the CSS management console. This topic provides the operation guide for the default UI (that is, the new UI). For the operation guide for the old UI, see Creating a Logstash Cluster.
To improve efficiency in creating clusters, CSS has made several enhancements on the cluster creation page in terms of ease-of-use. Being more intuitive, the new UI streamlines cluster creation procedures.
- Method 2: Create a Logstash cluster using CSS APIs. For details, see Creating a Cluster.
Prerequisites
- You have planned the Logstash cluster configuration by following the guidelines in Logstash Cluster Planning Suggestions, including the cluster size, node configuration, and storage capacity requirements.
- You have created a VPC, subnet, and security group beforehand, or you can create them when creating the cluster. For more information, see Virtual Private Cloud User Guide.
Opening the Create Cluster Page
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > Logstash.
- In the upper right corner, click Create Cluster. The new-version UI for creating a cluster is displayed by default.
Figure 1 Create Cluster (new version)
Cluster Configuration
Select the cluster type and version.
|
Parameter |
Description |
|---|---|
|
Cluster Type |
Select Logstash. |
|
Cluster Version |
Select a cluster version from the drop-down list. |
Basic Settings
Select the region and AZ of the cluster.
|
Parameter |
Description |
|---|---|
|
Region |
Select the region where the cluster is located. A region is the location of a physical data center. Regions are defined based on their geographical location and network latency. For lower network latency and quicker resource access, select the nearest region. |
|
AZ |
Select one or more AZs associated with the cluster region. An AZ is a physical region where resources use independent power supplies and networks. AZs are physically isolated but interconnected through an internal network. A maximum of two AZs can be configured. For details about the use of multiple AZs, see Suggestions on Multi-AZ Deployment. |
Configuring Logstash Nodes
Configure the nodes of the Logstash cluster.
|
Parameter |
Description |
|---|---|
|
CPU Architecture |
Select the CPU architecture of the Logstash nodes. x86 nodes are supported. The architectures actually supported may vary depending on the regional environment. |
|
Node Specifications |
Select the specifications of the Logstash nodes. Click Available. On the displayed page, select a flavor that suits your needs. The nodes of a Logstash cluster support EVS disks only. For more information about different node specifications, see ECS Types. In the node flavor list, vCPUs | Memory indicate the number of vCPUs and memory capacity available for each flavor, and Recommended Storage indicates the supported storage capacity range. The node flavors available may vary depending on the region you select. |
|
Node Storage Type and Capacity |
Select the storage type and capacity of the Logstash nodes.
|
|
Nodes |
Set the number of nodes for the Logstash cluster. The number of nodes determines the data migration speed. Select the number of Logstash nodes based on service needs. Value range: 1 to 100. |
Network Settings
Configure the VPC, subnet, IP addresses, and security group of the cluster.
|
Parameter |
Description |
|---|---|
|
VPC |
Select a VPC for the cluster for proper network isolation. If none of the existing VPCs meets your requirements, click Create VPC to create one. (Make sure you have the permission to create VPCs.) For details, see Creating a VPC and Subnet. |
|
Subnet |
Select a subnet for the cluster. A subnet improves network security by providing exclusive network resources that are isolated from other networks. Select a subnet in the current VPC. If no existing subnets meet your requirements, click Create Subnet to create a new subnet. For details, see Creating a VPC with a Subnet. |
|
IPv4 Address |
Assign IPv4 addresses to cluster nodes.
Default value: Assign automatically |
|
Security Group |
Select a security group for the cluster. A security group serves as a virtual firewall that provides access control policies for clusters. To select a security group that meets your requirements, click View Security Group to go to the security group list, where you can check the details of each security group. The selected security group must allow port 9600 in the inbound direction. Otherwise, the cluster may be inaccessible to external services. If none of the existing security groups meets your requirements, click Create Security Group to create one. (Make sure you have the permission to create security groups.) For details, see Creating a Security Group. |
Cluster Management
|
Parameter |
Description |
|---|---|
|
Cluster Name |
User-defined cluster name. The cluster name must start with a letter and can contain 4 to 32 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
|
Add Description |
Add a description for the cluster for easy recognition. Click + Add Description and then enter a description of the cluster. The value can contain 0 to 128 characters. |
|
Enterprise Project |
Associate the cluster with an enterprise project. An enterprise project groups cloud resources, so you can manage resources and members by project. The default project is default. If enterprise projects are enabled, you can select an enterprise project from the drop-down list. You can click View Enterprise Projects to go to the Enterprise Project Management Service console and check enterprise project details. For more information, see Accessing the Enterprise Center. |
Submit
- In the Configuration Summary panel on the right, review the cluster configuration details.
Any mandatory fields that are missing are displayed in red, and you have to set them in the configuration area.
- Click Create Now.
Cluster creation time depends on the number of nodes. Typically, this process takes less than 60 minutes, though clusters with a large number of nodes may require additional time.
- Return to the cluster list and check the newly created cluster.
If the cluster is created successfully, Cluster Status changes to Available.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.