Creating a Cluster (New Version)
Logstash clusters created in CSS can be used for efficient ingestion, parsing, processing, and transmission of vast quantities of data. They support real-time log analytics, data integration and transformation, and cross-system data transfers while providing capabilities for data monitoring, alarming, as well as in-depth analysis and visualization.
You can create a Logstash cluster in either of the following ways:
- Method 1: Create a Logstash cluster on the CSS management console. This topic provides the operation guide for the default UI (that is, the new UI). For the operation guide for the old UI, see Creating a Cluster.
To improve efficiency in creating clusters, CSS has made several enhancements on the cluster creation page in terms of ease-of-use. Being more intuitive, the new UI streamlines cluster creation procedures.
- Method 2: Create a Logstash cluster using CSS APIs. For details, see Creating a Cluster.
Prerequisites
- You have planned the Logstash cluster configuration by following the guidelines in Cluster Planning, including the cluster size, node configuration, and storage capacity requirements.
- You have created a VPC, subnet, and security group beforehand, or you can create them when creating the cluster. For more information, see Virtual Private Cloud User Guide.
Opening the Create Cluster Page
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > Logstash.
- In the upper right corner, click Create Cluster. The new-version UI for creating a cluster is displayed by default. Figure 1 Create Cluster (new version)
Cluster Configuration
Select the cluster type and version.
| Parameter | Description |
|---|---|
| Cluster Type | Select Logstash. |
| Cluster Version | Select a cluster version from the drop-down list. |
Basic Settings
Select the region and AZ of the cluster.
| Parameter | Description |
|---|---|
| Region | Select the region where the cluster is located. A region is the location of a physical data center. Regions are defined based on their geographical location and network latency. For lower network latency and quicker resource access, select the nearest region. |
| AZ | Select one or more AZs associated with the cluster region. An AZ is a physical region where resources use independent power supplies and networks. AZs are physically isolated but interconnected through an internal network. A maximum of two AZs can be configured. For details about the use of multiple AZs, see Suggestions on Multi-AZ Deployment. |
Configuring Logstash Nodes
Configure the nodes of the Logstash cluster.
| Parameter | Description |
|---|---|
| CPU Architecture | Select the CPU architecture of the Logstash nodes. x86 nodes are supported. The architectures actually supported may vary depending on the regional environment. |
| Node Specifications | Select the specifications of the Logstash nodes. Click Available. On the displayed page, select a flavor that suits your needs. The nodes of a Logstash cluster support EVS disks only. For more information about different node specifications, see ECS Types. In the node flavor list, vCPUs | Memory indicates the number of vCPUs and memory capacity available for each flavor, and Recommended Storage indicates the supported storage capacity range. The node flavors available may vary depending on the region you select. |
| Node Storage Type and Capacity | Select the storage type of the Logstash nodes.
|
| Nodes | Set the number of nodes for the Logstash cluster. The number of nodes determines the data migration speed. Select the number of Logstash nodes based on service needs. Value range: 1 to 100. |
Network Settings
Configure the VPC, subnet, IP addresses, and security group of the cluster.
| Parameter | Description |
|---|---|
| VPC | Select a VPC for the cluster for proper network isolation. If none of the existing VPCs meets your requirements, click Create VPC to create one. (Make sure you have the permission to create VPCs.) For details, see Creating a VPC and Subnet. |
| Subnet | Select a subnet for the cluster. A subnet improves network security by providing exclusive network resources that are isolated from other networks. Select a subnet in the current VPC. If no existing subnets meet your requirements, click Create Subnet to create a new subnet. For details, see Creating a VPC and Subnet. |
| IPv4 Address | Assign IPv4 addresses to cluster nodes.
|
| Security Group | Select a security group for the cluster. A security group serves as a virtual firewall that provides access control policies for clusters. To select a security group that meets your requirements, click View Security Group to go to the security group list, where you can check the details of each security group. The selected security group must allow port 9600 in the inbound direction. Otherwise, the cluster may be inaccessible to external services. If none of the existing security groups meets your requirements, click Create Security Group to create one. (Make sure you have the permission to create security groups.) For details, see Creating a Security Group. |
Cluster Management
| Parameter | Description |
|---|---|
| Cluster Name | User-defined cluster name. The cluster name must start with a letter and can contain 4 to 32 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
| Add Description | Add a description for the cluster for easy recognition. Click + Add Description and then enter a description of the cluster. The value can contain 0 to 128 characters. |
| Enterprise Project | Associate the cluster with an enterprise project. An enterprise project groups cloud resources, so you can manage resources and members by project. The default project is default. If enterprise projects are enabled, you can select an enterprise project from the drop-down list. You can click View Enterprise Projects to go to the Enterprise Project Management Service console and check enterprise project details. For more information, see Accessing the Enterprise Center. |
| Tags | Adding tags to clusters helps you identify and manage your cluster resources. Click + Add. You can select existing tag keys and values from the drop-down lists or enter new ones. If you want to use the same tag to identify multiple cloud resources for better resource grouping, we recommend that you predefine tags in Tag Management Service (TMS). For details, see Predefined Tags. Each cluster can have a maximum of 20 tags. If your organization has configured tag policies for CSS, add tags to clusters based on these policies. Tags that do not comply with predefined tag policies will cause a cluster creation failure. Contact the administrator to learn about the tag policies. |
Submit
- In the Configuration Summary panel, review the cluster configuration details.
Any mandatory fields that are missing are displayed in red, and you have to set them in the configuration area.
- Click Create Now.
Cluster creation time depends on the number of nodes. Clusters with a large number of nodes typically take longer to create.
- Return to the cluster list and check the newly created cluster.
If the cluster is created successfully, Cluster Status changes to Available.