Creating an Elasticsearch Cluster

Scenarios

Table 1 lists key parameters that differentiate between different types of clusters.

Prerequisites

You have planned the Elasticsearch cluster configuration by following the instructions in Elasticsearch Cluster Planning Suggestions.

Creating a Cluster

1. Log in to the CSS management console.
2. On the Dashboard page, click Create Cluster in the upper right corner. The Create Cluster page is displayed.

   Alternatively, choose Clusters > Elasticsearch in the navigation tree on the left. Click Create Cluster in the upper right corner. The Create Cluster page is displayed.

3. On the Basic Configuration page, configure basic information and resources for the Elasticsearch cluster.

   Table 2 Basic configuration of the Elasticsearch cluster

   Parameter	Description
   Region	Select the region where the cluster is located. ECSs in different regions cannot communicate with each other over an intranet. For lower network latency and quicker resource access, select the nearest region.
   AZ	Select one or more AZs associated with the cluster region. A maximum of three AZs can be configured. For details about the use of multiple AZs, see Suggestions on Multi-AZ Deployment.
   Type	Select Elasticsearch.
   Version	Select a cluster version from the drop-down list.
   Name	User-defined cluster name. The cluster name must start with a letter and can contain 4 to 32 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.
   Cluster Description	Add a description for the cluster for easy recognition. The value can contain 0 to 128 characters.
   Nodes	Number of nodes in the cluster. Select a number from 1 to 32. You are advised to configure three or more nodes to ensure high availability of the cluster. If the cluster has no client nodes, the data nodes will need to handle the additional task of query request distribution and data analysis. If the cluster has no master nodes, the data nodes will also need to handle cluster management. NOTE: If the number of data nodes in a cluster is not evenly divisible by the number of AZs, data in the cluster may be unevenly distributed, affecting data query or write performance.
   CPU Architecture	x86. The supported types depend on the actual regional environment.
   Node Specifications	Data node flavor. You can select a node flavor based on your needs. Each cluster supports only one specification. For details, see ECS Types.
   Node Storage Type	If you select EVS for node storage, you need to further select the EVS disk type for data nodes of the cluster. Options include Common I/O, High I/O, Ultra-high I/O. NOTE: If the type of storage in use is not supported, the storage type is not displayed.
   Node Storage Capacity	Data node storage capacity. Its value range varies with node specifications. The node storage capacity must be a multiple of 20.
   Master node	Master nodes manage cluster-wide operations, including metadata, indexes, and shard allocation. For large-scale deployments, using dedicated master nodes enhances cluster stability, service availability, and centralized control. After enabling master nodes, specify Node Specifications, Nodes, and Node Storage Type. The value of Nodes must be an odd number greater than or equal to 3. Up to nine nodes are supported. The value of Node Storage Capacity is fixed. You can select a storage type based on your needs.
   Client node	Client nodes route and coordinate search and index requests, offloading processing from data nodes for enhanced query performance and cluster scalability when there are heavy loads. After enabling client nodes, specify Node Specifications, Nodes and Node Storage Type. The value of Nodes ranges from 1 to 32. The value of Node Storage Capacity is fixed. You can select a storage type based on your needs.
   Cold data node	Cold data nodes are used to store and query large quantities of latency-insensitive data. They offer an effective way to manage large datasets while cutting storage costs. After enabling cold data nodes, configure Node Specifications, Nodes, Node Storage Type, and Node Storage Capacity. The value of Nodes ranges from 1 to 32. Select Node Storage Type and Node Storage Capacity as required. When cold data nodes are enabled, you can switch between cold and hot data nodes. For details, see Switching Between Hot and Cold Storage for an Elasticsearch Cluster. NOTE: If the number of cold data nodes in a cluster is not evenly divisible by the number of AZs, data in the cluster may be unevenly distributed, affecting data query or write performance.
   Enterprise Project	When creating a CSS cluster, you can bind an enterprise project to the cluster if you have enabled the enterprise project function. Select an enterprise project from the Enterprise Project drop-down list, or click View Enterprise Project to go to the Enterprise Project Management Service page and check existing enterprise projects.

4. Click Next: Network.
5. On the Network page, configure the network settings and security mode for the Elasticsearch cluster.

   Table 3 Network settings for the Elasticsearch cluster

   Parameter	Description
   VPC	Specify a VPC to isolate the cluster's network. Click View VPC to go to the VPC management console and check the created VPCs. If no VPC is available, contact the CSS administrator to create a new VPC. For details, see Creating a VPC with a Subnet. NOTE: The VPC must contain CIDRs. Otherwise, cluster creation will fail. By default, a created VPC contains CIDRs.
   Subnet	A subnet provides dedicated network resources that are isolated from other networks, improving network security. Select a subnet needed by the cluster in the current VPC. If no existing subnets meet your requirements, click Create Subnet to create a new subnet. For details, see Creating a VPC with a Subnet.
   Security Group	A security group serves as a virtual firewall that provides access control policies for clusters. Select a security group for the cluster. Click View Security Group to go to the security group list, where you can view details about security groups. NOTE: If your cluster version is 7.6.2 or later, ensure that all the ports used for communication between nodes in the same security group are allowed. If such settings cannot be configured, ensure at least the access to port 9300 is allowed. After the port 9300 is enabled, if the cluster disk usage is high, delete expired data to release the disk storage space. For details, see How Do I Clear Expired Data to Release Storage Space?
   Security Mode	Whether to enable the security mode for the cluster. The security mode is enabled by default. In security mode, a cluster's communication is encrypted and access to the cluster requires user authentication. This is why Administrator Username and Administrator Password must be configured for the cluster. The default administrator username is admin. Set and confirm the Administrator Password. This password will be required when you access this cluster. If Security Mode is disabled, a cluster in non-security mode will be created. With such a cluster, access to the cluster will require no user authentication, and data will be transmitted in plaintext using HTTP. Make sure the cluster is deployed in a secure environment. Do not expose the cluster's network interface to the public network.
   HTTPS Access	HTTPS access can be enabled only when security mode is enabled for the cluster. With HTTPS access enabled, communication will be encrypted when you access the cluster. NOTE: Compared with a non-security mode cluster that uses HTTP, a security-mode cluster that uses HTTPS has lower read performance. The performance loss is estimated at around 20% under high concurrency. If you want fast read performance as well as the isolation and permission control (such as indexes, documents, and fields) enabled by the security mode, you can disable HTTPS Access. After HTTPS Access is disabled, HTTP protocol is used for cluster communication. In this case, data security cannot be ensured and public IP address cannot be used.
   Public IP Address	This parameter is available only when Security Mode and HTTPS Access are enabled. When Public IP Address is enabled, a public IP address is automatically assigned, which will enable access to the security cluster from the Internet. For details, see Configuring Public Network Access for an Elasticsearch Cluster.

6. Click Next: Advanced Settings.
7. On the Advanced Settings page, configure a snapshot policy and other advanced settings for the Elasticsearch cluster.
   1. Set a cluster snapshot policy.

      Cluster snapshots are disabled by default. To enable them, toggle on Cluster Snapshots. To store automatically created snapshots in OBS, an agency will need to be created in order to access OBS. Fees will be incurred for using OBS storage (storage class: Standard).

      Table 4 Basic configuration for a cluster snapshot policy

      Parameter	Description
      OBS Bucket	From the drop-down list, select an OBS bucket for storing snapshots. You can also click Create Bucket on the right to create an OBS bucket. For details, see Creating a Bucket. The created or existing OBS bucket must meet the following requirements: Storage Class: Standard. Region must be the same as that of the created cluster.
      Backup Path	Storage path of the snapshot in the OBS bucket. The backup path cannot: Contain the following characters: \:*?"<>|'{} Start with a slash (/). Start or end with a period (.). Contain more than two consecutive slashes (/) or periods (.). Exceed 512 characters. CAUTION: Only standard OBS storage can be used to store snapshot files. Do not add OBS lifecycle rules to the backup path.
      Maximum Backup Rate (per Second)	The parameter sets the maximum backup rate per node. When it is exceeded, flow control is triggered to prevent excessive resource usage and ensure system stability. The actual backup rate may not reach the configured value, as it depends on factors such as OBS performance and disk I/O. Value format: number + unit Number range: 0–9999 Unit: KB, MB, GB, TB, PB, or B Default value: 40 MB The value 0MB means there is no limit on how fast data is backed up to snapshots. An overly high backup rate may lead to excessive resource usage, which may impact cluster stability. Therefore, set this parameter carefully.
      Maximum Recovery Rate (per Second)	The parameter sets the maximum recovery rate per node. When it is exceeded, flow control is triggered to prevent excessive resource usage and ensure system stability. The actual recovery rate may not reach the configured value, as it depends on factors such as OBS performance and disk I/O. Value format: number + unit Number range: 0–9999 Unit: KB, MB, GB, TB, PB, or B Default value: For Elasticsearch 7.6.2 clusters or earlier, the default value is 40 MB. For Elasticsearch clusters later than 7.6.2, the default value is 0 MB, indicating no limit. An overly high recovery rate may lead to excessive resource usage, which may impact cluster stability. Configure this parameter carefully to maintain optimal performance. For Elasticsearch clusters later than 7.6.2, the recovery rate is also limited by the indices.recovery.max_bytes_per_sec parameter. If Maximum Recovery Rate (per Second) is less than indices.recovery.max_bytes_per_sec, the former takes effect. If Maximum Recovery Rate (per Second) is greater than indices.recovery.max_bytes_per_sec, the latter takes effect. NOTE: To check the value of indices.recovery.max_bytes_per_sec, run the following command: GET _cluster/settings To modify indices.recovery.max_bytes_per_sec, run the following command: PUT _cluster/settings { "transient": { "indices.recovery.max_bytes_per_sec": "100mb" } }
      IAM Agency	IAM agency authorized by the current account for CSS to access or maintain data stored in OBS. If no agency is available, contact the CSS administrator to create one. For details, see Creating an Agency. The selected IAM agency must meet the following requirements: Agency Type must be Cloud service. Set Cloud Service to Elasticsearch or CSS. Mandatory policies: Tenant Administrator or OBS Administrator. WARNING: The agency name can contain only letters (case-sensitive), digits, underscores (_), and hyphens (-). Otherwise, the backup will fail.

      Table 5 Setting Automatic Snapshot Creation

      Parameter	Description
      Snapshot Name Prefix	The snapshot name prefix contains 1 to 32 characters and must start with a lowercase letter. Only lowercase letters, digits, hyphens (-), and underscores (_) are allowed. A snapshot name consists of a snapshot name prefix and a timestamp, for example, snapshot-1566921603720.
      Time Zone	Time zone for the backup time. Specify Backup Started Time based on the time zone.
      Backup Start Time	Specify the start time of auto backup. Select a time from the drop-down list. The interval can be Daily, Hourly, or weekly (by selecting a specific day of the week), and the backup time can be set to any hour from 00:00 to 23:00 (full hours only).

   2. Configure advanced settings for the cluster. Select Default or Custom.
      • Default: VPC Endpoint, Kibana Public Access are disabled by default. You can manually enable these settings after the cluster is created.
      • Custom: You can enable VPC Endpoint, Kibana Public Access as required.

      VPC Endpoint

      VPC Endpoint enables you to access resources across Virtual Private Clouds (VPCs) using a dedicated gateway, without exposing the network information of servers. When VPC Endpoint is enabled, a VPC endpoint will be created by default. You can select Create Private Domain Name if necessary. Users will be able to access this cluster across VPCs through node IP addresses or a private domain name.

      Table 6 Configuring VPC Endpoint

      Parameter	Description
      Create Private Domain Name	If Create Private Domain Name is selected, the system generates a node IP address and also automatically creates a private domain name, which enables users to access this cluster from within the same VPC. If it is not selected, only a node IP address is generated.
      VPC Endpoint Whitelist	In VPC Endpoint Whitelist, you can add accounts that are allowed to access the cluster using a node IP address or private domain name. Click Add to add accounts in Account ID. If no account is added or the account ID is set to *, all users are allowed to access the cluster through the VPC endpoint service. Click Delete in the Operation column to delete accounts. NOTE: To obtain your authorized account ID, point to your username in the upper right corner, and choose My Credentials. Copy the value of Account ID.

      Kibana Public Network Access

      This parameter is available only when security mode is enabled for the cluster. By enabling this option, you can obtain a public IP address for accessing Kibana.

      Table 7 Configuring public network access for Kibana

      Parameter	Description
      Bandwidth	Bandwidth for accessing Kibana through a public IP address. Value range: 1 to 100. Unit: Mbit/s
      Access Control	If you disable this function, all IP addresses can access Kibana through the public IP address. If you enable this function, only IP addresses or IP address ranges that are on the whitelist can access Kibana through from the public network.
      Whitelist	IP addresses or IP address ranges that are allowed to access the cluster from the public network. Use commas (,) to separate different values. This parameter can be configured only when Access Control is enabled. You are advised to enable the whitelist. NOTE: The whitelist that controls Kibana public network access depends on whitelist support by the ELB service. After you update the whitelist, the new settings take effect immediately for new connections. For existing persistent connections using the IP addresses that have been removed from the whitelist, the new settings take effect in approximately 1 minute after these connections are disconnected.

8. Click Next: Confirm Configuration. Check the configuration and click Next to create a cluster.
9. Return to the cluster list and check the newly created cluster. If the cluster is created successfully, Cluster Status changes to Available. Cluster creation time depends on the number of nodes. Typically, this process takes less than 60 minutes, though clusters with a large number of nodes may require additional time.

   If cluster creation fails, try creating the cluster again by rectifying the errors returned.