Changing the Node Specifications of an Elasticsearch Cluster

If the workloads on the data plane of an Elasticsearch or OpenSearch cluster change, you can scale the cluster by changing its node specifications or disk types.

Scenarios

**Table 1** Specifications change scenarios
Scenario	Description	Constraints
Changing node specifications	Change the node specifications of a cluster by node type. Changing node specifications is a rolling process, meaning nodes are stopped and restarted one at a time, not all at the same time. Applicable scenarios: If the allocation of new indexes or shards takes too long or the coordination and scheduling of nodes are inefficient, increase the master node specifications. If too many requests need to be handled or too many results need to be aggregated, increase the client node specifications. If data nodes are becoming slower in responding to data writing and query requests, increase the specifications of the data nodes. If cold data query becomes slow, increase the cold data node specifications.	Decreasing node specifications will decrease the cluster's data processing and storage capacities. Exercise caution. The node specifications and disk type cannot be changed at the same time. Kibana is unavailable during specifications change.
Changing the disk type	Change the node storage type of a cluster by node type. To do that, you need to delete old disks while rebuilding a node, and then attach disks of the new type.	The disk type can only be changed for data nodes and cold data nodes. Furthermore, the total number of data nodes plus cold data nodes must be at least 3. The disk type cannot be changed when local disks are used. The node specifications and disk type cannot be changed at the same time. Kibana is unavailable during disk type change.

If the data volume is large, it may take long to modify the node specifications. You are advised to modify node specifications during off-peak hours.

Prerequisites

The cluster is available and has no tasks in progress.
CSS has sufficient resources to expand cluster capacity.

Procedure

Log in to the CSS management console.
In the navigation pane, choose a cluster type. The cluster management page is displayed.
Check that all service data has replicas so that services will not be interrupted during the specifications change.
1. In the cluster list, select the target cluster, and click Access Kibana in the Operation column to log in to the Kibana console.
2. Click Dev Tools in the navigation tree on the left.
3. Run the GET _cat/indices?v command in Kibana.
  - If the returned rep value is greater than 0, data replicas exist. Go to the next step.
  - If the returned rep value is 0, there is no data replicas. You are advised to perform Manually Creating a Snapshot before changing node specifications for the cluster.
In the cluster list, select the target cluster, and choose More > Modify Configuration in the Operation column. The Modify Configuration page is displayed.
Click the Scale Cluster tab.

On the Modify Configuration page, click Change Specifications to set parameters.

**Table 2** Specifications change
Parameter	Description
Action	Select Change specifications.
Resources	Shows the change of resources for this operation.
Nodes	Select a node type in the Node Type column. Select new specifications in the Node Specifications column, or select a new disk type in the Node Storage Type column. The node specifications and disk type cannot be changed at the same time.

Figure 1 Changing specifications

Click Next.
Confirm the information and click Submit.

In the displayed dialog box, confirm check items, and click OK to start the specifications change.

Check items for node specifications change: Verify index copies and Cluster status check.
Check items for disk type change: Verify index copies and Check cluster load.

**Table 3** Check item description
Item	Description
Verify index copies	By default, CSS checks for indexes that do not have copies. You can skip this step, but the lack of index copies may impact service availability during a cluster specifications change. If you select Verify index copies and the cluster has no master node, each index is required to have at least one replica and the cluster must have at least three nodes. If you select Verify index copies and the cluster has a master node, each index is required to have at least one replica, but there is no requirement on how many nodes the cluster must have.
Cluster status check	By default, the cluster status is checked before specifications change. The specifications of nodes are changed one by one to ensure service availability and data security. If a cluster is overloaded and services are faulty, the request for a specifications change may fail to be delivered. In this case, you can skip cluster status check, but doing so may cause the cluster to become faulty and services become interrupted. Exercise caution when performing this operation.
Check cluster load	During a disk type change, data migration between nodes and the stopping and restarting of nodes will cause the cluster load to increase. A cluster load check can identify possible overload risks for a cluster and reduce the likelihood of a disk type change failure caused by an overload condition. The cluster load check items are as follows: nodes.thread_pool.search.queue < 1000: Check whether the maximum number of search queues is less than 1000. nodes.thread_pool.write.queue < 200: Check whether the maximum number of write queues is less than 200. nodes.process.cpu.percent < 90: Check whether the maximum CPU usage is less than 90%. nodes.os.cpu.load_average/Number of CPU cores < 80%: Check whether the ratio of the maximum load to the number of CPU cores is less than 80%.

If the change request fails to be submitted and a message is displayed indicating that the cluster needs to be upgraded, it means the current cluster version is too old and does not support a disk type change. Upgrade the cluster to the latest image version and then try again. For a detailed upgrade guide, see Upgrading the Version of an Elasticsearch Cluster.

Click Back to Cluster List to go to the Clusters page. The Cluster Status is Configuration modified. When Cluster Status changes to Available, the cluster specifications have been successfully changed.