Help Center/ Cloud Search Service/ User Guide/ OpenSearch/ Managing Clusters/ Restarting a Cluster
Updated on 2026-08-28 GMT+08:00
View PDF
Share

Restarting a Cluster

A cluster restart may be necessary for purposes such as configuration updates or fault recovery. CSS OpenSearch clusters support two restart modes: quick restart and rolling restart. To balance cluster availability and restart efficiency, select the mode best suited to your operational requirements.

Comparing the Two Restart Modes

Table 1 Introducing the two restart modes

Restart Mode

Quick Restart

Rolling Restart

Impact

High:

The cluster becomes unavailable for both read and write operations for the entire duration of the upgrade.

Low:

Only the node that is being restarted is unavailable. The cluster as a whole remains available all the time.

How It Works

Parallel operations: All selected nodes are stopped and restarted at once.

Serial operations: Restart node A > Wait for it to recover > Restart node B > Wait for it to recover, and so on

Data Security

There is risk of data loss if you forcibly restart nodes that are still serving write requests.

Data security is guaranteed as the system automatically handles shard allocation.

Time Required

All nodes are restarted simultaneously. The total restart time is typically shorter.

The selected nodes are restarted one at a time. After the first node, each node can restart only after the previous node has fully recovered. This process takes longer.

When to Use

Use this mode in development or testing environments, where clusters are completely deadlocked, or where service interruption is acceptable.

Use this model in production environments, where high availability takes top priority and clusters store large amounts of data.

Node Quantity

There is no limit on the number of nodes.

There are at least three nodes.

Constraints

Clusters whose Task Status is Frozen or that have ongoing tasks cannot be restarted.

Before You Start

  • Check the target cluster's health status. Check the Task Status column to ensure there are no ongoing tasks.
  • Pause high-concurrency data ingestion tasks to prevent write failures caused by primary shard changes during the restart.
  • Avoid restarting a cluster during peak hours, as doing so may impact data integrity and service availability.
  • Before performing a rolling restart, check that indexes have at least one replica configured.

Restart Considerations for Different Node Types

Before restarting nodes by node type or node name, review the following considerations based on the role of the target nodes.

Table 2 Restart considerations

Node Type

Restart Considerations

Data nodes and cold data nodes
  • In production environments, you are advised to perform rolling restart in the following sequence: Pause shard allocation > Restart one node > Wait for shard recovery to complete > Restart the next node. For details, see Related Operation: Managing Shard Allocation Policies.
  • For clusters with a large number of shards, you are advised to increase the delayed shard allocation timeout to prevent a node's temporary unavailability from triggering unnecessary shard reallocation and placing excessive load on the cluster. The default timeout is 1 minute and can be adjusted as needed: 
    PUT _all/_settings
{
  "settings": {
    "index.unassigned.node_left.delayed_timeout": "5m"
  }
}

Master nodes
  • For production clusters, do not restart all master nodes at once. Master nodes manage cluster metadata. Restarting all master nodes at once triggers metadata reallocation and prolongs cluster recovery.
  • You are advised to restart master nodes one at a time by node name, with the current master node restarted last.

    Run the following command to identify the current master node (the node with * in the master column is the current master node):

    GET /_cat/nodes?v

    Example response:

    ip             heap.percent ram.percent cpu load_1m  node.role master name
192.168.67.132           54          96   5    0.79  dimr      *      css-ess-esn-3-1
192.168.100.80           22          94   4    0.58  dimr      -      css-ess-esn-1-1

Client nodes
  • Client nodes are responsible only for request forwarding and preprocessing. Restarting a client node does not trigger shard reallocation, but requests routed to that node will fail. Before restart a client node, check whether any applications connect directly to the node's IP address.
  • Client nodes can be restarted in batches and recover quickly. You can also switch the application access IP addresses in advance to avoid request failures.

Quick Restart

  1. Log in to the CSS management console.
  2. In the navigation pane on the left, choose Clusters > OpenSearch.
  3. In the cluster list, find the target cluster, and choose More > Restart in the Operation column.
  4. On the Restart Cluster page, set parameters.
    Table 3 Configuring a quick restart

    Parameter

    Description

    Restart Mode

    Select Quick Restart.

    Select By

    Select Node type or Node name.

    • Node type: Select one or more node types for restart. For example, you can use this option to restart all client nodes to release connections.
    • Node name: Restart only one node at a time. Use this option to restart individual nodes, for example, to restore faulty nodes.

    Value

    Select one or more node types or a node name, depending on the setting of Select By. See Restart Considerations for Different Node Types for more information.
  5. Manually type in YES, and click OK to start the restart task.

    The cluster will be unavailable for the entire duration of a quick restart.

  6. When the cluster status changes from Processing to Available, the cluster has restarted successfully.

Rolling Restart

  1. Log in to the CSS management console.
  2. In the navigation pane on the left, choose Clusters > OpenSearch.
  3. In the cluster list, find the target cluster, and choose More > Restart in the Operation column.
  4. On the Restart Cluster page, set parameters.
    Table 4 Configuring a rolling restart

    Parameter

    Description

    Restart Mode

    Select Rolling Restart.

    This option is unavailable when the cluster has fewer than three nodes.

    Select By

    The default value is Node type.

    Value

    Select node types for the restart. See Restart Considerations for Different Node Types for more information.

    • Select all: Select all node types.
    • Select specific node types: For instance, to restart cold data nodes, select Cold Data Nodes only. The system will skip all other nodes.
  5. Manually type in YES, and click OK to start the restart task.
  6. When the cluster status changes from Processing to Available, the cluster has restarted successfully.

Related Operation: Managing Shard Allocation Policies

During a rolling restart, the system automatically handles shard allocation, so normally you do not need to manually change this setting. This operation is primarily intended for scenarios where nodes are manually restarted in batches by node type or node name during a quick restart.

Node restart can trigger shard reallocation, especially when nodes are manually restarted in batches by node type or node name during a quick restart. You are advised to disable automatic shard allocation before restarting the nodes and manually restore the automatic shard allocation policy after the restart is complete. This prevents a large number of shards from being reallocated when nodes go offline, which could otherwise cause the cluster to remain unavailable for an extended period.

  1. Disable automatic shard reallocation. 
    PUT /_cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "none"
  }
}
  2. After all target nodes have been restarted, restore the automatic shard allocation policy. 
    PUT /_cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "all"
  }
}

FAQ

  • Question 1: What should I do if a rolling restart is stuck in Restarting for a long time?

    During a rolling restart, after the last node is restarted, the system waits for the cluster to recover (health status turns green/yellow). If the cluster has extremely large indexes or index shards are still being restored, the task will be stuck in Restarting for a long time. You can run the GET _cat/recovery?v command to check the shard recovery progress.

  • Question 2: How do I perform a rolling-like restart on a cluster with just two nodes?

    The rolling restart option is unavailable for clusters with just two nodes, but you can achieve a similar process manually:

    1. Check the number of index replicas. To ensure service continuity, ensure that each index has at least one replica.
    2. Select Fast Restart, then select Node name for Select By. Select one of the nodes to restart it.
    3. Wait for this node to fully recover and the cluster status to change to Available. Then restart the other node.
Parent topic: Managing Clusters