Why All New Index Shards Are Allocated to the Same Node?
Possible Cause
The possible causes are as follows:
- Shards were unevenly distributed in previous index allocations, and the predominate parameter in the latest indexed shard allocation was balance.shard. To balance the shard distribution across nodes, the new shards were allocated to the node with only a small number of shards.
- After a new node was added to a cluster and before the automatic cluster rebalancing completes, the predominate parameter was balance.shard. The shards of a new index are allocated to the new node, where there are no shards yet.
The following two parameters are used to balance the shard allocation in a cluster:
cluster.routing.allocation.balance.index (default value: 0.45f)
cluster.routing.allocation.balance.shard (default value: 0.55f)
- balance.index: A larger value indicates that all the shards of an index are more evenly distributed across nodes. For example, if an index has six shards and there are three data nodes, two shards will be distributed on each node.
- balance.shard: A larger value indicates that all the shards of all the indexes are more evenly distributed across nodes. For example, if index a has two shards, index b has four, and there are three data nodes, two shards will be distributed on each node.
- You can specify both balance.index and balance.shard to balance the shard allocation.
Solution
To prevent the all the shards of an index from being allocated to a single node, use either of the following methods:
- To create an index during cluster scale-out, configure the following parameter:
"index.routing.allocation.total_shards_per_node": 2
That is, allow no more than two shards of an index to be allocated on each node. Determine the maximum number of shards allocated to each node based on the number of data nodes in your cluster and the number of index shards (both primary and secondary).
- If too many shards are distributed on only a few nodes, you can move some of the shards to other nodes to balance the distribution. Run the move command of POST _cluster/reroute. The rebalance module will automatically exchange the shard with a shard on the destination node. Determine the values of balance.index and balance.shard as needed.
Open Source Search Engine Consulting FAQs
- How Do I Set the Numbers of Index Copies to 0 in Batches?
- Why All New Index Shards Are Allocated to the Same Node?
- How Do I Create a Type Under an Index in an Elasticsearch 7.x Cluster?
- How Do I Configure a Two-Replica CSS Cluster?
- Can I Change the Number of Shards to Four with Two Replicas When There Is One Shard Set in the JSON File?
- What Are the Impacts If an Elasticsearch Cluster Has Too Many Shards?
- How Do I Check the Numbers of Shards and Replicas in a Cluster on the CSS Console?
- How Do I Query Index Data on Kibana in an ES Cluster?
- Can I Stop a CSS Cluster?
- Does the Value i of node.roles Indicate an Ingest Node?
- How Do I Set the Default Maximum Number of Records Displayed on a Page for an Elasticsearch Cluster
- How Do I Update the Lifecycle Policy of an Elasticsearch Cluster?
- How Do I Configure the Threshold for CSS Slow Query Logs?
- How Do I Delete Index Data?
- How Do I Clear the Cache of a CSS Cluster?
- Why Does the Disk Usage Increase After the delete_by_query Command Was Executed to Delete Data?
- How Do I Clear Expired Data to Release Storage Space?
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbotmore