Will CSS Cluster Services Be Affected If the Usage of a Single Node Gets Too High?

Symptom

According to the cluster monitoring information, the disk usage of an Elasticsearch cluster exceeds 80%. Does it affect cluster performance?

Impact on Services

• If the disk usage of a node exceeds 85%, the cluster will not allocate new shards to the node.
• If the disk usage of a node exceeds 90%, the cluster will migrate some of the shards on it to other data nodes with lower disk usage.
• If the disk usage of a node exceeds 95%, the read_only_allow_delete attribute will be enabled in its indexes. In this case, indexes on the node can only be read or deleted but data cannot be written in.

If the usage of a single node is too high, you can scale out the cluster by adding more nodes to the cluster or expanding the capacity of existing nodes. Indexes are not allocated to new nodes immediately. You can open the Cerebro file to check the index allocation of the nodes. You can also change the values of indices.recovery.max_bytes_per_sec and cluster.routing.allocation.cluster_concurrent_rebalance to speed up index allocation.