A Cluster Is Unavailable Due to Improper Security Group Policy

Symptom

The cluster status is Unavailable.

Click the cluster name to go to the cluster basic information page, choose Logs, and click the Log Search tab. The following error message is displayed: "master not discovered or elected yet, an election requires at least 2 nodes with ids [xxx, xxx, xxx, ...], have discovered [xxx...] which is not a quorum".

Figure 1 Node error logs

Possible Causes

In the preceding error log, nodes in the cluster cannot communicate with each other and the cluster cannot select the active node. A possible cause is that the security group selected for the cluster does not enable the port 9300.

In CSS 7.6.2 or later, port 9300 is enabled on the subnet of the user VPC by default. The security group selected for the cluster must enable the port 9300 in the subnet to ensure communication between nodes.

Procedure

On the Clusters page, click the name of the unavailable cluster. The Cluster Information page is displayed.
Choose Parameter Configurations and click the security group name. The basic information page of the security group is displayed.
On the Inbound Rules and Outbound Rules tab pages, check whether there is a security group rule whose Action is Allow, Protocol & Port is TCP:9300, and Type is IPv4.
- If yes, contact technical support to locate the problem.
- If no, go to the next step.
Modify the security group configuration and enable the communication port 9300.
1. On the basic information page of the security group, click the Inbound Rules tab.
2. Click Add Rule. In the Add Inbound Rule dialog box, set Priority to 100, Action to Allow, Protocol & Port to Protocols/TCP (Custom) and 9300, Type to IPv4, and Source to the name of the current security group.
  Figure 2 Adding a security group rule
3. Click OK to enable port 9300.
4. Repeat the preceding steps to enable the port 9300 on the Outbound Rules tab page.
After the port 9300 is enabled for the security group, wait until the cluster becomes available.