Performing an Active/Standby Switchover of MRS Role Instances
The active/standby switchover feature for MRS role instances is designed for cluster administrators and O&M engineers who need to switch role instances running in active/standby mode.
Typical applications include:
- Active node failover: If the active node has a high resource load or a process error, manually trigger an active/standby switchover to migrate services to the standby node for recovery.
- Planned maintenance: Before performing maintenance (such as system upgrades or hardware replacements) on the active node, trigger an active/standby switchover to transfer services to the standby node, ensuring zero downtime.
- Fault drills: Manually perform an active/standby switchover to verify cluster high availability, ensuring that services fail over correctly during a fault.
In an MRS cluster, an active/standby switchover between the active and standby instance nodes is a critical mechanism for ensuring component high availability and data reliability.
For role instances that are deployed in active/standby mode in an MRS cluster (for example, NameNode instances of the HDFS component), if the active instance node is faulty and cannot provide services, the system automatically triggers an active/standby switchover, and the standby instance node automatically becomes the active one.
For role instances that can be deployed in active/standby mode in an MRS cluster, see Overview.
Basic Concepts
- Active/Standby deployment: An HA deployment mode for key components (such as HDFS NameNode, YARN ResourceManager, and HBase HMaster) in an MRS cluster. One active instance and one or more standby instances run concurrently. The active instance processes external requests, while standby instances synchronize data from the active instance in real time and stand by for failover.
- Failover: A process of switching the active and standby instance roles. After the switchover, the standby instance becomes active and serves traffic, while the original active instance becomes standby and enters the standby state.
During failover, component services may temporarily disrupt ongoing read and write operations. We recommend performing this operation during off-peak hours or when you confirm that services can tolerate a brief interruption. Before the switchover, ensure the standby instance is healthy. Otherwise, services may fail to recover after the switchover.
Procedure
- Log in to FusionInsight Manager of the MRS cluster.
For details about how to log in to FusionInsight Manager, see Accessing MRS Manager.
- Choose Cluster > Services.
For MRS 2.x and earlier versions, click Services. The parameters vary depending on the version.
- Click the name of the component (for example, HDFS) where the role instance for which you want to perform an active/standby switchover is located.
On the Instances tab page, you can check the active/standby status of the role instances.
Figure 1 Checking the active/standby status of role instances
- On the service dashboard page, click More and select Perform Role Instance Switchover.
For example, click More and select Perform NameNode Switchover for HDFS.
- In the displayed dialog box, enter the password of the current login user and click OK.
In the dialog box that is displayed, confirm the operation impact and click OK to perform an active/standby switchover for role instances.
When an active/standby switchover is performed for NameNodes on HDFS, you must specify the NameService.
Verifying the active/standby switchover result:
After the role instance switchover succeeds, click Instances. You will see that the original active instance becomes the standby instance, the original standby instance becomes the active instance, and the Running Status of both instances is Normal.
FAQs
Q: What should I do if services are unavailable after an active/standby switchover?
A: Perform the following steps to troubleshoot the issue:
- On the MRS Manager instance page, check whether the running status of the new active instance is Normal.
- Check the new active instance logs to identify why it failed to start (such as incomplete data synchronization or port conflicts).
- If the new active instance fails to work properly, perform an active/standby switchover again to switch back to the original active instance.
Q: Can I perform an active/standby switchover if the standby instance is not in the Normal state?
A: No, doing so is not recommended. If you perform a switchover when the standby instance is abnormal, services may fail to recover. Identify the cause of the standby instance fault, resolve the issue, and then perform the switchover.
Q: Does an active/standby switchover cause data loss?
A: No. Data is not lost under normal conditions. The active and standby instances synchronize data in real time to ensure consistency. After a switchover, the new active instance contains complete data. However, if the active instance fails (for example, if it stops unexpectedly) before data synchronization completes, a small amount of unsynchronized data may be lost.
Q: Which components support active/standby switchover?
A: Active/standby switchover is supported for all components deployed in active/standby mode. For details about role instances that support active/standby deployment, see Overview.
