Configuring Multi-Active Instance Mode
Scenarios
Based on existing JDBCServers in the community, multi-active-instance HA is used to achieve the high availability. In this mode, multiple ThriftServers coexist in the cluster and the client can randomly connect any ThriftServer to perform service operations. When one or multiple ThriftServers stop working, a client can connect to another functional ThriftServer.
Compared with active/standby HA, multi-active instance HA eliminates the following restrictions:
- In active/standby mode, when the active/standby switchover occurs, the unavailable period cannot be controlled by JDBCServer, but determined by YARN service resources.
- In Spark, the Thrift JDBC similar to HiveServer2 provides services and users access services through the Beeline or JDBC API. Therefore, the processing capability of a JDBCServer cluster is limited by the single-node performance of its active server, resulting in poor scalability.
Multi-active instance HA not only prevents service interruption caused by switchover, but also enables cluster scale-out to secure high concurrency.
Configuration Description
- Log in to FusionInsight Manager.
For details, see Accessing FusionInsight Manager.
- Choose Cluster > Services > Spark2x or Spark, click Configurations and then All Configurations, and search for the following parameters and adjust their values.
Table 1 Parameter description Parameter
Description
Example Value
spark.thriftserver.zookeeper.connection.timeout
The maximum time, in milliseconds, that the JDBCServer will wait to establish a connection with the ZooKeeper client.
Value range: 1000 to 2147483647
6000
spark.thriftserver.zookeeper.session.timeout
The maximum amount of time, in milliseconds, that a JDBCServer session can remain active without receiving any heartbeats from the ZooKeeper client.
Value range: 0 to 2147483647
90000
spark.thriftserver.zookeeper.retry.times
The maximum number of retries allowed after the JDBCServer fails to connect to the ZooKeeper client.
Value range: 0 to 3600.
3
spark.yarn.queue
The name of the YARN queue to which the JDBCServer application should be submitted.
default
- After the parameter settings are modified, click Save, perform operations as prompted, and wait until the settings are saved successfully.
- After the Spark server configurations are updated, if Configure Status is Expired, restart the component for the configurations to take effect.
Figure 1 Modifying Spark configurationsOn the Spark dashboard page, choose More > Restart Service or Service Rolling Restart, enter the administrator password, and wait until the service restarts.
Components are unavailable during the restart, affecting upper-layer services in the cluster. To minimize the impact, perform this operation during off-peak hours or after confirming that the operation does not have adverse impact.
Helpful Links
For more information about Spark multi-active instances, see Spark2x Multi-Active Instance.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot