Overview

The analysis cluster, streaming cluster, and hybrid cluster provided by MRS use fixed templates to deploy cluster processes. Therefore, you cannot customize service processes on management nodes and control nodes.

If you want to customize the cluster deployment, set Cluster Type to Custom when creating a cluster. In this way, you can customize the deployment mode of process instances on cluster nodes. (Only MRS 3.x and later versions support the creation of custom topology clusters.)

A custom cluster provides the following functions:

Separated deployment of the management and control roles: The management role and control role are deployed on different Master nodes.
Co-deployment of the management and control roles: The management and control roles are co-deployed on the Master node.
Components are deployed separately to avoid resource contention.

Cluster Deployment Types

**Table 1** MRS cluster deployment types
Common Node	Description	Node Range
Compact	The management role and control role are deployed on the Master node, and data instances are deployed in the same node group. This deployment mode applies to scenarios where the number of control nodes is less than 100, reducing costs.	A minimum of 3 and a maximum of 11 Master nodes are required. The total number of node groups is less than or equal to 10, and the total number of nodes in non-Master node groups is less than or equal to 10,000.
Co-deployment of the management role, control role, and data instances	This template is not recommended in the production environment or commercial environment. If management, control, and data nodes are deployed together, cluster performance and reliability may deteriorate. If there are enough nodes, deploy data nodes separately.	A minimum of 3 and a maximum of 100 Master nodes are required. The total node groups are no more than 10.
OMS-separate	The management role and control role are deployed on different Master nodes, and data instances are deployed in the same node group. This deployment mode is applicable to a cluster with 100 to 500 nodes and delivers better performance in high-concurrency load scenarios.	A minimum of 5 and a maximum of 11 Master nodes are required. The total number of node groups is less than or equal to 10, and the total number of nodes in non-Master node groups is less than or equal to 10,000.
Full-size	The management role and control role are deployed on different Master nodes, and data instances are deployed in different node groups. This deployment mode is applicable to a cluster with more than 500 nodes. Components can be deployed separately, which can be used for a larger cluster scale.	A minimum of 9 and a maximum of 11 Master nodes are required. The total number of node groups is less than or equal to 10, and the total number of nodes in non-Master node groups is less than or equal to 10,000.

Cluster Role Deployment Rules

The MRS system consists of multiple services in logical architecture. Each service contains one or more roles. Each role can deploy one or more instances.

Service: Represents a specific capability provided by a cluster component. Each component provides its own service, which shares the component's name.
Role: Represents a functional sub-component of a service. Each service consists of one or more roles, which are deployed on nodes (servers) to run the service.
Instance: Formed when a service role is installed on a node. A service contains one or more role instances.

For details about the deployment principles of each service, see Overview.

Service A depending on service B: If service A is deployed in a cluster, service B must be deployed in advance. Service B provides basic capabilities for service A.
Service A associated with service B: Service A exchanges data with service B during service running but does not depend on service B during deployment.
Role A and role B deployed on the same server: If role A is deployed in a cluster, role B must also be deployed, and role A and role B must be deployed on the same node.

**Table 2** MRS role deployment rules
Service	Dependency	Role	Role Deployment Suggestions
OMSServer	-	OMSServer	Deploy this role only on the Master node. Deployment on other nodes is not allowed.
ClickHouse	Depends on ZooKeeper.	CHS (ClickHouseServer)	Deploy this role on any node in the cluster. Valid role instance range: 2 to 256 (even numbers only).
ClickHouse	Depends on ZooKeeper.	CLB (ClickHouseBalancer)	Deploy this role on any node in the cluster. Valid role instance range: 2 to 256.
Flink	Depends on ZooKeeper. Depends on KrbServer. Depends on DBService. Depends on Hadoop.	FR(FlinkResource)	Deploy this role on any node in the cluster. Valid role instance range: 1 to 10,000.
Flink		FS(FlinkServer)	Deploy this role on any node in the cluster. Valid role instance range: 0 or 2.
Flume	-	MS(MonitorServer)	Deploy this role only on the Master node. Valid role instance range: 1 to 2
Flume	-	F(Flume)	Deploy this role on any node in the cluster. Valid role instance range: 1 to 10,000.
Guardian	Depends on Hadoop. Depends on ZooKeeper.	TS(TokenServer)	Deploy this role in any node group. Valid role instance range: 2 to 100. The Guardian component needs to be installed only when OBS is connected.
Hadoop	Depends on ZooKeeper.	NN(NameNode)	Deploy this role only on the Master node. Valid role instance range: 2.
		ZKFC (Zkfc)	Deploy this role only on the Master node. Valid role instance range: 2.
		HFS (HttpFS)	Deploy this role only on the Master node. Valid role instance range: 0 to 10
		JN(JournalNode)	Deploy this role only on the Master node. Valid role instance range: 3 to 60, in increments of 2.
		DN(DataNode)	Deploy this role on any node in the cluster. Valid role instance range: 3 to 10,000.
		RM(ResourceManager)	Deploy this role only on the Master node. Valid role instance range: 2.
		NM(NodeManager)	Deploy this role on any node in the cluster. Valid role instance range: 3 to 10,000.
		JHS(JobHistoryServer)	Deploy this role only on the Master node. Valid role instance range: 1 to 2
		TLS(TimelineServer)	Deploy this role only on the Master node. Valid role instance range: 0 to 2.
HBase	Depends on Hadoop. Depends on ZooKeeper.	HM(HMaster)	Deploy this role only on the Master node. Valid role instance range: 2.
		TS(ThriftServer)	Deploy this role on any node in the cluster. Valid role instance range: 0 to 10,000.
		RT(RESTServer)	Deploy this role on any node in the cluster. Valid role instance range: 0 to 10,000.
		RS(RegionServer)	Deploy this role on any node in the cluster. Valid role instance range: 3 to 10,000.
		TS1(Thrift1Server)	Deploy this role on any node in the cluster. Valid role instance range: 0 to 10,000. If the Hue service is installed in a cluster and HBase needs to be used on the Hue web UI, install this instance for HBase.
		MC(MetricController)	Deploy this role only on the Master node. Valid role instance range: 0 to 2, in increments of 2. CAUTION: This role instance applies only to MRS 3.3.0 or later. This role is optional. It is used for HBase hotspotting auto recovery. This role must be deployed on two hosts in active/standby mode and must share the same nodes as the HMaster role.
HetuEngine	Depends on Hadoop. Depends on DBService. Depends on Hive. Depends on ZooKeeper. Depends on KrbServer. Depends on Yarn. Depends on HDFS.	HSB(HSBroker)	Deploy this role on any node in the cluster. Valid role instance range: Versions earlier than MRS 3.3.0: 2 to 10,000. MRS 3.3.0 or later: 2 to 50.
		HSC(HSConsole)	Deploy this role on any node in the cluster. Valid role instance range: 2.
		HSF(HSFabric)	Deploy this role on any node in the cluster. Valid role instance range: Versions earlier than MRS 3.3.0: 1 to 10,000. MRS 3.3.0 or later: 1 to 50.
		QAS (available for MRS 3.2.0-LTS.1 and later versions only)	Deploy this role on any node in the cluster. Valid role instance range: Versions earlier than MRS 3.3.0: 0 to 2. MRS 3.3.0 or later: 1 to 2.
Hive	Depends on Hadoop. Depends on DBService.	MS(MetaStore)	Deploy this role only on the Master node. Valid role instance range: 2 to 10.
		WH (WebHCat)	Deploy this role only on the Master node. Valid role instance range: 1 to 10.
		HS(HiveServer)	Deploy this role only on the Master node. Valid role instance range: 2 to 80.
Hue	Depends on DBService.	H(Hue)	Deploy this role only on the Master node. Valid role instance range: 2.
Impala	Depends on Hadoop. Depends on Hive. Depends on DBService. Depends on ZooKeeper.	StateStore	Deploy this role only on the Master node. Valid role instance range: 1.
		Catalog	Deploy this role only on the Master node. Valid role instance range: 1.
		Impalad	Deploy this role on any node in the cluster. Valid role instance range: 1 to 10,000.
IoTDB	Depends on KrbServer.	ConfigNode (CN)	Deploy this role only on the Master node. Valid role instance range: 3 to 9, in increments of 2.
IoTDB	Depends on KrbServer.	IoTDBServer (IoTDBS)	Deploy this role on any node in the cluster. Valid role instance range: 3 to 256.
Kafka	Depends on DBService. Depends on ZooKeeper.	B(Broker)	Deploy this role on any node in the cluster. Valid role instance range: 3 to 10,000.
		KafkaUI	Deploy this role on any node in the cluster. Valid role instance range: 0 to 2.
		MM(MirrorMaker)	Deploy this role on any node in the cluster. Valid role instance range: 0 to 1,000. This role instance applies only to MRS 3.3.0 or later.
Kudu	-	KuduMaster	Deploy this role only on the Master node. Valid role instance range: 1 to 3.
Kudu	-	KuduTserver	Deploy this role on any node in the cluster. Valid role instance range: 3 to 10,000.
Loader	Depends on Hadoop. Depends on DBService.	LS(LoaderServer)	Deploy this role only on the Master node. Valid role instance range: 2.
Oozie	Depends on Hadoop. Depends on DBService. Depends on ZooKeeper.	O(oozie)	Deploy this role only on the Master node. Valid role instance range: 2.
Presto	Depends on Hive.	PCD(Coordinator)	Deploy this role only on the Master node. Valid role instance range: 2.
Presto	Depends on Hive.	PWK(Worker)	Deploy this role on any node in the cluster. Valid role instance range: 1 to 10,000.
Ranger	Depends on DBService.	RA(RangerAdmin)	Deploy this role only on the Master node. Valid role instance range: 1 to 2
		USC(UserSync)	Deploy this role only on the Master node. Valid role instance range: 1.
		PSC(PolicySync)	Deploy this role only on the Master node. Valid role instance range: 0 to 2.
		TSC (TagSync)	Deploy this role on any node in the cluster. Valid role instance range: 0 to 1
		RK(RangerKMS)	Deploy this role only on the Master node. Valid role instance range: 0 to 2.
Spark	Depends on Hadoop. Depends on Hive. Depends on ZooKeeper.	JS(JDBCServer)	Deploy this role only on the Master node. Valid role instance range: 2 to 10.
		JH(JobHistory)	Deploy this role only on the Master node. Valid role instance range: 2.
		SR(SparkResource)	Deploy this role only on the Master node. Valid role instance range: 2 to 50.
		IS(IndexServer)	Deploy this role only on the Master node. Valid role instance range: 2 to 10, in increments of 2.
Spark2x	Depends on Hadoop. Depends on Hive. Depends on ZooKeeper.	JS2X(JDBCServer2x)	Deploy this role only on the Master node. Valid role instance range: 2 to 10.
		JH2X(JobHistory2x)	Deploy this role only on the Master node. Valid role instance range: 2.
		SR2X(SparkResource2x)	Deploy this role only on the Master node. Valid role instance range: 2 to 50.
		IS2X(IndexServer2x)	(Optional) Deploy this role only on the Master node. Valid role instance range: 0 to 2, in increments of 2.
Sqoop	Depends on Hadoop.	SC(SqoopClient)	Deploy this role on any node in the cluster. Valid role instance range: 1 to 10,000.
Tez	Depends on Hadoop. Depends on DBService. Depends on ZooKeeper.	TUI(TezUI)	Deploy this role only on the Master node. Valid role instance range: 1 to 2
ZooKeeper	-	QP(quorumpeer)	Deploy this role only on the Master node. Valid role instance range: 3 to 9, in increments of 2.
Doris	-	FE	Deploy this role only on the Master node. Valid role instance range: 1 to 3, in increments of 2. CAUTION: A single FE node applies only to the test or experiment environment. You are advised to deploy at least three FE nodes for the production environment.
		BE	Deploy this role on any node in the cluster. Valid role instance range: 3 to 200.
		DBroker	(Optional) Deploy this role on any node in the cluster. Valid role instance range: 0 to 200.
JobGateway	Depends on Hadoop. Depends on DBService.	JS(JobServer)	Deploy this role only in the Master node group. Valid role instance range: 2 to 10.
JobGateway	Depends on Hadoop. Depends on DBService.	JB(JobBalancer)	Deploy this role in any node group. Valid role instance range: 2.
MemArtsCC	Depends on ZooKeeper.	CCWK(CCWorker)	Deploy this role on Core nodes. Valid role instance range: 2 to 9999.
MemArtsCC	Depends on ZooKeeper.	CCSC(CCSidecar)	Deploy this role on the Core nodes. Ensure that this role and the CCWorker are deployed on the same node. Valid role instance range: 2 to 9999.

Parent Topic: MRS Cluster Deployment

Previous topic: MRS Cluster Deployment

Next topic: MRS Cluster Node Types

Overview

Cluster Deployment Types

Cluster Role Deployment Rules

What is your overall rating for this page?