Help Center/ MapReduce Service/ User Guide/ MRS Cluster Planning/ MRS Cluster Deployment/ Overview
Updated on 2025-12-10 GMT+08:00
View PDF

Overview

The analysis cluster, streaming cluster, and hybrid cluster provided by MRS use fixed templates to deploy cluster processes. Therefore, you cannot customize service processes on management nodes and control nodes.

If you want to customize the cluster deployment, set Cluster Type to Custom when creating a cluster. In this way, you can customize the deployment mode of process instances on cluster nodes. (Only MRS 3.x and later versions support the creation of custom topology clusters.)

A custom cluster provides the following functions:

  • Separated deployment of the management and control roles: The management role and control role are deployed on different Master nodes.
  • Co-deployment of the management and control roles: The management and control roles are co-deployed on the Master node.
  • Components are deployed separately to avoid resource contention.

Cluster Deployment Types

Table 1 MRS cluster deployment types

Common Node

Description

Node Range

Compact

The management role and control role are deployed on the Master node, and data instances are deployed in the same node group. This deployment mode applies to scenarios where the number of control nodes is less than 100, reducing costs.
  • A minimum of 3 and a maximum of 11 Master nodes are required.
  • The total number of node groups is less than or equal to 10, and the total number of nodes in non-Master node groups is less than or equal to 10,000.

Co-deployment of the management role, control role, and data instances

This template is not recommended in the production environment or commercial environment.

If management, control, and data nodes are deployed together, cluster performance and reliability may deteriorate.

If there are enough nodes, deploy data nodes separately.
  • A minimum of 3 and a maximum of 100 Master nodes are required.
  • The total node groups are no more than 10.

OMS-separate

The management role and control role are deployed on different Master nodes, and data instances are deployed in the same node group. This deployment mode is applicable to a cluster with 100 to 500 nodes and delivers better performance in high-concurrency load scenarios.
  • A minimum of 5 and a maximum of 11 Master nodes are required.
  • The total number of node groups is less than or equal to 10, and the total number of nodes in non-Master node groups is less than or equal to 10,000.

Full-size

The management role and control role are deployed on different Master nodes, and data instances are deployed in different node groups. This deployment mode is applicable to a cluster with more than 500 nodes. Components can be deployed separately, which can be used for a larger cluster scale.
  • A minimum of 9 and a maximum of 11 Master nodes are required.
  • The total number of node groups is less than or equal to 10, and the total number of nodes in non-Master node groups is less than or equal to 10,000.

Cluster Role Deployment Rules

The MRS system consists of multiple services in logical architecture. Each service contains one or more roles. Each role can deploy one or more instances.

  • Service: A service is a kind of service capabilities provided by a component in a cluster. Each component in the cluster provides a service and is named by the service.
  • Role: Roles are components of a service. Each service consists of one or more roles. Services are deployed on nodes (servers) through roles to ensure normal service running.
  • Instance: An instance is formed when a service role is installed on a node. A service has one or more role instances.

For details about the deployment principles of each service, see Overview.

  • Service A depending on service B: If service A is deployed in a cluster, service B must be deployed in advance. Service B provides basic capabilities for service A.
  • Service A associated with service B: Service A exchanges data with service B during service running but does not depend on service B during deployment.
  • Role A and role B deployed on the same server: If role A is deployed in a cluster, role B must also be deployed, and role A and role B must be deployed on the same node.
Table 2 MRS role deployment rules

Service

Dependency

Role

Role Deployment Suggestions

OMSServer

-

OMSServer

Deploy this role only on the Master node. Deployment on other nodes is not allowed.

ClickHouse

Depends on ZooKeeper.

CHS (ClickHouseServer)

Deploy this role on any node in the cluster.

Valid role instance range: 2 to 256 (even numbers only).

CLB (ClickHouseBalancer)

Deploy this role on any node in the cluster.

Valid role instance range: 2 to 256.

Flink
  • Depends on ZooKeeper.
  • Depends on KrbServer.
  • Depends on DBService.
  • Depends on Hadoop.

FR(FlinkResource)

Deploy this role on any node in the cluster.

Valid role instance range: 1 to 10,000.

FS(FlinkServer)

Deploy this role on any node in the cluster.

Valid role instance range: 0 to 2.

Flume

-

MS(MonitorServer)

Deploy this role only on the Master node.

Valid role instance range: 1 to 2

F(Flume)

Deploy this role on any node in the cluster.

Valid role instance range: 1 to 10,000.

Hadoop

Depends on ZooKeeper.

NN(NameNode)

Deploy this role only on the Master node.

Valid role instance range: 2.

HFS (HttpFS)

Deploy this role only on the Master node.

Valid role instance range: 0 to 10

JN(JournalNode)

Deploy this role only on the Master node.

Valid role instance range: 3 to 60, in increments of 2.

DN(DataNode)

Deploy this role on any node in the cluster.

Valid role instance range: 3 to 10,000.

RM(ResourceManager)

Deploy this role only on the Master node.

Valid role instance range: 2.

NM(NodeManager)

Deploy this role on any node in the cluster.

Valid role instance range: 3 to 10,000.

JHS(JobHistoryServer)

Deploy this role only on the Master node.

Valid role instance range: 1 to 2

TLS(TimelineServer)

Deploy this role only on the Master node.

Valid role instance range: 0 to 1

HBase

Depends on Hadoop.

HM(HMaster)

Deploy this role only on the Master node.

Valid role instance range: 2.

TS(ThriftServer)

Deploy this role on any node in the cluster.

Valid role instance range: 0 to 10,000.

RT(RESTServer)

Deploy this role on any node in the cluster.

Valid role instance range: 0 to 10,000.

RS(RegionServer)

Deploy this role on any node in the cluster.

Valid role instance range: 3 to 10,000.

TS1(Thrift1Server)

Deploy this role on any node in the cluster.

Valid role instance range: 0 to 10,000.

If the Hue service is installed in a cluster and HBase needs to be used on the Hue web UI, install this instance for HBase.

HetuEngine
  • Depends on Hadoop.
  • Depends on DBService.
  • Depends on Hive.
  • Depends on ZooKeeper.
  • Depends on KrbServer.
  • Depends on Yarn.
  • Depends on HDFS.

HSB(HSBroker)

Deploy this role on any node in the cluster.

Valid role instance range: 2 to 50.

HSC(HSConsole)

Deploy this role on any node in the cluster.

Valid role instance range: 2.

HSF(HSFabric)

Deploy this role on any node in the cluster.

Valid role instance range: 0 to 50.

QAS (available for MRS 3.2.0-LTS.1 and later versions only)

Deploy this role on any node in the cluster.

Valid role instance range: 0 to 2.

Hive
  • Depends on Hadoop.
  • Depends on DBService.

MS(MetaStore)

Deploy this role only on the Master node.

Valid role instance range: 2 to 10.

WH (WebHCat)

Deploy this role only on the Master node.

Valid role instance range: 1 to 10.

HS(HiveServer)

Deploy this role only on the Master node.

Valid role instance range: 2 to 80.

Hue

Depends on DBService.

H(Hue)

Deploy this role only on the Master node.

Valid role instance range: 2.

Impala
  • Depends on Hadoop.
  • Depends on Hive.
  • Depends on DBService.
  • Depends on ZooKeeper.

StateStore

Deploy this role only on the Master node.

Valid role instance range: 1.

Catalog

Deploy this role only on the Master node.

Valid role instance range: 1.

Impalad

Deploy this role on any node in the cluster.

Valid role instance range: 1 to 10,000.

IoTDB

Depends on KrbServer.

ConfigNode (CN)

Deploy this role only on the Master node.

Valid role instance range: 3 to 9, in increments of 2.

IoTDBServer (IoTDBS)

Deploy this role on any node in the cluster.

Valid role instance range: 3 to 256.

Kafka

Depends on ZooKeeper.

B(Broker)

Deploy this role on any node in the cluster.

Valid role instance range: 3 to 10,000.

Kudu

-

KuduMaster

Deploy this role only on the Master node.

Valid role instance range: 3 or 5.

KuduTserver

Deploy this role on any node in the cluster.

Valid role instance range: 3 to 10,000.

Loader
  • Depends on Hadoop.
  • Depends on DBService.

LS(LoaderServer)

Deploy this role only on the Master node.

Valid role instance range: 2.

Oozie
  • Depends on Hadoop.
  • Depends on DBService.
  • Depends on ZooKeeper.

O(oozie)

Deploy this role only on the Master node.

Valid role instance range: 2.

Presto

Depends on Hive.

PCD(Coordinator)

Deploy this role only on the Master node.

Valid role instance range: 2.

PWK(Worker)

Deploy this role on any node in the cluster.

Valid role instance range: 1 to 10,000.

Ranger

Depends on DBService.

RA(RangerAdmin)

Deploy this role only on the Master node.

Valid role instance range: 1 to 2

USC(UserSync)

Deploy this role only on the Master node.

Valid role instance range: 1.

TSC (TagSync)

Deploy this role on any node in the cluster.

Valid role instance range: 0 to 1

Spark
  • Depends on Hadoop.
  • Depends on Hive.
  • Depends on ZooKeeper.

JS(JDBCServer)

Deploy this role only on the Master node.

Valid role instance range: 1 to 2

JH(JobHistory)

Deploy this role only on the Master node.

Valid role instance range: 1 to 2

SR(SparkResource)

Deploy this role only on the Master node.

Valid role instance range: 1 to 2

Spark2x
  • Depends on Hadoop.
  • Depends on Hive.
  • Depends on ZooKeeper.

JS2X(JDBCServer2x)

Deploy this role only on the Master node.

Valid role instance range: 2 to 10.

JH2X(JobHistory2x)

Deploy this role only on the Master node.

Valid role instance range: 2.

SR2X(SparkResource2x)

Deploy this role only on the Master node.

Valid role instance range: 2 to 50.

IS2X(IndexServer2x)

(Optional) Deploy this role only on the Master node.

Valid role instance range: 0 to 2, in increments of 2.

Sqoop

Depends on Hadoop.

SC(SqoopClient)

Deploy this role on any node in the cluster.

Valid role instance range: 1 to 10,000.

Tez
  • Depends on Hadoop.
  • Depends on DBService.
  • Depends on ZooKeeper.

TUI(TezUI)

Deploy this role only on the Master node.

Valid role instance range: 1 to 2

ZooKeeper

-

QP(quorumpeer)

Deploy this role only on the Master node.

Valid role instance range: 3 to 9, in increments of 2.

Doris

-

FE

Deploy this role only on the Master node.

Valid role instance range: 1 to 3, in increments of 2.

NOTICE:

A single FE node applies only to the test or experiment environment. You are advised to deploy at least three FE nodes for the production environment.

BE

Deploy this role on any node in the cluster.

Valid role instance range: 3 to 200.

DBroker

(Optional) Deploy this role on any node in the cluster.

Valid role instance range: 0 to 200.

JobGateway
  • Depends on Hadoop.
  • Depends on DBService.

JS(JobServer)

Deploy this role only in the Master node group.

Valid role instance range: 2 to 10.

JB(JobBalancer)

Deploy this role in any node group.

Valid role instance range: 2.

MemArtsCC

Depends on ZooKeeper.

CCWorker

Deploy this role on Core nodes.

Valid role instance range: 2 to 9999.

CCSidecar

Deploy this role on the Core nodes. Ensure that this role and the CCWorker are deployed on the same node.

Valid role instance range: 2 to 9999.

MemArtsStore

Depends on ZooKeeper.

StoreWorker

Deploy this role on Core nodes.

Valid role instance range: 3 to 5000.

StoreSidecar

Do not deploy this role independently. Deploy it on the same nodes as the StoreWorker and StoreMaster roles.

StoreClientProxy

(Optional) Deploy this role on any node in the cluster. Deploy this role on the same nodes as YARN NodeManager.

Valid role instance range: 0 to 10,000.

StoreMaster

Deploy this role in the Master node group. Do not deploy it on the same node as StoreWorker.

Valid role instance range: 2 to 3.
Parent Topic: MRS Cluster Deployment