Manually Buying an MRS Cluster

This section describes how to create an MRS cluster on the MRS console.

MRS consists of multiple big data components, and you can select the cluster type that best fits your service requirements, data types, reliability expectations, and resource budget.

You can quickly buy a cluster using the preset cluster template or select the component list and advanced settings to manually buy a cluster.

Video Tutorial

This tutorial demonstrates how to create an MRS 3.1.0 cluster on the management console and configure software, hardware, and advanced settings.

The UI may vary depending on the version. This tutorial is for reference only.

Manually Buying an MRS Cluster

Go to the Buy Cluster page.
Click the Custom Config tab.

When creating a cluster, pay attention to quota notification. If a resource quota is insufficient, increase the resource quota as prompted and create a cluster.
In the basic configuration area, set basic information about the MRS cluster.
- Billing Mode: MRS provides two billing modes: yearly/monthly and pay-per-use.
- Region: Resources in different regions cannot communicate through intranet. To enhance access speed and minimize network latency, it is advised to choose the nearest region when selecting the location of your resources.
Configure the MRS cluster information as prompted.

When creating a cluster, you need to configure the cluster version, network, node specifications and quantity, and other advanced settings.
After the cluster information is configured, click Buy Now.

If Kerberos authentication is enabled for a cluster, check whether Kerberos authentication is required. If yes, click Continue. If no, click Back to disable Kerberos authentication and then create a cluster. This function cannot be changed after you buy a cluster.
- For any doubt about the pricing, click Pricing details in the lower left corner.
- If you select the pay-per-use billing mode, the order may fail to be placed because the account may be in risk. In this case, contact technical support.
Click Back to Cluster List to view the cluster status.

Wait for the cluster creation to complete. The initial status of the cluster is Starting. After the cluster is created, the cluster status becomes Running.

On the MRS console, a maximum of 10 clusters can be created concurrently, and a maximum of 100 clusters can be managed.

For details about cluster status during creation, see the description of the status parameters in Table 1.

MRS Cluster Version Configuration

**Table 1** MRS cluster parameters
Parameter	Description	Example Value
Cluster Type	Select a proper MRS cluster type based on service requirements. Analysis cluster: It is used for offline data analysis and comprises data analysis tools like Hadoop, Spark, HBase, Hive, Flink, Oozie, and Tez. Streaming cluster: It processes streaming data to quickly analyze real-time data sources, and it mainly includes streaming data processing tools such as Kafka and Flume. Hybrid cluster: It is utilized for both offline data analysis and stream processing. Custom: You can select from a variety of components that are supported by the corresponding version of the MRS cluster.	Custom
Version Type	MRS provides two types of clusters: LTS and Normal. Different versions provide different components. You can select a version as required. LTS: employs MRS's own components to provide highly reliable clusters with strong DR capabilities, making long-term support and evolution possible. Normal: integrates MRS's mature and stable features and functions with open-source capabilities, offering high performance and stability.	LTS
Cluster Version	Version of the MRS cluster. Different versions may contain different open source component versions and functions. You are advised to select the latest version.	MRS 3.2.0-LTS.1
Component	Select the components to be deployed in the MRS cluster. You can change components based on your needs. For some clusters, components cannot be added after creation.	-
Metadata	Whether to use external data sources to store Hive and Ranger metadata of the cluster. Local: Metadata is stored in the local cluster. External data connection: Metadata of external data sources is used. If the cluster is abnormal or deleted, metadata is not affected. This mode applies to scenarios where storage and compute are decoupled. When creating an MRS cluster, you can connect to LakeFormation instances to store metadata of components such as Hive and Spark. For details, see Configuring a LakeFormation Data Connection. After a cluster is created, you can manually store the component metadata to external data sources. For details, see Managing MRS Cluster Metadata.	Local
Component Port	Default communication port policy of each component in the MRS cluster. The LTS cluster supports the configuration. Open source: Use the port provided by the open source component. Custom: Customize a port for the component. For details about the differences between default open source port and default custom port, see Common Ports for MRS Cluster Services.	Open source

MRS Cluster Network Configuration

**Table 2** Network configuration parameters
Parameter	Description	Example Value
AZ	Resources are assigned to the AZ of the current region during creation. An AZ is a physical region that operates on independent power supplies and networks for resource usage.	AZ1
VPC	VPC to which the MRS cluster node belongs. If no VPC is available, click View VPC to access the network console and create a VPC.	-
Subnet	Subnet information in the VPC. If no subnet is available, click View Subnet to access the network console and create a subnet. A subnet provides dedicated network resources that are logically isolated from other networks for network security. For details about how to configure network ACL outbound rules, see How Do I Configure a Network ACL Outbound Rule?. NOTE: The number of IP addresses required by creating an MRS cluster depends on the number of cluster nodes and selected components, but not the cluster type. In MRS, IP addresses are automatically assigned to clusters during cluster creation basically based on the following formula: Quantity of IP addresses = Number of cluster nodes + 2 (one for Manager; one for the DB). In addition, if the Hadoop, Hue, Sqoop, and Presto or Loader and Presto components are selected during cluster deployment, one IP address is added for each component. To buy a ClickHouse cluster independently, the number of IP addresses required is calculated as follows: Number of IP addresses = Number of cluster nodes + 1 (for Manager).	-
Security Group	A security group is a set of ECS access rules. It provides access policies for ECSs that have the same security protection requirements and are mutually trusted in a VPC. When you create an MRS cluster, a security group is automatically created by default. You can also select an existing security group from the drop-down list. NOTE: When you select a security group created by yourself, ensure that the inbound rule contains a rule in which Protocol is set to All, Port is set to All, and Source is set to a trusted accessible IP address range. Do not use 0.0.0.0/0 as a source address. Otherwise, security risks may occur. If you do not know the trusted accessible IP address range, select Auto create.	Auto create
EIP	After binding an EIP to an MRS cluster, you can use the EIP to access the Manager web UI of the cluster. When creating a cluster, you can select an available EIP from the drop-down list and bind it. If no EIP is available in the drop-down list, click Create EIPs to buy an EIP. NOTE: The EIP must be in the same region as the cluster.	Bind later

MRS Cluster Node Configuration

**Table 3** Cluster node information
Parameter	Description	Example Value
CPU Architecture	CPU architecture type of an MRS cluster node. The value can be x86 or Kunpeng. This parameter is not available for MRS 3.1.0 and 3.1.5.	x86
Common Node Configurations	This parameter is available only when Cluster Type is set to Custom. Value options include Compact, Full-size, and OMS-separate. For details, see MRS Cluster Deployment Types.	Compact
Node Group	Name of the node group in the cluster. The name of the Master node group is fixed to master_node_default_group. The system automatically creates a Core node group based on the components contained in the cluster. For example, if you select the ClickHouse component, the system adds the ClickHouse node group and deploys the ClickHouseServer role in the node group by default. If Cluster Type is set to Custom, you can customize the names of other node groups. If the data volume does not change much but the service processing capability changes greatly, manually add a Task node group. For details, see Manually Adding a Task Node Group During MRS Cluster Creation.	node_group_1
Node Type	If Cluster Type is set to Custom, you can select the node type of a non-Master node group. If the node group type is set to Task, only the NodeManager role (except the mandatory roles of nodes) can be deployed in the node group.	Core
Payment Type	Billing mode of nodes in a cluster. The billing mode of the Master and Core node groups is the same as that of the cluster. The billing mode of the Task node group is fixed to pay-per-use.	Pay-per-use
Node Count	Configure node quantity in each node group. The number of nodes in a master node group ranges from 3 to 9. There must be at least one Core node and the total number of Core and Task nodes must not exceed 10,000. If Cluster Type is set to Custom, you can click Add Node Group to add node groups. NOTE: A small number of nodes may cause clusters to run slowly while a large number of nodes may be unnecessarily costly. Set an appropriate value based on data to be processed.	-
Instance Specifications	Select the instance specifications of the MRS cluster node. You can click to adjust the specifications. For details about the MRS cluster node specifications, see MRS Cluster Node Specifications. NOTE: More advanced instance specifications provide better data processing. However, they require higher cluster cost. Instance specifications may vary in different AZs. If no instance specifications in the current AZ can meet your requirements, switch to another AZ. If you select HDDs for Core nodes, there is no billing information for data disks. The fees are charged with ECSs. If you select non-HDD disks for Core nodes, the disk types of Master and Core nodes are determined by Data Disk. If Sold out appears next to an instance specification of a node, the node of this specification cannot be bought. You can only buy nodes of other specifications. The Master node specification (4 vCPUs 8 GB) is not within the SLA after-sales scope. It is applicable only to the test environment and is not recommended for the production environment. For MRS 3.x or later, the memory of the master node must be greater than 64 GB.	-
System Disk	You can adjust the storage type and space of the system disk on a node as required. For details about the MRS cluster storage, see Disk Roles.	-
Data Disk	Storage type and space of data disks on a node. To increase the data storage capacity, you can add disks during cluster creation. A maximum of 10 disks can be added to each Core or Task node. For more information about MRS cluster storage, see Disk Roles.	-
LVM	This parameter is valid when a streaming Core node is created only. Click this parameter to enable or disable the disk LVM management function. This parameter is not available in MRS 3.x and later versions. If LVM is enabled, all disks on a node are mounted as logical volumes. This delivers more proper disk planning to avoid data skew, thereby improving system stability.	Disable this function.
Topology Adjustment	If Cluster Type is set to Custom, you can adjust the deployment of each component in the cluster in the node group. Set Topology Adjustment to Enable and adjust the instance deployment mode based on service requirements. For details, see Role Deployment Rules for MRS Clusters.	Disable this function.

Other MRS Cluster Configuration Parameters

**Table 4** Other configuration parameters
Parameter	Description	Example Value
Kerberos Authentication	Whether to enable Kerberos authentication for each component in the MRS cluster. If Kerberos authentication is enabled, users can access component resources only after being authenticated. This function cannot be changed after you buy a cluster.	Switch this function on.
Username	Name of the administrator of Manager. admin is used by default.	admin
Password/Confirm Password	Password of the Manager administrator admin. Keep the password secure. Must contain 8 to 26 characters. Must contain at least four of the following: Lowercase letters Uppercase letters Digits At least one of the following special characters: `~!@#$%^&*()-_=+\|[{}];:',<.>/? Cannot be the same as the username or the username spelled backwards.	-
Login Mode	Method for logging in to a node in the MRS cluster. Password You can log in to the node as user root using a password. You need to customize the password of user root. Key Pair Select a key pair from the drop-down list. Select "I acknowledge that I have obtained private key file SSHkey-xxx and that without this file I will not be able to log in to my ECS." Click View Key Pair to create or import a key pair, and then obtain the private key file.	Password
Kerberos Encryption Type	Encryption algorithm and method used by Kerberos. (This parameter is supported in MRS 3.3.1-LTS and later versions. In earlier versions, the default Kerberos encryption type is aes256-sha1,aes128-sha1.) aes256-sha1,aes128-sha1: indicates that the encryption algorithm and mode are AES256-CTS-HMAC-SHA1-96 AES128-CTS-HMAC-SHA1-9. aes256-sha2,aes128-sha2: indicates that the encryption algorithm and mode are AES256-CTS-HMAC-SHA384-192 AES128-CTS-HMAC-SHA256-128. NOTE: If the encryption types of the two clusters are different, mutual trust cannot be set up across the Managers for the clusters. If you are using an external Java program to connect to the MRS cluster for Kerberos authentication and the Kerberos encryption type is aes256-sha2,aes128-sha2, the external Java program must use JDK 11 (or BiSheng JDK 1.8.0_392) or later. To use an instance created in DataArts Studio to connect to the MRS cluster, set the Kerberos encryption type to aes256-sha1,aes128-sha1.	aes256-sha2,aes128-sha2
Set Advanced Options	Advanced function parameters of the MRS cluster. For details, see Table 5.	-
Enterprise Project	Select the enterprise project to which the cluster belongs. To use an enterprise project, create one on the Enterprise > Project Management page. The Enterprise Management console is designed for resource management. It helps you manage cloud-based personnel, resources, permissions, and finance in a hierarchical manner, such as management of companies, departments, and projects.	default
Required Duration	This parameter is valid in Yearly/Monthly billing mode and indicates a cluster subscription duration. The minimum cluster duration is 1 month and the maximum available cluster duration is 1 year. If Auto-renew is selected, monthly subscriptions are automatically renewed every month and yearly subscriptions are automatically renewed every year.	-
Secure Communications	To allow the MRS console to access big data components in the user VPC, you need to activate the relevant security group rules. For details, see Configuring Secure Communication Authorization for an MRS Cluster.	Select this function.

**Table 5** MRS cluster advanced configuration
Parameter	Description	Example Value
Hostname Prefix	Enter the prefix for the computer hostname of an ECS or BMS in the cluster.	-
Cryptographic Algorithm	Algorithm used for encrypting and decrypting passwords in the cluster system. International: general encryption algorithm Chinese: SM series cryptographic algorithms are compatible with general cryptographic algorithms.	International
Tag	It is recommended that you use the tag function of TMS to add the same tag to different cloud resources. For details, see Adding a Tag to an MRS Cluster/Node.	-
Auto Scaling	Configure an auto scaling policy for the Task node group after the cluster is created. For details, see MRS Task Node Auto Scaling.	-
Bootstrap Action	Bootstrap actions run scripts on specified nodes in a cluster to install third-party software and modify the cluster's running environment. For details, see Adding MRS Node Bootstrap Actions and Installing Third-Party Software.	-
Agency	When an agency is bound to an ECS or BMS, it can manage designated resources in the cluster. It is essential to consider service requirements before deciding to configure an agency. For example, you can configure an ECS agency to automatically obtain the AK/SK to access OBS. For details, see Interconnecting an MRS Cluster with OBS Using an IAM Agency. To bind an agency to an MRS cluster, you need to create an IAM agency with required permissions in advance. By default, the system generates an MRS_ECS_DEFAULT_AGENCY agency. This agency has the OBSOperateAccess permission and the CESFullAccess (for users who have enabled fine-grained policies), CES Administrator, and KMS Administrator permissions in the region where the cluster is located.	Bind later
Metric Sharing	Monitoring metrics of big data components are collected. If a fault occurs when you use a cluster, share the monitoring metrics with Huawei Cloud technical support for troubleshooting.	Disable
System Disk Encryption	Whether to encrypt data in the system disk mounted to the MRS cluster node. This function is disabled by default. Keys used by encrypted system disks are provided by Key Management Service (KMS) in Data Encryption Workshop (DEW). You do not need to build and maintain the key management infrastructure. To enable this function, you must have the Security Administrator and KMS Administrator permissions. For more information about disk encryption, see Managing Encrypted EVS Disks. Configure the following parameters to enable this function: Data Disk Key ID: key ID of the selected key name. Data Disk Key Name: Select the name of the key used to encrypt the data disk. By default, the default master key named evs/default is selected. You can select another master key from the drop-down list. If cloud disks are encrypted using a master key and it is then disabled or scheduled for deletion, the cloud disks can no longer be read from or written to, and data on these disks may never be restored. Exercise caution when performing this operation. Click View the Key List to enter a page where you can create and manage keys.	Disable
Data Disk Encryption	Whether to encrypt data in the data disk mounted to the MRS cluster node. This function is disabled by default. MRS 3.1.0 and MRS 3.1.2-LTS.3 do not support this function. For more information about disk encryption, see Managing Encrypted EVS Disks. Keys used by encrypted system disks are provided by Key Management Service (KMS) in Data Encryption Workshop (DEW). You do not need to build and maintain the key management infrastructure. To enable this function, you must have the Security Administrator and KMS Administrator permissions. Configure the following parameters to enable this function: Data Disk Key ID: key ID of the selected key name. Data Disk Key Name: Select the name of the key used to encrypt the data disk. By default, the default master key named evs/default is selected. You can select another master key from the drop-down list. If cloud disks are encrypted using a master key and it is then disabled or scheduled for deletion, the cloud disks can no longer be read from or written to, and data on these disks may never be restored. Exercise caution when performing this operation. Click View the Key List to enter a page where you can create and manage keys.	Disable
Alarm	If the alarm function is enabled, the cluster maintenance personnel can be notified in a timely manner to locate faults when the cluster runs abnormally or the system is faulty. To send alarm messages, you need to enable the Simple Message Notification (SMN) service, set a notification rule, and bind the rule to an SMN topic. Rule Name: Name of the rule for sending alarm messages. The value can contain only digits, letters, hyphens (-), and underscores (_). Topic Name: Select an existing SMN topic or click Create Topic to create a topic. To deliver messages published to a topic, you need to add a subscriber to the topic. For details, see Adding Subscriptions to a Topic.	Enable
Install UniAgent	Install UniAgent on MRS cluster nodes to simplify plug-in management, provide AOM instructions, and enable script delivery and execution.	No
Logging	Whether to collect logs when cluster creation fails. After the logging function is enabled, system logs and component run logs are automatically collected and saved to the OBS file system in scenarios such as cluster creation failures and scale-out or scale-in failures for O&M personnel to quickly locate faults. The log information is retained for a maximum of seven days.	Disable

Manually Adding a Task Node Group During MRS Cluster Creation

To add a Task node group in the current MRS cluster for auto scaling, do as follows:

Add an analysis Task node group.

For a cluster whose Cluster Type is Analysis cluster or Hybrid cluster:
The system automatically adds the analysis Task node group task_node_analysis_group. Set the number of nodes, instance specifications, and node disk configurations as needed.

If the analysis Task node group is not required, you can delete it.
For a cluster whose Cluster Type is Custom:
1. In the cluster node configuration area, click Add Node Group to manually add a node group.
2. Set Topology Adjustment to Enable.
3. Select the role topology so that the node group contains only the NodeManager (NM) role.
Add a Task node group after the cluster is created. For details, see Adding a Task Node.

Adding streaming Task node group

For a streaming or hybrid cluster that contains the Storm component:

The system automatically adds the streaming Task node group task_node_streaming_group. Set the number of nodes, instance specifications, and node disk configurations as needed.

If the streaming Task node group is not required, you can manually delete it.

Viewing Failed MRS Cluster Creation Tasks

If a cluster fails to be created, the failed task will be managed on the Manage Failed Tasks page.

On the cluster list page, click to switch to the Manage Failed Tasks page. In the Task Status column, hover the cursor over the task status to view the failure cause.

Table 6 lists the error codes of MRS cluster creation failures.

**Table 6** Error codes
Error Code	Description
MRS.101	Insufficient quota to meet your request. Contact customer service to increase the quota.
MRS.102	The token cannot be null or invalid. Try again later or contact customer service.
MRS.103	Invalid request. Try again later or contact customer service.
MRS.104	Insufficient resources. Try again later or contact customer service.
MRS.105	Insufficient IP addresses in the existing subnet. Try again later or contact customer service.
MRS.201	Failed due to an ECS error. Try again later or contact customer service.
MRS.202	Failed due to an IAM error. Try again later or contact customer service.
MRS.203	Failed due to a VPC error. Try again later or contact customer service.
MRS.400	MRS system error. Try again later or contact customer service.