Buying a Custom MRS Cluster

Cluster Name

The cluster name must be unique. It can contain 1 to 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.

mrs_test

Cluster Type

Select a proper MRS cluster type based on service requirements.

• Analysis cluster: It is used for offline data analysis and comprises data analysis tools like Hadoop, Spark, HBase, Hive, Flink, Oozie, and Tez.
• Streaming cluster: It processes streaming data to quickly analyze real-time data sources, and it mainly includes streaming data processing tools such as Kafka and Flume.
• Hybrid cluster: It is suitable for both offline data analysis and stream processing.
• Custom: You can select from a variety of components that are supported by the corresponding version of the MRS cluster.

Custom

Version Type

MRS provides two cluster version types: LTS and Normal. Different version types provide different components. You can select a version type as required.

• LTS: employs MRS's own components to provide highly reliable clusters with strong DR capabilities, making long-term support and evolution possible.
• Normal: integrates MRS's mature and stable features and functions with open-source capabilities, offering high performance and stability. Currently, MRS clusters of the normal version type cannot be purchased directly. If you want to use clusters of this type, submit a service ticket.

LTS

Cluster Version

Version of the MRS cluster. Different versions may contain different open-source component versions and functions. You are advised to select the latest version. For details about the lifecycle of each cluster version, see MRS Cluster Version Lifecycle.

MRS 3.2.0-LTS.1

Component

Select the components to be deployed in the MRS cluster.

You can change components based on your needs. For some clusters, components cannot be added after creation.

-

Metadata

Whether to use external data sources to store Hive and Ranger metadata of the cluster.

• Local: Metadata is stored in the local cluster.
• External data connection: Metadata of external data sources is used. If the cluster is abnormal or deleted, metadata is not affected. This mode applies to scenarios where storage and compute are decoupled.

  When creating an MRS cluster, you can connect to LakeFormation instances to store metadata of components such as Hive and Spark. For details, see Configuring a LakeFormation Data Connection.

  After a cluster is created, you can manually store the component metadata to external data sources. For details, see Managing MRS Cluster Metadata.

Local

Component Port

Default communication port policy of each component in the MRS cluster. The LTS cluster supports the configuration.

• Open source: Use the port provided by the open source component.
• Custom: Customize a port for the component.

For details about the differences between default open source port and default custom port, see Common Ports for MRS Cluster Services.

Open source

AZ

AZ to which resources to be created belong in the current region. An AZ is a physical area where resources use independent power supply and networks.

AZ1

VPC

VPC to which the MRS cluster node belongs. If no VPC is available, click View VPC to access the network console and create a VPC.

For more information about VPCs, see What Is Virtual Private Cloud?

-

Subnet

Subnet information in the VPC. If no subnet is available, click View Subnet to access the network console and create a subnet.

• A subnet provides dedicated network resources that are logically isolated from other networks for network security. For details about how to configure network ACL outbound rules, see How Do I Configure a Network ACL Outbound Rule?.
• The number of IP addresses required by creating an MRS cluster depends on the number of cluster nodes and selected components, but not the cluster type.
• In MRS, IP addresses are automatically assigned to clusters during cluster creation basically based on the following formula: Quantity of IP addresses = Number of cluster nodes + 2 (one for Manager; one for the DB). In addition, if the Hadoop, Hue, Sqoop, and Presto or Loader and Presto components are selected during cluster deployment, one IP address is added for each component. To buy a ClickHouse cluster separately, the number of IP addresses required is calculated as follows: Number of IP addresses = Number of cluster nodes + 1 (for FusionInsight Manager).
• For more information about subnets, see Subnet.

-

Security Group

A security group is a set of ECS access rules. It provides access policies for ECSs that have the same security protection requirements and are mutually trusted in a VPC.

When you create an MRS cluster, a security group is automatically created by default. You can also select an existing security group from the drop-down list.

• When you select a security group created by yourself, ensure that there is an inbound rule that allows traffic from all protocols, all ports, and the trusted accessible IP address range. Do not use 0.0.0.0/0 as a source address. Otherwise, security risks may occur. If you do not know the trusted accessible IP address range, select Auto create.
• For more information about security groups, see Security Group.

Auto create

EIP

After binding an EIP to an MRS cluster, you can use the EIP to access the Manager web UI of the cluster.

• When creating a cluster, you can select an available EIP from the drop-down list and bind it. If no EIP is available in the drop-down list, click Create EIPs to buy an EIP.
• The EIP must be in the same region as the cluster.
• For more information about EIPs, see What Is Elastic IP?

Bind later

CPU Architecture

CPU architecture of an MRS cluster node. The value can be x86 or Kunpeng. This parameter is not available for MRS 3.1.0 and 3.1.5.

x86

Common Node Configurations

This parameter is available only when Cluster Type is set to Custom. Value options include Compact, Full-size, and OMS-separate. For details, see MRS Cluster Deployment Types.

Compact

Node Group

Name of the node group in the cluster.

• The name of the Master node group is fixed to master_node_default_group.
• The system automatically creates a Core node group based on the components contained in the cluster. For example, if you select the ClickHouse component, the system adds the ClickHouse node group and deploys the ClickHouseServer role in the node group by default.

• If Cluster Type is set to Custom, you can customize the names of other node groups.
• If the data volume does not change much but the service processing capability changes greatly, manually add a Task node group. For details, see How Do I Manually Add a Task Node Group During MRS Cluster Creation.

node_group_1

Node Type

If Cluster Type is set to Custom, you can select the node type of a non-Master node group. If the node group type is set to Task, only the NodeManager role, and certain mandatory node roles, can be deployed in the node group.

Nodes in a cluster can be classified into Master, Core, and Task nodes.

• Master node: MRS cluster management node. OMSServer is deployed on the node to manage and monitor the cluster.
• Core node: MRS cluster work node, which processes and analyzes data and stores process data.
• Task node: Compute node. When the compute resources in a cluster are insufficient, you can configure elastic scaling policies to increase nodes automatically.

For details about node types, see MRS Cluster Node Types.

Core

Payment Type

Billing mode of nodes in a cluster.

• The billing mode of the Master and Core node groups is the same as that of the cluster.
• The billing mode of the Task node group is fixed to pay-per-use.

Pay-per-use

Node Count

Configure node quantity in each node group.

• The number of nodes in a master node group ranges from 3 to 9.
• There must be at least one Core node and the total number of Core and Task nodes must not exceed 10,000.
• If Cluster Type is set to Custom, you can click Add Node Group to add node groups.

A small number of nodes may cause clusters to run slowly while a large number of nodes may be unnecessarily costly. Set an appropriate value based on data to be processed.

-

Instance Specifications

Select the instance specifications of the MRS cluster node. You can click to adjust the specifications.

• For details about the MRS cluster node specifications, see MRS Cluster Node Specifications.
• More advanced instance specifications provide better data processing. However, they require higher cluster cost.
• Instance specifications may vary in different AZs. If no instance specifications in the current AZ can meet your requirements, switch to another AZ.
• If you select HDDs for Core nodes, there is no billing information for data disks. The fees are charged with ECSs.
• If you select non-HDD disks for Core nodes, the disk types of Master and Core nodes are determined by Data Disk.
• If Sold out appears next to an instance specification of a node, the node of this specification cannot be bought. You can only buy nodes of other specifications.
• The Master node specification (4 vCPUs and 8 GB memory) is not within the after-sales SLA. It is applicable only to the test environment and is not recommended for the production environment.
• For MRS 3.x or later, the memory of the Master node must be greater than 64 GB.

-

System Disk

You can adjust the storage type and space of the system disk on a node as required. For details about the MRS cluster storage, see Cluster Node Disk Types.

-

Data Disk

Storage type and space of data disks on a node. To increase the data storage capacity, you can add disks during cluster creation. A maximum of 10 disks can be added to each Core or Task node. For more information about MRS cluster storage, see Cluster Node Disk Types.

-

LVM

This parameter is valid when a streaming Core node is created only. Click this parameter to enable or disable the disk LVM management function. This parameter is not available in MRS 3.x and later versions.

If LVM is enabled, all disks on a node are mounted as logical volumes. This delivers more proper disk planning to avoid data skew, thereby improving system stability.

Disable this function.

Topology Adjustment

If Cluster Type is set to Custom, you can adjust the deployment of each component in the cluster in the node group.

Set Topology Adjustment to Enable and adjust the instance deployment mode based on service requirements. For details, see Role Deployment Rules for MRS Clusters.

Disable this function.

Kerberos Authentication

Whether to enable Kerberos authentication for each component in the MRS cluster. If Kerberos authentication is enabled, users can access component resources only after being authenticated.

This option cannot be changed after you buy a cluster.

Kerberos authentication enabled

Username

Name of the administrator of Manager. admin is used by default.

admin

Password/Confirm Password

Password of the Manager administrator admin. Keep the password secure.

• Must contain 8 to 26 characters.
• Must contain at least four of the following:
  • Lowercase letters
  • Uppercase letters
  • Digits
  • At least one of the following special characters: `~!@#$%^&*()-_=+|[{}];:',<.>/?
• Cannot be the same as the username or the username spelled backwards.

-

Login Mode

Method for logging in to a node in the MRS cluster.

• Password

  You can log in to the node as user root using a password. You need to customize the password of user root.

• Key Pair

  Select a key pair from the drop-down list. Select "I acknowledge that I have obtained private key file SSHkey-xxx and that without this file I will not be able to log in to my ECS." Click View Key Pair to create or import a key pair, and then obtain the private key file.

Password

Kerberos Encryption Type

Encryption algorithm and method used by Kerberos. (This parameter is supported in MRS 3.3.1-LTS and later versions. In earlier versions, the default Kerberos encryption type is aes256-sha1,aes128-sha1.)

• aes256-sha1,aes128-sha1: indicates that the encryption algorithm and mode are AES256-CTS-HMAC-SHA1-96 AES128-CTS-HMAC-SHA1-96.
• aes256-sha2,aes128-sha2: indicates that the encryption algorithm and mode are AES256-CTS-HMAC-SHA384-192 AES128-CTS-HMAC-SHA256-128.

When setting this parameter, note the following:

• If the encryption types of two clusters are different, mutual trust cannot be set up across the FusionInsight Manager clusters.
• If you are using an external Java program to connect to the MRS cluster for Kerberos authentication and the Kerberos encryption type is aes256-sha2,aes128-sha2, the external Java program must use JDK 11 (or BiSheng JDK 1.8.0_392) or later.
• To use an instance created in DataArts Studio to connect to the MRS cluster, set the Kerberos encryption type to aes256-sha1,aes128-sha1.

aes256-sha2,aes128-sha2

Set Advanced Options

Advanced function parameters of the MRS cluster. For details, see Table 5.

-

Enterprise Project

Select the enterprise project to which the cluster belongs. To use an enterprise project, create one on the Enterprise > Project Management page.

The Enterprise Management console is designed for resource management. It helps you manage cloud-based personnel, resources, permissions, and finance in a hierarchical manner, such as management of companies, departments, and projects.

default

Required Duration

This parameter is valid in Yearly/Monthly billing mode and indicates a cluster subscription duration. The minimum cluster duration is 1 month and the maximum available cluster duration is 1 year.

If Auto-renew is selected, monthly subscriptions are automatically renewed every month and yearly subscriptions are automatically renewed every year.

-

Secure Communications

To allow the MRS console to access big data components in your VPC, you need to enable relevant security group rules to allow traffic to pass. For details, see Configuring Secure Communication Authorization for an MRS Cluster.

Select the check box.

Hostname Prefix

Enter the prefix for the computer hostname of an ECS or BMS in the cluster.

-

Cryptographic Algorithm

Algorithm used for encrypting and decrypting passwords in the cluster system.

• International: general cryptographic algorithm
• Chinese: SM series cryptographic algorithms are compatible with general cryptographic algorithms.

International

Tag

It is recommended that you use the tag function of TMS to add the same tag to different cloud resources. For details, see Adding a Tag to an MRS Cluster/Node.

For more information about TMS, see What Is Tag Management Service?

-

Auto Scaling

Configure an auto scaling policy for the Task node group after the cluster is created. For details, see MRS Task Node Auto Scaling.

-

Bootstrap Action

Bootstrap actions run scripts on specified nodes in a cluster to install third-party software and modify the cluster's running environment. For details, see Adding MRS Node Bootstrap Actions and Installing Third-Party Software.

-

Agency

When an agency is bound to an ECS or BMS, it can manage designated resources in the cluster. It is essential to consider service requirements before deciding to configure an agency.

For example, you can configure an ECS agency to automatically obtain the AK/SK to access OBS. For details, see Interconnecting an MRS Cluster with OBS Using an IAM Agency.

To bind an agency to an MRS cluster, you need to create an IAM agency with required permissions in advance. By default, the system generates an MRS_ECS_DEFAULT_AGENCY agency. This agency has the OBSOperateAccess permission and the CESFullAccess (for users who have enabled fine-grained policies), CES Administrator, and KMS Administrator permissions in the region where the cluster is located.

Bind later

Metric Sharing

Monitoring metrics of big data components are collected. If a fault occurs when you use a cluster, share the monitoring metrics with Huawei Cloud technical support for troubleshooting.

Disable

System Disk Encryption

Whether to encrypt data in the system disk mounted to the MRS cluster node. This function is disabled by default. This parameter is available only in MRS 3.3.0-LTS or later.

Keys used by encrypted system disks are provided by Key Management Service (KMS) in Data Encryption Workshop (DEW). You do not need to build and maintain the key management infrastructure. To enable this function, you must have the Security Administrator and KMS Administrator permissions.

For more information about disk encryption, see Managing Encrypted EVS Disks.

For more information about DEW, see What Is DEW?

Configure the following parameters to enable this function:

• Data Disk Key ID: Key ID of the selected key name.
• Data Disk Key Name: Select the name of the key used to encrypt the data disk. By default, the default master key named evs/default is selected. You can select another master key from the drop-down list.

  If cloud disks are encrypted using a master key and the key is then disabled or scheduled for deletion, the disks can no longer be read from or written to, and data on these disks may never be recovered. Exercise caution when performing relevant operations.

  Click View the Key List to enter a page where you can create and manage keys.

Disable

Data Disk Encryption

Whether to encrypt data in the data disk mounted to the MRS cluster node. This function is disabled by default.

For more information about disk encryption, see Managing Encrypted EVS Disks.

Keys used by encrypted system disks are provided by Key Management Service (KMS) in Data Encryption Workshop (DEW). You do not need to build and maintain the key management infrastructure. To enable this function, you must have the Security Administrator and KMS Administrator permissions.

Configure the following parameters to enable this function:

• Data Disk Key ID: Key ID of the selected key name.
• Data Disk Key Name: Select the name of the key used to encrypt the data disk. By default, the default master key named evs/default is selected. You can select another master key from the drop-down list.

  If cloud disks are encrypted using a master key and the key is then disabled or scheduled for deletion, the disks can no longer be read from or written to, and data on these disks may never be restored. Exercise caution when performing this operation.

  Click View the Key List to enter a page where you can create and manage keys.

Disable

Alarm

If the alarm function is enabled, the cluster maintenance personnel can be notified in a timely manner to locate faults when the cluster runs abnormally or the system is faulty. To send alarm messages, you need to enable the Simple Message Notification (SMN) service, set a notification rule, and bind the rule to an SMN topic.

• Rule Name: Name of the rule for sending alarm messages. The value can contain only digits, letters, hyphens (-), and underscores (_).
• Topic Name: Select an existing SMN topic or click Create Topic to create a topic.

  To receive messages published to a topic, you need to add a subscriber to the topic. For details, see Configuring Notifications for MRS Cluster Alarms and Events.

Enable

Install UniAgent

Install UniAgent on MRS cluster nodes to simplify plug-in management, provide AOM instructions, and enable script delivery and execution.

No

Logging

Whether to collect logs when cluster creation fails.

After the logging function is enabled, system logs and component run logs are automatically collected and saved to the OBS file system in scenarios such as cluster creation failures and scale-out or scale-in failures for O&M personnel to quickly locate faults.

The log information is retained for a maximum of seven days.

Disable

MRS.101

Insufficient quota to meet your request. Contact customer service to increase the quota.

MRS.102

The token cannot be null or invalid. Try again later or contact customer service.

MRS.103

Invalid request. Try again later or contact customer service.

MRS.104

Insufficient resources. Try again later or contact customer service.

MRS.105

Insufficient IP addresses in the existing subnet. Try again later or contact customer service.

MRS.201

Failed due to an ECS error. Try again later or contact customer service.

MRS.202

Failed due to an IAM error. Try again later or contact customer service.

MRS.203

Failed due to a VPC error. Try again later or contact customer service.

MRS.400

MRS system error. Try again later or contact customer service.