Buying a Custom MRS Cluster

Cluster Name

The cluster name must be unique. It can contain 1 to 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.

mrs_test

Cluster Type

Select a proper MRS cluster type based on service requirements.

• Analysis cluster: It is used for offline data analysis and comprises data analysis tools like Hadoop, Spark, HBase, Hive, Flink, Oozie, and Tez.
• Streaming cluster: It processes streaming data to quickly analyze real-time data sources, and it mainly includes streaming data processing tools such as Kafka and Flume.
• Hybrid cluster: It is utilized for both offline data analysis and stream processing.
• Custom: You can select from a variety of components that are supported by the corresponding version of the MRS cluster.

Custom

Version Type

MRS provides two types of clusters: LTS and Normal. Different versions provide different components. You can select a version as required.

• LTS: employs MRS's own components to provide highly reliable clusters with strong DR capabilities, making long-term support and evolution possible.
• Normal: integrates MRS's mature and stable features and functions with open-source capabilities, offering high performance and stability.

LTS

Cluster Version

Version of the MRS cluster. Different versions may contain different open source component versions and functions. You are advised to select the latest version.

MRS 3.2.0-LTS.1

Component

Select the components to be deployed in the MRS cluster.

You can change components based on your needs. For some clusters, components cannot be added after creation.

-

Metadata

Whether to use external data sources to store Hive and Ranger metadata of the cluster.

• Local: Metadata is stored in the local cluster.
• External data connection: Metadata of external data sources is used. If the cluster is abnormal or deleted, metadata is not affected. This mode applies to scenarios where storage and compute are decoupled.

  After a cluster is created, you can manually store the component metadata to external data sources. For details, see Managing MRS Cluster Metadata.

Local

Component Port

Default communication port policy of each component in the MRS cluster. The LTS cluster supports the configuration.

• Open source: Use the port provided by the open source component.
• Custom: Customize a port for the component.

For details about the differences between default open source port and default custom port, see Common Ports for MRS Cluster Services.

Open source

AZ

Resources are assigned to the AZ of the current region during creation. An AZ is a physical region that operates on independent power supplies and networks for resource usage.

AZ1

VPC

VPC to which the MRS cluster node belongs. If no VPC is available, click View VPC to access the network console and create a VPC.

-

Subnet

Subnet information in the VPC. If no subnet is available, click View Subnet to access the network console and create a subnet.

A subnet provides dedicated network resources that are logically isolated from other networks for network security. For details about how to configure network ACL outbound rules, see How Do I Configure a Network ACL Outbound Rule?.

-

Security Group

A security group is a set of ECS access rules. It provides access policies for ECSs that have the same security protection requirements and are mutually trusted in a VPC.

When you create an MRS cluster, a security group is automatically created by default. You can also select an existing security group from the drop-down list.

Auto create

EIP

After binding an EIP to an MRS cluster, you can use the EIP to access the Manager web UI of the cluster.

When creating a cluster, you can select an available EIP from the drop-down list and bind it. If no EIP is available in the drop-down list, click Create EIPs to buy an EIP.

Bind later

CPU Architecture

CPU architecture type of an MRS cluster node. The value can be x86 or Kunpeng. This parameter is not available for MRS 3.1.0 and 3.1.5.

x86

Common Node Configurations

This parameter is available only when Cluster Type is set to Custom. Value options include Compact, Full-size, and OMS-separate. For details, see MRS Cluster Deployment Types.

Compact

Node Group

Name of the node group in the cluster.

• The name of the Master node group is fixed to master_node_default_group.
• The system automatically creates a Core node group based on the components contained in the cluster. For example, if you select the ClickHouse component, the system adds the ClickHouse node group and deploys the ClickHouseServer role in the node group by default.

• If Cluster Type is set to Custom, you can customize the names of other node groups.
• If the data volume does not change much but the service processing capability changes greatly, manually add a Task node group.

node_group_1

Node Type

If Cluster Type is set to Custom, you can select the node type of a non-Master node group. If the node group type is set to Task, only the NodeManager role (except the mandatory roles of nodes) can be deployed in the node group.

Core

Billing Mode

Billing mode of nodes in a cluster.

• The billing mode of the Master and Core node groups is the same as that of the cluster.
• The billing mode of the Task node group is fixed to pay-per-use.

Pay-per-use

Instance Specifications

Select the instance specifications of the MRS cluster node. You can click to adjust the specifications.

For details about the MRS cluster node specifications, see MRS Cluster Node Specifications.

-

System Disk

You can adjust the storage type and space of the system disk on a node as required. For details about the MRS cluster storage, see Disk Roles.

-

Data Disk

Storage type and space of data disks on a node. To increase the data storage capacity, you can add disks during cluster creation. A maximum of 10 disks can be added to each Core or Task node. For more information about MRS cluster storage, see Disk Roles.

-

Instance Count

Number of Master and Core nodes.

• Master Node Groups: The number of Master instances ranges from 3 to 9.
• At least one Core node must exist and the total number of Core and Task nodes cannot exceed 10,000.

  Click to add a node group, click to modify the node instance specifications, and click to delete the added node group.

-

LVM

This parameter is valid when a streaming Core node is created only. Click this parameter to enable or disable the disk LVM management function. This parameter is not available in MRS 3.x and later versions.

If LVM is enabled, all disks on a node are mounted as logical volumes. This delivers more proper disk planning to avoid data skew, thereby improving system stability.

Disable this function.

Topology Adjustment

If Cluster Type is set to Custom, you can adjust the deployment of each component in the cluster in the node group.

Set Topology Adjustment to Enable and adjust the instance deployment mode based on service requirements. For details, see Role Deployment Rules for MRS Clusters.

Disable this function.

Kerberos Authentication

Whether to enable Kerberos authentication for each component in the MRS cluster. If Kerberos authentication is enabled, users can access component resources only after being authenticated.

This function cannot be changed after you buy a cluster.

Switch this function on.

Username

Name of the administrator of Manager. admin is used by default.

admin

Password/Confirm Password

Password of the Manager administrator admin. Keep the password secure.

• Must contain 8 to 26 characters.
• Must contain at least four of the following:
  • Lowercase letters
  • Uppercase letters
  • Digits
  • At least one of the following special characters: `~!@#$%^&*()-_=+|[{}];:',<.>/?
• Cannot be the same as the username or the username spelled backwards.

-

Login Mode

Method for logging in to a node in the MRS cluster.

• Password

  You can log in to the node as user root using a password. You need to customize the password of user root.

• Key Pair

  Select a key pair from the drop-down list. Select "I acknowledge that I have obtained private key file SSHkey-xxx and that without this file I will not be able to log in to my ECS." Click View Key Pair to create or import a key pair, and then obtain the private key file.

Password

Kerberos Encryption Type

Encryption algorithm and method used by Kerberos. (This parameter is supported in MRS 3.3.1-LTS and later versions. In earlier versions, the default Kerberos encryption type is aes256-sha1,aes128-sha1.)

• aes256-sha1,aes128-sha1: indicates that the encryption algorithm and mode are AES256-CTS-HMAC-SHA1-96 AES128-CTS-HMAC-SHA1-96.
• aes256-sha2,aes128-sha2: indicates that the encryption algorithm and mode are AES256-CTS-HMAC-SHA384-192 AES128-CTS-HMAC-SHA256-128.
  NOTE:

  • If the encryption types of the two clusters are different, mutual trust cannot be set up across the Managers for the clusters.
  • If you are using an external Java program to connect to the MRS cluster for Kerberos authentication and the Kerberos encryption type is aes256-sha2,aes128-sha2, the external Java program must use JDK 11 (or BiSheng JDK 1.8.0_392) or later.
  • To use an instance created in DataArts Studio to connect to the MRS cluster, set the Kerberos encryption type to aes256-sha1,aes128-sha1.

aes256-sha2,aes128-sha2

Set Advanced Options

Advanced function parameters of the MRS cluster. For details, see Table 5.

-

Enterprise Project

Select the enterprise project to which the cluster belongs. To use an enterprise project, create one on the Enterprise > Project Management page.

The Enterprise Management console is designed for resource management. It helps you manage cloud-based personnel, resources, permissions, and finance in a hierarchical manner, such as management of companies, departments, and projects.

default

Required Duration

This parameter is valid in Yearly/Monthly billing mode and indicates a cluster subscription duration. The minimum cluster duration is 1 month and the maximum available cluster duration is 1 year.

If Auto-renew is selected, monthly subscriptions are automatically renewed every month and yearly subscriptions are automatically renewed every year.

-

Secure Communications

To allow the MRS console to access big data components in the user VPC, you need to activate the relevant security group rules. For details, see Configuring Secure Communication Authorization for an MRS Cluster.

Select this function.

Hostname Prefix

Enter the prefix for the computer hostname of an ECS or BMS in the cluster.

-

Cryptographic Algorithm

Algorithm used for encrypting and decrypting passwords in the cluster system.

• International: general encryption algorithm
• Chinese: SM series cryptographic algorithms are compatible with general cryptographic algorithms.

International

Tag

It is recommended that you use the tag function of TMS to add the same tag to different cloud resources. For details, see Adding a Tag to an MRS Cluster/Node.

-

Auto Scaling

Configure an auto scaling policy for the Task node group after the cluster is created. For details, see MRS Task Node Auto Scaling.

-

Bootstrap Action

Bootstrap actions run scripts on specified nodes in a cluster to install third-party software and modify the cluster's running environment. For details, see Adding MRS Node Bootstrap Actions and Installing Third-Party Software.

-

Agency

When an agency is bound to an ECS or BMS, it can manage designated resources in the cluster. It is essential to consider service requirements before deciding to configure an agency.

For example, you can configure an ECS agency to automatically obtain the AK/SK to access OBS. For details, see Interconnecting an MRS Cluster with OBS Using an IAM Agency.

To bind an agency to an MRS cluster, you need to create an IAM agency with required permissions in advance. By default, the system generates an MRS_ECS_DEFAULT_AGENCY agency. This agency has the OBSOperateAccess permission and the CESFullAccess (for users who have enabled fine-grained policies), CES Administrator, and KMS Administrator permissions in the region where the cluster is located.

Bind later

Metric Sharing

Monitoring metrics of big data components are collected. If a fault occurs when you use a cluster, share the monitoring metrics with Huawei Cloud technical support for troubleshooting.

Disable

System Disk Encryption

Whether to encrypt data in the system disk mounted to the MRS cluster node. This function is disabled by default.

Keys used by encrypted system disks are provided by Key Management Service (KMS) in Data Encryption Workshop (DEW). You do not need to build and maintain the key management infrastructure. To enable this function, you must have the Security Administrator and KMS Administrator permissions.

For more information about disk encryption, see Managing Encrypted EVS Disks.

Configure the following parameters to enable this function:

• Data Disk Key ID: key ID of the selected key name.
• Data Disk Key Name: Select the name of the key used to encrypt the data disk. By default, the default master key named evs/default is selected. You can select another master key from the drop-down list.

  If cloud disks are encrypted using a master key and it is then disabled or scheduled for deletion, the cloud disks can no longer be read from or written to, and data on these disks may never be restored. Exercise caution when performing this operation.

  Click View the Key List to enter a page where you can create and manage keys.

Disable

Data Disk Encryption

Whether to encrypt data in the data disk mounted to the MRS cluster node. This function is disabled by default. MRS 3.1.0 and MRS 3.1.2-LTS.3 do not support this function.

For more information about disk encryption, see Managing Encrypted EVS Disks.

Keys used by encrypted system disks are provided by Key Management Service (KMS) in Data Encryption Workshop (DEW). You do not need to build and maintain the key management infrastructure. To enable this function, you must have the Security Administrator and KMS Administrator permissions.

Configure the following parameters to enable this function:

• Data Disk Key ID: key ID of the selected key name.
• Data Disk Key Name: Select the name of the key used to encrypt the data disk. By default, the default master key named evs/default is selected. You can select another master key from the drop-down list.

  If cloud disks are encrypted using a master key and it is then disabled or scheduled for deletion, the cloud disks can no longer be read from or written to, and data on these disks may never be restored. Exercise caution when performing this operation.

  Click View the Key List to enter a page where you can create and manage keys.

Disable

Alarm

If the alarm function is enabled, the cluster maintenance personnel can be notified in a timely manner to locate faults when the cluster runs abnormally or the system is faulty. To send alarm messages, you need to enable the Simple Message Notification (SMN) service, set a notification rule, and bind the rule to an SMN topic.

• Rule Name: Name of the rule for sending alarm messages. The value can contain only digits, letters, hyphens (-), and underscores (_).
• Topic Name: Select an existing SMN topic or click Create Topic to create a topic.

  To deliver messages published to a topic, you need to add a subscriber to the topic. For details, see Adding Subscriptions to a Topic.

Enable

Install UniAgent

Install UniAgent on MRS cluster nodes to simplify plug-in management, provide AOM instructions, and enable script delivery and execution.

No

Logging

Whether to collect logs when cluster creation fails.

After the logging function is enabled, system logs and component run logs are automatically collected and saved to the OBS file system in scenarios such as cluster creation failures and scale-out or scale-in failures for O&M personnel to quickly locate faults.

The log information is retained for a maximum of seven days.

Disable