How Do I Use MRS?
MapReduce Service (MRS) is a service you can use to deploy and manage Hadoop-based components on the Huawei Cloud. It enables you to deploy Hadoop clusters with a few clicks. MRS provides enterprise-ready big data clusters in the cloud. Tenants can fully control the clusters and easily run big data components such as Hadoop, Spark, HBase, Kafka, and Storm in the clusters.
MRS is easy to use. You can execute various tasks and process or store PB-scale data using computers connected in a cluster. To use MRS, do as follows:
- Develop a data processing program. For details about how to quickly develop such a program and execute it properly, see the sample code and tutorials provided in Method of Building an MRS Sample Project.
- Upload local programs and data files to OBS.
- Create a cluster. You need to specify the cluster type (for example, analysis or streaming), and set ECS instance specifications, number of instances, data disk type (common I/O, high I/O, and ultra-high I/O), and components to be installed, such as Hadoop, Spark, HBase, Hive, Kafka, and Storm, in a cluster. You can use a bootstrap action to install third-party software or modify the cluster running environment on a node before or after the cluster is started.
- Use MRS to submit, execute, and monitor your programs.
- Manage clusters on MRS Manager, an enterprise-level unified management platform of big data clusters. You can learn about the health status of services and hosts, obtain critical system information in a timely manner from graphical metric monitoring and customization, modify service attributes based on performance requirements, and start or stop clusters, services, and role instances.
- Terminate any MRS cluster that you do not require after job execution is complete. The terminated cluster is no longer billed.
MRS Overview FAQs
- What Is MRS Used For?
- What Types of Distributed Storage Does MRS Support?
- How Do I Create an MRS Cluster Using a Custom Security Group?
- How Do I Use MRS?
- Region and AZ
- Can I Configure a Phoenix Connection Pool?
- Does MRS Support Change of the Network Segment?
- Can I Downgrade the Specifications of an MRS Cluster Node?
- What Is the Relationship Between Hive and Other Components?
- Does an MRS Cluster Support Hive on Spark?
- What Are the Differences Between Hive Versions?
- Which MRS Cluster Version Supports Hive Connection and User Synchronization?
- What Are the Differences Between OBS and HDFS in Data Storage?
- How Do I Obtain the Hadoop Pressure Test Tool?
- What Is the Relationship Between Impala and Other Components?
- Statement About the Public IP Addresses in the Open-Source Third-Party SDK Integrated by MRS
- What Is the Relationship Between Kudu and HBase?
- Does MRS Support Running Hive on Kudu?
- What Are the Solutions for processing 1 Billion Data Records?
- Can I Change the IP address of DBService?
- Can I Clear MRS sudo Logs?
- Is the Storm Log also limited to 20 GB in MRS cluster 2.1.0?
- What Is Spark ThriftServer?
- What Access Protocols Are Supported by Kafka?
- What If Error 408 Is Reported When an MRS Node Accesses OBS?
- What Is the Compression Ratio of zstd?
- Why Are the HDFS, YARN, and MapReduce Components Unavailable When an MRS Cluster Is Bought?
- Why Is the ZooKeeper Component Unavailable When an MRS Cluster Is Bought?
- Which Python Versions Are Supported by Spark Tasks in an MRS 3.1.0 Cluster?
- How Do I Enable Different Service Programs to Use Different YARN Queues?
- Differences and Relationships Between the MRS Management Console and Cluster Manager
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.
more