Glossary_MapReduce Service

On this page

Show all

Help Center/ MapReduce Service/ Glossary

A-E

A

AM

See ApplicationMaster

ApplicationMaster

Manages the life cycle of applications.
B

Block

The smallest logical unit of storage in HDFS. Each HDFS file is stored as one or more blocks. All blocks are stored on DataNodes.
C

Cell

A {row, column} tuple exactly specifies a cell in HBase. Cell content is uninterpreted bytes.

Column

A column is one of HBase table dimensions. A column name has the form "family:label" where family and label can be arbitrary byte arrays. A table enforces its set of families (called "column families"). Each column in an HBase table belongs to a column family.

Column Family

A column family is a predefined arbitrary set of columns and stored in HBase Schema. To create some columns in a column family, you should create the column family first. A column family regroups data of a same nature in HBase and has no constraint on the type. Every row of data in a column family is physically stored on the same server. Each column family is as an attribute, such as a compression package, a timestamp, block cache, and so on.

Container

A container isolates CPU and memory resources on Java virtual machines (VMs).

Core Node

A core node in a MapReduce Service cluster processes data and stores process data in the HDFS.
D

Data Definition Language

A language used to define the data structure and database objects in the HiveQL set. It consists of three types of syntax: CREATE, ALTER, and DROP. Derived from the Conference on Data Systems Languages (Codasyl) model, the DDL has become a subset of SQL.

Data Manipulation Language

An instruction set for accessing objects in databases. The core instructions in this set are INSERT, UPDATE, and DELETE, which respectively mean inserting, updating, and deleting. These instructions are indispensable for developing data-centered applications.

DataNode

DataNodes, usually one per node in the HDFS cluster, manage storage attached to the nodes that they run on.

DDL

See Data Definition Language

DML

See Data Manipulation Language

DN

See DataNode
E

Executor

A process launched for an application on a worker node, that runs tasks and keeps data in memory or disk storage across them. Each application has its own executors.

F-J

H

Hadoop Distributed File System

HDFS provides high-throughput data access and is applicable to the processing of large data sets. MRS cluster data is stored in HDFS.

HBase

HBase is a column-oriented distributed cloud storage system that features enhanced reliability, excellent performance, and elastic scalability. It applies to the storage of massive data and distributed computing. Users can use HBase to build a storage system capable of storing TB- or even PB-level data. With HBase, users can filter and analyze data with ease and get responses in milliseconds, rapidly mining data value.

HDFS

See Hadoop Distributed File System

HFile

File format for HBase. A file of sorted key/value pairs. Both keys and values are byte arrays.

Hive

A data warehouse tool running on Hadoop. Hive maps structured data files to a database table and provides simple SQL search function that converts SQL statements into MapReduce tasks.

Hive Query Language

Hive Query Language, a standard data query language used for Hive data warehouses.

HiveQL

See Hive Query Language

HMaster

Also known as Master. HMaster manages the RegionServer in the HBase, including the load balancing of the RegionServer, and the split, distribution, and migration of Regions. In an HA mode, HMaster includes a primary HMaster and a secondary HMaster.
I

Index

A data structure that improves the speed of data retrieval operations on a database table at the cost of slower writes and increased storage space. Indices can be created using one or more columns of a database table, providing the basis for both rapid random lookups and efficient access of ordered records.
J

Job

MRS provides users with an application execution platform, which enables users to submit applications they have developed to a MRS cluster, execute the applications, and obtain results. MRS also enables users to submit SQL statements online as well as query and resolve structured data.

K-O

M

Map

A processing model function that processes a key-value pair to generate a set of intermediate key-value pairs.

MapReduce

As a programming model that simplifies parallel computing, MapReduce gets its name from two key operations: Map and Reduce. Map divides one task into multiple tasks, and Reduce summarizes the processing results of these tasks and produces the final analysis result. MRS clusters allow users to submit self-developed MapReduce programs, execute the programs, and obtain the result.

MapReduce Service

MapReduce Service builds a reliable, secure, and easy-to-use operation and maintenance (O&M) platform and provides storage and analysis capabilities for massive data, helping address enterprise data storage and processing demands. Users can independently apply for and use the hosted Hadoop, Spark, HBase and Hive services to quickly create clusters on a host and provide storage and computing capabilities for massive data that has low requirements on realtime processing.

Master Node

A master node in a MapReduce Service cluster manages the cluster, assigns MapReduce executable files to core nodes, traces the execution status of each job, and monitors DataNode running status.

MRS

See MapReduce Service
N

NameNode

A master server that manages the file system namespace and regulates access to files by clients. In HA mode, both NameNode and Secondary NameNode are provided.

Namespace

A namespace is a logical grouping of tables analogous to a database in relational database systems.

NM

See NodeManager

NN

See NameNode

NodeManager

Launches and monitors the compute containers on machines in the cluster.

P-T

R

RDD

See Resilient Distributed Datasets

Reduce

A processing model function that merges all intermediate values associated with the same intermediate key.

Region

Similar to partitions in a traditional database, each RegionServer can have multiple regions.

RegionServer

RegionServer is a service of HBase on each working node. It manages Regions, uploads Region load information, and facilitates HMaster in distributed, coordinated management.

Replication factor

The number of copies of a file is called the replication factor of that file.

Resilient Distributed Datasets

Resilient Distributed Datasets, a distributed memory abstraction that lets programmers perform in-memory computations on large clusters in a fault-tolerant manner.

ResourceManager

Manages the global assignment of compute resources to applications.

RM

See ResourceManager

Row

A row key is one of HBase table dimensions. It is an arbitrary array of bytes. Tables are sorted in lexicographical order by their row keys.

Row key

A row key is the HBase primary key. Tables in HBase are lexicographically sorted in ascending order based on row key.

RS

See RegionServer
S

Secondary NameNode

Performs periodic checkpoints of the namespace and helps keep the size of file containing log of HDFS modifications within certain limits at the NameNode.

Shuffle

A process of outputting data from a Map task to a Reduce task.

Spark

MRS deploys and hosts Apache Spark clusters in the cloud, and Spark is a distributed and parallel data processing framework.

Spark SQL

Spark SQL is an important component of Apache Spark and subsumes Shark. It helps engineers who understand conventional databases but do not know MapReduce quickly get started.

Stage

Each job gets divided into smaller sets of tasks called stages that depend on each other.
T

Table

An HBase table is a three dimensional sorted map. It maps from Cartesian product of row key, column key and timestamp to cell value. All HBase data is stored in cells of tables.

Task

A task is an arithmetic unit bearing service logic and a unit of work that will be sent to one executor.

Timestamp

Different versions of the same data for the index, the timestamp type is 64-bit integer. Timestamp can be automatically assigned by the customer or by the explicit assignment when data is written to HBase.

U-Z

U-Z

WAL

See Write Ahead Log

Write Ahead Log

An efficient database algorithm. For the same amount of data, when using WAL log, the database system during the transaction commits disk writes only about half of the traditional rollback log, greatly improving the efficiency of the database disk I/O operations, thereby improving the performance of the database. Each incremental data loads in HBase are written to WAL.

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot

A-E

AM

ApplicationMaster

Block

Cell

Column

Column Family

Container

Core Node

Data Definition Language

Data Manipulation Language

DataNode

DDL

DML

DN

Executor

F-J

Hadoop Distributed File System

HBase

HDFS

HFile

Hive

Hive Query Language

HiveQL

HMaster

Index

Job

K-O

Map

MapReduce

MapReduce Service

Master Node

MRS

NameNode

Namespace

NM

NN

NodeManager

P-T

RDD

Reduce

Region

RegionServer

Replication factor

Resilient Distributed Datasets

ResourceManager

RM

Row

Row key

RS

Secondary NameNode

Shuffle

Spark

Spark SQL

Stage

Table

Task

Timestamp

U-Z

WAL

Write Ahead Log

Feedback

Was this page helpful?

Feedback