Help Center/ MapReduce Service/ What's New

What's New

Updated on 2024/10/29 GMT+08:00
Share

The tables below describe the functions released in each MapReduce Service version and corresponding documentation updates. New features will be successively launched in each region.

October 2024

No.

Feature

Description

Phase

Document

1

Cloud service authorization supports permission change agency.

You can manually change a permission agency for MRS Service authorization. After you agree to the authorization, the system will create and grant permissions to mrs_service_agency, mrs_cluster_agency, or mrs_admin_agency based on the agency and policy you have selected.

Commercial use

Configuring MRS Cloud Service Authorization

July 2024

No.

Feature

Description

Phase

Document

1

Optimized the MRS Getting Started.

The MRS Getting Started Guide is optimized as follows:

  • Optimized the title and added the guide to creating a ClickHouse cluster.
  • The operation video is embedded in the scenario introduction case to help novice users quickly get started with the product.
  • Added the entry-level practice list to quickly view MRS practice cases in various scenarios.

Commercial use

Creating and Using a ClickHouse Cluster for Columnar Store

Creating and Using a Kafka Cluster for Stream Processing

Best Practices for Beginners

January 2024

No.

Feature

Description

Phase

Document

1

Added the IAM synchronization management APIs.

The IAM synchronization management APIs are added, which provide the following functions:

  • Obtain the synchronized IAM users and user groups.
  • Synchronize IAM users and user groups to Manager.
  • Cancel synchronization of specified users and user groups.

Commercial use

IAM Synchronization

December 2023

No.

Feature

Description

Phase

Document

1

Added auto scaling APIs.

Added auto scaling APIs, including the following functions:

  • View information about all auto scaling policies of a specified cluster.
  • Update an auto scaling policy.
  • Delete an auto scaling policy.
  • Create an auto scaling policy.

Commercial use

Auto Scaling APIs

October 2023

No.

Feature

Description

Phase

Document

1

IAM user synchronization supports specified users

When synchronizing IAM users in an MRS cluster, you can specify a user or user group to synchronize IAM users.

Commercial use

Synchronizing IAM Users to MRS

May 2023

No.

Feature

Description

Phase

Document

1

MRS 3.2.0-LTS.1

  • Added the CDL and IoTDB components.
  • Upgraded the kernels of ClickHouse, Flink, Hadoop, and Hudi components. 

Commercial use

Component Operation Guide

List of MRS Component Versions

March 2023

No.

Feature

Description

Phase

Document

1

MRS 3.1.5

  • Upgraded the kernels of Hudi, CarbonData, Flink, Spark, and ZooKeeper components. 
  • Now you can access OBS from a client that is not deployed in the cluster.
  • When you configure auto scaling rules, you can also configure resource pool policies.

Commercial use

List of MRS Component Versions

Accessing OBS from a Client on a Node Outside the Cluster

Creating an Auto Scaling Policy for an Existing Cluster

January 2023

No.

Feature

Description

Phase

Document

1

The effective date can be configured when a scaling policy is configured for an MRS cluster.

You can set the effective date of a resource plan as required.

Commercial use

Auto Scaling of Task Nodes in the MRS Cluster

2

Added the function of viewing alarm details.

You can use this function to quickly view alarm details.

Commercial use

Viewing the Alarm List

July 2022

No.

Feature

Description

Phase

Document

1

Added the API for obtaining the file list of a specified directory.

The list of files in a specified directory can be obtained through an API in an MRS cluster.

Commercial use

Obtaining Files from a Specified Directory

May 2022

No.

Feature

Description

Phase

Document

1

Supported MRS clusters of the LTS version.

LTS:

  • In addition to basic cluster operations, the LTS version supports version upgrade.
  • Supports multi-AZ deployment.
  • Supports the HetuEngine component.

Commercial use

Using HetuEngine from Scratch

March 2022

No.

Feature

Description

Phase

Document

1

Scaled up the specifications of multiple Master nodes.

Added the conditions for scaling up the specifications of multiple Master nodes.

Commercial use

Scaling Up Master Node Specifications

February 2022

No.

Feature

Description

Phase

Document

1

Suppored unsubscription from nodes in yearly/monthly clusters.

You can reduce the number of specific nodes to scale in a cluster so that MRS delivers better storage and computing capabilities at lower O&M costs based on service requirements.

Commercial use

Unsubscribing from a Specified Node in a Yearly/Monthly Cluster

September 2021

No.

Feature

Description

Phase

Document

1

Supported custom node names.

During MRS cluster creation, you can customize prefixes for the hostnames of ECSs or BMSs in the cluster.

Commercial use

Purchasing a Custom Cluster

July 2021

No.

Feature

Description

Phase

Document

1

Upgraded ClickHouse to 21.3.4.25 and supported data redistribution.

After new ClickHouse nodes are added, you can balance data in the ClickHouse cluster through data migration.

Commercial use

Using the ClickHouse Data Migration Tool

2

Added Hudi in MRS.

Hudi is the file organization layer of the data lake. It manages Parquet files, provides the data lake capability, and supports multiple compute engines. It also provides insert, update, and deletion (IUD) interfaces and streaming primitives for inserting, updating, and incremental pulling on HDFS datasets.

Commercial use

Hudi

Quick Start

March 2021

No.

Feature

Description

Phase

Document

1

Brought MRS 2.1.0 offline.

Purchase MRS 3.1.0.

Commercial use

List of MRS Component Versions

2

Brought MRS 1.8.10 offline.

Purchase MRS 1.9.2.

Commercial use

List of MRS Component Versions

3

Supported scale-in of specified nodes.

If a node is faulty and is no longer needed, you can use this function to remove the faulty node. When the node is removed, the role instance of the component will not be decommissioned. Before deleting the node, ensure that the data on the node has been backed up.

Commercial use

Manually Scaling In a Cluster

November 2020

No.

Feature

Description

Phase

Document

1

Supported ClickHouse clusters.

MRS 3.x allows you to create ClickHouse clusters and provides capabilities such as smooth scale-out, HA, Kunpeng, flexible configuration, simple O&M, and security and reliability.

Commercial use

ClickHouse

Quick Purchase of a ClickHouse Cluster

Using ClickHouse from Scratch

Introduction to ClickHouse

2

Upgraded new big data components.

MRS 3.x provides the latest capabilities of open-source components and enhances functions, performance, and reliability based on the community versions.

Commercial use

List of MRS Component Versions

September 2020

No.

Feature

Description

Phase

Document

1

Supported multiple heterogeneous VM specifications.

MRS clusters support hybrid deployment of x86 and Arm servers, and hybrid deployment of Elastic Cloud Servers (ECSs) and Bare Metal Servers (BMSs).

Commercial use

Purchasing a Custom Cluster

2

Added the V2 API for creating a cluster.

The API of V2 is more powerful and easy to use. If you need to create a cluster, you are advised to use V2 API.

Commercial use

Creating Clusters

August 2020

No.

Feature

Description

Phase

Document

1

Added automatic synchronization between IAM accounts and cluster users.

IAM user synchronization is to synchronize IAM users bound with MRS policies to the MRS system and create accounts sharing same names with the IAM users but different passwords. Then, you can use an IAM username (the password needs to be reset by user admin of Manager) to log in to Manager for cluster management.

Commercial use

Synchronizing IAM Users to MRS

June 2020

No.

Feature

Description

Phase

Document

1

Supported user-defined cluster topologies.

MRS allows you to customize the deployment mode of cluster process instances on cluster nodes.

Commercial use

Purchasing a Custom Topology Cluster

March 2020

No.

Feature

Description

Phase

Document

1

Added Presto HA in MRS.

MRS supports the installation of multiple Presto instances for a large-scale cluster by default. That is, multiple Worker instances, such as Worker1, Worker2, and Worker3, are installed on a Core/Task node. Multiple Worker instances interact with a Coordinator to execute computing tasks, greatly improving node resource utilization and computing efficiency.

Commercial use

Presto

2

Added Tez in MRS.

Tez is Apache's latest open source computing framework that supports Directed Acyclic Graph (DAG) jobs. It can convert multiple dependent jobs into one job, greatly improving the performance of DAG jobs. If projects like Hive and Pig use Tez instead of MapReduce as the backbone of data processing, response time will be significantly reduced. Tez is built on Yarn and can run MapReduce jobs without any modification.

Commercial use

Tez

3

Added Alluxio in MRS.

Alluxio is data orchestration technology for analytics and artificial intelligence (AI) for the cloud. In the MRS big data ecosystem, Alluxio lies between computing and storage. It provides a data abstraction layer for computing frameworks including Apache Spark, Presto, MapReduce, and Apache Hive, so that upper-layer computing applications can access persistent storage systems including HDFS and OBS through unified client APIs and a global namespace. In this way, computing and storage are separated.

Commercial use

Alluxio

4

Added Ranger in MRS.

Apache Ranger offers a centralized security framework and supports authorization and auditing. It manages fine-grained access control over Hadoop and related components, such as HDFS, Hive, HBase, Kafka, and Storm. You can use the frontend web UI console provided by Ranger to configure policies to control users' access to these components.

Commercial use

Ranger

Show allHide