Help Center/ MapReduce Service/ User Guide/ MRS Cluster O&M/ MRS Cluster Patch Description/ MRS 1.9.0.8, 1.9.0.9, and 1.9.0.10 Patch Description
Updated on 2024-09-23 GMT+08:00

MRS 1.9.0.8, 1.9.0.9, and 1.9.0.10 Patch Description

Basic Information

Table 1 Basic information

Resolved Issues

Patch number: MRS 1.9.0.10

Release date: January 17, 2023

Resolved issues

MRS big data components

OBSA supports flow control retry.

Patch number: MRS 1.9.0.9

Release date: August 10, 2022

Resolved issues

MRS big data components

Superior scheduling algorithm optimization

Patch number: MRS 1.9.0.8

Release date: February 20, 2021

Resolved issues

MRS big data components

Added the monitoring on the exception that occurs when the ECS getSecuritykey API is called.

Optimized the temporary AK/SK process.

Resolved the issue of ResourceManager memory leakage.

List of resolved issues in MRS 1.9.0.7:

MRS Manager

Resolved the issue of queue information loss due to queue update during capacity expansion.

MRS big data components

Resolved the issue that the Hive on Spark task is suspended because the block ID is displayed as garbled characters.

Solve the problem that the Hadoop task fails to be executed due to insufficient space.

Self-developed APIs are added to Hive.

Resolved the issue that the map.xml file cannot be read.

Resolved the issue that YARN is unavailable due to ZooKeeper dirty data.

Resolved the issue of ResourceManager memory leakage.

Optimized the Hive Har feature.

Upgraded the OBS packages.

Upgraded the JDK version.

List of resolved issues in MRS 1.9.0.6:

MRS Manager

MRS Manager supports scale-in of specified nodes in a yearly/monthly cluster.

MRS big data components

Resolved the issue of slow response when HiveSE delivers SQL statements.

Supported the JobHistory query failure information interface.

Resolved the issue that fine-grained permissions do not take effect.

Fixed the exception that occurs when Hive on Spark reads data.

Resolved the issue that data volume increases when the Hive on mrs task is executed twice.

Resolved the issue that the performance of some strings is poor when vector-based vectorized query is enabled in Hive.

List of resolved issues in MRS 1.9.0.5:

MRS Manager

Optimized the service restart process during configuration saving on MRS Manager.

Rectified the periodic backup failure on MRS Manager.

MRS big data components

Private patch of Ranger

Resolved the JVM Create GC thread failed issue of YARN.

Added the HiveServer2 task stacking alarm.

Added the alarm indicating that the GC time of HiveServer HiveMetastore exceeds 5s.

Added the alarm indicating that HiveServer2 uncomment ZooKeeper.

Added the alarm indicating that YARN tasks failed and the number of killed tasks exceeds 5 within 20 minutes.

Corrected time zone of Spark JobHistory.

Optimized MetaStore restart mechanism.

Resolved the HIVE-22771 open-source issue.

Resolved the Hive beeline log printing errors.

Corrected the number of active nodes displayed on the YARN page.

Resolved the slow response of RM page display, which is caused by large number of RM threads.

Supported OBS monitoring.

Upgraded the OBS packages.

Resolved the issue that some data is not inserted when 10 data records are concurrently inserted into hive-jdbc.

Resolved the issue that Hive occasionally reports a Kryo deserialization failure.

Resolved the issue of Spark JobHistory memory leakage.

Resolved the issue that the application list cannot be displayed occasionally in Spark JobHistory.

List of resolved issues in MRS 1.9.0.3:

MRS Manager

Upgraded Arm JDK on MRS Manager.

Resolved the issue that the system disk is fully occupied by logs of the Core node on MRS Manager.

MRS big data components

Resolved the issue that the number of Ranger logs cannot be set, which may cause full disk occupation.

List of resolved issues in MRS 1.9.0.2:

MRS Manager

Resolved the mutual trust lost between some Core nodes in the cluster.

Resolved the issue that instances fail to be added after the patch is installed.

Resolved the issue that the rolling restart timeout interval of HiveServer cannot be modified on MRS Manager.

MRS big data components

Upgraded the OBS packages.

List of resolved issues in MRS 1.9.0.1:

MRS Manager

Resolved the issue that MRS Manager does not support rolling patch installation without restarting services.

MRS big data components

Resolved the issue that the OBS entrusted access frequency is not limited to 140 times within 5 minutes.

Resolved the issue that Kafka does not support open-source access.

Resolved the SPARK-27637 open-source issue.

Optimized the Hive rolling restart.

Upgraded the OBS packages.

Compatibility with Other Patches

The MRS 1.9.0.10 patch can resolve all problems that have been resolved by the MRS 1.9.0 patch.

Impact of Patch Installation

  • During the installation of the MRS 1.9.0.10 patch, MRS Manager will be restarted, and the components such as Hadoop, Hive, Spark, Presto, and related dependent services will be restarted in rolling mode. During the restart of MRS Manager, services are temporarily unavailable, and services are not interrupted during the rolling restart.
  • After installing the MRS 1.9.0.10 patch, you need to download and install all clients again, including the original clients of Master nodes and the clients used by other nodes of VPC (that is, the clients that you set up).
    • You are advised to back up the old clients before reinstalling the new ones.
    • If you have modified client configurations based on the service scenario, modify them again after reinstalling the clients.
  • (Optional) In the scenario where a temporary AK/SK is obtained by using an agency to access OBS, configure the fs.obs.auth.node-cache-short-circuit.enable parameter to determine whether to allow access to the ECS metadata API, thereby determining whether to trigger ECS flow control.

    The MRS cluster can access OBS using a temporary AK/SK obtained by an agency. The temporary AK/SK is obtained using the ECS metadata API. The ECS metadata API has a flow control threshold of 140 times within 5 minutes for a single node. After the flow control is triggered, the node is added to the blacklist and cannot call the metadata API again within 30 minutes. To prevent flow control, MRS provides the node-level cross-process cache service meta to cache temporary AK/SK.

    Application scenario: YARN jobs such as Spark and Hadoop that access OBS using a temporary AK/SK obtained by an agency. This parameter is configured in the core-site.xml file on the client.

    The default value is true. The YARN application process in the MRS cluster obtains the temporary AK/SK from the node-level cache service meta. If meta is abnormal, obtain the temporary AK/SK from the ECS metadata API.

    If you do not want to directly access the ECS metadata API when the meta is abnormal, set this parameter to false to prevent the node from being added to the blacklist due to flow control.