MRS 1.9.0.8, 1.9.0.9, and 1.9.0.10 Patch Description
Basic Information
Resolved Issues |
Patch number: MRS 1.9.0.10 Release date: January 17, 2023 Resolved issues MRS big data components OBSA supports flow control retry. |
---|---|
Patch number: MRS 1.9.0.9 Release date: August 10, 2022 Resolved issues MRS big data components Superior scheduling algorithm optimization |
|
Patch number: MRS 1.9.0.8 Release date: February 20, 2021 Resolved issues MRS big data components Added the monitoring on the exception that occurs when the ECS getSecuritykey API is called. Optimized the temporary AK/SK process. Resolved the issue of ResourceManager memory leakage. |
|
List of resolved issues in MRS 1.9.0.7: MRS Manager Resolved the issue of queue information loss due to queue update during capacity expansion. MRS big data components Resolved the issue that the Hive on Spark task is suspended because the block ID is displayed as garbled characters. Solve the problem that the Hadoop task fails to be executed due to insufficient space. Self-developed APIs are added to Hive. Resolved the issue that the map.xml file cannot be read. Resolved the issue that Yarn is unavailable due to ZooKeeper dirty data. Resolved the issue of ResourceManager memory leakage. Optimized the Hive Har feature. Upgraded the OBS packages. Upgraded the JDK version. |
|
List of resolved issues in MRS 1.9.0.6: MRS Manager MRS Manager supports scale-in of specified nodes in a yearly/monthly cluster. MRS big data components Resolved the issue of slow response when HiveSE delivers SQL statements. Supported the JobHistory query failure information interface. Resolved the issue that fine-grained permissions do not take effect. Fixed the exception that occurs when Hive on Spark reads data. Resolved the issue that data volume increases when the Hive on mrs task is executed twice. Resolved the issue that the performance of some strings is poor when vector-based vectorized query is enabled in Hive. |
|
List of resolved issues in MRS 1.9.0.5: MRS Manager Optimized the service restart process during configuration saving on MRS Manager. Rectified the periodic backup failure on MRS Manager. MRS big data components Private patch of Ranger Resolved the JVM Create GC thread failed issue of Yarn. Added the HiveServer2 task stacking alarm. Added the alarm indicating that the GC time of HiveServer HiveMetastore exceeds 5s. Added the alarm indicating that HiveServer2 uncomment ZooKeeper. Added the alarm indicating that Yarn tasks failed and the number of killed tasks exceeds 5 within 20 minutes. Corrected time zone of Spark JobHistory. Optimized MetaStore restart mechanism. Resolved the HIVE-22771 open-source issue. Resolved the Hive beeline log printing errors. Corrected the number of active nodes displayed on the Yarn page. Resolved the slow response of RM page display, which is caused by large number of RM threads. Supported OBS monitoring. Upgraded the OBS packages. Resolved the issue that some data is not inserted when 10 data records are concurrently inserted into hive-jdbc. Resolved the issue that Hive occasionally reports a Kryo deserialization failure. Resolved the issue of Spark JobHistory memory leakage. Resolved the issue that the application list cannot be displayed occasionally in Spark JobHistory. |
|
List of resolved issues in MRS 1.9.0.3: MRS Manager Upgraded ARM JDK on MRS Manager. Resolved the issue that the system disk is fully occupied by logs of the Core node on MRS Manager. MRS big data components Resolved the issue that the number of Ranger logs cannot be set, which may cause full disk occupation. |
|
List of resolved issues in MRS 1.9.0.2: MRS Manager Resolved the mutual trust lost between some Core nodes in the cluster. Resolved the issue that instances fail to be added after the patch is installed. Resolved the issue that the rolling restart timeout interval of HiveServer cannot be modified on MRS Manager. MRS big data components Upgraded the OBS packages. |
|
List of resolved issues in MRS 1.9.0.1: MRS Manager Resolved the issue that MRS Manager does not support rolling patch installation without restarting services. MRS big data components Resolved the issue that the OBS entrusted access frequency is not limited to 140 times within 5 minutes. Resolved the issue that Kafka does not support open-source access. Resolved the SPARK-27637 open-source issue. Optimized the Hive rolling restart. Upgraded the OBS packages. |
|
Compatibility with Other Patches |
The MRS 1.9.0.10 patch can resolve all problems that have been resolved by the MRS 1.9.0 patch. |
Impact of Patch Installation
- During the installation of the MRS 1.9.0.10 patch, MRS Manager will be restarted, and the components such as Hadoop, Hive, Spark, Presto, and related dependent services will be restarted in rolling mode. During the restart of MRS Manager, services are temporarily unavailable, and services are not interrupted during the rolling restart.
- After installing the MRS 1.9.0.10 patch, you need to download and install all clients again, including the original clients of Master nodes and the clients used by other nodes of VPC (that is, the clients that you set up).
- For details about how to fully update the original client of the active Master node, see Fully Updating the Original Client of the Active Master Node.
- For details about how to fully update the original client of the standby Master node, see Fully Updating the Original Client of the Standby Master Node.
- For details about how to fully install the clients you set up, see Installing a Client (Versions Earlier Than 3.x).
- You are advised to back up the old clients before reinstalling the new ones.
- If you have modified client configurations based on the service scenario, modify them again after reinstalling the clients.
- (Optional) In the scenario where a temporary AK/SK is obtained by using an agency to access OBS, configure the fs.obs.auth.node-cache-short-circuit.enable parameter to determine whether to allow access to the ECS metadata API, thereby determining whether to trigger ECS flow control.
The MRS cluster can access OBS using a temporary AK/SK obtained by an agency. The temporary AK/SK is obtained using the ECS metadata API. The ECS metadata API has a flow control threshold of 140 times within 5 minutes for a single node. After the flow control is triggered, the node is added to the blacklist and cannot call the metadata API again within 30 minutes. To prevent flow control, MRS provides the node-level cross-process cache service meta to cache temporary AK/SK.
Application scenario: Yarn jobs such as Spark and Hadoop that access OBS using a temporary AK/SK obtained by an agency. This parameter is configured in the core-site.xml file on the client.
The default value is true. The Yarn application process in the MRS cluster obtains the temporary AK/SK from the node-level cache service meta. If meta is abnormal, obtain the temporary AK/SK from the ECS metadata API.
If you do not want to directly access the ECS metadata API when the meta is abnormal, set this parameter to false to prevent the node from being added to the blacklist due to flow control.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.