Help Center/ MapReduce Service/ Component Operation Guide (LTS)/ Using YARN/ Common Issues About Yarn/ Why Do Reduce Tasks Fail to Run in Some OSs After the Native Task Feature is Enabled?
Updated on 2025-11-13 GMT+08:00
View PDF

Why Do Reduce Tasks Fail to Run in Some OSs After the Native Task Feature is Enabled?

Question

When -Dmapreduce.job.map.output.collector.class=org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator is executed to enable the Native Task feature during the running of MapReduce tasks that contain Reduce tasks, the tasks fail to run in some OSs, and the error message "version 'GLIBCXX_3.4.20' not found" is displayed in logs.

Answer

The cause is that the GLIBCXX version of the OSs is too early. As a result, the libnativetask.so.1.0.0 library on which the feature depends cannot be loaded, leading to task failures.

Workaround:

  1. Log in to FusionInsight Manager.
  2. Choose Cluster > Services > Yarn > Configurations > All Configurations.
  3. In the navigation pane, choose ResourceManager(Role) > Customization.
  4. Add the following content to the resourcemanager.mapred-site.customized.configs parameter:

    • Name: Set it to mapreduce.job.map.output.collector.class.
    • Value: Set it to org.apache.hadoop.mapred.MapTask$MapOutputBuffer.

  5. Save the settings. Restart the expired service or instance for the configuration to take effect.
  6. After the modification is complete, download the client again or update the client configuration.
Parent topic: Common Issues About Yarn