Configuring Speculative Execution for MapReduce Tasks
Scenario
Clusters with hundreds or thousands of nodes are more prone to software and hardware issues, resulting in longer task execution times. This can cause tasks to be delayed until a malfunctioning device is fixed. In speculative execution, a task is allocated to multiple machines for running, and the node that completes the task first is selected. You can disable speculative execution for small clusters.
Procedure
Navigation path for setting parameters:
On the All Configurations page of the Yarn service, enter a parameter name in the search box. For details, see Modifying Cluster Service Configuration Parameters.
|
Parameter |
Description |
Default Value |
|---|---|---|
|
mapreduce.map.speculative |
Sets whether to execute multiple instances of some map tasks concurrently. true indicates that speculative execution is enabled. |
false |
|
mapreduce.reduce.speculative |
Sets whether to execute multiple instances of some reduce tasks concurrently. true indicates that speculative execution is enabled. |
false |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.
