Improving MapReduce Client Task Reliability

When the network is unstable or the cluster I/O and CPU are overloaded, client applications might encounter running failures.

Adjust the following parameters in the mapred-site.xml configuration file on the client to reduce the client application failure rate:

The mapred-site.xml configuration file is in the conf directory of the client installation path, for example, /opt/client/Yarn/config.

**Table 1** Parameter description
Parameter	Description	Default Value
mapreduce.reduce.shuffle.max-host-failures	Indicates the number of allowed failures of an MR task to read remote shuffle data in the Reduce process. When the number is set to be over 5, the client application failure rate can be reduced. This parameter applies to MRS 3.x.	5
mapreduce.client.submit.file.replication	Indicates the backup of job files on HDFS. MR tasks are dependent on the job files during running. When the number of backups is set to be over 10, the client application failure rate can be reduced.	10