Configuring Reserved Percentage of Disk Usage on DataNodes
Scenario
When the YARN local directory and DataNode directory are on the same disk, the larger disk can handle more tasks, storing more intermediate data in the YARN local directory.
You can configure the reserved disk space for DataNodes by setting dfs.datanode.du.reserved. Its default value is 0. Set a proper value based on the site requirements. A smaller value may not meet the requirements and a larger value may waste resources.
To avoid this problem, you can configure the percentage of the reserved disk space to the total disk space on MRS.
Configuration Description
- Log in to FusionInsight Manager.
For details about how to log in to FusionInsight Manager, see Accessing MRS Manager.
- Choose Cluster > Services > HDFS > Configurations > All Configurations.
- Search for the following parameters and change their values as required.
Table 1 Parameters Parameter
Description
Default Value
dfs.datanode.du.reserved.percentage
Percentage of the reserved disk space on DataNodes to the total disk space. The DataNode permanently reserves the disk space calculated using this percentage.
The value is an integer ranging from 0 to 100.
- If dfs.datanode.du.reserved.percentage and dfs.datanode.du.reserved are configured at the same time, the larger value of the two parameters is used as the percentage of the reserved DataNode space.
- You are advised to set the dfs.datanode.du.reserved or dfs.datanode.du.reserved.percentage parameter based on the disk space.
10
- Save the settings. Restart the expired service or instance for the configuration to take effect.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot