Spark on OBS Performance Tuning
Scenarios
In the scenario where a small number of requests are frequently sent from Spark on OBS to OBS, you can disable OBS monitoring to improve performance.
Configuration
- Install the Spark client.
For details, see Installing a Client.
- Log in to the Spark client node as the client installation user.
Modify the following parameters in the Client installation directory/Spark/spark/conf/core-site.xml file on the Spark client.
Table 1 Parameter description Parameter
Description
Default Value
fs.obs.metrics.switch
Specifies whether to report OBS monitoring metrics.
- true: enable
- false: disable
true
fs.obs.metrics.consumer
Specifies the processing mode of OBS monitoring metrics.
- org.apache.hadoop.fs.obs.metrics.OBSAMetricsProvider: indicates that OBS monitoring metrics are collected.
- org.apache.hadoop.fs.obs.DefaultMetricsConsumer: indicates that OBS monitoring metrics are not collected.
To use the OBS monitoring function, ensure that the function of reporting OBS monitoring metrics is enabled.
org.apache.hadoop.fs.obs.metrics.OBSAMetricsProvider
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot