Configuring the Number of Lost Executors Displayed on the Web UI
Scenario
On the Spark web UI, the Executor page can display information about lost executors. Executors of JDBCServer long tasks are dynamically recycled. Therefore, the number of displayed lost executors must be configured to fit the Executor page.
Configuration Description
- Install the Spark client.
For details, see Installing a Client.
- Log in to the Spark client node as the client installation user.
Modify the following parameters in the {Client installation directory}/Spark/spark/conf/spark-defaults.conf file on the Spark client.
Table 1 Parameter description Parameter
Description
Example Value
spark.ui.retainedDeadExecutors
Specifies the maximum number of lost executors displayed on the Spark web UI.
When an executor exits because a task is complete, resources are released, or an exception occurs, the Spark web UI retains the executor information for monitoring and analysis. This parameter specifies the maximum number of historical executor records that can be retained.
100
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot
