How Do I View the Resource Usage of DLI Spark Jobs?
Viewing the Configuration of a Spark Job
Log in to the DLI console. In the navigation pane, choose Job Management > Spark Jobs. In the job list, locate the target job and click next to Job ID to view the parameters of the job.
![](https://support.huaweicloud.com/intl/en-us/dli_faq/public_sys-resources/note_3.0-en-us.png)
The content is displayed only when the parameters in Advanced Settings are configured during Spark job creation. For details about how to create a Spark job, see Creating a Spark Job.
![Click to enlarge](https://support.huaweicloud.com/intl/en-us/dli_faq/en-us_image_0000001103932820.png)
Viewing Real-Time Resource Usage of a Spark Job
Perform the following operations to view the number of running CUs occupied by a Spark job in real time:
- Log in to the DLI console. In the navigation pane, choose Job Management > Spark Jobs. In the job list, locate the target job and click SparkUI in the Operation column.
- On the Spark UI page, view the real-time running resources of the Spark job.
Figure 2 SparkUI
- On the Spark UI page, view the original configuration of the Spark job (available only to new clusters).
On the Spark UI page, click Environment to view Driver and Executor information.
Figure 3 Driver informationFigure 4 Executor information
Job Development FAQs
- How Do I Use Spark to Write Data into a DLI Table?
- How Do I Set the AK/SK for a Queue to Operate an OBS Table?
- How Do I View the Resource Usage of DLI Spark Jobs?
- How Do I Use Python Scripts to Access the MySQL Database If the pymysql Module Is Missing from the Spark Job Results Stored in MySQL?
- How Do I Run a Complex PySpark Program in DLI?
- How Does a Spark Job Access a MySQL Database?
- How Do I Use JDBC to Set the spark.sql.shuffle.partitions Parameter to Improve the Task Concurrency?
- How Do I Read Uploaded Files for a Spark Jar Job?
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbotmore