Help Center/ MapReduce Service/ Developer Guide (LTS)/ Spark2x Development Guide (Normal Mode)/ Commissioning the Application/ Commissioning an Application in Linux/ Checking the Commissioning Result
Updated on 2022-11-18 GMT+08:00
View PDF

Checking the Commissioning Result

Scenario

After a Spark application is run, you can check the running result through one of the following methods:

  • Viewing the command output.
  • Logging in to the Spark WebUI.
  • Viewing Spark logs.

Procedure

  • Check the operating result data of the Spark application.

    The data storage directory and format are specified by users in the Spark application. You can obtain the data in the specified file.

  • Check the status of the Spark application.
    The Spark contains the following two Web UIs:
    • The Spark UI displays the status of applications being executed.

      The Spark UI contains the Spark Jobs, Spark Stages, Storage, Environment, and Executors parts. Besides these parts, Streaming is displayed for the Streaming application.

      Access to the interface: On the Web UI of the YARN, find the corresponding Spark application, and click the final column ApplicationMaster of the application information to access the Spark UI.

    • The History Server UI displays the status of all Spark applications.

      The History Server UI displays information such as the application ID, application name, start time, end time, execution time, and user to whom the application belongs. After the application ID is clicked, the Spark UI of the application is displayed.

  • View Spark logs to learn application running conditions.

    The logs of Spark offers immediate visibility into application running conditions. You can adjust application programs based on the logs.