Viewing the Spark Application Commissioning Result

Scenario

After a Spark application is run, you can view the running result or view the application running status on the Spark web UI.

Procedure

• Viewing the running result of the Spark application

  The storage path and format of execution results are specified by the Spark application. You can obtain the results from the specified file.

• Viewing the running status of the Spark application

  Spark has two web UIs.

  • Spark UI: used to display the status of running applications.

    Spark UI consists of the following parts: Jobs, Stages, Storage, Environment, Executors, and SQL. The Streaming application has one more Streaming tab.

    Entry: Log in to MRS Manager by referring to Logging in to MRS Manager, choose Services > Yarn, and click ResourceManager corresponding to ResourceManager Web UI. On the page that is displayed, find the corresponding Spark application. Click ApplicationMaster in the last column of the application information. The Spark UI page is displayed.

  • History Server UI: used to display the running status of Spark applications that are complete or incomplete.

    History Server UI involves the application ID, application name, start time, end time, execution time, and owner information.

    Entry: Log in to MRS Manager by referring to Logging in to MRS Manager, choose Services > Spark, and click JobHistory corresponding to Spark Web UI.

• Viewing Spark logs to learn application running status

  View Spark logs to learn application running status, and adjust applications based on log information.