Updated on 2024-08-10 GMT+08:00

Checking the Commissioning Result

Scenario

After a MapReduce application is run, you can check the running result through one of the following methods:

  • Viewing the command output.
  • Logging in to the MapReduce WebUI
  • Logging in to the Yarn WebUI
  • Viewing MapReduce logs

You must have permission to access WebUI. If not, contact the admin to obtain an account and password.

Procedure

  • Check the running result by using MapReduce WebUI.

    Log in to FusionInsight Manager as a user who has the permission to view tasks and choose Cluster > Name of the desired cluster > Services > Mapreduce > JobHistoryServer. On the web page that is displayed, view the task execution status.

    Figure 1 JobHistory Web UI
  • Check the running result by using YARN WebUI.
    Log in to FusionInsight Manager as a user who has the permission to view tasks and choose Cluster > Name of the desired cluster > Services > Yarn > ResourceManager(Active). On the web page that is displayed, view the task execution status.
    Figure 2 ResourceManager Web UI
  • Check the running result of the MapReduce application.
    • After running the yarn jar MRTest-XXX.jar command in the Linux environment, you can check the running status of the application by the returned information about the command.
      linux1:/opt # yarn jar MRTest-XXX.jar /user/mapred/example/input/ /output6
      16/02/24 15:45:40 INFO security.UserGroupInformation: Login successful for user admin@<system domain name> using keytab file user.keytab
      Login success!!!!!!!!!!!!!!
      16/02/24 15:45:40 INFO hdfs.PeerCache: SocketCache disabled.
      16/02/24 15:45:41 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 28 for admin on ha-hdfs:hacluster
      16/02/24 15:45:41 INFO security.TokenCache: Got dt for hdfs://hacluster; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster, 
      Ident: (HDFS_DELEGATION_TOKEN token 28 for admin)
      16/02/24 15:45:41 INFO input.FileInputFormat: Total input files to process : 2
      16/02/24 15:45:41 INFO mapreduce.JobSubmitter: number of splits:2
      16/02/24 15:45:42 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1455853029114_0027
      16/02/24 15:45:42 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster, 
      Ident: (HDFS_DELEGATION_TOKEN token 28 for admin)
      16/02/24 15:45:42 INFO impl.YarnClientImpl: Submitted application application_1455853029114_0027
      16/02/24 15:45:42 INFO mapreduce.Job: The url to track the job: https://linux1:8090/proxy/application_1455853029114_0027/
      16/02/24 15:45:42 INFO mapreduce.Job: Running job: job_1455853029114_0027
      16/02/24 15:45:50 INFO mapreduce.Job: Job job_1455853029114_0027 running in uber mode : false
      16/02/24 15:45:50 INFO mapreduce.Job:  map 0% reduce 0%
      16/02/24 15:45:56 INFO mapreduce.Job:  map 100% reduce 0%
      16/02/24 15:46:03 INFO mapreduce.Job:  map 100% reduce 100%
      16/02/24 15:46:03 INFO mapreduce.Job: Job job_1455853029114_0027 completed successfully
      16/02/24 15:46:03 INFO mapreduce.Job: Counters: 49
    • In the Linux environment, run the yarn application -status <ApplicationID> command to check the running result of the current application. Example:
      linux1:/opt # yarn application -status application_1455853029114_0027
      Application Report : 
              Application-Id : application_1455853029114_0027
              Application-Name : Collect Female Info
              Application-Type : MAPREDUCE
              User : admin
              Queue : default
              Start-Time : 1456299942302
              Finish-Time : 1456299962343
              Progress : 100%
              State : FINISHED
              Final-State : SUCCEEDED
              Tracking-URL : https://linux1:26014/jobhistory/job/job_1455853029114_0027
              RPC Port : 27100
              AM Host : SZV1000044726
              Aggregate Resource Allocation : 114106 MB-seconds, 42 vcore-seconds
              Log Aggregation Status : SUCCEEDED
              Diagnostics : Application finished execution. 
              Application Node Label Expression : <Not set>
              AM container Node Label Expression : <DEFAULT_PARTITION>
  • View MapReduce logs to learn application running conditions.

    MapReduce logs offers immediate visibility into application running conditions. You can adjust application programs based on the logs.