Checking the Commissioning Result

Scenario

After a MapReduce application is run, you can check the running result through one of the following methods:

• Viewing the command output.
• Logging in to the MapReduce WebUI
• Logging in to the Yarn WebUI
• Viewing MapReduce logs

You must have permission to access WebUI. If not, contact the admin to obtain an account and password.

Procedure

• Check the running result by using MapReduce WebUI.

  Log in to FusionInsight Manager as a user who has the permission to view tasks and choose Cluster > Name of the desired cluster > Services > Mapreduce > JobHistoryServer. On the web page that is displayed, view the task execution status.

  Figure 1 JobHistory Web UI
  
• Check the running result by using YARN WebUI.
  Log in to FusionInsight Manager as a user who has the permission to view tasks and choose Cluster > Name of the desired cluster > Services > Yarn > ResourceManager(Active). On the web page that is displayed, view the task execution status.

  Figure 2 ResourceManager Web UI
  

• Check the running result of the MapReduce application.
  • After running the yarn jar MRTest-XXX.jar command in the Linux environment, you can check the running status of the application by the returned information about the command.

    linux1:/opt # yarn jar MRTest-XXX.jar /user/mapred/example/input/ /output6
    16/02/24 15:45:40 INFO security.UserGroupInformation: Login successful for user admin@<system domain name> using keytab file user.keytab
    Login success!!!!!!!!!!!!!!
    16/02/24 15:45:40 INFO hdfs.PeerCache: SocketCache disabled.
    16/02/24 15:45:41 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 28 for admin on ha-hdfs:hacluster
    16/02/24 15:45:41 INFO security.TokenCache: Got dt for hdfs://hacluster; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster, 
    Ident: (HDFS_DELEGATION_TOKEN token 28 for admin)
    16/02/24 15:45:41 INFO input.FileInputFormat: Total input files to process : 2
    16/02/24 15:45:41 INFO mapreduce.JobSubmitter: number of splits:2
    16/02/24 15:45:42 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1455853029114_0027
    16/02/24 15:45:42 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster, 
    Ident: (HDFS_DELEGATION_TOKEN token 28 for admin)
    16/02/24 15:45:42 INFO impl.YarnClientImpl: Submitted application application_1455853029114_0027
    16/02/24 15:45:42 INFO mapreduce.Job: The url to track the job: https://linux1:8090/proxy/application_1455853029114_0027/
    16/02/24 15:45:42 INFO mapreduce.Job: Running job: job_1455853029114_0027
    16/02/24 15:45:50 INFO mapreduce.Job: Job job_1455853029114_0027 running in uber mode : false
    16/02/24 15:45:50 INFO mapreduce.Job:  map 0% reduce 0%
    16/02/24 15:45:56 INFO mapreduce.Job:  map 100% reduce 0%
    16/02/24 15:46:03 INFO mapreduce.Job:  map 100% reduce 100%
    16/02/24 15:46:03 INFO mapreduce.Job: Job job_1455853029114_0027 completed successfully
    16/02/24 15:46:03 INFO mapreduce.Job: Counters: 49

  • In the Linux environment, run the yarn application -status <ApplicationID> command to check the running result of the current application. Example:

    linux1:/opt # yarn application -status application_1455853029114_0027
    Application Report : 
            Application-Id : application_1455853029114_0027
            Application-Name : Collect Female Info
            Application-Type : MAPREDUCE
            User : admin
            Queue : default
            Start-Time : 1456299942302
            Finish-Time : 1456299962343
            Progress : 100%
            State : FINISHED
            Final-State : SUCCEEDED
            Tracking-URL : https://linux1:26014/jobhistory/job/job_1455853029114_0027
            RPC Port : 27100
            AM Host : SZV1000044726
            Aggregate Resource Allocation : 114106 MB-seconds, 42 vcore-seconds
            Log Aggregation Status : SUCCEEDED
            Diagnostics : Application finished execution. 
            Application Node Label Expression : <Not set>
            AM container Node Label Expression : <DEFAULT_PARTITION>

• View MapReduce logs to learn application running conditions.

  MapReduce logs offers immediate visibility into application running conditions. You can adjust application programs based on the logs.