Checking the Commissioning Result

Updated on 2022-09-14 GMT+08:00

View PDF

Scenario

After a MapReduce application is run, you can check the running result through one of the following methods:

Viewing the command output.
Logging in to the MapReduce WebUI
Logging in to the Yarn WebUI
Viewing MapReduce logs

NOTE:

You must have permission to access WebUI. If not, contact the admin to obtain an account and password.

Procedure

Check the running result by using MapReduce WebUI.
Log in to FusionInsight Manager as a user who has the permission to view tasks and choose Cluster > Name of the desired cluster > Services > Mapreduce > JobHistoryServer. On the web page that is displayed, view the task execution status.

Figure 1 JobHistory Web UI
Check the running result by using YARN WebUI.
Log in to FusionInsight Manager as a user who has the permission to view tasks and choose Cluster > Name of the desired cluster > Services > Yarn > ResourceManager(Active). On the web page that is displayed, view the task execution status.
Figure 2 ResourceManager Web UI

Check the running result of the MapReduce application.

After running the yarn jar MRTest-XXX.jar command in the Linux environment, you can check the running status of the application by the returned information about the command.

linux1:/opt # yarn jar MRTest-XXX.jar /user/mapred/example/input/ /output6
16/02/24 15:45:40 INFO security.UserGroupInformation: Login successful for user admin@<system domain name> using keytab file user.keytab
Login success!!!!!!!!!!!!!!
16/02/24 15:45:40 INFO hdfs.PeerCache: SocketCache disabled.
16/02/24 15:45:41 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 28 for admin on ha-hdfs:hacluster
16/02/24 15:45:41 INFO security.TokenCache: Got dt for hdfs://hacluster; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster, 
Ident: (HDFS_DELEGATION_TOKEN token 28 for admin)
16/02/24 15:45:41 INFO input.FileInputFormat: Total input files to process : 2
16/02/24 15:45:41 INFO mapreduce.JobSubmitter: number of splits:2
16/02/24 15:45:42 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1455853029114_0027
16/02/24 15:45:42 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster, 
Ident: (HDFS_DELEGATION_TOKEN token 28 for admin)
16/02/24 15:45:42 INFO impl.YarnClientImpl: Submitted application application_1455853029114_0027
16/02/24 15:45:42 INFO mapreduce.Job: The url to track the job: https://linux1:8090/proxy/application_1455853029114_0027/
16/02/24 15:45:42 INFO mapreduce.Job: Running job: job_1455853029114_0027
16/02/24 15:45:50 INFO mapreduce.Job: Job job_1455853029114_0027 running in uber mode : false
16/02/24 15:45:50 INFO mapreduce.Job:  map 0% reduce 0%
16/02/24 15:45:56 INFO mapreduce.Job:  map 100% reduce 0%
16/02/24 15:46:03 INFO mapreduce.Job:  map 100% reduce 100%
16/02/24 15:46:03 INFO mapreduce.Job: Job job_1455853029114_0027 completed successfully
16/02/24 15:46:03 INFO mapreduce.Job: Counters: 49

In the Linux environment, run the yarn application -status <ApplicationID> command to check the running result of the current application. Example:

linux1:/opt # yarn application -status application_1455853029114_0027
Application Report : 
        Application-Id : application_1455853029114_0027
        Application-Name : Collect Female Info
        Application-Type : MAPREDUCE
        User : admin
        Queue : default
        Start-Time : 1456299942302
        Finish-Time : 1456299962343
        Progress : 100%
        State : FINISHED
        Final-State : SUCCEEDED
        Tracking-URL : https://linux1:26014/jobhistory/job/job_1455853029114_0027
        RPC Port : 27100
        AM Host : SZV1000044726
        Aggregate Resource Allocation : 114106 MB-seconds, 42 vcore-seconds
        Log Aggregation Status : SUCCEEDED
        Diagnostics : Application finished execution. 
        Application Node Label Expression : <Not set>
        AM container Node Label Expression : <DEFAULT_PARTITION>

View MapReduce logs to learn application running conditions.
MapReduce logs offers immediate visibility into application running conditions. You can adjust application programs based on the logs.