Checking the Commissioning Result

Updated on 2022-09-14 GMT+08:00

View PDF

Scenario

After a MapReduce application is run, you can check the running result through one of the following methods:

Viewing the program running status in IntelliJ IDEA.
Viewing MapReduce logs.
Logging in to the MapReduce WebUI.
Logging in to the Yarn WebUI.

NOTE:

You must have permission to access WebUI. If not, contact the admin to obtain an account and password.

Procedure

Check the running result by obtaining the MapReduce log.

As shown below, the console outputs the application running result.

1848 [main] INFO  org.apache.hadoop.security.UserGroupInformation  - Login successful for user admin@<system domain name> using keytab file 
Login success!!!!!!!!!!!!!!
7093 [main] INFO  org.apache.hadoop.hdfs.PeerCache  - SocketCache disabled.
9614 [main] INFO  org.apache.hadoop.hdfs.DFSClient  - Created HDFS_DELEGATION_TOKEN token 45 for admin on ha-hdfs:hacluster
9709 [main] INFO  org.apache.hadoop.mapreduce.security.TokenCache  - Got dt for hdfs://hacluster; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster, Ident: 
(HDFS_DELEGATION_TOKEN token 45 for admin)
10914 [main] INFO  org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider  - Failing over to 53
12136 [main] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat  - Total input files to process : 2
12731 [main] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - number of splits:2
13405 [main] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Submitting tokens for job: job_1456738266914_0006
13405 [main] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster, Ident: (HDFS_DELEGATION_TOKEN token 45 for admin)
16019 [main] INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl  - Application submission is not finished, 
submitted application application_1456738266914_0006 is still in NEW
16975 [main] INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl  - Submitted application application_1456738266914_0006
17069 [main] INFO  org.apache.hadoop.mapreduce.Job  - The url to track the job: https://linux2:8090/proxy/application_1456738266914_0006/
17086 [main] INFO  org.apache.hadoop.mapreduce.Job  - Running job: job_1456738266914_0006
29811 [main] INFO  org.apache.hadoop.mapreduce.Job  - Job job_1456738266914_0006 running in uber mode : false
29811 [main] INFO  org.apache.hadoop.mapreduce.Job  -  map 0% reduce 0%
41492 [main] INFO  org.apache.hadoop.mapreduce.Job  -  map 100% reduce 0%
53161 [main] INFO  org.apache.hadoop.mapreduce.Job  -  map 100% reduce 100%
53265 [main] INFO  org.apache.hadoop.mapreduce.Job  - Job job_1456738266914_0006 completed successfully
53393 [main] INFO  org.apache.hadoop.mapreduce.Job  - Counters: 50

NOTE:

In the Windows environment, the following exception occurs but does not affect services.

java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

Check the running result by using MapReduce WebUI.
Log in to FusionInsight Manager as a user who has the permission to view tasks and choose Cluster > Name of the desired cluster > Services > Mapreduce > JobHistoryServer. On the web page that is displayed, view the task execution status.

Figure 1 JobHistory Web UI
Check the running result by using YARN WebUI.
Log in to FusionInsight Manager as a user who has the permission to view tasks and choose Cluster > Name of the desired cluster > Services > Yarn > ResourceManager (Active). On the web page that is displayed, view the task execution status.
Figure 2 ResourceManager Web UI
View MapReduce logs to learn application running conditions.
MapReduce logs offers immediate visibility into application running conditions. You can adjust application programs based on the logs.