Checking the Commissioning Result

Scenario

After a MapReduce application is run, you can check the running result through one of the following methods:

• Check the running status of the application in IntelliJ IDEA.
• Viewing MapReduce logs
• Logging in to the MapReduce web UI
• Logging in to the Yarn web UI

Contact the administrator to obtain a service account that has the right to access the web UI and its password.

Procedure

• Viewing running results to learn application running status

  View the application running status in the command output on the console.

  1848 [main] INFO  org.apache.hadoop.security.UserGroupInformation  - Login successful for user admin@<system domain name> using keytab file 
  Login success!!!!!!!!!!!!!!
  7093 [main] INFO  org.apache.hadoop.hdfs.PeerCache  - SocketCache disabled.
  9614 [main] INFO  org.apache.hadoop.hdfs.DFSClient  - Created HDFS_DELEGATION_TOKEN token 45 for admin on ha-hdfs:hacluster
  9709 [main] INFO  org.apache.hadoop.mapreduce.security.TokenCache  - Got dt for hdfs://hacluster; Kind: HDFS_DELEGATION_TOKEN,
  Service: ha-hdfs:hacluster, Ident: 
  (HDFS_DELEGATION_TOKEN token 45 for admin)
  10914 [main] INFO  org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider  - Failing over to 53
  12136 [main] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat  - Total input files to process : 2
  12731 [main] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - number of splits:2
  13405 [main] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Submitting tokens for job: job_1456738266914_0006
  13405 [main] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster, 
  Ident: (HDFS_DELEGATION_TOKEN token 45 for admin)
  16019 [main] INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl  - Application submission is not finished, 
  submitted application application_1456738266914_0006 is still in NEW
  16975 [main] INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl  - Submitted application application_1456738266914_0006
  17069 [main] INFO  org.apache.hadoop.mapreduce.Job  - The url to track the job: 
  https://linux2:8090/proxy/application_1456738266914_0006/
  17086 [main] INFO  org.apache.hadoop.mapreduce.Job  - Running job: job_1456738266914_0006
  29811 [main] INFO  org.apache.hadoop.mapreduce.Job  - Job job_1456738266914_0006 running in uber mode : false
  29811 [main] INFO  org.apache.hadoop.mapreduce.Job  -  map 0% reduce 0%
  41492 [main] INFO  org.apache.hadoop.mapreduce.Job  -  map 100% reduce 0%
  53161 [main] INFO  org.apache.hadoop.mapreduce.Job  -  map 100% reduce 100%
  53265 [main] INFO  org.apache.hadoop.mapreduce.Job  - Job job_1456738266914_0006 completed successfully
  53393 [main] INFO  org.apache.hadoop.mapreduce.Job  - Counters: 50

  

  The following exception may occur when the sample code is running in the Windows OS, but it will not affect services.

  java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

• Viewing application running status by using the MapReduce web UI

  Log in to FusionInsight Manager as a user who has the permission to view tasks. Choose Cluster > Service > Mapreduce > JobHistoryServer to go to the web page and view the task execution status.

  Figure 1 JobHistory web UI
  
• Viewing job execution status by using Yarn web UI
  Log in to FusionInsight Manager as a user who has the permission to view tasks. Choose Cluster > Service > Yarn > ResourceManager(Active) to go to the web page and view the task execution status.

  Figure 2 ResourceManager web UI
  
• Viewing MapReduce Logs to Obtain the Application Running Status

  View MapReduce logs to learn application running status, and adjust applications based on log information.