Help Center/ MapReduce Service/ Developer Guide (LTS)/ MapReduce Development Guide (Security Mode)/ Commissioning the Application/ Commissioning the Application in the Windows Environment/ Checking the Commissioning Result
Updated on 2024-08-10 GMT+08:00
View PDF
Share

Checking the Commissioning Result

Scenario

After a MapReduce application is run, you can check the running result through one of the following methods:

  • Viewing the program running status in IntelliJ IDEA.
  • Viewing MapReduce logs.
  • Logging in to the MapReduce WebUI.
  • Logging in to the Yarn WebUI.

You must have permission to access WebUI. If not, contact the admin to obtain an account and password.

Procedure

  • Check the running result by obtaining the MapReduce log.

    As shown below, the console outputs the application running result.

    1848 [main] INFO  org.apache.hadoop.security.UserGroupInformation  - Login successful for user admin@<system domain name> using keytab file 
Login success!!!!!!!!!!!!!!
7093 [main] INFO  org.apache.hadoop.hdfs.PeerCache  - SocketCache disabled.
9614 [main] INFO  org.apache.hadoop.hdfs.DFSClient  - Created HDFS_DELEGATION_TOKEN token 45 for admin on ha-hdfs:hacluster
9709 [main] INFO  org.apache.hadoop.mapreduce.security.TokenCache  - Got dt for hdfs://hacluster; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster, Ident: 
(HDFS_DELEGATION_TOKEN token 45 for admin)
10914 [main] INFO  org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider  - Failing over to 53
12136 [main] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat  - Total input files to process : 2
12731 [main] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - number of splits:2
13405 [main] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Submitting tokens for job: job_1456738266914_0006
13405 [main] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:hacluster, Ident: (HDFS_DELEGATION_TOKEN token 45 for admin)
16019 [main] INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl  - Application submission is not finished, 
submitted application application_1456738266914_0006 is still in NEW
16975 [main] INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl  - Submitted application application_1456738266914_0006
17069 [main] INFO  org.apache.hadoop.mapreduce.Job  - The url to track the job: https://linux2:8090/proxy/application_1456738266914_0006/
17086 [main] INFO  org.apache.hadoop.mapreduce.Job  - Running job: job_1456738266914_0006
29811 [main] INFO  org.apache.hadoop.mapreduce.Job  - Job job_1456738266914_0006 running in uber mode : false
29811 [main] INFO  org.apache.hadoop.mapreduce.Job  -  map 0% reduce 0%
41492 [main] INFO  org.apache.hadoop.mapreduce.Job  -  map 100% reduce 0%
53161 [main] INFO  org.apache.hadoop.mapreduce.Job  -  map 100% reduce 100%
53265 [main] INFO  org.apache.hadoop.mapreduce.Job  - Job job_1456738266914_0006 completed successfully
53393 [main] INFO  org.apache.hadoop.mapreduce.Job  - Counters: 50

    In the Windows environment, the following exception occurs but does not affect services.

    java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

  • Check the running result by using MapReduce WebUI.

    Log in to FusionInsight Manager as a user who has the permission to view tasks and choose Cluster > Name of the desired cluster > Services > Mapreduce > JobHistoryServer. On the web page that is displayed, view the task execution status.

    Figure 1 JobHistory Web UI
  • Check the running result by using YARN WebUI.
    Log in to FusionInsight Manager as a user who has the permission to view tasks and choose Cluster > Name of the desired cluster > Services > Yarn > ResourceManager (Active). On the web page that is displayed, view the task execution status.
    Figure 2 ResourceManager Web UI
  • View MapReduce logs to learn application running conditions.

    MapReduce logs offers immediate visibility into application running conditions. You can adjust application programs based on the logs.