Help Center> MapReduce Service> Developer Guide (3.x)> HDFS Development Guide (Security Mode)> Commissioning the Application> Commissioning an Application in the Windows Environment> Checking the Commissioning Result
View PDF

Checking the Commissioning Result

Scenario

After an HDFS application is run, you can learn the application running conditions by viewing the running result or HDFS logs.

Procedure

  • Learn the application running conditions by viewing the running result.
    • The running result of the HDFS windows example application is shown as follows: 
      1308 [main] INFO  org.apache.hadoop.security.UserGroupInformation  - Login successful for user hdfsDevelop using keytab file 
1308 [main] INFO  com.huawei.hadoop.security.LoginUtil  - Login success!!!!!!!!!!!!!!
2040 [main] WARN  org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory  - The short-circuit local reads feature cannot be used because UNIX Domain sockets are not available on Windows.
3006 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples
3131 [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
3598 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
4408 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
5015 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
5015 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
5077 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples\test.txt
5186 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples
5311 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples/hdfs_example_0
5311 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples/hdfs_example_1
5669 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
5669 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
7258 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
7741 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
7896 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
7896 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
7959 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples/hdfs_example_1\test.txt
8068 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples/hdfs_example_1
8364 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
8364 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
8426 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples/hdfs_example_0\test.txt
8535 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples/hdfs_example_0

      In the Windows environment, the following exception occurs but does not affect services.

      java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

    • The running result of the Colocation windows example application is shown as follows: 
      1417 [main] INFO  org.apache.hadoop.security.UserGroupInformation  - Login successful for user hdfsDevelop using keytab file 
1417 [main] INFO  com.huawei.hadoop.security.LoginUtil  - Login success!!!!!!!!!!!!!!
2102 [main] WARN  org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory  - The short-circuit local reads feature cannot be used because UNIX Domain sockets are not available on Windows.
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:zookeeper.version=V100R002C30, built on 10/19/2017 04:21 GMT
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:host.name=siay7l004300541.china.huawei.com
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.version=1.8.0_131
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.vendor=Oracle Corporation
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.home=D:\Program Files\Java\jre1.8.0_131
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.class.path=D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\bin;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\commons-cli-1.2.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\commons-codec-1.4.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\commons-collections-3.2.2.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\commons-configuration-1.6.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\commons-io-2.4.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\commons-lang-2.6.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\commons-logging-1.1.3.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\dynalogger-V100R002C30.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\guava-11.0.2.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\hadoop-annotations-3.1.1.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\hadoop-auth-3.1.1.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\hadoop-common-3.1.1.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\hadoop-hdfs-3.1.1.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\hadoop-hdfs-client-3.1.1.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\hadoop-hdfs-colocation-3.1.1.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\hadoop-hdfs-datamovement-3.1.1.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\hadoop-hdfs-nfs-3.1.1.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\hadoop-hdfs-restore-3.1.1.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\hadoop-mapreduce-client-core-3.1.1.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\hadoop-nfs-3.1.1.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\htrace-core-3.1.0-incubating.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\log4j-1.2.17.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\protobuf-java-2.5.0.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\slf4j-api-1.7.10.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\slf4j-log4j12-1.7.10.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security\lib\zookeeper-3.5.1.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\SmallFS\FusionInsight-SmallFS-1.0.0.tar.gz\smallfs\share\datasight\smallfs\smallfs-common-V100R002C30.jar;D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\SmallFS\FusionInsight-SmallFS-1.0.0.tar.gz\smallfs\share\datasight\smallfs\smallfs-main-V100R002C30.jar
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.library.path=D:\Program Files\Java\jre1.8.0_131\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;D:/Program Files/Java/jre1.8.0_131/bin/server;D:/Program Files/Java/jre1.8.0_131/bin;D:/Program Files/Java/jre1.8.0_131/lib/amd64;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;D:\Program Files\Java\jdk1.8.0_131\bin;D:\Program Files\Java\jdk1.8.0_131\jre\bin;D:\Program Files (x86)\GitExtensions\;D:\Program Files\Git\cmd;D:\soft\apache-maven-3.2.2\bin;D:\soft\gnubin;D:\soft\protoc-2.5.0-win32;;D:\soft\TMSS;D:\installation package\eclipse-jee-mars-R-win32-x86_64\eclipse;;.
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.io.tmpdir=C:\Users\L00430~1\AppData\Local\Temp\
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.compiler=<NA>
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.name=Windows 7
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.arch=amd64
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.version=6.1
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:user.name=l00430054
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:user.home=C:\Users\l00430054
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:user.dir=D:\FIClient\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-security
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.memory.free=99MB
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.memory.max=1819MB
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.memory.total=123MB
2149 [main] INFO  org.apache.zookeeper.ZooKeeper  - Initiating client connection, connectString=192-168-33-94:2181,192-168-34-145:2181,192-168-35-218:2181 sessionTimeout=45000 watcher=com.huawei.hadoop.oi.colocation.ZooKeeperWatcher@58e1d9d
2196 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.request.timeout is not configured. Using default value 120000.
2196 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.client.bind.port.range is not configured.
2196 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.client.bind.address is not configured.
2211 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.client.FourLetterWordMain  - connecting to 192-168-33-94 2181
2320 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Got server principal from the server and it is zookeeper/hadoop.<system domain name>
2320 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Using server principal zookeeper/hadoop.<system domain name>
2429 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.Login  - successfully logged in.
2429 [Thread-3] INFO  org.apache.zookeeper.Login  - TGT refresh thread started.
2445 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.client.ZooKeeperSaslClient  - Client will use GSSAPI as SASL mechanism.
2445 [Thread-3] INFO  org.apache.zookeeper.Login  - TGT valid starting at:        Wed Oct 25 20:30:47 GMT+08:00 2017
2445 [Thread-3] INFO  org.apache.zookeeper.Login  - TGT expires:                  Thu Oct 26 20:30:47 GMT+08:00 2017
2445 [Thread-3] INFO  org.apache.zookeeper.Login  - TGT refresh sleeping until: Thu Oct 26 16:51:02 GMT+08:00 2017
2460 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Opening socket connection to server 192-168-33-94/192.168.33.94:2181. Will attempt to SASL-authenticate using Login Context section 'Client'
2507 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Socket connection established, initiating session, client: /192.168.35.189:51125, server: 192-168-33-94/192.168.33.94:2181
2569 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Session establishment complete on server 192-168-33-94/192.168.33.94:2181, sessionid = 0x12001da5db861647, negotiated timeout = 45000
2881 [main] INFO  com.huawei.hadoop.oi.colocation.ZKUtil  - ZooKeeper colocation znode : /hadoop/colocationDetails. Will publish colocation details under this znode hierarchy.
Create Group is running...
6821 [main] INFO  org.apache.zookeeper.ZooKeeper  - Initiating client connection, connectString=192-168-33-94:2181,192-168-34-145:2181,192-168-35-218:2181 sessionTimeout=45000 watcher=com.huawei.hadoop.oi.colocation.ZooKeeperWatcher@74bf1791
6821 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.request.timeout is not configured. Using default value 120000.
6821 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.client.bind.port.range is not configured.
6821 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.client.bind.address is not configured.
6821 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.client.FourLetterWordMain  - connecting to 192-168-33-94 2181
6930 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Got server principal from the server and it is zookeeper/hadoop.<system domain name>
6930 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Using server principal zookeeper/hadoop.<system domain name>
7039 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.Login  - successfully logged in.
7039 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.client.ZooKeeperSaslClient  - Client will use GSSAPI as SASL mechanism.
7039 [Thread-7] INFO  org.apache.zookeeper.Login  - TGT refresh thread started.
7039 [Thread-7] INFO  org.apache.zookeeper.Login  - TGT valid starting at:        Wed Oct 25 20:30:51 GMT+08:00 2017
7039 [Thread-7] INFO  org.apache.zookeeper.Login  - TGT expires:                  Thu Oct 26 20:30:51 GMT+08:00 2017
7039 [Thread-7] INFO  org.apache.zookeeper.Login  - TGT refresh sleeping until: Thu Oct 26 15:53:00 GMT+08:00 2017
7039 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Opening socket connection to server 192-168-33-94/192.168.33.94:24002. Will attempt to SASL-authenticate using Login Context section 'Client'
7102 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Socket connection established, initiating session, client: /192.168.35.189:51134, server: 192-168-33-94/192.168.33.94:2181
7164 [main-SendThread(192-168-33-94:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Session establishment complete on server 192-168-33-94/192.168.33.94:2181, sessionid = 0x12001da5db861648, negotiated timeout = 45000
7444 [main] INFO  com.huawei.hadoop.oi.colocation.ZKUtil  - ZooKeeper colocation znode : /hadoop/colocationDetails. Will publish colocation details under this znode hierarchy.
Create Group has finished.
Put file is running...
9438 [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Put file has finished.
Delete file is running...
Delete file has finished.
Delete Group is running...
Delete Group has finished.
10403 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn  - EventThread shut down for session: 0x12001da5db861647
10403 [main] INFO  org.apache.zookeeper.ZooKeeper  - Session: 0x12001da5db861647 closed
10450 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn  - EventThread shut down for session: 0x12001da5db861648
10450 [main] INFO  org.apache.zookeeper.ZooKeeper  - Session: 0x12001da5db861648 closed.
    • .(Optional) The running result of the SmallFS Windows example application is shown as follows: 
      1310 [main] INFO  org.apache.hadoop.security.UserGroupInformation  - Login successful for user hdfsDevelop using keytab file 
1310 [main] INFO  com.huawei.hadoop.security.LoginUtil  - Login success!!!!!!!!!!!!!!
2012 [main] WARN  org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory  - The short-circuit local reads feature cannot be used because UNIX Domain sockets are not available on Windows.
3057 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples
3914 [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
4382 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
5848 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
6862 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
6862 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
6940 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples\test.txt
7127 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples
7345 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples/hdfs_example_0
7345 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples/hdfs_example_1
7767 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
7767 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
9045 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
9482 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
9700 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
9700 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
9763 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples/hdfs_example_1\test.txt
9919 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples/hdfs_example_1
10153 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
10153 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
10231 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples/hdfs_example_0\test.txt
10387 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples/hdfs_example_0
  • Learn the application running conditions by viewing HDFS logs.

    The NameNode logs of HDFS offer immediate visibility into application running conditions. You can adjust application programs based on the logs.