On this page

Checking the Commissioning Result

Updated on 2022-11-18 GMT+08:00

Scenario

After an HDFS application is run, you can learn the application running conditions by viewing the running result or HDFS logs.

Procedure

  • Learn the application running conditions by viewing the running result.
    • The running result of the HDFS windows example application is shown as follows:
      1654 [main] WARN  org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory  - The short-circuit local reads feature cannot be used because UNIX Domain sockets are not available on Windows.
      2013 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples
      2137 [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      2590 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
      3245 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
      4447 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
      4447 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
      4509 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples\test.txt
      4618 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples
      4743 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples/hdfs_example_1
      4743 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples/hdfs_example_0
      5087 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
      5087 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
      6507 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
      6553 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
      7505 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
      7505 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
      7568 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples/hdfs_example_1\test.txt
      7583 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
      7583 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
      7630 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples/hdfs_example_0\test.txt
      7677 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples/hdfs_example_1
      7739 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples/hdfs_example_0
      NOTE:

      In the Windows environment, the following exception occurs but does not affect services.

      java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

    • The running result of the Colocation windows example application is shown as follows:
      1623 [main] WARN  org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory  - The short-circuit local reads feature cannot be used because UNIX Domain sockets are not available on Windows.
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:zookeeper.version=V100R002C30, built on 10/19/2017 04:21 GMT
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:host.name=siay7user1.china.huawei.com
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.version=1.8.0_131
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.vendor=Oracle Corporation
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.home=D:\Program Files\Java\jre1.8.0_131
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.class.path=D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\bin;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-cli-1.2.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-codec-1.4.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-collections-3.2.2.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-configuration-1.6.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-io-2.4.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-lang-2.6.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-logging-1.1.3.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\dynalogger-V100R002C30.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\guava-11.0.2.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-annotations-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-auth-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-common-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-hdfs-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-hdfs-client-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-hdfs-colocation-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-hdfs-datamovement-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-hdfs-nfs-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-hdfs-restore-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-mapreduce-client-core-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-nfs-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\htrace-core-3.1.0-incubating.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\log4j-1.2.17.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\protobuf-java-2.5.0.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\slf4j-api-1.7.10.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\slf4j-log4j12-1.7.10.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\zookeeper-3.5.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\SmallFS\FusionInsight-SmallFS-1.0.0.tar.gz\smallfs\share\datasight\smallfs\smallfs-common-V100R002C30.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\SmallFS\FusionInsight-SmallFS-1.0.0.tar.gz\smallfs\share\datasight\smallfs\smallfs-main-V100R002C30.jar
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.library.path=D:\Program Files\Java\jre1.8.0_131\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;D:/Program Files/Java/jre1.8.0_131/bin/server;D:/Program Files/Java/jre1.8.0_131/bin;D:/Program Files/Java/jre1.8.0_131/lib/amd64;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;D:\Program Files\Java\jdk1.8.0_131\bin;D:\Program Files\Java\jdk1.8.0_131\jre\bin;D:\Program Files (x86)\GitExtensions\;D:\Program Files\Git\cmd;D:\soft\apache-maven-3.2.2\bin;D:\soft\gnubin;D:\soft\protoc-2.5.0-win32;;D:\soft\TMSS;D:\installation package\eclipse-jee-mars-R-win32-x86_64\eclipse;;.
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.io.tmpdir=C:\Users\L00430~1\AppData\Local\Temp\
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.compiler=<NA>
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.name=Windows 7
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.arch=amd64
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.version=6.1
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:user.name=user
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:user.home=C:\Users\user
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:user.dir=D:\FIClient\nonSafety\FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig\HDFS\hdfs-example-normal
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.memory.free=107MB
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.memory.max=1819MB
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.memory.total=123MB
      1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Initiating client connection, connectString=192-168-32-144:2181,192-168-32-67:2181,192-168-33-190:2181 sessionTimeout=45000 watcher=com.huawei.hadoop.oi.colocation.ZooKeeperWatcher@5f9b2141
      1794 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.request.timeout is not configured. Using default value 120000.
      1794 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.client.bind.port.range is not configured.
      1794 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.client.bind.address is not configured.
      1794 [main-SendThread(192-168-32-67:2181)] INFO  org.apache.zookeeper.client.FourLetterWordMain  - connecting to 192-168-32-67 2181
      1904 [main-SendThread(192-168-32-67:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Got server principal from the server and it is null
      1904 [main-SendThread(192-168-32-67:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Using server principal zookeeper/192-168-32-67
      1904 [main-SendThread(192-168-32-67:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Opening socket connection to server 192-168-32-67/192.168.32.67:2181. Will not attempt to authenticate using SASL (unknown error)
      1966 [main-SendThread(192-168-32-67:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Socket connection established, initiating session, client: /192.168.35.189:50954, server: 192-168-32-67/192.168.32.67:2181
      2029 [main-SendThread(192-168-32-67:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Session establishment complete on server 192-168-32-67/192.168.32.67:2181, sessionid = 0x13000074b7e464b7, negotiated timeout = 45000
      2169 [main] INFO  com.huawei.hadoop.oi.colocation.ZKUtil  - ZooKeeper colocation znode : /hadoop/colocationDetails. Will publish colocation details under this znode hierarchy.
      Create Group is running...
      5212 [main] INFO  org.apache.zookeeper.ZooKeeper  - Initiating client connection, connectString=192-168-32-144:2181,192-168-32-67:2181,192-168-33-190:2181 sessionTimeout=45000 watcher=com.huawei.hadoop.oi.colocation.ZooKeeperWatcher@2438dcd
      5212 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.request.timeout is not configured. Using default value 120000.
      5212 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.client.bind.port.range is not configured.
      5212 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.client.bind.address is not configured.
      5212 [main-SendThread(192-168-33-190:2181)] INFO  org.apache.zookeeper.client.FourLetterWordMain  - connecting to 192-168-33-190 2181
      5321 [main-SendThread(192-168-33-190:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Got server principal from the server and it is null
      5321 [main-SendThread(192-168-33-190:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Using server principal zookeeper/192-168-33-190
      5321 [main-SendThread(192-168-33-190:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Opening socket connection to server 192-168-33-190/192.168.33.190:2181. Will not attempt to authenticate using SASL (unknown error)
      5368 [main-SendThread(192-168-33-190:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Socket connection established, initiating session, client: /192.168.35.189:50962, server: 192-168-33-190/192.168.33.190:2181
      5430 [main-SendThread(192-168-33-190:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Session establishment complete on server 192-168-33-190/192.168.33.190:2181, sessionid = 0x14000073f13b657b, negotiated timeout = 45000
      5540 [main] INFO  com.huawei.hadoop.oi.colocation.ZKUtil  - ZooKeeper colocation znode : /hadoop/colocationDetails. Will publish colocation details under this znode hierarchy.
      Create Group has finished.
      Put file is running...
      5930 [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      Put file has finished.
      Delete file is running...
      Delete file has finished.
      Delete Group is running...
      Delete Group has finished.
      6866 [main] INFO  org.apache.zookeeper.ZooKeeper  - Session: 0x13000074b7e464b7 closed
      6866 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn  - EventThread shut down for session: 0x13000074b7e464b7
      6928 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn  - EventThread shut down for session: 0x14000073f13b657b
      6928 [main] INFO  org.apache.zookeeper.ZooKeeper  - Session: 0x14000073f13b657b closed
    • (Optional) The running result of the SmallFS Windows example application is shown as follows:
      2513 [main] WARN  org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory  - The short-circuit local reads feature cannot be used because UNIX Domain sockets are not available on Windows.
      3465 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples
      3667 [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      4214 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
      4885 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
      6242 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
      6242 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
      6320 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples\test.txt
      6445 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples
      6663 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples/hdfs_example_1
      6679 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples/hdfs_example_0
      7069 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
      7069 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
      8614 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
      8754 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
      9566 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
      9566 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
      9644 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples/hdfs_example_1\test.txt
      9706 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
      9706 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
      9753 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples/hdfs_example_0\test.txt
      9784 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples/hdfs_example_1
      9894 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples/hdfs_example_0
  • Learn the application running conditions by viewing HDFS logs.

    The NameNode logs of HDFS offer immediate visibility into application running conditions. You can adjust application programs based on the logs.

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback