在Linux环境中调测HDFS应用

操作场景

HDFS应用程序支持在Linux环境中运行。在程序代码完成开发后，可以上传Jar包至准备好的Linux环境中运行。

HDFS应用程序运行完成后，可直接通过运行结果查看应用程序运行情况，也可以通过HDFS日志获取应用运行情况。

前提条件

已安装客户端时：
- 已安装HDFS客户端。
- 当客户端所在主机不是集群中的节点时，需要在客户端所在节点的hosts文件中设置主机名和IP地址映射。主机名和IP地址请保持一一对应。
未安装客户端时：
- Linux环境已安装JDK，版本号需要和IDEA导出Jar包使用的JDK版本一致。
- 当Linux环境所在主机不是集群中的节点时，需要在Linux环境所在节点的hosts文件中设置主机名和IP地址映射。主机名和IP地址请保持一一对应。

已安装客户端时编译并运行程序

进入样例工程本地根目录，在Windows命令提示符窗口中执行下面命令进行打包。

mvn -s "{maven_setting_path}" clean package
- 上述打包命令中的{maven_setting_path}为本地Maven的“settings.xml”文件路径。
- 打包成功之后，在工程根目录的target子目录下获取打好的jar包，例如“HDFSTest-XXX.jar”，jar包名称以实际打包结果为准。
将导出的Jar包上传至Linux客户端运行环境的任意目录下，例如“/opt/client”。
配置环境变量：

cd /opt/client

source bigdata_env
运行此样例代码需要设置运行用户，设置运行用户有两种方式，添加环境变量HADOOP_USER_NAME或者修改代码设置运行用户。若在没有修改代码的场景下，执行以下语句添加环境变量：
```
export HADOOP_USER_NAME=test
```
用户可向管理员咨询运行用户。test在这里只是举例，若需运行Colocation相关操作的样例代码，则此用户需属supergroup用户组。
执行如下命令，运行Jar包。

hadoop jar HDFSTest-XXX.jar com.huawei.bigdata.hdfs.examples.HdfsExample

hadoop jar HDFSTest-XXX.jar com.huawei.bigdata.hdfs.examples.ColocationExample

在运行com.huawei.bigdata.hdfs.examples.ColocationExample时，HDFS的配置项“fs.defaultFS”不能配置为“viewfs://ClusterX”。

未安装客户端时编译并运行程序

进入工程本地根目录，在Windows命令提示符窗口中执行下面命令进行打包。

mvn -s "{maven_setting_path}" clean package
- 上述打包命令中的{maven_setting_path}为本地Maven的“settings.xml”文件路径。
- 打包成功之后，在工程根目录的target子目录下获取打好的jar包。
将导出的Jar包上传至Linux运行环境的任意目录下，例如“/optclient”。
将工程中的“lib”文件夹和“conf”文件夹上传至和Jar包相同的Linux运行环境目录下，例如“/opt/client”（其中“lib”目录汇总包含了工程中依赖的所有的Jar包，“conf”目录包含运行jar包所需的集群相关配置文件，请参考准备运行环境）。
运行此样例代码需要设置运行用户，设置运行用户有两种方式，添加环境变量HADOOP_USER_NAME或者修改代码设置运行用户。若在没有修改代码的场景下，执行以下语句添加环境变量：
```
export HADOOP_USER_NAME=test
```
用户可向管理员咨询运行用户。test在这里只是举例，若需运行Colocation相关操作的样例代码，则此用户需属supergroup用户组。
执行如下命令运行Jar包。

java -cp HDFSTest-XXX.jar:conf/:lib/* com.huawei.bigdata.hdfs.examples.HdfsExample

java -cp HDFSTest-XXX.jar:conf/:lib/* com.huawei.bigdata.hdfs.examples.ColocationExample

在运行com.huawei.bigdata.hdfs.examples.ColocationExample:时，HDFS的配置项“fs.defaultFS”不能配置为“viewfs://ClusterX”。

查看调测结果

查看运行结果获取应用运行情况

HdfsExample Linux样例程序运行结果如下所示。

[root@192-168-32-144 client]#hadoop jar HDFSTest-XXX.jar com.huawei.bigdata.hdfs.examples.HdfsExample
WARNING: Use "yarn jar" to launch YARN applications.
17/10/26 19:11:44 INFO examples.HdfsExample: success to create path /user/hdfs-examples
17/10/26 19:11:44 INFO examples.HdfsExample: success to write.
17/10/26 19:11:45 INFO examples.HdfsExample: success to append.
17/10/26 19:11:45 INFO examples.HdfsExample: result is : hi, I am bigdata. It is successful if you can see me.I append this content.
17/10/26 19:11:45 INFO examples.HdfsExample: success to read.
17/10/26 19:11:45 INFO examples.HdfsExample: success to delete the file /user/hdfs-examples/test.txt
17/10/26 19:11:45 INFO examples.HdfsExample: success to delete path /user/hdfs-examples
17/10/26 19:11:45 INFO examples.HdfsExample: success to create path /user/hdfs-examples/hdfs_example_1
17/10/26 19:11:45 INFO examples.HdfsExample: success to create path /user/hdfs-examples/hdfs_example_0
17/10/26 19:11:45 INFO examples.HdfsExample: success to write.
17/10/26 19:11:45 INFO examples.HdfsExample: success to write.
17/10/26 19:11:46 INFO examples.HdfsExample: success to append.
17/10/26 19:11:46 INFO examples.HdfsExample: result is : hi, I am bigdata. It is successful if you can see me.I append this content.
17/10/26 19:11:46 INFO examples.HdfsExample: success to read.
17/10/26 19:11:46 INFO examples.HdfsExample: success to delete the file /user/hdfs-examples/hdfs_example_1/test.txt
17/10/26 19:11:46 INFO examples.HdfsExample: success to delete path /user/hdfs-examples/hdfs_example_1
17/10/26 19:11:46 INFO examples.HdfsExample: success to append.
17/10/26 19:11:46 INFO examples.HdfsExample: result is : hi, I am bigdata. It is successful if you can see me.I append this content.
17/10/26 19:11:46 INFO examples.HdfsExample: success to read.
17/10/26 19:11:46 INFO examples.HdfsExample: success to delete the file /user/hdfs-examples/hdfs_example_0/test.txt
17/10/26 19:11:46 INFO examples.HdfsExample: success to delete path /user/hdfs-examples/hdfs_example_0

ColocationExample Linux样例程序运行结果如下所示。

[root@192-168-32-144 client]#hadoop jar HDFSTest-XXX.jar com.huawei.bigdata.hdfs.examples.ColocationExample
WARNING: Use "yarn jar" to launch YARN applications.
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=V100R002C30, built on 10/19/2017 04:21 GMT
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:host.name=192-168-32-144
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_144
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:java.home=/opt/client/JDK/jdk1.8.0_144/jre
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/opt/client/Yarn/config:/opt/client/HDFS/hadoop/etc/hadoop:/opt/client/HDFS/hadoop/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jsr305-3.0.0.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/crypter-0.0.6.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/dynalogger-V100R002C30.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/curator-client-2.7.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/nimbus-jose-jwt-3.9.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/json-smart-1.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/zookeeper-3.5.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/cas-client-core-hw-3.3.3.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/opensaml-2.6.5.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/htrace-core-3.1.0-incubating.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/wc2frm-v1r2c60-20160429.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-collections-3.2.2.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/asm-3.2.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/hadoop-auth-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/gson-2.2.4.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/apache-log4j-extras-1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/hadoop-crypto-adapter-0.0.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jsch-0.1.54.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/hamcrest-core-1.3.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/hadoop-annotations-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/activation-1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/junit-4.11.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/httpcore-4.4.4.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/httpclient-4.5.2.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/xz-1.0.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jcip-annotations-1.0.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/slf4j-api-1.7.10.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/curator-framework-2.7.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/xmlsec-1.5.7.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/jetty-sslengine-6.1.26.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/SSO-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/opt/client/HDFS/hadoop/share/hadoop/common/datasight-hadoop-trace-ping-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/hadoop-common-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/common/hadoop-nfs-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/javaluator-3.0.1.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/hdfs-inode-provider-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/hadoop-hdfs-client-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/hadoop-hdfs-colocation-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/hadoop-hdfs-datamovement-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/superior-client-1.0.0.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/spark-3.1.1-yarn-shuffle.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/commons-io-2.4.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jetty-6.1.26.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/dynalogger-V100R002C30.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jersey-server-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/commons-codec-1.4.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/zookeeper-3.5.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/javaluator-3.0.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/xercesImpl-2.9.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/htrace-core-3.1.0-incubating.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jettison-1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/servlet-api-2.5.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/spark-3.1.1-yarn-shuffle.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jersey-json-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/apache-log4j-extras-1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jersey-client-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/commons-cli-1.2.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/activation-1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/xml-apis-1.3.04.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/superior-yarn-scheduler-1.0.0.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/netty-all-4.0.23.Final.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/commons-daemon-1.0.13.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/xmlenc-0.52.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/commons-lang-2.6.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jersey-core-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/lib/guava-11.0.2.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-datasight-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-yarn-client-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-yarn-registry-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/yarn/hadoop-yarn-api-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/junit-4.11.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.1.jar:/opt/client/HDFS/hadoop/contrib/capacity-scheduler/*.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/apacheds-i18n-2.0.0-M15.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-math3-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jackson-xc-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jaxb-impl-2.2.3-1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-openstack-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-io-2.4.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-digester-1.8.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jetty-6.1.26.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jackson-core-asl-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/api-asn1-api-1.0.0-M20.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-datajoin-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/avro-1.7.4.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jsr305-3.0.0.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/dynalogger-V100R002C30.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jersey-server-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-codec-1.4.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/curator-client-2.7.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-gridmix-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jaxb-api-2.2.2.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/nimbus-jose-jwt-3.9.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/json-smart-1.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/zookeeper-3.5.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-rumen-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/htrace-core-3.1.0-incubating.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jettison-1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-collections-3.2.2.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/asm-3.2.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jetty-util-6.1.26.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/servlet-api-2.5.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-auth-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/log4j-1.2.17.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jackson-databind-2.2.3.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/curator-recipes-2.7.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/gson-2.2.4.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-beanutils-1.7.0.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jets3t-0.9.0.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/mockito-all-1.8.5.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jersey-json-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/apache-log4j-extras-1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jackson-mapper-asl-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/protobuf-java-2.5.0.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/api-util-1.0.0-M20.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jsch-0.1.54.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-archives-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hamcrest-core-1.3.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-logging-1.1.3.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/java-xmlbuilder-0.4.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/joda-time-2.9.7.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/netty-3.6.2.Final.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-distcp-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-cli-1.2.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/activation-1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jackson-core-2.2.3.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/junit-4.11.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jsp-api-2.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/httpcore-4.4.4.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-hdfs-restore-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/httpclient-4.5.2.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/metrics-core-3.0.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-beanutils-core-1.8.0.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-azure-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jackson-annotations-2.2.3.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-streaming-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/xz-1.0.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jcip-annotations-1.0.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-lang3-3.3.2.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/paranamer-2.3.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/curator-framework-2.7.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-sls-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-httpclient-3.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/stax-api-1.0-2.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-compress-1.4.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/xmlenc-0.52.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-lang-2.6.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/azure-storage-2.0.0.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jersey-core-1.9.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-extras-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-net-3.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/hadoop-ant-3.1.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jackson-jaxrs-1.9.13.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/jetty-sslengine-6.1.26.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/guava-11.0.2.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/snappy-java-1.0.4.1.jar:/opt/client/HDFS/hadoop/share/hadoop/tools/lib/commons-configuration-1.6.jar
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/opt/client/HDFS/hadoop/lib/native
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-504.el6.x86_64
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:user.name=root
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:user.home=/root
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:user.dir=/opt/client
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:os.memory.free=97MB
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:os.memory.max=123MB
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Client environment:os.memory.total=123MB
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=192-168-32-144:2181,192-168-32-67:2181,192-168-33-190:2181 sessionTimeout=20000 watcher=com.huawei.hadoop.oi.colocation.ZooKeeperWatcher@60410cd
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: zookeeper.request.timeout is not configured. Using default value 120000.
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: zookeeper.client.bind.port.range is not configured.
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: zookeeper.client.bind.address is not configured.
17/10/26 19:12:38 INFO client.FourLetterWordMain: connecting to 192-168-32-67 2181
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: Got server principal from the server and it is null
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: Using server principal zookeeper/192-168-32-67
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: Opening socket connection to server 192-168-32-67/192.168.32.67:2181. Will not attempt to authenticate using SASL (unknown error)
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /192.168.32.144:52882, server: 192-168-32-67/192.168.32.67:2181
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: Session establishment complete on server 192-168-32-67/192.168.32.67:2181, sessionid = 0x13000074b7e4687f, negotiated timeout = 20000
17/10/26 19:12:38 INFO colocation.ZKUtil: ZooKeeper colocation znode : /hadoop/colocationDetails. Will publish colocation details under this znode hierarchy.
Create Group is running...
17/10/26 19:12:38 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=192-168-32-144:2181,192-168-32-67:2181,192-168-33-190:2181 sessionTimeout=20000 watcher=com.huawei.hadoop.oi.colocation.ZooKeeperWatcher@f80945f
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: zookeeper.request.timeout is not configured. Using default value 120000.
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: zookeeper.client.bind.port.range is not configured.
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: zookeeper.client.bind.address is not configured.
17/10/26 19:12:38 INFO client.FourLetterWordMain: connecting to 192-168-32-144 2181
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: Got server principal from the server and it is null
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: Using server principal zookeeper/192-168-32-144
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: Opening socket connection to server 192-168-32-144/192.168.32.144:2181. Will not attempt to authenticate using SASL (unknown error)
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /192.168.32.144:40383, server: 192-168-32-144/192.168.32.144:2181
17/10/26 19:12:38 INFO zookeeper.ClientCnxn: Session establishment complete on server 192-168-32-144/192.168.32.144:2181, sessionid = 0x12000059699f69e1, negotiated timeout = 20000
17/10/26 19:12:38 INFO colocation.ZKUtil: ZooKeeper colocation znode : /hadoop/colocationDetails. Will publish colocation details under this znode hierarchy.
Create Group has finished.
Put file is running...
Put file has finished.
Delete file is running...
Delete file has finished.
Delete Group is running...
Delete Group has finished.
17/10/26 19:12:39 INFO zookeeper.ZooKeeper: Session: 0x13000074b7e4687f closed
17/10/26 19:12:39 INFO zookeeper.ClientCnxn: EventThread shut down for session: 0x13000074b7e4687f
17/10/26 19:12:39 INFO zookeeper.ZooKeeper: Session: 0x12000059699f69e1 closed
17/10/26 19:12:39 INFO zookeeper.ClientCnxn: EventThread shut down for session: 0x12000059699f69e1

查看HDFS日志获取应用运行情况

可以查看HDFS的namenode日志了解应用运行情况，并根据日志信息调整应用程序。

父主题： 调测HDFS应用

上一篇：在本地Windows中调测HDFS程序

下一篇：HDFS应用开发常见问题

意见反馈

文档内容是否对您有帮助？

有帮助没帮助

提供反馈

提交成功！非常感谢您的反馈，我们会继续努力做到更好！

系统繁忙，请稍后重试

在使用文档中是否遇到以下问题

内容与产品页面不一致

内容不易理解

缺失示例代码

步骤不可操作

搜不到想要的内容

缺少最佳实践

意见反馈（选填）

0/500

请至少选择一项反馈信息并填写问题反馈

字符长度不能超过500

直接提交取消