文档首页/ MapReduce服务 MRS/ 开发指南（LTS版）/ HDFS开发指南（普通模式）/ 调测HDFS应用/ 在本地Windows中调测HDFS程序
更新时间：2025-12-16 GMT+08:00
查看PDF
分享

在本地Windows中调测HDFS程序

操作场景

在代码完成开发后，您可以在Windows开发环境中运行应用。本地和集群业务平面网络互通时，您可以直接在本地进行调测。

HDFS应用程序运行完成后，可直接通过运行结果查看应用程序运行情况，也可以通过HDFS日志获取应用运行情况。

在本地Windows中调测HDFS程序

  • 编译并运行SpringBoot对接HDFS样例：
    1. 登录HDFS时会使用到如表1所示的配置文件。这些文件均已导入到“hdfs-examples”工程的“src/main/resource”目录。
      表1 配置文件

      文件名称

      作用

      core-site.xml

      配置HDFS详细参数。

      hdfs-site.xml

      配置HDFS详细参数。

      application.properties

      Spring boot的配置信息，目前需要配置下面信息：

      • “hdfs-examples”工程的“src/main/resource”目录的绝对地址。
      • 登录hdfs所需要的用户名
    2. 修改application.properties中的参数：
      • spring.hdfs.config.dir：访问HDFS的配置文件目录地址，其中配置文件包括：hdfs-site.xml、core-site.xml、krb5.conf、user.keytab。
      • hdfs.user：访问HDFS的业务用户名称。
    3. 在开发环境中（例如IntelliJ IDEA中），选中SpringDataHDFSApplication.java，右键工程。选中“Run 'SpringDataHDFSApplication.main()'”运行工程。
    4. 在浏览器中依次访问如下对应链接。
      • http://127.0.0.1:8080/save?fileName=test.txt ： 向HDFS的目录“/tmp/examples”中保存test.txt数据。
      • http://127.0.0.1:8080/read?fileName=test.txt ：读取HDFS的目录“/tmp/examples”中保存test.txt数据。
      • http://127.0.0.1:8080/remove?fileName=test.txt：删除HDFS的目录“/tmp/examples”中test.txt文件。
  • 编译并运行除SpringBoot外其他样例程序：
    1. （可选）在开发环境中（例如IntelliJ IDEA中），运行此样例代码需要设置运行用户，设置运行用户有两种方式，添加环境变量HADOOP_USER_NAME或者修改代码。设置环境变量方法具体如下：
      选中需要运行的样例程序HdfsExample.java或者ColocationExample.java，右键工程，选择“Run Configurations”，在对话框中选择“JavaApplication > HdfsExample”进行运行参数设置。在IntelliJ IDEA的菜单栏依次选择“Run > Edit Configurations”，在弹出的会话框中设置运行用户。 
      -DHADOOP_USER_NAME=test
      图1 参数配置

      用户可向管理员咨询运行用户。test在这里只是举例，若需运行Colocation相关操作的样例代码，则此用户需属于supergroup用户组。

    2. 若已按照1设置环境变量，则直接单击Run，运行应用工程。否则分别选中以下两个工程运行程序：
      • 选中HdfsExample.java，右键工程，选择“Run 'HdfsExample.main()'"”运行应用工程。
      • 选中ColocationExample.java，右键工程，选择“Run 'ColocationExample.main()'”运行应用工程。
      • 在HDFS任务运行过程中禁止重启HDFS服务，否则可能会导致任务失败。
      • 在运行Colocation工程时，HDFS的配置项fs.defaultFS不能配置为viewfs://ClusterX。

查看调测结果

  • 查看运行结果获取应用运行情况
    • HdfsExample Windows样例程序运行结果如下所示。 
      1654 [main] WARN  org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory  - The short-circuit local reads feature cannot be used because UNIX Domain sockets are not available on Windows.
2013 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples
2137 [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2590 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
3245 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
4447 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
4447 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
4509 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples\test.txt
4618 [main] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples
4743 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples/hdfs_example_1
4743 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to create path /user/hdfs-examples/hdfs_example_0
5087 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
5087 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to write.
6507 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
6553 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to append.
7505 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
7505 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
7568 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples/hdfs_example_1\test.txt
7583 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - result is : hi, I am bigdata. It is successful if you can see me.I append this content.
7583 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to read.
7630 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete the file /user/hdfs-examples/hdfs_example_0\test.txt
7677 [hdfs_example_1] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples/hdfs_example_1
7739 [hdfs_example_0] INFO  com.huawei.bigdata.hdfs.examples.HdfsExample  - success to delete path /user/hdfs-examples/hdfs_example_0

      在Windows环境运行样例代码时会出现下面的异常，但是不影响业务：

      java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

    • ColocationExample Windows样例程序运行结果如下所示。 
      1623 [main] WARN  org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory  - The short-circuit local reads feature cannot be used because UNIX Domain sockets are not available on Windows.
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:zookeeper.version=***, built on 10/19/2017 04:21 GMT
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:host.name=siay7user1.china.huawei.com
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.version=***
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.vendor=Oracle Corporation
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.home=D:\Program Files\Java\jre1.8.0_131
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.class.path=D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\bin;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-cli-1.2.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-codec-1.4.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-collections-3.2.2.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-configuration-1.6.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-io-2.4.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-lang-2.6.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\commons-logging-1.1.3.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\dynalogger-V100R002C30.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\guava-11.0.2.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-annotations-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-auth-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-common-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-hdfs-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-hdfs-client-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-hdfs-colocation-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-hdfs-datamovement-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-hdfs-nfs-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-hdfs-restore-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-mapreduce-client-core-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\hadoop-nfs-3.1.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\htrace-core-3.1.0-incubating.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\log4j-1.2.17.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\protobuf-java-2.5.0.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\slf4j-api-1.7.10.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\slf4j-log4j12-1.7.10.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal\lib\zookeeper-3.5.1.jar;D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\SmallFS\FusionInsight-SmallFS-1.0.0.tar.gz\smallfs\share\datasight\smallfs\smallfs-main-V100R002C30.jar
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.library.path=D:\Program Files\Java\jre1.8.0_131\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;D:/Program Files/Java/jre1.8.0_131/bin/server;D:/Program Files/Java/jre1.8.0_131/bin;D:/Program Files/Java/jre1.8.0_131/lib/amd64;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;D:\Program Files\Java\jdk1.8.0_131\bin;D:\Program Files\Java\jdk1.8.0_131\jre\bin;D:\Program Files (x86)\GitExtensions\;D:\Program Files\Git\cmd;D:\soft\apache-maven-3.2.2\bin;D:\soft\gnubin;D:\soft\protoc-2.5.0-win32;;D:\soft\TMSS;D:\安装包\eclipse-jee-mars-R-win32-x86_64\eclipse;;.
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.io.tmpdir=C:\Users\L00430~1\AppData\Local\Temp\
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:java.compiler=<NA>
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.name=Windows 7
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.arch=amd64
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.version=6.1
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:user.name=user
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:user.home=C:\Users\user
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:user.dir=D:\FIClient\nonSafety\FusionInsight_Cluster_<集群ID>_Services_ClientConfig\HDFS\hdfs-example-normal
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.memory.free=107MB
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.memory.max=1819MB
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Client environment:os.memory.total=123MB
1670 [main] INFO  org.apache.zookeeper.ZooKeeper  - Initiating client connection, connectString=192-168-32-144:2181,192-168-32-67:2181,192-168-33-190:2181 sessionTimeout=45000 watcher=com.huawei.hadoop.oi.colocation.ZooKeeperWatcher@5f9b2141
1794 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.request.timeout is not configured. Using default value 120000.
1794 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.client.bind.port.range is not configured.
1794 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.client.bind.address is not configured.
1794 [main-SendThread(192-168-32-67:2181)] INFO  org.apache.zookeeper.client.FourLetterWordMain  - connecting to 192-168-32-67 2181
1904 [main-SendThread(192-168-32-67:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Got server principal from the server and it is null
1904 [main-SendThread(192-168-32-67:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Using server principal zookeeper/192-168-32-67
1904 [main-SendThread(192-168-32-67:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Opening socket connection to server 192-168-32-67/192.168.32.67:2181. Will not attempt to authenticate using SASL (unknown error)
1966 [main-SendThread(192-168-32-67:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Socket connection established, initiating session, client: /192.168.35.189:50954, server: 192-168-32-67/192.168.32.67:2181
2029 [main-SendThread(192-168-32-67:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Session establishment complete on server 192-168-32-67/192.168.32.67:2181, sessionid = 0x13000074b7e464b7, negotiated timeout = 45000
2169 [main] INFO  com.huawei.hadoop.oi.colocation.ZKUtil  - ZooKeeper colocation znode : /hadoop/colocationDetails. Will publish colocation details under this znode hierarchy.
Create Group is running...
5212 [main] INFO  org.apache.zookeeper.ZooKeeper  - Initiating client connection, connectString=192-168-32-144:2181,192-168-32-67:2181,192-168-33-190:2181 sessionTimeout=45000 watcher=com.huawei.hadoop.oi.colocation.ZooKeeperWatcher@2438dcd
5212 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.request.timeout is not configured. Using default value 120000.
5212 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.client.bind.port.range is not configured.
5212 [main] INFO  org.apache.zookeeper.ClientCnxn  - zookeeper.client.bind.address is not configured.
5212 [main-SendThread(192-168-33-190:2181)] INFO  org.apache.zookeeper.client.FourLetterWordMain  - connecting to 192-168-33-190 2181
5321 [main-SendThread(192-168-33-190:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Got server principal from the server and it is null
5321 [main-SendThread(192-168-33-190:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Using server principal zookeeper/192-168-33-190
5321 [main-SendThread(192-168-33-190:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Opening socket connection to server 192-168-33-190/192.168.33.190:2181. Will not attempt to authenticate using SASL (unknown error)
5368 [main-SendThread(192-168-33-190:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Socket connection established, initiating session, client: /192.168.35.189:50962, server: 192-168-33-190/192.168.33.190:2181
5430 [main-SendThread(192-168-33-190:2181)] INFO  org.apache.zookeeper.ClientCnxn  - Session establishment complete on server 192-168-33-190/192.168.33.190:2181, sessionid = 0x14000073f13b657b, negotiated timeout = 45000
5540 [main] INFO  com.huawei.hadoop.oi.colocation.ZKUtil  - ZooKeeper colocation znode : /hadoop/colocationDetails. Will publish colocation details under this znode hierarchy.
Create Group has finished.
Put file is running...
5930 [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Put file has finished.
Delete file is running...
Delete file has finished.
Delete Group is running...
Delete Group has finished.
6866 [main] INFO  org.apache.zookeeper.ZooKeeper  - Session: 0x13000074b7e464b7 closed
6866 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn  - EventThread shut down for session: 0x13000074b7e464b7
6928 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn  - EventThread shut down for session: 0x14000073f13b657b
6928 [main] INFO  org.apache.zookeeper.ZooKeeper  - Session: 0x14000073f13b657b closed
    • SpringBoot对接HDFS样例运行结果。

      Spring Boot启动日志：

      SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/example/spring-boot-hdfs-1.0-SNAPSHOT.jar!/BOOT-INF/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/example/spring-boot-hdfs-1.0-SNAPSHOT.jar!/BOOT-INF/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.4.2)
2025-01-21 10:43:58.049  INFO 3628949 --- [           main] c.h.s.hdfs.SpringDataHDFSApplication     : Starting SpringDataHDFSApplication v1.0-SNAPSHOT using Java 1.8.0_422 on hdcore-common-001 with PID 3628949 (/opt/example/spring-boot-hdfs-1.0-SNAPSHOT.jar started by root in /opt/example)
2025-01-21 10:43:58.054  INFO 3628949 --- [           main] c.h.s.hdfs.SpringDataHDFSApplication     : No active profile set, falling back to default profiles: default
2025-01-21 10:43:58.928  WARN 3628949 --- [kground-preinit] o.s.h.c.j.Jackson2ObjectMapperBuilder    : For Jackson Kotlin classes support please add "com.fasterxml.jackson.module:jackson-module-kotlin" to the classpath
2025-01-21 10:43:59.695  INFO 3628949 --- [           main] org.eclipse.jetty.util.log               : Logging initialized @3808ms to org.eclipse.jetty.util.log.Slf4jLog
2025-01-21 10:43:59.845  INFO 3628949 --- [           main] o.s.b.w.e.j.JettyServletWebServerFactory : Server initialized with port: 8080
2025-01-21 10:43:59.848  INFO 3628949 --- [           main] org.eclipse.jetty.server.Server          : jetty-9.4.35.v20201120; built: 2020-11-20T21:17:03.964Z; git: bdc54f03a5e0a7e280fab27f55c3c75ee8da89fb; jvm 1.8.0_422-BiSheng_JDK8_Enterprise_202.1.0.420.B002-b05
2025-01-21 10:43:59.886  INFO 3628949 --- [           main] o.e.j.s.h.ContextHandler.application     : Initializing Spring embedded WebApplicationContext
2025-01-21 10:43:59.886  INFO 3628949 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1180 ms
2025-01-21 10:44:00.018  INFO 3628949 --- [           main] org.eclipse.jetty.server.session         : DefaultSessionIdManager workerName=node0
2025-01-21 10:44:00.018  INFO 3628949 --- [           main] org.eclipse.jetty.server.session         : No SessionScavenger set, using defaults
2025-01-21 10:44:00.021  INFO 3628949 --- [           main] org.eclipse.jetty.server.session         : node0 Scavenging every 660000ms
2025-01-21 10:44:00.035  INFO 3628949 --- [           main] o.e.jetty.server.handler.ContextHandler  : Started o.s.b.w.e.j.JettyEmbeddedWebAppContext@7ce3cb8e{application,/,[file:///tmp/jetty-docbase.8080.5072591134057219935/],AVAILABLE}
2025-01-21 10:44:00.036  INFO 3628949 --- [           main] org.eclipse.jetty.server.Server          : Started @4150ms
2025-01-21 10:44:00.643  INFO 3628949 --- [           main] o.a.h.security.UserGroupInformation      : Login successful for user admintest using keytab file user.keytab. Keytab auto renewal enabled : false
2025-01-21 10:44:00.643  INFO 3628949 --- [           main] com.huawei.spring.hdfs.LoginUtil         : Login success!!!!!!!!!!!!!!
2025-01-21 10:44:01.464  WARN 3628949 --- [           main] org.apache.hadoop.util.NativeCodeLoader  : Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2025-01-21 10:44:01.471  WARN 3628949 --- [           main] o.a.h.h.s.DomainSocketFactory            : The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
2025-01-21 10:44:01.711  INFO 3628949 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2025-01-21 10:44:01.975  INFO 3628949 --- [           main] o.e.j.s.h.ContextHandler.application     : Initializing Spring DispatcherServlet 'dispatcherServlet'
2025-01-21 10:44:01.975  INFO 3628949 --- [           main] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2025-01-21 10:44:01.976  INFO 3628949 --- [           main] o.s.web.servlet.DispatcherServlet        : Completed initialization in 1 ms
2025-01-21 10:44:02.004  INFO 3628949 --- [           main] o.e.jetty.server.AbstractConnector       : Started ServerConnector@2133814f{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2025-01-21 10:44:02.005  INFO 3628949 --- [           main] o.s.b.web.embedded.jetty.JettyWebServer  : Jetty started on port(s) 8080 (http/1.1) with context path '/'
2025-01-21 10:44:02.023  INFO 3628949 --- [           main] c.h.s.hdfs.SpringDataHDFSApplication     : Started SpringDataHDFSApplication in 4.513 seconds (JVM running for 6.138)

      访问链接结果如下：

  • 查看HDFS日志获取应用运行情况

    可以查看HDFS的NameNode日志了解应用运行情况，并根据日志信息调整应用程序。

父主题： 调测HDFS应用

相关文档