What Should I Do If an Error Is Reported When I Run DistCp Commands?

Updated on 2024-12-11 GMT+08:00

View PDF

Question

DistCp commands fail to run in a security cluster and exceptions are thrown.

The following client exception is reported:

Invalid arguments:Unexpected end of file from server

The following server exception is reported:

javax.net.ssl.SSLException:Unrecognized SSL message, plaintext connection?

Answer

When a user uses webhdfs:// in a DistCp command, the preceding exceptions are thrown because the cluster uses HTTPS, that is, the dfs.http.policy value in the hdfs-site.xml file configured in Client installation directory/HDFS/hadoop/etc/hadoop is HTTPS_ONLY. To avoid this exception, replace webhdfs:// with swebhdfs://.

For example:

./hadoop distcpswebhdfs://IP:PORT/testfile hdfs://IP:PORT/testfile1

Parent topic: Common Issues About HDFS

Previous topic: Common Issues About HDFS

Next topic: When Does a Balance Process in HDFS, Shut Down and Fail to be Executed Again?