Help Center/ MapReduce Service/ Component Operation Guide (Normal)/ Using HDFS/ FAQ/ Why Does the Distcp Command Fail in the Secure Cluster, Causing an Exception?
Updated on 2022-09-15 GMT+08:00
View PDF

Why Does the Distcp Command Fail in the Secure Cluster, Causing an Exception?

Question

Why distcp command fails in the secure cluster with the following error displayed?

Client side exception

Invalid arguments: Unexpected end of file from server

Server side exception

javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?

Answer

The preceding error may occur if webhdfs:// is used in the distcp command. The reason is that the big data cluster uses the HTTPS mechanism, that is, dfs.http.policy is set to HTTPS_ONLY in core-site.xml file. To avoid the error, replace webhdfs:// with swebhdfs:// in the file.

For example:

./hadoop distcp swebhdfs://IP:PORT/testfile hdfs://IP:PORT/testfile1

Parent topic: FAQ