Failed to Query or Delete HDFS Files

Symptom

A file fails to be queried or deleted using the HDFS shell client. The file (invisible characters) can be viewed in the parent directory.

Figure 1 List of files in the parent directory

Cause Analysis

The possible cause is that invisible characters are written to the file. You can write the file name to the local text and run the vi command to open the file.

hdfs dfs -ls parent directory > /tmp/t.txt

vi /tmp/t.txt

Run the :set list command to display invisible characters in the file name. For example, the file name contains ^M, which is invisible.

Figure 2 Displaying invisible characters

Solution

1. Run the Shell command to read the file name recorded in the text. Ensure that the following command output contains the full path of the file in HDFS.

   cat /tmp/t.txt |awk '{print $8}'

   Figure 3 File path
   

2. Run the following command to delete the file:

   hdfs dfs -rm $(cat /tmp/t.txt |awk '{print $8}')

3. Verify that the file has been deleted.

   hdfs dfs -ls Parent directory