Help Center/ Cloud Data Migration/ FAQs/ Troubleshooting/ What Should I Do If an Error Message Is Displayed Indicating that the Block Is Missing During HDFS File Migration?
Updated on 2023-06-21 GMT+08:00
View PDF
Share

What Should I Do If an Error Message Is Displayed Indicating that the Block Is Missing During HDFS File Migration?

Symptom

  1. A connector is created to link the on-premises HDFS and OBS, and the link connectivity is normal.
  2. The following error message is displayed when the migration is started:
Error: java.io.IOException: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-787476470-192.168.152.10-1573351961380:blk_1073742171_1347 file=/user/hive/warehouse/db_hive.db/emp/emp.txt (state=,code=0)

Possible Cause

  1. Files can be obtained using the HDFS client, which means that the file block is not lost.
  2. Check whether all DataNode instances of the HDFS service are started. If DataNode instances are stopped, the block cannot be obtained. Check the CDM cluster can communicate with the DataNode instances properly.

    Note: DataNode instances in the HDFS use port 9866 to transmit data.

Solution

The firewall is enabled for DataNode instances. As a result, CDM fails to connect to the DataNode instances and obtain the blocks. Disable the firewall for DataNode instances, and this issue is rectified.

Parent topic: Troubleshooting