Using CDM to Migrate Data to HDFS
Issue
A user failed to use CDM to migrate data from an old cluster to HDFS of a new cluster.
Symptom
When CDM is used to import data from the source HDFS to the destination HDFS, the destination MRS cluster is faulty and the NameNode cannot be started.
The logs show that the Java heap space error is reported during the startup. The JVM parameter of the NameNode needs to be modified.
![Click to enlarge](https://support.huaweicloud.com/intl/en-us/eu-west-0-usermanual-mrs/en-us_image_0000001433624001.png)
Cause Analysis
When the user uses CDM to migrate data, the HDFS data volume is too large. As a result, a stack exception occurs when metadata is merged.
Procedure
- Go to the HDFS service configuration page.
- For versions earlier than MRS 2.0.1: Log in to MRS Manager, choose Services > HDFS > Service Configuration, and select All from the Basic drop-down list.
- For MRS 2.0.1 or later: Click the cluster name on the MRS console, choose Components > HDFS > Service Configuration, and select All from the Basic drop-down list.
- For MRS 3.x or later: Log in to FusionInsight Manager and choose Cluster. Click the name of the target cluster and choose Services > HDFS > Configurations > All Configurations.
- Search for the GC_OPTS parameter in HDFS->NameNode and increase the values of -Xms512M and -Xmx512M based on service requirements.
- Save the configuration and restart the affected services or instances.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot