On this page

Show all

Help Center/ MapReduce Service/ Best Practices/ Data Migration/ Preparing the Network Before Data Migration to MRS

Preparing the Network Before Data Migration to MRS

Updated on 2022-12-09 GMT+08:00

View PDF

During big data migration, ensure that the network connection between the source cluster and the destination cluster is normal. For example, when you run the Hadoop distcp command to copy data across clusters, all DataNodes must be connected. Based on different migration scenarios, you need to use different methods to connect the network between the two clusters.

You can migrate data from your local data center to an MRS cluster on HUAWEI CLOUD and use Direct Connect to establish a dedicated connection between your local data center and a VPC on the cloud. You can create a Direct Connect service on HUAWEI CLOUD or use a third-party Direct Connect service to connect to HUAWEI CLOUD.
Figure 1 Offline data center migration
If you want to migrate your created big data clusters (or MRS clusters of earlier versions) to HUAWEI CLOUD MRS clusters, you need to switch to the same region and VPC subnet. In this way, your created clusters and MRS clusters can use the same security group, VPC, and subnet to ensure network connectivity.
Figure 2 Online migration in the same region and VPC
The customer builds a big data cluster (or an MRS cluster of an earlier version) on HUAWEI CLOUD and needs to migrate it to an MRS cluster on HUAWEI CLOUD. The two clusters are in the same region but use different VPC subnets. You need to use a VPC peering connection to configure network connectivity.
Figure 3 Online migration between different VPCs in the same region
You need to migrate your big data clusters (or MRS clusters of earlier versions) created on HUAWEI CLOUD to MRS clusters on HUAWEI CLOUD. In different regions, you can use Cloud Connect (CC) to build cross-region VPC network connections.
Figure 4 Online migration between different regions