Migrating Big Data Without Using the Internet

Step 1: Configure a Huawei Cloud VPN

1. Log in to the Huawei Cloud console and, in the service list, choose Networking > Virtual Private Network.
2. Configure a VPN gateway
   1. In the navigation pane, choose Virtual Private Network > Enterprise – VPN Gateways.
   2. Click Buy S2C VPN Gateway and set parameters by following the on-screen instructions.
   3. Configure all required parameters and click Buy Now.

3. Configure customer gateways. You need to create an active and a standby customer gateway.
   1. In the navigation pane on the left, choose Virtual Private Network > Enterprise – Customer Gateways.
   2. Click Create Customer Gateway and set parameters by following the on-screen instructions. Select IP Address for Identifier and enter the public IP address of the Alibaba Cloud gateway.
   3. Click Create Now.

4. Create VPN connections. Create two VPN connections to connect to the Huawei Cloud VPN gateway and Alibaba Cloud customer gateway, respectively.
   1. In the navigation pane on the left, choose Virtual Private Network > Enterprise – VPN Connections.
   2. Click Create VPN Connection. On the displayed page, select the created VPN gateway and a customer gateway, and enter the subnet address of the customer gateway. Ensure that the subnet addresses do not overlap.

      

      Select Custom for Policy Settings and ensure that the settings are the same as those on Alibaba Cloud.

      

   3. Configure all required parameters and click Buy Now.

Step 2: Configure an Alibaba Cloud VPN

1. Sign in to the Alibaba Cloud console and choose Products and Services > Networking and CDN > Hybrid Cloud Network > VPN Gateway.
2. Configure a VPN gateway
   1. Click Create VPN Gateway and set parameters by following the on-screen instructions.
   2. Configure all required parameters and click Buy Now.

3. Configure the customer gateway.
   1. In the navigation pane, choose VPN > Customer Gateways.
   2. Click Create Customer Gateway and set parameters by following the on-screen instructions.
   3. Click OK.

4. Create a VPN connection.
   1. In the navigation pane, choose VPN > IPsec Connections.
   2. Click Create IPsec Connection, select the VPN gateway configured in step 2, and keep the policy settings the same as those on Huawei Cloud.
   3. Click OK.

5. Configure a route to the Huawei Cloud VPC subnet.
   1. In the navigation pane, choose VPN > VPN Gateways.
   2. Click the VPN gateway name. On the Destination-based Route Table tab, click Add Route Entry and set parameters based on the instructions.

Step 3: Configure an Alibaba Cloud NAT Gateway

Create an Alibaba Cloud NAT gateway and configure SNAT and DNAT entries. For details, see Creating and Managing an Internet Public NAT Gateway.

1. Sign in to the Alibaba Cloud console and choose Products and Services > Networking and CDN > Hybrid Cloud Network > VPN Gateway.
2. Create an Internet NAT gateway.
   1. On the Internet NAT Gateway page, click Create Internet NAT Gateway and configure parameters based on the instructions.
   2. Configure all required parameters and click Buy Now.

3. Configure an SNAT entry.
   1. On the Internet NAT Gateway page, locate the Internet NAT gateway created in step 2 and click Configure SNAT in the Actions column.
   2. On the SNAT Management tab, click Create SNAT Entry and set parameters based on the instructions.
   3. Click OK.

4. Configure a DNAT entry.
   1. On the Internet NAT Gateway page, locate the Internet NAT gateway created in step 2 and click Configure DNAT in the Actions column.
   2. On the DNAT Management tab, click Create DNAT Entry and set parameters based on the instructions.
   3. Click OK.

Step 4: Configure Security Groups

You need to configure security groups on Huawei Cloud and Alibaba Cloud.

1. On the Huawei Cloud console, configure the involved security group to allow access from the private IP address of the server where the MgC Agent is installed.
   1. Sign in to the Huawei Cloud console.
   2. In the Service List, choose Networking > Virtual Private Cloud.
   3. In the navigation pane, choose Access Control > Security Groups.
   4. In the security group list, locate the security group where the target big data cluster is managed and click Manage Rules in the Operation column.
   5. On the Inbound Rules tab, click Add Rule.
   6. In the displayed dialog box, add a rule that allows TCP traffic to the open port of the MRS cluster. Enter the private IP address of the server where the MgC Agent is installed in the Source text box. For example, the default open port of a secured Hive cluster is 9083.
   7. Click OK.

2. On the Alibaba Cloud console, configure the involved security group to allow access from the private IP address of the server where the MgC Agent is installed.
   1. Sign in to the Alibaba Cloud ECS console.
   2. In the navigation pane, choose Network & Security > Security Groups.
   3. Locate the security group that the server with the MgC Agent installed belongs to and click Manage Rules in the Operation column.
   4. On the Inbound tab, click Quick Add. Set Action to Allow, Authorization Object to the public IP address of the server where the MgC Agent is installed, and Port Range to All.
   5. Click OK.

Step 5: Set Up a Migration Environment

Set up a migration environment by referring to Preparations. Purchase an ECS on Alibaba Cloud. Configure an SNAT rule for the NAT gateway to allow the ECS to access the Internet using its private IP address. Install the MgC Agent on the ECS, register an account, and connect the MgC Agent to MgC.

Step 6: Create a Big Data Migration Task

Review and understand the notes about big data verification tasks. Perform the following steps to create a big data migration task:

1. Create a connection to MaxCompute.
2. Create a connection to DLI.
3. Create a metadata migration task or Create a data migration task and execute it.