Example Typical Scenario: Connecting DLI to a Data Source on a Public Network

Updated on 2024-12-28 GMT+08:00

View PDF

Scenario

A public network data source is one that is accessible over the Internet and has a public IP address. By connecting DLI to the public network, you can access these data sources.

This section explains how to connect DLI to a public network by setting up SNAT rules and configuring routing information.

Procedure

Figure 1 Configuration process

Step 1: Create a VPC

For details about how to create a VPC, see Creating a VPC.

Figure 2 Creating a VPC

Step 2: Create an Elastic Resource Pool and Add Queues Within It

Log in to the DLI management console.
In the navigation pane on the left, choose Resources > Resource Pool.
On the displayed page, click Buy Resource Pool in the upper right corner.

On the displayed page, set the parameters.

**Table 1** Parameter descriptions
Parameter	Description
Region	Select a region where you want to buy the elastic resource pool.
Project	Project uniquely preset by the system for each region
Name	Name of the elastic resource pool
Specifications	Specifications of the elastic resource pool
CU Range	The maximum and minimum CUs allowed for the elastic resource pool
CIDR Block	CIDR block the elastic resource pool belongs to. If you use an enhanced datasource connection, this CIDR block cannot overlap that of the data source. Once set, this CIDR block cannot be changed.
Enterprise Project	Select an enterprise project for the elastic resource pool.

Click Buy.
Click Submit.
In the elastic resource pool list, locate the pool you just created and click Add Queue in the Operation column.

Set the basic parameters listed below.

**Table 2** Basic parameters for adding a queue
Parameter	Description
Name	Name of the queue to add
Type	Type of the queue To execute SQL jobs, select For SQL. To execute Flink or Spark jobs, select For general purpose.
Engine	SQL queue engine. The options are Spark and HetuEngine.
Enterprise Project	Select an enterprise project for the elastic resource pool.

Click Next and configure scaling policies for the queue.

Click Create to add a scaling policy with varying priority, period, minimum CUs, and maximum CUs.

Figure 3 shows the scaling policy configured in this example.

Figure 3 Configuring a scaling policy when adding a queue

**Table 3** Scaling policy parameters
Parameter	Description	Example Value
Priority	Priority of the scaling policy in the current elastic resource pool. A larger value indicates a higher priority. In this example, only one scaling policy is configured, so its priority is set to 1 by default.	1
Period	The first scaling policy is the default policy, and its Period parameter configuration cannot be deleted or modified. The period for the scaling policy is from 00 to 24.	00-24
Min CU	Minimum number of CUs allowed by the scaling policy	16
Max CU	Maximum number of CUs allowed by the scaling policy	64

Click OK.

Step 3: Create an Enhanced Datasource Connection Between the Queue and a VPC

In the navigation pane of the DLI management console, choose Datasource Connections.
In the Enhanced tab, click Create.
Enter the connection name, select the created queue, VPC, and subnet, and enter the host information (optional).

Figure 4 Creating an enhanced datasource connection

Step 4: Buy an EIP

Log in to the EIPs page of the network console, click Buy EIP.
In the displayed page, configure the parameters as required.
For details about how to set the parameters, see Buy EIP.

Step 5: Configure a NAT Gateway

Create a NAT gateway.
1. Log in to the console and search for NAT Gateway in the Service List. The Public NAT Gateways page of the network console is displayed.
2. Click Buy Public NAT Gateway and configure the required parameters.
  For details, see Buying a Public NAT Gateway.
  
  Figure 5 Buying a NAT gateway
3. Click Next, confirm the configurations, and click Submit.
  NOTE:
  
  During the configuration, set VPC to the one created in Step 1: Create a VPC.
Add a route.

In the navigation pane on the left of the network console, choose Virtual Private Cloud > Route Tables. After a NAT gateway instance is created, a route to that gateway is automatically created. Click the route table name to view the automatically created route.

The destination address is the public IP address you want to access, and the next hop is the NAT gateway.

Figure 6 Viewing the route
Add an SNAT rule.
You need to add SNAT rules for the new NAT gateway to allow the hosts in the subnet to communicate with the Internet.
1. Click the name of the created NAT gateway on the Public NAT Gateways page of the network console.
2. On the SNAT Rules tab, click Add SNAT Rule.
  For details, see Adding an SNAT Rule.
3. Scenario: Select Direct Connect/Cloud Connect.
4. Subnet: Select the subnet where the queue you want to connect locates.
5. EIP: Select the target EIP.
  Figure 7 Adding an SNAT rule
6. Click OK.

Step 6: Add a Custom Route

Add a custom route for the enhanced datasource connection you have created. Specify the route information of the IP address you want to access.

For details, see Custom Route Information.

Figure 8 Adding route information for test

Step 7: Test the Connectivity to the Public Network

Test the connectivity between the queue and the public network. Click More > Test Address Connectivity in the Operation column of the target queue and enter the public IP address you want to access.

Figure 9 Testing address connectivity