Help Center/ ModelArts/ Best Practices/ Model Inference/ Enabling an Inference Service to Access the Internet
Updated on 2024-01-09 GMT+08:00
View PDF
Share

Enabling an Inference Service to Access the Internet

This section describes how to enable an inference service to access the Internet.

Application Scenarios

An inference service accesses the Internet in the following scenarios:

  • After an image is input, the inference service calls OCR on the Internet and then processes data using NLP.
  • The inference service downloads files from the Internet and analyzes the files.
  • The inference service sends back the analysis result to the terminal on the Internet.

Solution Design

Use the algorithm on the instance where the inference service is deployed to access the Internet.

Figure 1 Networking for an inference service to access the Internet

Procedure

  1. Configure the network for the ModelArts resource pool.

    When purchasing a dedicated resource pool, you can select inference services in Job Type. In this case, the selected network must be accessible to the target VPC.
    Figure 2 Purchasing a dedicated resource pool
    Figure 3 Interconnecting the VPC

    Interconnecting a VPC enables the ModelArts resource pool to exchange data with your VPC.

  2. Install and configure a forward proxy for your VPC.

    Before installing a forward proxy, purchase an ECS with the latest Ubuntu image and bind an EIP to the ECS. Then, log in to the ECS, and install and configure a squid forward proxy.

    1. If Docker is not installed, run the following command to install it: 
      curl -sSL https://get.daocloud.io/docker | sh
    2. Pull the squid image. 
      docker pull ubuntu/squid
    3. Create a host directory and configure whitelist.conf and squid.conf.

      Create a host directory:

      mkdir –p /etc/squid/

      Add the whitelist.conf configuration file. The content is the addresses that can be accessed. For example:

      .apig.cn-east-3.huaweicloudapis.com

      Add the squid.conf configuration file, which includes the following:

      # An ACL named 'whitelist'
acl whitelist dstdomain '/etc/squid/whitelist.conf'
 
# Allow whitelisted URLs through
http_access allow whitelist
 
# Block the rest
http_access deny all
 
# Default port
http_port 3128

      Set the permissions on the host directory and configuration files:

      chmod 640 -R /etc/squid
    4. Start a squid instance. 
      docker run -d --name squid -e TZ=UTC -v /etc/squid:/etc/squid -p 3128:3128 ubuntu/squid:latest
    5. If whitelist.conf or squid.conf is updated, go to the container and update the squid. 
      docker exec –it squid bash
root@{container_id}:/# squid -k reconfigure

  3. Configure the DNS proxy and Internet access URL in the algorithm image.

    1. Set the proxy.

      In the code, specify the private IP address and port of the proxy server, as shown in the following:

      proxies = {
  "http": "http://{proxy_server_private_ip}:3128",
  "https": "http://{proxy_server_private_ip}:3128"
}

      The following figure shows how to obtain the private IP address of a server.

      Figure 4 Private IP address
    2. Configure the Internet access URL.

      In the inference code, use the service URL to send a service request, for example:

      https://e8a048ce25136addbbac23ce6132a.apig.cn-east-3.huaweicloudapis.com

Parent topic: Model Inference