Full-Process Development of WebSocket Real-Time Services

Context

WebSocket is a network transmission protocol that supports full-duplex communication over a single TCP connection. It is located at the application layer in an OSI model. The WebSocket communication protocol was established by IETF in 2011 as standard RFC 6455 and supplemented by RFC 7936. The WebSocket API in the Web IDL is standardized by W3C.

WebSocket simplifies data exchange between the client and the server and allows the server to proactively push data to the client. In the WebSocket API, if the initial handshake between the client and the server is successful, a persistent connection will be established between them and data can be transferred bidirectionally.

Prerequisites

• You are experienced in developing Java and familiar with JAR packaging.
• You have basic knowledge and calling methods of WebSocket.
• You are familiar with the method of creating an image using Docker.

Constraints

• WebSocket supports only the deployment of real-time services.
• It supports only real-time services deployed using models imported from custom images.

Preparations

Before using WebSocket in ModelArts for inference, bring your own custom image. The custom image must be able to provide complete WebSocket services in a standalone environment, for example, completing WebSocket handshakes and exchanging data between the client to the server. The model inference is implemented in the custom image, including downloading the model, loading the model, performing preprocessing, completing inference, and assembling the response body.

Procedure

To develop a WebSocket real-time service, perform the following operations:

• Uploading the Image to SWR
• Creating a Model Using an Image
• Deploying the Model as a Real-Time Service
• Calling the WebSocket Real-Time Service

Uploading the Image to SWR

Upload the local image to SWR.

Creating a Model Using an Image

1. Log in to the ModelArts console. In the navigation pane on the left, choose Model Management (AI Applications). On the displayed page, click Create Model.
2. Configure the AI application.
   • Meta Model Source: Select Container image.
   • Container Image Path: Select the path specified in Uploading the Image to SWR.
   • Container API: Configure this parameter based on site requirements.
   • Health Check: Retain default settings. If health check has been configured in the image, configure the health check parameters based on those configured in the image.
     Figure 1 Model configuration parameters
     
3. Click Create now. Wait until the model status changes to Normal.

Deploying the Model as a Real-Time Service

1. Log in to the ModelArts management console. In the navigation pane on the left, choose Model Deployment > Real-Time Services. On the displayed page, click Deploy.
2. Configure the service.
   • Model and Version: Select the model and version created in Creating a Model Using an Image.
   • WebSocket: Enable this function.
     Figure 2 WebSocket
     
3. Click Next, confirm the configuration, and click Submit. In the real-time service list you will be redirected to, check the service status. When it changes to Running, the real-time service has been deployed.

Calling a WebSocket Real-Time Service

WebSocket itself does not require additional authentication. ModelArts WebSocket is WebSocket Secure-compliant, regardless of whether WebSocket or WebSocket Secure is enabled in the custom image. WebSocket Secure supports only one-way authentication, from the client to the server.

You can use one of the following authentication methods provided by ModelArts:

• Token-based Authentication
• AK/SK
• App Authentication

The following section uses GUI software Postman for prediction and token authentication as an example to describe how to call WebSocket.

1. Establish a WebSocket connection.
2. Exchange data between the WebSocket client and the server.

1. Establish a WebSocket connection.
   1. Open Postman of a version later than 8.5, for example, 10.12.0. Click in the upper left corner and choose File > New. In the displayed dialog box, select WebSocket Request (beta version currently).
      Figure 3 WebSocket Request
      
   2. Configure parameters for the WebSocket connection.

      Select Raw in the upper left corner. Do not select Socket.IO (a type of WebSocket implementation, which requires that both the client and the server run on Socket.IO). In the address box, enter the API Address obtained on the Usage Guides tab on the service details page. If there is a finer-grained URL in the custom image, add the URL to the end of the address. If queryString is available, add this parameter in the params column. Add authentication information into the header. The header varies depending on the authentication mode, which is the same as that in the HTTPS-compliant inference service. Click Connect in the upper right corner to establish a WebSocket connection.

      Figure 4 Obtaining the API address
      
      

      • If the information is correct, CONNECTED will be displayed in the lower right corner.
      • If establishing the connection failed and the status code is 401, check the authentication.
      • If a keyword such as WRONG_VERSION_NUMBER is displayed, check whether the port configured in the custom image is the same as that configured in WebSocket or WebSocket Secure.

      The following shows an established WebSocket connection.

      Figure 5 Connection established
      
      

      Preferentially check the WebSocket service provided by the custom image. The type of implementing WebSocket varies depending on the tool you used. Possible issues are as follows: A WebSocket connection can be established but cannot be maintained, or the connection is interrupted after one request and needs to be reconnected. ModelArts only ensures that it will not affect the WebSocket status in a custom image (the API address and authentication mode may be changed on ModelArts).

2. Exchange data between the WebSocket client and the server.

   After the connection is established, WebSocket uses TCP for full-duplex communication. The WebSocket client sends data to the server. The implementation types vary depending on the client, and the lib package may also be different for the same language. Different implementation types are not considered here.

   The format of the data sent by the client is not limited by the protocol. Postman supports text, JSON, XML, HTML, and Binary data. Take text as an example. Enter the text data in the text box and click Send on the right to send the request to the server. If the text is oversized, Postman may be suspended.

   Figure 6 Sending data