Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Situation Awareness
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive
Help Center/ ModelArts/ Best Practices/ Model Inference/ Full-Process Development of WebSocket Real-Time Services

Full-Process Development of WebSocket Real-Time Services

Updated on 2024-03-05 GMT+08:00

Context

WebSocket is a network transmission protocol that supports full-duplex communication over a single TCP connection. It is located at the application layer in an OSI model. The WebSocket communication protocol was established by IETF in 2011 as standard RFC 6455 and supplemented by RFC 7936. The WebSocket API in the Web IDL is standardized by W3C.

WebSocket simplifies data exchange between the client and the server and allows the server to proactively push data to the client. In the WebSocket API, if the initial handshake between the client and the server is successful, a persistent connection will be established between them and data can be transferred bidirectionally.

Prerequisites

  • You are experienced in developing Java and familiar with JAR packaging.
  • You have basic knowledge and calling methods of WebSocket.
  • You are familiar with the method of creating an image using Docker.

Constraints

  • WebSocket supports only the deployment of real-time services.
  • WebSocket supports only real-time services deployed using AI applications imported from custom images.

Preparations

Before using WebSocket in ModelArts for inference, bring your own custom image. The custom image must be able to provide complete WebSocket services in a standalone environment, for example, completing WebSocket handshakes and exchanging data between the client to the server. The model inference is implemented in the custom image, including downloading the model, loading the model, performing preprocessing, completing inference, and assembling the response body.

Uploading the Image to SWR

Upload the local image to SWR. For details, see How Can I Log In to SWR and Upload Images to It?

Creating an AI Application Using the Image

  1. Log in to the ModelArts management console, choose AI Application Management > AI Applications, and click Create under My AI Applications. The page for creating an AI application is displayed.
  2. Configure the AI application.
    • Meta Model Source: Select Container image.
    • Container Image Path: Select the path specified in Uploading the Image to SWR.
    • Container API: Configure this parameter based on site requirements.
    • Health Check: Retain default settings. If health check has been configured in the image, configure the health check parameters based on those configured in the image.
      Figure 1 AI application parameters
  3. Click Create now. In the AI application list that is displayed, check the AI application status. When it changes to Normal, the AI application has been created.

Deploying the AI Application as a Real-Time Service

  1. Log in to the ModelArts management console, choose Service Deployment > Real-Time Services, and click Deploy.
  2. Configure the service.
  3. Click Next, confirm the configuration, and click Submit. In the real-time service list you will be redirected to, check the service status. When it changes to Running, the real-time service has been deployed.

Calling a WebSocket Real-Time Service

WebSocket itself does not require additional authentication. ModelArts WebSocket is WebSocket Secure-compliant, regardless of whether WebSocket or WebSocket Secure is enabled in the custom image. WebSocket Secure supports only one-way authentication, from the client to the server.

You can use one of the following authentication methods provided by ModelArts:

The following section uses GUI software Postman for prediction and token authentication as an example to describe how to call WebSocket.

  1. Establish a WebSocket connection.
  2. Exchange data between the WebSocket client and the server.
  1. Establish a WebSocket connection.

    1. Open Postman of a version later than 8.5, for example, 10.12.0. Click in the upper left corner and choose File > New. In the displayed dialog box, select WebSocket Request (beta version currently).
      Figure 3 WebSocket Request
    2. Configure parameters for the WebSocket connection.

      Select Raw in the upper left corner. Do not select Socket.IO (a type of WebSocket implementation, which requires that both the client and the server run on Socket.IO). In the address box, enter the API Address obtained on the Usage Guides tab on the service details page. If there is a finer-grained URL in the custom image, add the URL to the end of the address. If queryString is available, add this parameter in the params column. Add authentication information into the header. The header varies depending on the authentication mode, which is the same as that in the HTTPS-compliant inference service. Click Connect in the upper right corner to establish a WebSocket connection.

      Figure 4 Obtaining the API address
      NOTE:
      • If the information is correct, CONNECTED will be displayed in the lower right corner.
      • If establishing the connection failed and the status code is 401, check the authentication.
      • If a keyword such as WRONG_VERSION_NUMBER is displayed, check whether the port configured in the custom image is the same as that configured in WebSocket or WebSocket Secure.

      The following shows an established WebSocket connection.

      Figure 5 Connection established
      NOTICE:

      Preferentially check the WebSocket service provided by the custom image. The type of implementing WebSocket varies depending on the tool you used. Possible issues are as follows: A WebSocket connection can be established but cannot be maintained, or the connection is interrupted after one request and needs to be reconnected. ModelArts only ensures that it will not affect the WebSocket status in a custom image (the API address and authentication mode may be changed on ModelArts).

  2. Exchange data between the WebSocket client and the server.

    After the connection is established, WebSocket uses TCP for full-duplex communication. The WebSocket client sends data to the server. The implementation types vary depending on the client, and the lib package may also be different for the same language. Different implementation types are not considered here.

    The format of the data sent by the client is not limited by the protocol. Postman supports text, JSON, XML, HTML, and Binary data. Take text as an example. Enter the text data in the text box and click Send on the right to send the request to the server. If the text is oversized, Postman may be suspended.

    Figure 6 Sending data

We use cookies to improve our site and your experience. By continuing to browse our site you accept our cookie policy. Find out more

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback