Accessing a Real-Time Service Using Server-Sent Events

Context

Server-Sent Events (SSE) is a server push technology enabling a server to push events to a client via an HTTP connection. This technology is usually used to enable a server to push real-time data to a client, for example, a chat application or a real-time news update.

SSE primarily facilitates unidirectional real-time communication from the server to the client, such as streaming ChatGPT responses. In contrast to WebSockets, which provide bidirectional real-time communication, SSE is designed to be more lightweight and simpler to implement.

Prerequisites

The image for importing the AI application is SSE-compliant.

Constraints

• SSE supports only the deployment of real-time services.
• It supports only real-time services deployed using AI applications imported from custom images.
• When you call an API to access a real-time service, the size of the prediction request body and the prediction time are subject to the following limitations:
  • The size of a request body cannot exceed 12 MB. Otherwise, the request will fail.
  • Due to the limitation of API Gateway, the prediction duration of each request does not exceed 40 seconds.

Calling an SSE Real-Time Service

The SSE protocol itself does not introduce new authentication mechanisms; it relies on the same methods as HTTP requests.

You can use one of the following authentication methods provided by ModelArts:

• Accessing a Real-Time Service Through Token-based Authentication
• Accessing a Real-Time Service Through AK/SK-based Authentication
• Accessing a Real-Time Service Through App Authentication

The following section uses GUI software Postman for prediction and token authentication as an example to describe how to call an SSE service.

Figure 1 Calling an SSE service

Figure 2 Response header Content-Type

In normal cases, the value of Content-Type in the response header is text/event-stream;charset=UTF-8.