How Do I Check Whether an Error Is Caused by a Real-Time Inference Service When It Is Running But Prediction Failed?

Symptom

A running real-time inference service is used for prediction. After a prediction request is initiated, the received response does not meet the expectation. It is difficult to determine whether the issue is caused by the inference service.

Possible Causes

After a real-time inference service is started, either of the following methods can be used for prediction:

• Method 1: Perform prediction in the Prediction tab of the service details page on the ModelArts console.
• Method 2: Obtain the call information in the Service tab of the service details page on the ModelArts console and use cURL or Postman to perform prediction.

This issue may occur after an inference request is initiated, regardless of whether method 1 or 2 is used.

An inference request is finally sent to the inference service. The issue may be caused by an error occurred when the service processed the inference request. Determine whether the issue is caused by the service, which facilitates rapid fault locating.

Solution

No matter whether method 1 or 2 is used, obtain the response header and body of the inference request.

• If method 1 is used, obtain the response to the inference request through the developer tool of the browser. Take Google Chrome as an example. Press F12 to open the developer tool, click the Network tab and then Predict. The response to the inference request is displayed in the Network tab.

  Find the inference request in the Name pane. The URL of the inference request contains keyword /v2/infers. View the complete URL in the Headers pane. Obtain the request and response in Headers and Response.

• If method 2 is used, obtain the response header and body through different tools. For example, run the cURL command and use -I to obtain the response header.

If Server in the obtained response header is ModelArts and the response body does not contain a ModelArts.XXXX error code, the response is returned by the model. If the response is not as expected, the issue is caused by the model.

If the error code is ModelArts.8909, check whether the access failure is caused by service connection timeout or other exceptions.