How Do I Fill in the Request Header and Request Body of an Inference Request When a Real-Time Service Is Running?

Symptom

After a real-time inference service is deployed, you can obtain its inference request address in the Usage Guides tab of the service details page when the service is running. However, there is no instruction for filling in the header and body of an inference request.

Possible Causes

After a real-time inference service is deployed and is running, you can send a prediction request to the service using the URL in the Service tab. For security purposes, ModelArts takes authentication and authorization measures to prevent unauthorized calling of the real-time inference service. Therefore, the header of a prediction request contains the identity information of the request initiator, and the body contains the content to be predicted.

The header must be added with authentication information by following Huawei Cloud authentication rules. The body must be entered based on the format requirements of the inference service.

Solution

• Header:

  Real-time inference services offer three authentication options: none, token, and API key. Find the API call URL in network settings on the Service tab page. Headers differ based on the chosen authentication method. The details are as follows:

  • IAM token authentication: In the header, enter the domain-level token of the tenant in the target region in the X-Auth-Token field. For details, see Obtaining a User Token Through Password Authentication.
  • API key authentication: Enter the API key of the tenant in the Authorization field of the header.
  • None: Authentication information is not required.
• Body:

  The body varies depending on the model source.

  • If the model is imported from a container image, the body must be configured based on the custom image requirements. For details, contact the image creator.
  • If the model is imported from OBS, the requirements on the body are reflected in inference code preprocessing, which will convert the input HTTP body into the input required by the model. For details, see Specifications for Model Inference Coding.
  • If the model is obtained from AI Gallery, check the call description in AI Gallery or consult the model provider.