How Do I Fill in the Request Header and Request Body of an Inference Request When a Real-Time Service Is Running?

Symptom

After a real-time service is deployed, you can obtain its inference request address on the Usage Guides tab of the service details page when the service is running. However, there is no instruction for filling in the header and body of an inference request.

Possible Cause

The inference request address on the Usage Guides tab of the service details page can be called for inference. For security purposes, ModelArts takes authentication and authorization measures to prevent unauthorized calling of the real-time service. Therefore, the header of a prediction request contains the identity information of the request initiator, and the body contains the content to be predicted.

The header must be authenticated by following HUAWEI CLOUD authentication rules. The body must be configured based on model requirements, such as the requirements of pre-processing scripts or custom images.

Solution

• Header:

  On the Usage Guides tab of the service details page, you can obtain a maximum of two API addresses, one for IAM or AK/SK authentication and the other for application authentication. The header structure varies depending on the authentication mode.

  • IAM or AK/SK authentication: In the header, enter the domain-level token of the tenant in the target region in the X-Auth-Token field. For details, see Obtaining a User Token Through Password Authentication.
  • Application authentication: Application authentication can be further classified as AppCode authentication and application signature authentication.
    • For AppCode authentication, enter the AppCode of the application associated with the real-time service in the X-Apig-AppCode field of the header.
    • For application signature authentication, in the header, enter the X-Sdk-Date and Authorization values generated using the AppKey and AppSecret of the application associated with the real-time service through the SDK or tool to authenticate the signature of the request. For details, see Access Authenticated Using an Application.
• Body:

  The body varies depending on the model source.

  • If the model is imported from a container image, the body must be configured based on the custom image requirements. For details, contact the image creator.
  • If the model is imported from OBS, the requirements on the body are reflected in inference code preprocessing, which will convert the input HTTP body into the input required by the model. For details, see Specifications for Model Inference Coding.
  • If the model is obtained from AI Gallery, check the calling description in AI Gallery or consult the model provider.

Summary and Suggestions

None