Updated on 2025-07-28 GMT+08:00

(Optional) Creating an API

The API management module can manage APIs of the evaluation services. You can select an evaluation service when creating an evaluation job. You can create, edit, and delete APIs.

  1. Log in to ModelArts Studio. In the My Spaces area, click the required workspace.
    Figure 1 My Spaces
  2. In the navigation pane, choose Evaluation Center > API Management. Click Create API in the upper right corner.
  3. Set the API name and organization, select a model type, and click submit.
    Figure 2 Creating an API
  4. After an API is created, add versions on the right to define the API information. You can add multiple versions.
  5. Click New Version and configure the model version, Concurrency Level, QPM, URL, Param, Body, and Response.
    1. model version: user-defined.
    2. Concurrency Level: Set it based on the model performance. The value ranges from 1 to 32.
    3. QPM: Set it based on the model performance. The value ranges from 1 to 5000.
    4. URL: Set a model inference address.
    5. Param: You can set the parameters based on the API requirements. You can define the header and request parameters.
    6. Body: inference request body. You can customize the body or select the preset OpenAI or TGI structure.
    7. Response: defines how to obtain the content in the response body in JsonPath format. The JsonPath syntax is used to extract required data from the JSON field in the response body.
    Figure 3 Adding a version for an API
  6. After creating an API version, you need to verify whether the API is available. Only verified APIs can be evaluated. On the API Management page, click the name of the created API task, click Interface verification, enter a question, and check whether the API response content is displayed.
    Figure 4 Interface verification

Creating an API Service

In this example, we use a deployed and properly running Pangu NLP model to create an API service.

  1. On the Model Development > Model Deployment page, find the deployed NLP model, click the service name, and copy the API URL on the details page.
  2. In the navigation pane, choose Evaluation Center > API Management. Click Create API in the upper right corner. Define the API name and organization structure, and set the model type to NLP.
  3. After an API is created, add versions on the right to define the API information. You can add multiple versions. Click New Version and set parameters based on Table 1.
    Table 1 Configuring the API service

    Parameter

    Description

    model version

    The value can be customized.

    Concurrency Level

    Set this parameter to 1.

    QPM

    Set this parameter to 1.

    URL

    Enter the obtained API URL.

    Param

    Set two parameters of the header type.

    Parameter 1: The parameter name is X-AUTH-TOKEN, and the parameter value is the token value. For details about how to obtain the token, see "Calling REST APIs" > "Authentication" in API Reference.

    Parameter 2: The parameter name is Content-Type and the parameter value is application/json.

    Body

    Select custom and enter the following content:

    {"model": "model","messages": [{"role": "user","content": "{{message}}"}],"max_tokens": 1024, "temperature": 0.7,"top_p": 0.95,"presence_penalty": 0.5,"frequency_penalty": 0.5,"stream": false}

    Response

    Enter the following content:

    $.choices[0].message.content

    Figure 5 Configuring the API service