Help Center/ ModelArts/ Model Inference/ Inference Specifications/ Model Package Specifications/ Specifications for Editing a Model Configuration File
Updated on 2024-06-13 GMT+08:00
View PDF

Specifications for Editing a Model Configuration File

A model developer needs to edit a configuration file config.json when publishing a model. The model configuration file describes the model usage, computing framework, precision, inference code dependency package, and model API.

Configuration File Format

The configuration file is in JSON format. Table 1 describes the parameters.

Table 1 Parameters

Parameter

Mandatory

Data Type

Description

model_algorithm

Yes

String

Model algorithm, which is set by the model developer to help model users understand the usage of the model. The value must start with a letter and contain no more than 36 characters. Chinese characters and special characters (&!'\"<>=) are not allowed. Common model algorithms include image_classification (image classification), object_detection (object detection), and predict_analysis (prediction analysis).

model_type

Yes

String

Model AI engine, which indicates the computing framework used by a model. Common AI engines and Image are supported.

runtime

No

String

Model runtime environment. Python3.6 is used by default The value of runtime depends on the value of model_type. If model_type is set to Image, you do not need to set runtime. If model_type is set to another mainstream framework, select the engine and runtime environment. For details about the supported running environments, see Supported AI Engines for ModelArts Inference.

If your model must run on specified CPUs or GPUs, select the CPUs or GPUs based on the runtime suffix. If the runtime does not contain the CPU or GPU information, check the runtime description in Supported AI Engines for ModelArts Inference.

metrics

No

Object

Model precision information, including the average value, recall rate, precision, and accuracy. For details about the metrics object structure, see Table 2.

The result is displayed in the model precision area on the AI application details page.

apis

No

api array

Format of the requests received and returned by a model. The value is structure data.

It is the RESTful API array provided by a model. For details about the API data structure, see Table 3. For details about the code example, see Code Example of apis Parameters.

  • If model_type is set to Image, the AI application is created using a custom image.
  • When model_type is not Image, only one API whose request path is / can be declared in apis because the preconfigured AI engine exposes only one inference API whose request path is /.

dependencies

No

dependency array

Package on which the model inference code depends, which is structure data.

Model developers need to provide the package name, installation mode, and version constraints. Only the pip installation mode is supported. Table 6 describes the dependency array.

If the model package does not contain the customize_service.py file, you do not need to set this parameter. Dependency packages cannot be installed for custom image models.

health

No

health data structure

Configuration of an image health interface. This parameter is mandatory only when model_type is set to Image.

If services cannot be interrupted during a rolling upgrade, a health check API must be provided for ModelArts to call. For details about the health data structure, see Table 8.
Table 2 metrics object description

Parameter

Mandatory

Data Type

Description

f1

No

Number

F1 score. The value is rounded to 17 decimal places.

recall

No

Number

Recall rate. The value is rounded to 17 decimal places.

precision

No

Number

Precision. The value is rounded to 17 decimal places.

accuracy

No

Number

Accuracy. The value is rounded to 17 decimal places.
Table 3 api array

Parameter

Mandatory

Data Type

Description

url

No

String

Request path. The default value is a slash (/). For a custom image model (model_type is Image), set this parameter to the actual request path exposed in the image. For a non-custom image model (model_type is not Image), the URL can only be /.

method

No

String

Request method. The default value is POST.

request

No

Object

Request body. For details, see Table 4.

response

No

Object

Response body. For details, see Table 5.
Table 4 request description

Parameter

Mandatory

Data Type

Description

Content-type

No for real-time services

Yes for batch services

String

Data is sent in a specified content format. The default value is application/json.

The options are as follows:

  • application/json: JSON data is uploaded.
  • multipart/form-data: A file is uploaded.
NOTE:

For machine learning models, only application/json is supported.

data

No for real-time services

Yes for batch services

String

The request body is described in JSON schema. For details about the parameter description, see the official guide.
Table 5 response description

Parameter

Mandatory

Data Type

Description

Content-type

No for real-time services

Yes for batch services

String

Data is sent in a specified content format. The default value is application/json.

NOTE:

For machine learning models, only application/json is supported.

data

No for real-time services

Yes for batch services

String

The response body is described in JSON schema. For details about the parameter description, see the official guide.
Table 6 dependency array

Parameter

Mandatory

Data Type

Description

installer

Yes

String

Installation method. Only pip is supported.

packages

Yes

package array

Dependency package collection. For details about the package structure array, see Table 7.
Table 7 package array

Parameter

Mandatory

Type

Description

package_name

Yes

String

Dependency package name. Chinese characters and special characters (&!'"<>=) are not allowed.

package_version

No

String

Dependency package version. If the dependency package does not rely on package versions, leave this field blank. Chinese characters and special characters (&!'"<>=) are not allowed.

restraint

No

String

Version restriction. This parameter is mandatory only when package_version is configured. Possible values are EXACT, ATLEAST, and ATMOST.

  • EXACT indicates that a specified version is installed.
  • ATLEAST indicates that the version of the installation package is not earlier than the specified version.
  • ATMOST indicates that the version of the installation package is not later than the specified version.
    NOTE:
    • If there are specific requirements on the version, preferentially use EXACT. If EXACT conflicts with the system installation packages, you can select ATLEAST.
    • If there is no specific requirement on the version, retain only the package_name parameter and leave restraint and package_version blank.
Table 8 health data structure description

Parameter

Mandatory

Type

Description

check_method

Yes

String

Health check method. The value can be HTTP or EXEC.

  • HTTP: Use an HTTP request.
  • EXEC: Execute a command.

command

No

String

Health check command. This parameter is mandatory when check_method is set to EXEC.

url

No

String

Request URL of a health check API. This parameter is mandatory when check_method is set to HTTP.

protocol

No

String

Request protocol of a health check API. The default value is http. This parameter is mandatory when check_method is set to HTTP.

initial_delay_seconds

No

String

Delay for initializing the health check.

timeout_seconds

No

String

Health check timeout.

period_seconds

Yes

String

Health check period, in seconds. Enter an integer greater than 0 and no more than 2147483647.

failure_threshold

Yes

String

Maximum number of health check failures. Enter an integer greater than 0 and no more than 2147483647.

Code Example of apis Parameters

[{
    "url": "/",
    "method": "post",
    "request": {
        "Content-type": "multipart/form-data",
        "data": {
            "type": "object",
            "properties": {
                "images": {
                    "type": "file"
                }
            }
        }
    },
    "response": {
        "Content-type": "applicaton/json",
        "data": {
            "type": "object",
            "properties": {
                "mnist_result": {
                    "type": "array",
                    "item": [
                        {
                            "type": "string"
                        }
                    ]
                }
            }
        }
    }
}]

Example of the Object Detection Model Configuration File

The following code uses the TensorFlow engine as an example. You can modify the model_type parameter based on the actual engine type.

  • Model input

    Key: images

    Value: image files

  • Model output 
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
    		 
    {
    "detection_classes": [
        "face",
        "arm"
    ],
    "detection_boxes": [
        [
            33.6,
            42.6,
            104.5,
            203.4
        ],
        [
            103.1,
            92.8,
            765.6,
            945.7
        ]
    ],
    "detection_scores": [0.99, 0.73]
}
  • Configuration file 
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
    		 
    {
    "model_type": "TensorFlow",
    "model_algorithm": "object_detection",
    "metrics": {
        "f1": 0.345294,
        "accuracy": 0.462963,
        "precision": 0.338977,
        "recall": 0.351852
    },
    "apis": [{
        "url": "/",
        "method": "post",
        "request": {
            "Content-type": "multipart/form-data",
            "data": {
                "type": "object",
                "properties": {
                    "images": {
                        "type": "file"
                    }
                }
            }
        },
        "response": {
            "Content-type": "application/json",
            "data": {
                "type": "object",
                "properties": {
                    "detection_classes": {
                        "type": "array",
                        "items": [{
                            "type": "string"
                        }]
                    },
                    "detection_boxes": {
                        "type": "array",
                        "items": [{
                            "type": "array",
                            "minItems": 4,
                            "maxItems": 4,
                            "items": [{
                                "type": "number"
                            }]
                        }]
                    },
                    "detection_scores": {
                        "type": "array",
                        "items": [{
                            "type": "number"
                        }]
                    }
                }
            }
        }
    }],
    "dependencies": [{
        "installer": "pip",
        "packages": [{
                "restraint": "EXACT",
                "package_version": "1.15.0",
                "package_name": "numpy"
            },
            {
                "restraint": "EXACT",
                "package_version": "5.2.0",
                "package_name": "Pillow"
            }
        ]
    }]
}

Example of the Image Classification Model Configuration File

The following code uses the TensorFlow engine as an example. You can modify the model_type parameter based on the actual engine type.

  • Model input

    Key: images

    Value: image files

  • Model output 
    1
2
3
4
5
6
7
    		 
    {
    "predicted_label": "flower",
    "scores": [
       ["rose", 0.99],
       ["begonia", 0.01]
    ]
}
  • Configuration file 
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
    		 
    {
    "model_type": "TensorFlow",
    "model_algorithm": "image_classification",
    "metrics": {
        "f1": 0.345294,
        "accuracy": 0.462963,
        "precision": 0.338977,
        "recall": 0.351852
    },
    "apis": [{
        "url": "/",
        "method": "post",
        "request": {
            "Content-type": "multipart/form-data",
            "data": {
                "type": "object",
                "properties": {
                    "images": {
                        "type": "file"
                    }
                }
            }
        },
        "response": {
            "Content-type": "application/json",
            "data": {
                "type": "object",
                "properties": {
                    "predicted_label": {
                        "type": "string"
                    },
                    "scores": {
                        "type": "array",
                        "items": [{
                            "type": "array",
                            "minItems": 2,
                            "maxItems": 2,
                            "items": [
                                {
                                    "type": "string"
                                },
                                {
                                    "type": "number"
                                }
                            ]
                        }]
                    }
                }
            }
        }
    }],
    "dependencies": [{
        "installer": "pip",
        "packages": [{
                "restraint": "ATLEAST",
                "package_version": "1.15.0",
                "package_name": "numpy"
            },
            {
                "restraint": "",
                "package_version": "",
                "package_name": "Pillow"
            }
        ]
    }]
}

The following code uses the MindSpore engine as an example. You can modify the model_type parameter based on the type of the engine you use.

  • Model input

    Key: images

    Value: image files

  • Model output 
    1
    		 
    "[[-2.404526   -3.0476532  -1.9888215   0.45013925 -1.7018927   0.40332815\n  -7.1861157  11.290332   -1.5861531   5.7887416 ]]"
  • Configuration file 
    {
     "model_algorithm": "image_classification",
     "model_type": "MindSpore",
     "metrics": {
         "f1": 0.124555,
         "recall": 0.171875,
         "precision": 0.0023493892851938493,
         "accuracy": 0.00746268656716417
     },
     "apis": [{
             "url": "/",
             "method": "post",
             "request": {
                 "Content-type": "multipart/form-data",
                 "data": {
                     "type": "object",
                     "properties": {
                         "images": {
                             "type": "file"
                         }
                     }
                 }
             },
             "response": {
                 "Content-type": "applicaton/json",
                 "data": {
                     "type": "object",
                     "properties": {
                         "mnist_result": {
                             "type": "array",
                             "item": [{
                                 "type": "string"
                             }]
                         }
                     }
                 }
             }
         }
     ],
    "dependencies": []
 }

Example of the Predictive Analytics Model Configuration File

The following code uses the TensorFlow engine as an example. You can modify the model_type parameter based on the actual engine type.

  • Model input 
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
    		 
    {
    "data": {
        "req_data": [
            {
                "buying_price": "high",
                "maint_price": "high",
                "doors": "2",
                "persons": "2",
                "lug_boot": "small",
                "safety": "low",
                "acceptability": "acc"
            },
            {
                "buying_price": "high",
                "maint_price": "high",
                "doors": "2",
                "persons": "2",
                "lug_boot": "small",
                "safety": "low",
                "acceptability": "acc"
            }
        ]
    }
}
  • Model output 
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    		 
    {
    "data": {
        "resp_data": [
            {
                "predict_result": "unacc"
            },
            {
                "predict_result": "unacc"
            }
        ]
    }
}
  • Configuration file

    In the code, the data parameter in the request and response structures is described in JSON Schema. The content in data and properties corresponds to the model input and output.

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
    		 
    {
    "model_type": "TensorFlow",
    "model_algorithm": "predict_analysis",
    "metrics": {
        "f1": 0.345294,
        "accuracy": 0.462963,
        "precision": 0.338977,
        "recall": 0.351852
    },
    "apis": [
        {
            "url": "/",
            "method": "post",
            "request": {
                "Content-type": "application/json",
                "data": {
                    "type": "object",
                    "properties": {
                        "data": {
                            "type": "object",
                            "properties": {
                                "req_data": {
                                    "items": [
                                        {
                                            "type": "object",
                                            "properties": {}
                                        }
                                    ],
                                    "type": "array"
                                }
                            }
                        }
                    }
                }
            },
            "response": {
                "Content-type": "application/json",
                "data": {
                    "type": "object",
                    "properties": {
                        "data": {
                            "type": "object",
                            "properties": {
                                "resp_data": {
                                    "type": "array",
                                    "items": [
                                        {
                                            "type": "object",
                                            "properties": {}
                                        }
                                    ]
                                }
                            }
                       }
                    }
                }
            }
        }
    ],
    "dependencies": [
        {
            "installer": "pip",
            "packages": [
                {
                    "restraint": "EXACT",
                    "package_version": "1.15.0",
                    "package_name": "numpy"
                },
                {
                    "restraint": "EXACT",
                    "package_version": "5.2.0",
                    "package_name": "Pillow"
                }
            ]
        }
    ]
}

Example of the Custom Image Model Configuration File

The model input and output are similar to those in Example of the Object Detection Model Configuration File.

  • If the input is an image, the request example is as follows.

    In the example, a model prediction request containing the parameter images with the parameter type of file is received. For this example, the file upload button is displayed on the inference page, and the inference is performed in file format.

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
    		 
    {
    "Content-type": "multipart/form-data",
    "data": {
        "type": "object",
        "properties": {
            "images": {
                "type": "file"
            }
        }
    }
}
  • If the input is JSON data, the request example is as follows.

    In this example, the model prediction JSON request body is received. In the request, there is only one prediction request containing the parameter input with the parameter type of string. On the inference page, a text box is displayed for you to enter the prediction request.

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
    		 
    {
    "Content-type": "application/json",
    "data": {
        "type": "object",
        "properties": {
            "input": {
                "type": "string"
            }
        }
    }
}

A complete request example is as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
 
{
    "model_algorithm": "image_classification",
    "model_type": "Image",
    "metrics": {
        "f1": 0.345294,
        "accuracy": 0.462963,
        "precision": 0.338977,
        "recall": 0.351852
    },
    "apis": [{
        "url": "/",
        "method": "post",
        "request": {
            "Content-type": "multipart/form-data",
            "data": {
                "type": "object",
                "properties": {
                    "images": {
                        "type": "file"
                    }
                }
            }
        },
        "response": {
            "Content-type": "application/json",
            "data": {
                "type": "object",
                "required": [
                    "predicted_label",
                    "scores"
                ],
                "properties": {
                    "predicted_label": {
                        "type": "string"
                    },
                    "scores": {
                        "type": "array",
                        "items": [{
                            "type": "array",
                            "minItems": 2,
                            "maxItems": 2,
                            "items": [{
                                    "type": "string"
                                },
                                {
                                    "type": "number"
                                }
                            ]
                        }]
                    }
                }
            }
        }
    }]
}

Example of the Machine Learning Model Configuration File

The following uses XGBoost as an example:

  • Model input
{
    "req_data": [
        {
            "sepal_length": 5,
            "sepal_width": 3.3,
            "petal_length": 1.4,
            "petal_width": 0.2
        },
        {
            "sepal_length": 5,
            "sepal_width": 2,
            "petal_length": 3.5,
            "petal_width": 1
        },
        {
            "sepal_length": 6,
            "sepal_width": 2.2,
            "petal_length": 5,
            "petal_width": 1.5
        }
    ]
}
  • Model output
{
    "resp_data": [
        {
            "predict_result": "Iris-setosa"
        },
        {
            "predict_result": "Iris-versicolor"
        }
    ]
}
  • Configuration file
{
    "model_type": "XGBoost",
    "model_algorithm": "xgboost_iris_test",
    "runtime": "python2.7",
    "metrics": {
        "f1": 0.345294,
        "accuracy": 0.462963,
        "precision": 0.338977,
        "recall": 0.351852
    },
    "apis": [
        {
            "url": "/",
            "method": "post",
            "request": {
                "Content-type": "application/json",
                "data": {
                    "type": "object",
                    "properties": {
                        "req_data": {
                            "items": [
                                {
                                    "type": "object",
                                    "properties": {}
                                }
                            ],
                            "type": "array"
                        }
                    }
                }
            },
            "response": {
                "Content-type": "applicaton/json",
                "data": {
                    "type": "object",
                    "properties": {
                        "resp_data": {
                            "type": "array",
                            "items": [
                                {
                                    "type": "object",
                                    "properties": {
                                        "predict_result": {}
                                    }
                                }
                            ]
                        }
                    }
                }
            }
        }
    ]
}

Example of a Model Configuration File Using a Custom Dependency Package

The following example defines the NumPy 1.16.4 dependency environment.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
 
{
    "model_algorithm": "image_classification",
    "model_type": "TensorFlow",
    "runtime": "python3.6",
    "apis": [
        {
            "url": "/",
            "method": "post",
            "request": {
                "Content-type": "multipart/form-data",
                "data": {
                    "type": "object",
                    "properties": {
                        "images": {
                            "type": "file"
                        }
                    }
                }
            },
            "response": {
                "Content-type": "applicaton/json",
                "data": {
                    "type": "object",
                    "properties": {
                        "mnist_result": {
                            "type": "array",
                            "item": [
                                {
                                    "type": "string"
                                }
                            ]
                        }
                    }
                }
            }
        }
    ],
    "metrics": {
        "f1": 0.124555,
        "recall": 0.171875,
        "precision": 0.00234938928519385,
        "accuracy": 0.00746268656716417
    },
    "dependencies": [
        {
            "installer": "pip",
            "packages": [
                {
                    "restraint": "EXACT",
                    "package_version": "1.16.4",
                    "package_name": "numpy"
                }
            ]
        }
    ]
}