Specifications for Using a Custom Engine to Create an AI Model

When using a custom engine to create an AI application, you can select your image stored in SWR as the engine and specify a file directory in OBS as the model package. In this way, bring-your-own images can be used to meet your dedicated requirements.

Before deploying such an AI application as a service, ModelArts downloads the SWR image to the cluster and starts the image as a container as the user whose UID is 1000 and GID is 100. Then, ModelArts downloads the OBS file to the /home/mind/model directory in the container and runs the boot command preset in the SWR image. The service available to port 8080 in the container is automatically registered with APIG. You can access the service through the APIG URL.

Specifications for Using a Custom Engine to Create an AI Application

To use a custom engine to create an AI application, ensure the SWR image, OBS model package, and file size comply with the following requirements:

SWR image specifications
- A common user named ma-user in group ma-group must be built in the SWR image. Additionally, the UID and GID of the user must be 1000 and 100, respectively. The following is the dockerfile command for the built-in user:
```
groupadd -g 100 ma-group && useradd -d /home/ma-user -m -u 1000 -g 100 -s /bin/bash ma-user
```
- Specify a command for starting the image. In the dockerfile, specify cmd. The following shows an example:
```
CMD sh /home/mind/run.sh
```
  Customize the boot file run.sh. The following is an example:
```
#!/bin/bash

# User-defined script content
...

# run.sh calls app.py to start the server. For details about app.py, see "HTTPS Example".
python app.py
```
- The service must be HTTPS enabled, and it is available on port 8080. For details, see the HTTPS example.
- (Optional) On port 8080, enable health check with URL /health. (The health check URL must be /health.)

OBS model package specifications
The name of the model package must be model. For details about the model package specifications, see Model Package Specifications.
File size specifications
When a public resource pool is used, the total size of the downloaded SWR image (not the compressed image displayed on the SWR page) and the OBS model package cannot exceed 30 GB.

HTTPS Example

Use Flask to start HTTPS. The following is an example of the web server code:

from flask import Flask, request
import json 

app = Flask(__name__)

@app.route('/greet', methods=['POST'])
def say_hello_func():
    print("----------- in hello func ----------")
    data = json.loads(request.get_data(as_text=True))
    print(data)
    username = data['name']
    rsp_msg = 'Hello, {}!'.format(username)
    return json.dumps({"response":rsp_msg}, indent=4)

@app.route('/goodbye', methods=['GET'])
def say_goodbye_func():
    print("----------- in goodbye func ----------")
    return '\nGoodbye!\n'


@app.route('/', methods=['POST'])
def default_func():
    print("----------- in default func ----------")
    data = json.loads(request.get_data(as_text=True))
    return '\n called default func !\n {} \n'.format(str(data))

@app.route('/health', methods=['GET'])
def healthy():
    return "{\"status\": \"OK\"}"

# host must be "0.0.0.0", port must be 8080
if __name__ == '__main__':
    app.run(host="0.0.0.0", port=8080, ssl_context='adhoc')

Debugging on a Local Computer

Perform the following operations on a local computer with Docker installed to check whether a custom engine complies with specifications:

Download the custom image, for example, custom_engine:v1 to the local computer.
Copy the model package folder model to the local computer.
Run the following command in the same directory as the model package folder to start the service:
```
docker run --user 1000:100 -p 8080:8080 -v model:/home/mind/model    custom_engine:v1
```
This command is used for simulation only because the directory mounted to -v is assigned the root permission. In the cloud environment, after the model file is downloaded from OBS to /home/mind/model, the file owner will be changed to ma-user.
Start another terminal on the local computer and run the following command to obtain the expected inference result:
```
curl https://127.0.0.1:8080/${Request path to the inference service}
```

Deployment Example

The following section describes how to use a custom engine to create an AI application.

Create an AI application and view its details.
Log in to the ModelArts console, choose AI Applications from the navigation pane and click Create Applications. Configure the following parameters:
- Meta Model Source: Select OBS.
- Meta Model: Select a model package from OBS.
- AI Engine: Select Custom.
- Engine Package: Select a container image from SWR.
  Retain default settings for other parameters.
Click Create now. Wait until the application status changes to Normal.

Figure 1 Creating an AI application

Click the AI application name. On the displayed page, view details about the AI application.
Deploy the AI application as a service and view service details.
On the AI application details page, choose Deploy > Real-Time Services in the upper right corner. On the Deploy page, select a proper compute node flavor (for example, CPU: 2 vCPUs 8 GB), retain default settings for other parameters, and click Next. When the service status changes to Running, the service has been deployed.

Click the service name. On the displayed page, view the service details. Click the Logs tab to view the service logs.

Figure 2 Service logs
Use the service for prediction.
On the service details page, click the Prediction tab to use the service for prediction.

Figure 3 Service prediction