Overview

Developed AI models can be used to create AI applications, which can then be quickly deployed as inference services. These services can be integrated into your IT platform by calling APIs or generate batch results.

Figure 1 Introduction to inference

1. Train a model: Models can be trained in ModelArts or your local development environment. A locally developed model must be uploaded to Huawei Cloud OBS.
2. Create an AI application: Import the model file and inference file to the ModelArts model repository and manage them by version. Use these files to build an executable AI application.
3. Deploy a service: Deploy the AI application as a service type based on your needs.
   • Deploying an AI Application as Real-Time Inference Jobs

     Deploy an AI application as a web service with real-time UI and monitoring. This service provides you a callable API.

   • Deploying an AI Application as a Batch Inference Service

     Deploy an AI application as a batch service that performs inference on batch data and automatically stops after data processing is complete.

     Figure 2 Different inference scenarios