Service Developers: Building Models Using ExeML

Preparations

• Your HUAWEI CLOUD account is not in arrears or frozen.
• Access authorization has been configured for your account. For details, see Configuring Agency Authorization. If you have been authorized using access keys, clear the authorization and configure agency authorization.
• OBS buckets and folders are ready for model data storage. For details about how to create OBS buckets and folders, see Creating a Bucket and Creating a Folder. For normal data access, ensure that the created OBS bucket and ModelArts are in the same region.
• Data management is required for creating ExeML projects. Therefore, obtain the permission to access OBS from the Data Management module before using ExeML.

  On the ModelArts management console, choose Data Management > Datasets in the left navigation pane. On the page that is displayed, click Service Authorization to apply for permission authorization. If you log in using an account, a dialog box is displayed, asking you to accept the authorization. If you log in as an IAM user (member account), your master account or a user with admin permissions grants authorization to you.

Step 1: Prepare Data

ModelArts provides a sample dataset of Yunbao named Yunbao-Data-Custom. This example uses this dataset to build a model. Perform the following operations to upload the dataset to the OBS directory test-modelarts/dataset-yunbao created in preparation. The OBS bucket name test-modelarts is for reference only. You need to customize an OBS bucket name.

If you want to use your own dataset, skip this step, upload the dataset to the OBS folder, and select this directory in Step 2: Create an Object Detection Project.

1. Download the Yunbao-Data-Custom dataset to the local PC.
2. Decompress the Yunbao-Data-Custom.zip file to the Yunbao-Data-Custom directory on the local PC.
3. Batch upload all files from the Yunbao-Data-Custom folder to the test-modelarts/dataset-yunbao directory on OBS. For details, see Uploading a File.

   The obtained dataset has two directories: eval and train. The data stored in train is used for model training, and the data stored in eval is used for model prediction.

Step 2: Create an Object Detection Project

1. On the ModelArts management console, click ExeML in the left navigation pane.
2. In the Object Detection box, click Create Project. On the Create Object Detection Project page that is displayed, enter a project name and a dataset name, and select an input dataset path. The OBS path of the Yunbao dataset is /test-modelarts/dataset-yunbao/train/. Select an empty directory in Output Dataset Path.
   

   The Yunbao dataset has two directories: eval and train. Select the data in the train directory for training. If the upper-layer directory of train is selected, an error message is displayed, indicating that OBS has invalid data. As a result, the project will fail to be created.

   Figure 1 Creating an object detection project
   
3. Click Create Project. The object detection project is created. After the project is created, the ExeML > Label Data page is displayed and data source synchronization is automatically performed.

Step 3: Label Data

For an object detection project, labeling data is to locate an object in an image and assign a label to the object. The labeled data is used for model training. In the Yunbao dataset, part of data has been labeled. You can label the unlabeled data for trial use.

Data source synchronization is automatically performed when you create an ExeML project. Data source synchronization takes a certain period of time. If the synchronization fails, you can click Synchronize Data Source to manually execute the synchronization.

1. On the ExeML > Label Data page, click the Unlabeled tab. All unlabeled images are displayed. Click an image to go to the labeling page.
2. Left-click and drag the mouse to select the area where Yunbao is located. In the dialog box that is displayed, enter the label name, for example, yunbao, and press Enter. After the labeling is completed, the status of the image changes to Labeled in the left Image Catalog pane.
   You can select another image from the image catalog in the lower part of the page and repeat the preceding steps to label the image. If an image contains more than one Yunbao, you can label all. You are advised to label all images in the dataset to train a model with better precision.

   Figure 2 Image labeling for object detection
   
3. After all images in the image directory are labeled, click the project name in the upper left corner. In the dialog box that is displayed, click OK to save the labeling information. On the Labeled tab page, you can view the labeled images and view the label names and quantity in the right pane.

Step 4: Generate a Model with ExeML

1. After data labeling is completed, click Train in the upper right corner of the data labeling page. In the Training Configuration dialog box that is displayed, set related parameters. For details, see Figure 3.
   Figure 3 Setting training parameters
   
2. Click Next. On the configuration page that is displayed, confirm the specifications and click Submit to start auto model training. The training takes a certain period of time. If you close or exit the page, the system continues training until it is completed.
   After the training is completed, you can view the training details on the page, such as the accuracy, evaluation result, training parameters, and classification statistics.

   Figure 4 Model training
   

Step 5: Deploy the Model as a Real-Time Service

1. On the Train Model tab page, wait until the training status changes to Completed. Click Deploy in the Version Manager pane.
   Figure 5 Deploying a service
   
2. In the displayed Deploy dialog box, set Specifications and Auto Stop, and click OK to deploy the object detection model as a real-time service.

   If you select free specifications, you do not need to set Auto Stop, because the node will be stopped one hour later.

   Figure 6 Deployment settings
   
3. After the deployment is started, the system automatically switches to the Deploy Service tab page. This page displays the deployment progress and status.
   The deployment takes a certain period of time. After the deployment is completed, the status in the Version Manager pane changes to Running.

   Figure 7 Successful deployment
   

Step 6: Test the Service

After the model is deployed, you can test the service using an image.

1. On the Deployment Online tab page, select a running service version, and click Upload to upload a local image.
   Figure 8 Uploading an image
   
2. Select an image from a local environment. The image must contain Yunbao. Click Predict to perform the test.

   After the prediction is completed, the label name yunbao, location coordinates, and confidence score are displayed in the prediction result pane on the right. In the prediction result, detection_boxes indicates the location of the object, detection_scores indicates the detection score of yunbao.

   If the model accuracy does not meet your expectation, add images on the Label Data tab page, label the images, and train and deploy the model again.

   

   A running real-time service keeps consuming the resources. If you do not need to use the real-time service, click Stop in the Version Manager pane to stop the service and avoid unnecessary billing. If you want to use the service again, click Start.

   Figure 9 Test result