Help Center> ModelArts> User Guide (Senior AI Engineers)> Model Management> Model Evaluation and Diagnosis> Model Optimization Suggestions> Analysis on Class Errors of Object Detection Models and Solution
View PDF

Analysis on Class Errors of Object Detection Models and Solution

Symptom

During the model evaluation phase of object detection, false positive analysis and false negative analysis are important.

  • In false positive analysis, the class error is mainly reflected by the probability of class false positive.
  • In false negative analysis, the class error is mainly reflected by the probability of class false negative.

The model evaluation results in ModelArts are as follows:

Figure 1 Model evaluation results

The preceding figure shows an example of analysis on the object detection inference results. Graph a shows the analysis on the position false positive, class false positive, and background false positive. Graph b shows the analysis on the position false negative, class false negative, and background false negative.

In the preceding figure, class errors account for a large proportion of false positive and false negative errors. You are advised to refer to the following algorithms and technical description to understand how to reduce the class error probability and improve model inference accuracy.

Solution

Multiscale is a common data processing mode in object detection. You can define the scales in your models. During model training, you are advised to perform scaling at a fixed interval (steps) to ensure the multiscale input of the images in the training of a single epoch. The multiscale training process is as follows:

Figure 2 Multiscale training process

Verification

A dataset for camera type identification is used for verification. The dataset has 14 classes, which represent 14 different cameras. The comparison between the false positive analysis before and after multiscale is used shows that multiscale significantly reduces the class false positive ratio.

Figure 3 Analysis and comparison of false positives after multiscale is used

The comparison between the false negative analysis before and after multiscale is used shows that multiscale significantly reduces the class false negative ratio.

Figure 4 Analysis and comparison of false negatives after multiscale is used

Suggestions

In the model inference result, if the class error ratio is large, you are advised to use multiscale to optimize model training.