Symptom

During the model evaluation phase of object detection, false positive analysis and false negative analysis are important. In false positive analysis, the background error is mainly reflected by the probability of background false positive. In false negative analysis, the background error is mainly reflected by the probability of background false negative.

The model evaluation results in ModelArts are as follows:

Figure 1 Model evaluation results

The preceding figure shows an example of analysis on the object detection inference results. Graph a shows the analysis on the position false positive, class false positive, and background false positive. Graph b shows the analysis on the position false negative, class false negative, and background false negative.

In the preceding figure, background errors account for a large proportion of false positive and false negative errors. You are advised to refer to the following algorithms and technical description to understand how to reduce the background error probability and improve model inference accuracy.

Solution

Object detection involves multi-task training. In addition to accurate class identification, class instances need to be accurately located. The model training loss includes the class loss and bounding box loss. The most common bounding box loss is the Smooth L1 loss. The calculation formula is as follows:

The balanced loss was first proposed in Libra R-CNN. Compared with the traditional Smooth L1 loss, the balanced loss has a more smooth curve and better convergence. The specific calculation formula and derivation formula are as follows:

Compared with the Smooth L1 loss, the balanced loss has a slightly larger gradient at the boundary of inliers. A smaller Alpha value means more obvious increase. In this way, the balanced loss can increase the probability of identifying positive samples during the update of the model reverse gradient.

Verification

The open source dataset Canine Coccidiosis Parasite is used for verification. The following figure shows the comparison of false positives before and after the balanced loss is used in the dataset. The balanced loss significantly reduces the background false positive ratio.

Figure 2 Analysis and comparison of false positives before and after the balanced loss is used

The following figure shows the comparison of false negatives before and after the balanced loss is used in Canine Coccidiosis Parasite. The balanced loss significantly reduces the background false negative ratio.

Figure 3 Analysis and comparison of false negatives before and after the balanced loss is used

Suggestions

In the model inference result, if the background error ratio is large, you are advised to use the balanced loss for model optimization and enhancement during training.