Principles

Dataset details

This is a classification problem. The model must identify the defects on the surface of industrial parts and determine whether they are defective parts. The following figure shows an example.

Figure 1 Defects on the surface of industrial parts

This figure shows the surfaces of two PV modules. The left one is normal, and the right one is abnormal. A model is required to distinguish the two classes of images to help locate the abnormal PV modules. There are 754 normal samples and 358 abnormal samples. The validation set has 754 normal samples and 357 abnormal samples. There are about 2000 samples in total, which are far less than the samples required to achieve accuracy of 95% or higher. Use PyTorch to load the ResNet50 model for training based on this dataset. The overall precision is about 86.06%. The recall of normal classes is 97.3%, and that of abnormal classes is 62.9%. The result does not meet the customer's expectation.

Few-Shot Learning (FSL) is a common method of feeding a learning model with a very small amount of training data. It focuses on data and model training, that is, features extracted from images. In this dataset, the images are 300 x 300 grayscale images and are the front view images of the PV module surfaces. They are efficiently pre-processed images. Flipping this type of images has little impact on the overall image structure. Therefore, flip these images to increase data diversity. The following figure shows the flipping result.

Figure 2 Flipping result

In this way, the number of images in the dataset increases from 1100 to 2200. There is no other way available for expanding the dataset. In this case, you need to use the ModelArts model evaluation function to evaluate the data generalization capability of the model. Call the model evaluation API, that is, the analyse API under deep_moxing.model_analysis.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

def validate(val_loader, model, criterion, args):
batch_time = AverageMeter('Time', ':6.3f')
losses = AverageMeter('Loss', ':.4e')
top1 = AverageMeter('Acc@1', ':6.2f')
top5 = AverageMeter('Acc@5', ':6.2f')
progress = ProgressMeter(
len(val_loader),
[batch_time, losses, top1, top5],
prefix='Test: ')
pred_list = []
target_list = []
# switch to evaluate mode
model.eval()
with torch.no_grad():
end = time.time()
for i, (images, target) in enumerate(val_loader):
if args.gpu is not None:
images = images.cuda(args.gpu, non_blocking=True)
target = target.cuda(args.gpu, non_blocking=True)

# compute output
output = model(images)
loss = criterion(output, target)
# Obtain the model inference result pred and the actual target target.
pred_list += output.cpu().numpy()[:, :2].tolist()
target_list += target.cpu().numpy().tolist()
# measure accuracy and record loss
acc1, acc5 = accuracy(output, target, topk=(1, 5), i=i)
losses.update(loss.item(), images.size(0))
top1.update(acc1[0], images.size(0))
top5.update(acc5[0], images.size(0))

# measure elapsed time
batch_time.update(time.time() - end)
end = time.time()

if i % args.print_freq == 0:
progress.display(i)
# TODO: this should also be done with the ProgressMeter
print(' * Acc@1 {top1.avg:.3f} Acc@5 {top5.avg:.3f}'
.format(top1=top1, top5=top5))
# Obtain the image storage path name.
name_list = val_loader.dataset.samples
for idx in range(len(name_list)):
name_list[idx] = name_list[idx][0]
analyse(task_type='image_classification', save_path='/home/image_labeled/',
pred_list=pred_list, label_list=target_list, name_list=name_list)
return top1.avg

Most of the preceding code is the validation code in the PyTorch training ImageNet. Obtain three lists: pred (model reference results), target (actual image classes), and name (image storage paths). Call the analyse API according to the preceding code. A JSON file is generated in the save_path directory and saved to the ModelArts training output directory. Then, you can view the model analysis result in the evaluation result. You can view the result visually after uploading the JSON file generated offline to the online directory. Table 1 and Table 2 list the sensitivity analysis results.

The preceding two tables indicate the test precision of images with different feature value ranges. For example, the test precision of class 0 when the brightness range is 0%-20% is much lower than that when the brightness is in other ranges. This model is used to detect images of class 1. Images of class 1 are sensitive to both brightness and clarity. That is, the model cannot properly process the images whose brightness and clarity change.

ModelArts provides the data augmentation function to directly expand datasets.

After data augmentation, the dataset contains 2,210 normal images and 1,174 abnormal images. Import the dataset into PyTorch for training. The result is as follows:

The preceding results show that the precision does not improve significantly. Analyze the dataset again. The datasets for industry scenarios often have uneven samples. Although the ratio is close to 2:1 in this dataset, the detection requirements are high for the abnormal class. Therefore, the model should be oriented to the abnormal class. The results of class 1 indicate that the normal and abnormal samples are unbalanced. Therefore, the following two augmentation methods are used only for the abnormal class, that is, the abnormal images. Finally, about 3000 images (1508 normal images and 1432 abnormal images) are obtained. In this way, the samples are balanced. Import the model to ResNet50 for training. The following table lists the precision.