Scenario

Model robustness has always been a research focus, and there have been many excellent practices of model attack defense. However, most model attacks require the models, which is impossible in most cases. The model attack method described in this section focuses on preprocessing (the resizing phase), which is necessary for all models. This method comes from Seeing is Not Believing: Camouflage Attacks on Image Scaling Algorithms.

Principles

Most models have requirements on the size of input samples, for example, 224 x 224 or a custom size. Moreover, these sizes are enumerable and exhaustive, which make black-box attacks possible. Before a sample enters a model, the sample is preprocessed. The basic operation is to resize the sample to the size required by the model,

which causes information loss. If most reliable information is lost but attack information is retained during model resizing, an attack may be launched.

There are several common sample resizing algorithms, including nearest neighbor interpolation (INTER_NEAREST), bilinear interpolation (INTER_LINEAR), bicubic interpolation (INTER_CUBIC), Lanczos interpolation (INTER_LANCZOS4), and local pixel-based resampling (INTER_AREA). The nearest neighbor interpolation algorithm is also a sampling algorithm. If it is used to resize a sample, the pixels that are reserved during sampling may be calculated based on the size of original samples and the size of attack images. You only need to place the attack images on these pixels to hide the attack information on the images that look normal. The logic of bilinear interpolation is similar.

Figure 1 Attack effect

Figure 2 Effect after resizing

The larger the original samples, the better the attack effect. Partial modification is allowed. For example, a certificate sample can be attacked only by modifying the key words.

Defense

This attack method has obvious advantages and disadvantages.

• Advantage: This method is simple, and can be used for black-box attacks. A smaller model input sample size means the model is more vulnerable to attacks. A larger difference between the attack sample size and the model input sample size means a better attack effect.
• Disadvantage: This attack is easy to defend.
• Defense method: During resizing, use the more complex Lanczos interpolation algorithm (INTER_LANCZOS4), which may compromise some performance but can directly prevent such attacks. Alternatively, randomly tailor the samples and then resize them. In this case, the attack pixels that have been specially arranged will not be selected.