Updated on 2022-03-13 GMT+08:00

Configuring AIPPs

AIPP is a hardware image pre-processing function provided by Ascend 310. The pre-processing includes CSC, image normalization (by subtracting the mean value or multiplying a coefficient), image cropping (by specifying the start point of cropping and cropping the image to the size required by the neural network), and much more.

AIPP supports static and dynamic modes.
  • Static AIPP: In this mode, parameters are set during model conversion. The model inference process uses fixed AIPP pre-processing (cannot be modified). For details about the keywords and configuration file template of static AIPP, see "Configuration File Template" in the Model Conversion Guide.
  • Dynamic AIPP: During model conversion, AIPP is set to dynamic mode. Before model inference, set AIPP pre-processing parameters as required. Dynamic AIPP is used when pre-processing parameters have to be changed based on service requirements. For example, cameras use different normalization parameters, and the input image format must be compatible with YUV420 and RGB. For details about how to use dynamic AIPP, see "AIPP Configuration APIs" in the Matrix API Reference.
  • The input format of AIPP is YUV420SP_U8 (the default format is YUV420SP_UV). If the format is YUV420SP_VU, change the value of rbuv_swap_switch. Otherwise, the output result will be affected.
  • The model input is in either RGB_U8 or BGR_U8 format, corresponding to different color gamut conversion matrices.

Images output by the DVPP module are the aligned YUV420SP images. The output does not support the RGB format. Therefore, the service flow needs to use converted and aligned YUV420SP images of the AIPP module, and crop the image to the size in line with the model input.

For example, the model requires the input of a 300 x 300 RGB image. After DVPP APIs are called for processing (such as JPEG decoding and scaling), the DVPP module outputs a 384 x 304 image YUV420SP_UV image (the valid region size is 300 x 300, and 0s are padded to rightward and downward.

The following shows the static AIPP configuration. The file configures the start coordinates of the image to be cropped. The length and width of the image to be cropped are set according to the model input by default. The image normalization parameters are the mean value and the reciprocal of the variance (The final value is obtained by subtracting the mean value and multiplying this coefficient).

aipp_op{
# Sets AIPP to static mode.
aipp_mode: static
# Enables image cropping.
crop: true
# Sets the format and size for an input image.
input_format : YUV420SP_U8
src_image_size_w : 384
src_image_size_h : 304
# Sets the start coordinates for cropping. The width and height of the cropped region are in line with the model input by default.
load_start_pos_h : 0
load_start_pos_w : 0

# Enables format conversion. The conversion matrix converts YUV420SP_UV to RGB888.
csc_switch : true
matrix_r0c0 : 298
matrix_r0c1 : 516
matrix_r0c2 : 0
matrix_r1c0 : 298
matrix_r1c1 : -100
matrix_r1c2 : -208
matrix_r2c0 : 298
matrix_r2c1 : 0
matrix_r2c2 : 409
input_bias_0 : 16
input_bias_1 : 128
input_bias_2 : 128

# Enables data normalization and configure the mean value and the reciprocal of variance.
mean_chn_0 : 125
mean_chn_1 : 125
mean_chn_2 : 125
var_reci_chn_0 : 0.0039
var_reci_chn_1 : 0.0039
var_reci_chn_2 : 0.0039
}