Description of the Configuration Parameters

Inference mode

enum

USE_CPU

USE_CPU: Uses the CPU for inference.

USE_CPU

Mandatory

Quantization mode

enum

• NON_OFFSET
• HALF_OFFSET

• NON_OFFSET: Both the weight and data are offset.
• HALF_OFFSET: The data is offset while the weight is not.

There are two quantization mapping formulas:

• With offset: q_uint8 = round (d_float/scale) – offset
• Without offset: q_int8 = round (d_float/scale)

HALF_OFFSET

Mandatory

Weight quantization mode

enum

• VECTOR_TYPE
• SCALAR_TYPE

A convolution operator may have multiple filters, and the corresponding quantization parameters may be different.

• VECTOR_TYPE: Each filter uses a separate group of quantization parameters
• SCALAR_TYPE: The filters share the same group of quantization parameters.

VECTOR_TYPE

Mandatory

Pre-processing and input parameters

Struct

None

Pre-processing and input settings. The number of preprocess_parameter parameters must be the same as the number of operators input to the network.

preprocess_parameter contains the following parameters:

• input_type
• image_format
• input_file_path
• mean_value
• standard_deviation

None

Mandatory

Input data type

enum

• IMAGE
• BINARY

None

Mandatory

File header (20 bytes in total)

0000

32-bit int

510

Magic number.

When the magic number is 510, it is used to verify the validity of a file.

0004

32-bit int

50

Input num

0008

32-bit int

3

Input channels

0012

32-bit int

28

Input height

0016

32-bit int

28

Input width

Data

...

float

126

The number of data records is equal to the product of (num x channels x height x width).

Input data address, that is, path of the calibration set images

string

None

Folder or file to be quantified. The path cannot contain Chinese characters, special characters (including | ; & $ > < `), or spaces.

Set this parameter based on the actual network configuration.

• For IMAGE input, specify the folder.
• For BINARY input, specify the file.

Set this parameter to an image folder or a binary file as required.

Mandatory

Order of the three channels of the image input

enum

• BGR
• RGB

Order of the three channels of the image input for model training

This parameter is mandatory when input_type is set to IMAGE. This parameter is determined based on the order of the input channels for network training. A typical order is BGR.

BGR

Mandatory

Mean value for image preprocessing

float

[0, 255.0]

Mean value of a channel for image preprocessing

This parameter is mandatory when input_type is set to IMAGE. The count of mean values is determined by the channel count. Three mean values need to be configured for the RGB channels respectively as follows:

• mean_value: 104.0
• mean_value: 117.0
• mean_value: 123.0

None

Mandatory

Standard deviation for image preprocessing

float

[0, FLOAT_MAX]

Standard deviation for image preprocessing

This parameter is mandatory when input_type is set to IMAGE. Channels share the same standard deviation.

If the input range is beyond the range that can be represented by the FLOAT type, the quantization precision of the model cannot be guaranteed.

If 0 ≤ standard_deviation ≤ 0.00001, the value of standard_deviation is 1.0.

1.0: The image value range (Source image value – Mean value) does not change.

255.0: If the image value range is [0, 0,255], the value can be condensed to [0,1, 1.0].

Mandatory

Data range of the mapping histogram

uint32

[0, 1000]

Data range of the histogram statistics

The statistics histogram is required during the divergence computation. The value of this parameter determines the maximum value of the histogram. If this parameter is not set or set to 0, the default value 150 is used.

100/150/200/250

Optional

Type of the divergence computation

enum

• KL
• SYMKL
• JSD

• KL: Kullback-Leibler Divergence
• SYMKL: Symmetric Kullback-Leibler Divergence
• JSD: Jensen-Shannon Divergence

Different divergence types correspond to different computation methods. The default value is KL.

None

Optional

Whether to use the quantized data as the inference input

bool

• true
• false

Whether to use the quantized data as the inference input

When this parameter is set to true, the quantization of the input data is simulated. The default value is false.

false

Optional

Whether to use the quantized weight as the inference input

bool

• true
• false

Whether to use the quantized weight as the inference input

When this parameter is set to true, the quantization and de-quantization of the weight data are simulated. The default value is true.

true

Optional

Search parameters

Struct

None

Search parameters

super_parameter contains the following parameters:

• min_percentile
• max_percentile
• start_ratio
• end_ratio
• step_ratio

None

Optional

Searched minimum number

enum

• PERCENTILE_HIGH
• PERCENTILE_MID
• PERCENTILE_LOW

Indicates the minimum number to be considered as the search result among a group of numbers in ascending order. For example, if there are 100 numbers, the value 1.0 indicates that number 0 (100 – 100 x 1.0) is considered as the minimum, that is, the smallest number.

• PERCENTILE_HIGH: 1.0
• PERCENTILE_MID: 0.99999
• PERCENTILE_LOW: 0.9999

PERCENTILE_HIGH

Optional

Searched maximum number

enum

• PERCENTILE_HIGH
• PERCENTILE_MID
• PERCENTILE_LOW

Indicates the maximum number to be considered as the search result among a group of numbers in descending order. For example, if there are 100 numbers, the value 1.0 indicates that number 0 (100 – 100 x 1.0) is considered as the maximum, that is, the largest number.

• PERCENTILE_HIGH: 1.0
• PERCENTILE_MID: 0.99999
• PERCENTILE_LOW: 0.9999

PERCENTILE_MID/PERCENTILE_HIGH

Optional

Parameter

float

end_ratio > start_ratio > 0, step_ratio > 0 (If the input range is beyond the range that can be represented by the FLOAT type, the quantization accuracy of the model cannot be guaranteed.)

• start_ratio indicates the search start.
• end_ratio indicates the search end.
• step_ratio indicates the search step.

After d_max and d_min are found in the algorithm, the ranges taken before and after d_max and d_min are determined based on start_ratio and end_ratio. Then, the step incremented each time is determined by step_ratio. For example:

If d_max is 100, start_ratio is 0.8, end_ratio is 1.2, and step_ratio is 0.01, the defined search range boundaries before and after d_max are 80 (100 x 0.8) and 120 (100 x 1.2) at a step of 1 (100 x 0.01). That is, there are 41 d_max search results in total.

The following two sets of configurations are recommended:

• start_ratio: 0.7 end_ratio: 1.3 step_ratio: 0.01
• start_ratio: 0.3 end_ratio: 1.7 step_ratio: 0.01

Optional

Number of images in the quantization calibration set to be processed

uint32

[0, UINT32_MAX)

Number of images in the quantization calibration set to be processed

If this parameter is not set or set to 0, all images in the calibration set are used. If the value is greater than 0, the smaller one in the total number of images in the calibration set path and the value of this parameter is used as the actual number of images in the calibration set. It is recommended that the number of images in the calibration set do not exceed 500.

50

Optional

Quantization operator blacklist

string

Operator name

Operators in the list are not used for quantization.

• Only the Convolution, Full Connection, and ConvolutionDepthwise operators are supported.
• To blacklist multiple operators, multiple exclude_op parameters need to be set. Each parameter occupies a line. For example:

  exclude_op:'aaa'

  exclude_op:'bbb'

None

Optional