Conversion Templates

ModelArts provides the following conversion templates based on different AI frameworks:

Caffe to Ascend

Convert the model trained by the Caffe framework. The converted model can run on the Ascend chip.

This template has no advanced settings.

TensorFlow frozen_graph to TFLite

Convert the model trained by the TensorFlow framework and saved in frozen_graph format. The converted model can run on the ARM.

Table 1 Advanced settings of TensorFlow frozen_graph to TFLite

Parameter

Description

Input Tensor Name

Enter the model input tensors in a character string, in the format of input1:input2.

Output Tensor Name

Enter the model output tensors in a character string, in the format of output1:output2.

Quantization Precision

The value can be 8bit or 32bit. 32bit indicates that the model is directly converted, and 8bit indicates that the model is quantized.

Batch Size

Enter the quantization batch size as a numeric value. The value must be an integer.

TensorFlow saved_model to TFLite

Convert the model trained by the TensorFlow framework and saved in saved_model format. The converted model can run on the ARM.

Table 2 Advanced settings of TensorFlow saved_model to TFLite

Parameter

Description

Model Signature

Enter the tensor signatures in a character string. By default, the first signature is selected.

Model Input Label

Enter the model input labels in a character string. By default, the first label is selected.

Quantization Precision

The value can be 8bit or 32bit. 32bit indicates that the model is directly converted, and 8bit indicates that the model is quantized.

Batch Size

Enter the quantization batch size as a numeric value. The value must be an integer.

TensorFlow frozen_graph to TensorRT

Convert the model trained by the TensorFlow framework and saved in frozen_graph format. The converted model can run on the GPU.

Table 3 Advanced settings of TensorFlow frozen_graph to TensorRT

Parameter

Description

Input Tensor Name

Enter the model input tensors in a character string, in the format of input1:input2.

Output Tensor Name

Enter the model output tensors in a character string, in the format of output1:output2.

Quantization Precision

The value can be 8bit or 32bit. 32bit indicates that the model is directly converted, and 8bit indicates that the model is quantized.

Batch Size

Enter the quantization batch size as a numeric value. The value must be an integer.

TensorFlow saved_model to TensorRT

Convert the model trained by the TensorFlow framework and saved in saved_model format. The converted model can run on the GPU.

Table 4 Advanced settings of TensorFlow saved_model to TensorRT

Parameter

Description

Model Signature

Enter the tensor signatures in a character string. By default, the first signature is selected.

Model Input Label

Enter the model input labels in a character string. By default, the first label is selected.

Quantization Precision

The value can be 8bit or 32bit. 32bit indicates that the model is directly converted, and 8bit indicates that the model is quantized.

Batch Size

Enter the quantization batch size as a numeric value. The value must be an integer.

TensorFlow frozen_graph to Ascend

Convert the model trained by the TensorFlow framework and saved in frozen_graph format. The converted model can run on the Ascend.

Table 5 Advanced settings of TensorFlow frozen_graph to Ascend

Parameter

Description

input_shape

Enter the shape of the input data of the model. The input data format is NHWC, for example, input_name:1,224,224,3. This parameter is mandatory. input_name must be the node name in the network model before model conversion. This parameter is mandatory when the model has dynamic shape input. For example, in input_name1:? ,h,w,c, the question mark (?) indicates the batch size, that is, the number of images processed at a time. It is used to convert the original model with a dynamic shape into an offline model with a fixed shape. Currently, the batch feature is not supported. The batch value of the input_shape can only be 1.

TF-FrozenGraph-To-Ascend

Convert the model trained by the TensorFlow framework and saved in frozen_graph format. The converted model can run on the Ascend. The custom operators (TE operators) developed based on TE can be used for conversion.

Table 6 Advanced settings of the custom operator conversion template

Parameter

Description

input_shape

Enter the shape of the input data of the model, for example, input_name1:n1,c1,h1,w1;input_name2:n2,c2,h2,w2. input_name must be the node name in the network model before model conversion. This parameter is mandatory when the model has dynamic shape input. For example, in input_name1:? ,h,w,c, the question mark (?) indicates the batch size, that is, the number of images processed at a time. It is used to convert the original model with a dynamic shape into an offline model with a fixed shape. Currently, the batch feature is not supported. The batch value of the input_shape can only be 1. During the conversion, the system parses the input model to obtain the input tensor and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

input_format

NCHW and NHWC are supported. The default format is NHWC. For the TensorFlow framework, the default value is NHWC. To use the NCHW format, you need to specify NCHW. For the Caffe framework, only the NCHW format is supported.

out_nodes

Specifies the output node, for example, node_name1:0;node_name1:1;node_name2:0. node_name must be the node name in the network model before model conversion. The digit after each colon (:) indicates the sequence number of the output. For example, node_name1:0 indicates the 0th output of node_name1. If the output node is not specified, the output of the last operator layer serves as the model output by default. To check the parameters of a specific operator layer, specify the operator layer by using this parameter. During the conversion, the system parses the input model to obtain the output node and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

net_format

Specifies the preferred data format for network operators. Possible values are ND (N cannot be more than 4) and 5D. This parameter only takes effect if the input data of operators on the network supports both ND and 5D formats. ND indicates that operators in the model are converted into the NCHW format. 5D indicates that operators in the model are converted into the Huawei-developed 5D format. 5D is the default value.

fp16_high_prec

Specifies whether to generate a high-precision FP16 Davinci model. 0 is the default value, indicating that a common FP16 Da Vinci model with better inference performance is generated. The value 1 indicates that a high-precision FP16 Da Vinci model with better inference precision is generated. Currently, high-precision models support only Caffe operators (Convolution, Pooling, and FullConnection) and TensorFlow operators (tf.nn.conv2d and tf.nn.max_poo).

output_type

FP32 is the default value and is recommended for classification and detection networks. For image super-resolution networks, UINT8 is recommended for better inference performance.

TF-SavedModel-To-Ascend

Convert the model trained by the TensorFlow framework and saved in saved_model format. The converted model can run on the Ascend. The custom operators (TE operators) developed based on TE can be used for conversion.

Table 7 Advanced settings of the custom operator conversion template

Parameter

Description

input_shape

Enter the shape of the input data of the model, for example, input_name1:n1,c1,h1,w1;input_name2:n2,c2,h2,w2. input_name must be the node name in the network model before model conversion. This parameter is mandatory when the model has dynamic shape input. For example, in input_name1:? ,h,w,c, the question mark (?) indicates the batch size, that is, the number of images processed at a time. It is used to convert the original model with a dynamic shape into an offline model with a fixed shape. Currently, the batch feature is not supported. The batch value of the input_shape can only be 1. During the conversion, the system parses the input model to obtain the input tensor and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

input_format

NCHW and NHWC are supported. The default format is NHWC. For the TensorFlow framework, the default value is NHWC. To use the NCHW format, you need to specify NCHW. For the Caffe framework, only the NCHW format is supported.

out_nodes

Specifies the output node, for example, node_name1:0;node_name1:1;node_name2:0. node_name must be the node name in the network model before model conversion. The digit after each colon (:) indicates the sequence number of the output. For example, node_name1:0 indicates the 0th output of node_name1. If the output node is not specified, the output of the last operator layer serves as the model output by default. To check the parameters of a specific operator layer, specify the operator layer by using this parameter. During the conversion, the system parses the input model to obtain the output node and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

net_format

Specifies the preferred data format for network operators. Possible values are ND (N cannot be more than 4) and 5D. This parameter only takes effect if the input data of operators on the network supports both ND and 5D formats. ND indicates that operators in the model are converted into the NCHW format. 5D indicates that operators in the model are converted into the Huawei-developed 5D format. 5D is the default value.

fp16_high_prec

Specifies whether to generate a high-precision FP16 Davinci model. 0 is the default value, indicating that a common FP16 Da Vinci model with better inference performance is generated. The value 1 indicates that a high-precision FP16 Da Vinci model with better inference precision is generated. Currently, high-precision models support only Caffe operators (Convolution, Pooling, and FullConnection) and TensorFlow operators (tf.nn.conv2d and tf.nn.max_poo).

output_type

FP32 is the default value and is recommended for classification and detection networks. For image super-resolution networks, UINT8 is recommended for better inference performance.

Onnx-To-Ascend-TBE

Convert the model saved in the ONNX format to the frozen_graph format and then to a model that can run on Ascend. The custom operators (TBE operators) developed based on Tensor Based Engine (TBE) can be used for conversion.

Table 8 Advanced settings of the custom operator conversion template

Parameter

Description

input_shape

Enter the shape of the input data of the model, for example, input_name1:n1,c1,h1,w1;input_name2:n2,c2,h2,w2. input_name must be the node name in the network model before model conversion. This parameter is mandatory when the model has dynamic shape input. For example, in input_name1:? ,h,w,c, the question mark (?) indicates the batch size, that is, the number of images processed at a time. It is used to convert the original model with a dynamic shape into an offline model with a fixed shape. Currently, the batch feature is not supported. The batch value of the input_shape can only be 1. During the conversion, the system parses the input model to obtain the input tensor and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

input_format

NCHW and NHWC are supported. The default format is NHWC. For the TensorFlow framework, the default value is NHWC. To use the NCHW format, you need to specify NCHW. For the Caffe framework, only the NCHW format is supported.

out_nodes

Specifies the output node, for example, node_name1:0;node_name1:1;node_name2:0. node_name must be the node name in the network model before model conversion. The digit after each colon (:) indicates the sequence number of the output. For example, node_name1:0 indicates the 0th output of node_name1. If the output node is not specified, the output of the last operator layer serves as the model output by default. To check the parameters of a specific operator layer, specify the operator layer by using this parameter. During the conversion, the system parses the input model to obtain the output node and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

net_format

Specifies the preferred data format for network operators. Possible values are ND (N cannot be more than 4) and 5D. This parameter only takes effect if the input data of operators on the network supports both ND and 5D formats. ND indicates that operators in the model are converted into the NCHW format. 5D indicates that operators in the model are converted into the Huawei-developed 5D format. 5D is the default value.

fp16_high_prec

Specifies whether to generate a high-precision FP16 Davinci model. 0 is the default value, indicating that a common FP16 Da Vinci model with better inference performance is generated. The value 1 indicates that a high-precision FP16 Da Vinci model with better inference precision is generated. Currently, high-precision models support only Caffe operators (Convolution, Pooling, and FullConnection) and TensorFlow operators (tf.nn.conv2d and tf.nn.max_poo).

output_type

FP32 is the default value and is recommended for classification and detection networks. For image super-resolution networks, UINT8 is recommended for better inference performance.

TF-FrozenGraph-To-Ascend-C32

Convert the model trained by the TensorFlow framework and saved in frozen_graph format. The converted model can run on the Ascend. The custom operators (TBE operators) developed based on Tensor Based Engine (TBE) can be used for conversion.

Table 9 Advanced settings of the custom operator conversion template

Parameter

Description

input_shape

Enter the shape of the input data of the model, for example, input_name1:n1,c1,h1,w1;input_name2:n2,c2,h2,w2. input_name must be the node name in the network model before model conversion. This parameter is mandatory when the model has dynamic shape input. For example, in input_name1:? ,h,w,c, the question mark (?) indicates the batch size, that is, the number of images processed at a time. It is used to convert the original model with a dynamic shape into an offline model with a fixed shape. Currently, the batch feature is not supported. The batch value of the input_shape can only be 1. During the conversion, the system parses the input model to obtain the input tensor and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

input_format

NCHW and NHWC are supported. The default format is NHWC. For the TensorFlow framework, the default value is NHWC. To use the NCHW format, you need to specify NCHW. For the Caffe framework, only the NCHW format is supported.

out_nodes

Specifies the output node, for example, node_name1:0;node_name1:1;node_name2:0. node_name must be the node name in the network model before model conversion. The digit after each colon (:) indicates the sequence number of the output. For example, node_name1:0 indicates the 0th output of node_name1. If the output node is not specified, the output of the last operator layer serves as the model output by default. To check the parameters of a specific operator layer, specify the operator layer by using this parameter. During the conversion, the system parses the input model to obtain the output node and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

net_format

Specifies the preferred data format for network operators. Possible values are ND (N cannot be more than 4) and 5D. This parameter only takes effect if the input data of operators on the network supports both ND and 5D formats. ND indicates that operators in the model are converted into the NCHW format. 5D indicates that operators in the model are converted into the Huawei-developed 5D format. 5D is the default value.

fp16_high_prec

Specifies whether to generate a high-precision FP16 Davinci model. 0 is the default value, indicating that a common FP16 Da Vinci model with better inference performance is generated. The value 1 indicates that a high-precision FP16 Da Vinci model with better inference precision is generated. Currently, high-precision models support only Caffe operators (Convolution, Pooling, and FullConnection) and TensorFlow operators (tf.nn.conv2d and tf.nn.max_poo).

output_type

FP32 is the default value and is recommended for classification and detection networks. For image super-resolution networks, UINT8 is recommended for better inference performance.

TF-SavedModel-To-Ascend-C32

Convert the model trained by the TensorFlow framework and saved in saved_model format. The converted model can run on the Ascend. The custom operators (TE operators) developed based on TE can be used for conversion.

Table 10 Advanced settings of the custom operator conversion template

Parameter

Description

input_shape

Enter the shape of the input data of the model, for example, input_name1:n1,c1,h1,w1;input_name2:n2,c2,h2,w2. input_name must be the node name in the network model before model conversion. This parameter is mandatory when the model has dynamic shape input. For example, in input_name1:? ,h,w,c, the question mark (?) indicates the batch size, that is, the number of images processed at a time. It is used to convert the original model with a dynamic shape into an offline model with a fixed shape. Currently, the batch feature is not supported. The batch value of the input_shape can only be 1. During the conversion, the system parses the input model to obtain the input tensor and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

input_format

NCHW and NHWC are supported. The default format is NHWC. For the TensorFlow framework, the default value is NHWC. To use the NCHW format, you need to specify NCHW. For the Caffe framework, only the NCHW format is supported.

out_nodes

Specifies the output node, for example, node_name1:0;node_name1:1;node_name2:0. node_name must be the node name in the network model before model conversion. The digit after each colon (:) indicates the sequence number of the output. For example, node_name1:0 indicates the 0th output of node_name1. If the output node is not specified, the output of the last operator layer serves as the model output by default. To check the parameters of a specific operator layer, specify the operator layer by using this parameter. During the conversion, the system parses the input model to obtain the output node and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

net_format

Specifies the preferred data format for network operators. Possible values are ND (N cannot be more than 4) and 5D. This parameter only takes effect if the input data of operators on the network supports both ND and 5D formats. ND indicates that operators in the model are converted into the NCHW format. 5D indicates that operators in the model are converted into the Huawei-developed 5D format. 5D is the default value.

fp16_high_prec

Specifies whether to generate a high-precision FP16 Davinci model. 0 is the default value, indicating that a common FP16 Da Vinci model with better inference performance is generated. The value 1 indicates that a high-precision FP16 Da Vinci model with better inference precision is generated. Currently, high-precision models support only Caffe operators (Convolution, Pooling, and FullConnection) and TensorFlow operators (tf.nn.conv2d and tf.nn.max_poo).

output_type

FP32 is the default value and is recommended for classification and detection networks. For image super-resolution networks, UINT8 is recommended for better inference performance.

TF-FrozenGraph-To-Ascend-HiLens

This template is intended for Huawei HiLens. If the firmware version of your HiLens Kit system is earlier than 2.2.200.011, you are advised to use this template for model conversion. TensorFlow frozen_graph models can be converted into models that can run on Ascend chips. During the conversion, custom operators developed based on the TE can be used.

Table 11 Advanced settings of the custom operator conversion template

Parameter

Description

input_shape

Enter the shape of the input data of the model, for example, input_name1:n1,c1,h1,w1;input_name2:n2,c2,h2,w2. input_name must be the node name in the network model before model conversion. This parameter is mandatory when the model has dynamic shape input. For example, in input_name1:? ,h,w,c, the question mark (?) indicates the batch size, that is, the number of images processed at a time. It is used to convert the original model with a dynamic shape into an offline model with a fixed shape. Currently, the batch feature is not supported. The batch value of the input_shape can only be 1. During the conversion, the system parses the input model to obtain the input tensor and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

input_format

NCHW and NHWC are supported. The default format is NHWC. For the TensorFlow framework, the default value is NHWC. To use the NCHW format, you need to specify NCHW. For the Caffe framework, only the NCHW format is supported.

out_nodes

Specifies the output node, for example, node_name1:0;node_name1:1;node_name2:0. node_name must be the node name in the network model before model conversion. The digit after each colon (:) indicates the sequence number of the output. For example, node_name1:0 indicates the 0th output of node_name1. If the output node is not specified, the output of the last operator layer serves as the model output by default. To check the parameters of a specific operator layer, specify the operator layer by using this parameter. During the conversion, the system parses the input model to obtain the output node and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

net_format

Specifies the preferred data format for network operators. Possible values are ND (N cannot be more than 4) and 5D. This parameter only takes effect if the input data of operators on the network supports both ND and 5D formats. ND indicates that operators in the model are converted into the NCHW format. 5D indicates that operators in the model are converted into the Huawei-developed 5D format. 5D is the default value.

fp16_high_prec

Specifies whether to generate a high-precision FP16 Davinci model. 0 is the default value, indicating that a common FP16 Da Vinci model with better inference performance is generated. The value 1 indicates that a high-precision FP16 Da Vinci model with better inference precision is generated. Currently, high-precision models support only Caffe operators (Convolution, Pooling, and FullConnection) and TensorFlow operators (tf.nn.conv2d and tf.nn.max_poo).

output_type

FP32 is the default value and is recommended for classification and detection networks. For image super-resolution networks, UINT8 is recommended for better inference performance.

TF-SavedModel-To-Ascend-HiLens

This template is intended for Huawei HiLens. If the firmware version of your HiLens Kit system is earlier than 2.2.200.011, you are advised to use this template for model conversion. TensorFlow saved_model models can be converted into models that can run on Ascend chips. During the conversion, custom operators developed based on the TE can be used.

Table 12 Advanced settings of the custom operator conversion template

Parameter

Description

input_shape

Enter the shape of the input data of the model, for example, input_name1:n1,c1,h1,w1;input_name2:n2,c2,h2,w2. input_name must be the node name in the network model before model conversion. This parameter is mandatory when the model has dynamic shape input. For example, in input_name1:? ,h,w,c, the question mark (?) indicates the batch size, that is, the number of images processed at a time. It is used to convert the original model with a dynamic shape into an offline model with a fixed shape. Currently, the batch feature is not supported. The batch value of the input_shape can only be 1. During the conversion, the system parses the input model to obtain the input tensor and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

input_format

NCHW and NHWC are supported. The default format is NHWC. For the TensorFlow framework, the default value is NHWC. To use the NCHW format, you need to specify NCHW. For the Caffe framework, only the NCHW format is supported.

out_nodes

Specifies the output node, for example, node_name1:0;node_name1:1;node_name2:0. node_name must be the node name in the network model before model conversion. The digit after each colon (:) indicates the sequence number of the output. For example, node_name1:0 indicates the 0th output of node_name1. If the output node is not specified, the output of the last operator layer serves as the model output by default. To check the parameters of a specific operator layer, specify the operator layer by using this parameter. During the conversion, the system parses the input model to obtain the output node and prints it in the log. If you do not know the input tensor of the used model, refer to the parsing result in the log.

net_format

Specifies the preferred data format for network operators. Possible values are ND (N cannot be more than 4) and 5D. This parameter only takes effect if the input data of operators on the network supports both ND and 5D formats. ND indicates that operators in the model are converted into the NCHW format. 5D indicates that operators in the model are converted into the Huawei-developed 5D format. 5D is the default value.

fp16_high_prec

Specifies whether to generate a high-precision FP16 Davinci model. 0 is the default value, indicating that a common FP16 Da Vinci model with better inference performance is generated. The value 1 indicates that a high-precision FP16 Da Vinci model with better inference precision is generated. Currently, high-precision models support only Caffe operators (Convolution, Pooling, and FullConnection) and TensorFlow operators (tf.nn.conv2d and tf.nn.max_poo).

output_type

FP32 is the default value and is recommended for classification and detection networks. For image super-resolution networks, UINT8 is recommended for better inference performance.