Help Center> ModelArts> User Guide (Senior AI Engineers)> Model Management> Model Compression and Conversion> Compressing and Converting Models
View PDF

Compressing and Converting Models

To obtain higher computing power, you can deploy the models created on ModelArts or a local PC on the Ascend chip, ARM, or GPU. In this case, you need to compress or convert the models to the required formats before deploying them.

ModelArts supports model conversion, allowing you to convert a model to a required format before deploying the model on a chip with higher computing power and performance.

Model conversion is applicable to the following scenarios:

  • If you use the Caffe (in .caffemodel format) or TensorFlow framework (in frozen_graph or saved_model format) to train a model, you can convert the model to the .om format. The converted model can be deployed and run on Ascend chips.
  • If you use the TensorFlow framework to train a model (in frozen_graph or saved_model format), you can convert the model to the .tflite format. The converted model can be deployed and run on ARM.
  • If you use the TensorFlow framework to train a model (in frozen_graph or saved_model format), you can convert the model to the TensorRT format. The converted model can be deployed and run on the NVIDIA Tesla P4 GPU.

Constraints

  • Only the following types of chips are supported for model conversion: Ascend, ARM, and GPU.
  • Currently, only Caffe and TensorFlow models can be converted. For a Caffe model, the input data is of the FLOAT type. For a TensorFlow model, the input data type is of the INT32, BOOL, UINT8, or FLOAT type.
  • ModelArts provides conversion templates for you to choose. For details about the supported templates, see Conversion Templates.
  • The .tflite and TensorRT formats support fewer operators and quantization operators. Therefore, some models may fail to be converted. If the conversion fails, view the log dialog box or check error logs in the conversion output directory.
  • An OBS directory must be specified in compression/conversion tasks. Ensure that the OBS directory you use and ModelArts are in the same region.
  • When importing the converted model to ModelArts, you need to use the model template.
  • In Training Management > Training Jobs > Built-in Algorithms, only yolov3_resnet18 supports model conversion. In AI Gallery > Algorithms, all officially published algorithms support model conversion.
  • ONNX model files can be converted. During the conversion, the model files are first converted into the FrozenGraphDef format of the TensorFlow framework, and then converted into the OM format. The ONNX version must be 1.6.0 and the opset version must be 9+.
  • For a Caffe model, the op name and op type in the model file (.prototxt) must be the consistent (case sensitive) with those in the weight file (.caffemodel).
  • For a Caffe model, the top names of all layers must be the same except for the layers with the same top and bottom (such as BatchNorm, Scale, and ReLU).
  • For a TensorFlow model, the FrozenGraphDef and SavedModel formats are supported. If a model is in the SavedModel format, the model is converted to the FrozenGraphDef format and then to the OM format.
  • Inputs with dynamic shapes are not supported, for example, NHWC = [?,?,?,3]. A fixed value needs to be specified during model conversion.
  • The input can be up to 4-dimensional. Operators involving dimension changes (such as reshape and expanddim) cannot output five dimensions.
  • Except the const operator, the input and output at all layers in a model must meet the condition dim!=0.
  • Model conversion does not support models that contain training operators.
  • A UINT8 quantized model cannot be converted.
  • Model operators support only 2D convolution but do not support 3D convolution. The batch_normalization_1 and FusedBatchNorm operators cannot be converted in batches.
  • Only operators in the Caffe and TensorFlow operator lists are supported, and the operator restrictions must be met.

Creating a Model Compression/Conversion Task

  1. Log in to the ModelArts management console, and choose Model Management > Compression/Conversion in the left navigation pane. The Compression/Conversion page is displayed.
  2. Click Create Task in the upper left corner to create a task.
  3. On the Create Task page, set the conversion task parameters.
    1. Enter the name and description of the conversion task.
    2. Set the parameters of the conversion task. For details, see Table 1.
      Table 1 Parameter description

      Parameter

      Description

      Input Framework

      Currently, Caffe and TensorFlow frameworks support conversion.

      Conversion Input Path

      Path to the model to be converted. The path must be an OBS path and comply with the ModelArts specifications. For details about the specifications, see Model Input Path Specifications.

      Output Framework

      When Input Framework is set to Caffe, the output framework can be MindSpore.

      When Input Framework is set to TensorFlow, the output framework can be TFLite, MindSpore, or TensorRT.

      Conversion Output Path

      Path to the converted model. The path must comply with the ModelArts specifications. For details about the specifications, see Model Output Path Description.

      Conversion Template

      ModelArts provides various templates to define model conversion and the parameters required during the conversion. Conversion Templates details the supported model conversion templates. The drop-down list on the right of the conversion template displays the matching templates based on the selected input and output frameworks.

      After you select a template from the drop-down list, the advanced parameters of the template are displayed in the lower part of the page, for example, the precision.

      Different conversion templates support different advanced settings. For details about the parameters supported by each template, see Conversion Templates.
      Figure 1 Creating a model compression/conversion task
  4. After entering the task information, click Next in the lower right corner.

    After the task is created, the system automatically switches to the Compression/Conversion page. The created conversion task is displayed on the page and is in the Initializing status. The conversion task takes several minutes to complete. When the task status changes to Successful, the task is complete and the model has been converted.

    If the task status changes to Failed, click the task name to go to the task details page, view the log information, adjust task parameters based on the log information, and create another conversion task.

    The converted model can be used in the following scenarios:

    • You can import the converted model on the HiLens management console and install the model on the HiLens Kits device.
    • Go to the OBS path corresponding to the conversion output path, download the model (in .om format), and deploy it on your device.

Deleting a Model Compression/Conversion Task

You can delete unnecessary conversion tasks. However, tasks in the Running or Initializing status cannot be deleted.

Deleted tasks cannot be recovered. Exercise caution when performing this operation.

  • Deleting a single task:

    On the Compression/Conversion page, click Delete in the Operation column of the target task.

  • Deleting a batch of tasks:

    On the Compression/Conversion page, select multiple tasks to be deleted and click Delete in the upper left corner.