Help Center> ModelArts> FAQs> Model Evaluation> What Are the Restrictions on Using Model Evaluation?
View PDF

What Are the Restrictions on Using Model Evaluation?

Using model evaluation, users can fully understand the adaptability of models to different data features, so that model optimization can be targeted.

Note the following restrictions when using model evaluation:

  • Currently, model evaluation and diagnosis support the following types of models and datasets: image classification, object detection, and image segmentation.
  • Models for evaluation and diagnosis use TensorFlow or PyTorch as AI engines. Only TF-1.13.0-python3.6, TF-2.1.0-python3.6, and PyTorch-1.4.0-python3.6 can be used to compile evaluation code.
  • Only GPU resources are supported. In addition, the resource pool supports only the single-node running mode and does not support the distributed mode.
  • Models generated by ExeML do not support model evaluation.
  • For the built-in TensorFlow algorithms of ModelArts, you can set parameters to start evaluation after training, or import the models to the model management module and create an evaluation job.
  • You can view the evaluation result of a training job created using a built-in algorithm without compiling evaluation code.
  • You can subscribe to the following built-in algorithms in AI Gallery: ResNet_v1_50 for image classification, FasterRCNN_ResNet50 for object detection, and EfficientDet for object detection.

Parent topic: Model Evaluation