Help Center/ ModelArts/ ModelArts User Guide (Standard)/ Using ModelArts Standard to Train Models/ Preparing a Model Training Image
Updated on 2025-06-06 GMT+08:00
View PDF
Share

Preparing a Model Training Image

ModelArts provides deep learning-powered base images such as TensorFlow, PyTorch, and MindSpore images. In these images, the software mandatory for running training jobs has been installed. If the software in the base images cannot meet your service requirements, create new images based on the base images and use the new images to create training jobs.

Preset Training Images

The following table lists the preset training base images of ModelArts.

Table 1 ModelArts training base images

Engine

Version

PyTorch

pytorch_1.8.0-cuda_10.2-py_3.7-ubuntu_18.04-x86_64

TensorFlow

tensorflow_2.1.0-cuda_10.1-py_3.7-ubuntu_18.04-x86_64

Horovod

horovod_0.20.0-tensorflow_2.1.0-cuda_10.1-py_3.7-ubuntu_18.04-x86_64

horovod_0.22.1-pytorch_1.8.0-cuda_10.2-py_3.7-ubuntu_18.04-x86_64

MPI

mindspore_1.3.0-cuda_10.1-py_3.7-ubuntu_1804-x86_64

Ascend-Powered-Engine

tensorflow_1.15-cann_5.1.0-py_3.7-euler_2.8.3-aarch64

mindspore_1.7.0-cann_5.1.0-py_3.7-euler_2.8.3-aarch64

Procedure

Figure 1 Creating a custom image for a training job

Scenario 1: If the preset images meet ModelArts training constraints but lack necessary code dependencies, install additional software packages.

For details, see Creating a Custom Training Image Using a Preset Image.

Scenario 2: If the local images meet code dependency requirements but not ModelArts training constraints, adapt them to ModelArts.

For details, see Migrating Existing Images to ModelArts.

Scenario 3: If neither the preset nor local images meet your needs, create an image that has necessary code dependencies and meet ModelArts constraints. For details, see the following cases:

Creating a Custom Training Image (PyTorch + CPU/GPU)

Creating a Custom Training Image (MPI + CPU/GPU)

Creating a Custom Training Image (Tensorflow + GPU)

Installing pip Dependencies in an Image

Before creating distributed training jobs, pre-install all required pip dependencies. If there are more than 10 nodes, the system automatically deletes the pip source configuration. Executing pip install commands during training may cause training failures.

Install all dependency packages before training. This stops failures from missing pip source configurations when using many nodes, making training more stable and efficient.

Install pip dependencies in either of the following ways:

  • Method 1: Install dependencies in a notebook and save it as an image.
    1. Run the target image in the notebook environment.
    2. Run the pip install command to install all dependency packages.
    3. Save the image.
    4. Obtain the SWR address on the image details page for future training.
  • Method 2: Install dependencies in a local container and export an image.
    1. Run the container in the local environment and run the pip install command to install all dependencies.
    2. Run the following command to save the container as an image: 
      docker commit <Container ID> <Image name>

      Example:

      docker commit my_container my_image:v1
    3. Run the following command to export the image as a .tar file: 
      docker save -o <.tar file name>.tar <Image name>:<Tag>

      Example:

      docker save -o my_image_v1.tar my_image:v1
    4. Upload the image to SWR for future training jobs.