Help Center/ ModelArts/ Model Training/ Preparing a Model Training Image
Updated on 2026-07-02 GMT+08:00
View PDF
Share

Preparing a Model Training Image

ModelArts offers base images for deep learning, including TensorFlow, PyTorch, and MindSpore. These images have the necessary software for running training jobs already installed. If the software in the base images cannot meet your service requirements, create new images based on the base images and use the new images to create training jobs.

Preset Training Images

The following table lists the preset training base images of ModelArts.

Table 1 ModelArts training base images

Engine

Version

PyTorch

pytorch_1.8.0-cuda_10.2-py_3.7-ubuntu_18.04-x86_64

pytorch_2.1.0-cuda_12.1-py_3.10.6-ubuntu_22.04-x86_64

TensorFlow

tensorflow_2.1.0-cuda_10.1-py_3.7-ubuntu_18.04-x86_64

Horovod

horovod_0.20.0-tensorflow_2.1.0-cuda_10.1-py_3.7-ubuntu_18.04-x86_64

horovod_0.22.1-pytorch_1.8.0-cuda_10.2-py_3.7-ubuntu_18.04-x86_64

MPI

mindspore_1.3.0-cuda_10.1-py_3.7-ubuntu_18.04-x86_64

Ascend-Powered-Engine

tensorflow_1.15-cann_5.1.0-py_3.7-euler_2.8.3-aarch64

mindspore_1.7.0-cann_5.1.0-py_3.7-euler_2.8.3-aarch64

pytorch_2.1.0-cann_7.0.1.1-py_3.9-euler_2.10.7-aarch64-snt3p

mindspore_2.2.12-cann_7.0.1.1-py_3.9-euler_2.10.7-aarch64-snt3p

Creating a Custom Training Image

Figure 1 Creating a custom image for a training job

Scenario 1: If preset images meet ModelArts training constraints but lack necessary code dependencies, install additional software packages.

For details, see Creating a Custom Training Image Using a Preset Image.

Scenario 2: If local images meet code needs but not ModelArts training constraints, adjust them for ModelArts.

For details, see Migrating Existing Images to ModelArts.

Scenario 3: If neither the preset nor local images meet your needs, create an image that meets both code dependency and ModelArts training constraints. For details, see the following cases:

Creating a Custom Training Image (PyTorch + Ascend)

Creating a Custom Training Image (PyTorch + CPU/GPU)

Creating a Custom Training Image (MPI + CPU/GPU)

Creating a Custom Training Image (Tensorflow + GPU)

Installing pip Dependencies in an Image

Before creating distributed training jobs, pre-install all required pip dependencies. If there are more than 10 nodes, the system automatically deletes the pip source configuration. Executing pip install commands during training may cause training failures.

Install all dependency packages before training. This stops failures from missing pip source configurations when using many nodes, making training more stable and efficient.

Install pip dependencies in either of the following ways:

  • Method 1: Install dependencies in a notebook and save it as an image.
    1. Run the target image in the notebook environment.
    2. Run the pip install command to install all dependency packages.
    3. Save the image.
    4. Obtain the SWR address on the image details page for future training.
  • Method 2: Install dependencies in a local container and export an image.
    1. Run the container in the local environment and run the pip install command to install all dependencies.
    2. Save the container as an image: 
      docker commit <container-ID> <image-name>:<tag>

      Example:

      docker commit my_container my_image:v1
    3. Export the image as a .tar file: 
      docker save -o <.tar-file-name>.tar <image-name>:<tag>

      Example:

      docker save -o my_image_v1.tar my_image:v1
    4. Upload the image to SWR for future training jobs.