Help Center/ ModelArts/ Image Management/ Using a Custom Image to Train Models (Model Training)/ Preparing a Training Image/ Using a Base Image to Create a Training Image
Updated on 2024-06-15 GMT+08:00
View PDF
Share

Using a Base Image to Create a Training Image

ModelArts provides deep learning-powered base images such as TensorFlow, PyTorch, and MindSpore images. In these images, the software mandatory for running training jobs has been installed. If the software in the base images cannot meet your service requirements, create new images based on the base images and use the new images to create training jobs.

Procedure

Perform the following operations to create an image using a training base image:

  1. Install Docker. If the docker images command is executed, Docker has been installed. In this case, skip this step.

    The following uses Linux x86_64 as an example to describe how to obtain the Docker installation package. Run the following command to install Docker:

    curl -fsSL get.docker.com -o get-docker.sh
sh get-docker.sh
  2. Create a folder named context. 
    mkdir -p context
  3. Obtain the pip.conf file. 
    [global]
index-url = https://repo.huaweicloud.com/repository/pypi/simple
trusted-host = repo.huaweicloud.com
timeout = 120
  4. Create a new image based on a training base image provided by ModelArts. Save the edited Dockerfile in the context folder. For details about how to obtain a training base image, see Available Training Base Images. 
    FROM {Path to the training base image provided by ModelArts}

# Configure pip.
RUN mkdir -p /home/ma-user/.pip/
COPY --chown=ma-user:ma-group pip.conf /home/ma-user/.pip/pip.conf

# Configure the preset environment variables of the container image.
# Add the Python interpreter path to the PATH environment variable.
# Set PYTHONUNBUFFERED to 1 to prevent log loss.
ENV PATH=${ANACONDA_DIR}/envs/${ENV_NAME}/bin:$PATH \
    PYTHONUNBUFFERED=1

RUN /home/ma-user/anaconda/bin/pip install --no-cache-dir numpy
  5. Run the following command in the directory where the Dockerfile is stored to create a container image, for example, training:v1: 
    docker build . -t training:v1
  6. Upload the new image to SWR. For details, see How Can I Log In to SWR and Upload Images to It?.
  7. Use the custom image to create a training job on ModelArts. For details, see Using a Custom Image to Create a CPU- or GPU-based Training Job.