Updated on 2024-06-12 GMT+08:00

Specifications for Custom Images for Training Jobs

When you use a locally developed model and training script to create a custom image, ensure that the custom image complies with the specifications defined by ModelArts.

In both new-version and old-version training management, custom images can be used to create training jobs. This document describes training management of the new version. The old version will be discontinued soon. You are advised to use the new version.

Specifications

  • The size of a custom image cannot exceed 30 GB. It is recommended that the size be less than or equal to 15 GB. An oversized image affects the startup of a training job.
  • The uid of the default user of a custom image must be 1000.
  • The GPU or Ascend driver cannot be installed in a custom image. When you select GPU resources to run training jobs, ModelArts automatically places the GPU driver in the /usr/local/nvidia directory in the training environment. When you select Ascend resources to run training jobs, ModelArts automatically places the Ascend driver in the /usr/local/Ascend/driver directory.
  • x86- or Arm-based custom images can run only with specifications corresponding to their architecture.
    • Run the following command to check the CPU architecture of a custom image:
      docker inspect {Custom image path} | grep Architecture
      The following is the command output for an Arm-based custom image:
      "Architecture": "arm64"
    • If the name of a specification contains Arm, this specification is an Arm-based CPU architecture.

    • If the name of a specification does not contain Arm, this specification is an x86-based CPU architecture.

  • ModelArts does not support the download of open source installation packages. Install the dependency packages required by the training job in the custom image.