Help Center/ ModelArts/ ModelArts User Guide (Standard)/ Using ModelArts Standard to Train Models/ Managing Model Training Jobs/ Saving the Image of a Debug Training Job
Updated on 2025-08-18 GMT+08:00
View PDF
Share

Saving the Image of a Debug Training Job

During development and debugging, you may modify and optimize the image environment. After modifying the image, you can save the image on the console for future use.

Notes for saving an image:

  1. The installed dependency packages will be retained, but the training data and code that need to be persistently stored will not be saved in the generated container image.
  2. VS Code keeps the plug-ins you install on the server during remote development.

Key settings and dependencies are preserved, but dynamic data and code require separate handling.

Prerequisites

  • The debug job is created in a dedicated resource pool.
  • The job is in the Running state.

Notes and Constraints

  • Only the image of the training job's worker-0 can be saved on the console.
  • The image to be saved should not be larger than 35 GB and there should be no more than 125 layers. Otherwise, the image may fail to be saved due to the rootfs difference between node containers.

Procedure

  1. Click the target training job to access its details page. On the displayed page, click Save Image in the upper right corner.
  2. In the Save Image dialog box, configure parameters. Click OK to save the image.

    Choose an organization from the Organization drop-down list. If no organization is available, click Create on the right to create one.

    Users in an organization can share all images in the organization.

    The image will be saved as a snapshot, which will take a while. During which, the image status is Saving.

    The time required for saving an image as a snapshot will be counted in the job runtime. If the job runtime expires before the snapshot is saved, saving the image will fail.

  3. The saved image can be used to create a training job.

FAQs

  1. What Can I Do If an Image Fails to Be Saved?
    1. If the image fails to be saved, check the event on the training job details page. For details, see Viewing Training Job Events.
    2. If a dedicated resource pool is used, you can adjust the container engine space on the Resource Management > Lite Cluster page as required. For details, see Changing the Number of Instances.
    3. If the problem still persists, contact technical engineers.
  2. Why Cannot I Check Real-Time Logs During Image Saving?

    Saving an image may affect checking real-time logs. The Cloud Shell connection may be interrupted. After the image is saved, the connection is automatically restored.