Training Job Creation

In the old version, you can use custom algorithms, built-in algorithms, common frameworks, and custom images to create training jobs.

In the new version, you can use the algorithms subscribed in AI Gallery and the custom algorithms to create training jobs.

In the new version, algorithms can be selected by category when you create a training job. This does not affect existing training jobs.

If you use custom algorithms or common frameworks to create training jobs in the old version, you can use custom algorithms to do so in the new version.

If you use built-in algorithms to create training jobs in the old version, you can use subscribed algorithms to do so in the new version.

If you use custom images to create training jobs in the old version, you can use DevEnviron to create custom images and import them to the training platform for training.

Training Code Adaptation

In the old version, you are required to configure data input and output as follows:

# Parse CLI parameters.
import argparse
parser = argparse.ArgumentParser(description='MindSpore Lenet Example')
parser.add_argument('--data_url', type=str, default="./Data",
                    help='path where the dataset is saved')
parser.add_argument('--train_url', type=str, default="./Model", help='if is test, must provide\
                    path where the trained ckpt file')
args = parser.parse_args()
...
# Download data parameters to your local container. In the code, local_data_path specifies the training input path.
mox.file.copy_parallel(args.data_url, local_data_path)
...
# Upload the local container data to the OBS path.
mox.file.copy_parallel(local_output_path, args.train_url)

Built-in Training Engines

• In the new version, MoXing 2.0.0 or later is installed by default for built-in training engines.
• In the new version, Python 3.7 or later is used for built-in training engines.
• In the new image, the default home directory has been changed from /home/work to /home/ma-user. Check whether the training code contains hard coding of /home/work.
• Built-in training engines are different between the old and new versions. Commonly used built-in training engines have been upgraded in the new version.
  To use a training engine in the old version, switch to the old version. Table 1 lists the differences between the built-in training engines in the old and new versions. For details about the training engine versions, see Which AI Frameworks Does ModelArts Support?

  Table 1 Differences between the built-in training engines in the old and new versions

  Work Environment	Built-in Training Engine and Version	Old Version	New Version
  TensorFlow	TensorFlow-1.8.0	√	x
  	TensorFlow-1.13.1	√	Coming soon
  	TensorFlow-2.1.0	√	√
  MXNet	MXNet-1.2.1	√	x
  Caffe	Caffe-1.0.0	√	x
  Spark MLlib	Spark-2.3.2	√	x
  Ray	Ray-0.7.4	√	x
  XGBoost with scikit-learn	Scikit_Learn-0.18.1	√	x
  PyTorch	PyTorch-1.0.0	√	x
  	PyTorch-1.3.0	√	x
  	PyTorch-1.4.0	√	x
  	PyTorch-1.8.0	x	√
  Ascend-Powered-Engine	MindSpore-1.1.1	√	x
  	MindSpore-1.3.0	x	√
  	TensorFlow-1.15	√	√
  MPI	MindSpore-1.1.0	√	x
  	MindSpore-1.3.0	x	√
  Horovod	Horovod_0.20.0-TensorFlow_2.1.0	x	√
  	Horovod_0.20.0-PyTorch_1.8.0	x	√