Creating an Algorithm Using a Custom Image

Your locally developed algorithms or algorithms developed using other tools can be uploaded to ModelArts for unified management.

Entries for Creating an Algorithm

You can create an algorithm using a custom image on ModelArts in either of the following ways:

Entry 1: On the ModelArts console, choose Algorithm Management > My algorithms. Then, create an algorithm and use it in training jobs or publish it to AI Gallery.
Entry 2: On the ModelArts console, choose Model Training > Training Jobs, and click Create Training Job to create a custom algorithm and submit a training job. For details, see Using a Custom Image to Create a CPU- or GPU-based Training Job.

Parameters for creating an algorithm

**Table 1** Parameters for creating an algorithm
Parameter	Description
Boot Mode	Select Custom images. This parameter is mandatory.
Image	URL of an SWR image. This parameter is mandatory. Private images or shared images: Click Select on the right to select an SWR image. Ensure that the image has been uploaded to SWR. For details, see How Can I Log In to SWR and Upload Images to It?. Public images: You can also manually enter the image path in the format of "<Organization to which your image belongs>/<Image name>" on SWR. Do not contain the domain name (swr.<region>.xxx.com) in the path because the system will automatically add the domain name to the path. For example: modelarts-job-dev-image/pytorch_1_8:train-pytorch_1.8.0-cuda_10.2-py_3.7-euleros_2.10.1-x86_64-8.1.1
Code Directory	OBS path for storing the training code. This parameter is optional. Take OBS path obs://obs-bucket/training-test/demo-code as an example. The content in the OBS path will be automatically downloaded to ${MA_JOB_DIR}/demo-code in the training container, and demo-code (customizable) is the last-level directory of the OBS path.
Boot Command	Command for booting an image. This parameter is mandatory. The boot command will be automatically executed after the code directory is downloaded. If the training boot script is a .py file, train.py for example, the boot command can be python ${MA_JOB_DIR}/demo-code/train.py. If the training boot script is an .sh file, main.sh for example, the boot command can be bash ${MA_JOB_DIR}/demo-code/main.sh. Semicolons (;) and ampersands (&&) can be used to combine multiple boot commands, but line breaks are not supported. demo-code (customizable) in the boot command is the last-level directory of the OBS path.

Configuring Pipelines

A preset image-based algorithm obtains data from an OBS bucket or dataset for model training. The training output is stored in an OBS bucket. The input and output parameters in your algorithm code must be parsed to enable data exchange between ModelArts and OBS. For details about how to develop code for training on ModelArts, see Developing a Custom Script.

When you use a preset image to create an algorithm, configure the input and output pipelines.

Input configurations

**Table 2** Input configurations
Parameter	Description
Parameter Name	Set the name based on the data input parameter in your algorithm code. The code path parameter must be the same as the training input parameter parsed in your algorithm code. Otherwise, the algorithm code cannot obtain the input data. For example, If you use argparse in the algorithm code to parse data_url into the data input, set the data input parameter to data_url when creating the algorithm.
Description	Customizable description of the input parameter,
Obtained from	Source of the input parameter. You can select Hyperparameters (default) or Environment variables.
Constraints	Whether data is obtained from a storage path or ModelArts dataset. If you select the ModelArts dataset as the data source, the following constraints are added: Labeling Type: For details, see Creating a Labeling Job. Data Format, which can be Default, CarbonData, or both. Default indicates the manifest format. Data Segmentation: available only for image classification, object detection, text classification, and sound classification datasets. Possible values are Segmented dataset, Dataset not segmented, and Unlimited. For details, see Publishing a Data Version.
Add	Multiple data input sources are allowed.

Output configurations

**Table 3** Output configurations
Parameter	Description
Parameter Name	Set the name based on the data output parameter in your algorithm code. The code path parameter must be the same as the training output parameter parsed in your algorithm code. Otherwise, the algorithm code cannot obtain the output path. For example, If you use argparse in the algorithm code to parse train_url into the data output, set the data output parameter to train_url when creating the algorithm.
Description	Customizable description of the output parameter,
Obtained from	Source of the output parameter. You can select Hyperparameters (default) or Environment variables.
Add	Multiple data output paths are allowed.

Defining Hyperparameters

When you use a preset image to create an algorithm, ModelArts allows you to customize hyperparameters so you can view or modify them anytime. After the hyperparameters are defined, they are displayed in the startup command and transferred to your boot file as CLI parameters.

Import hyperparameters.
You can click Add hyperparameter to manually add hyperparameters.

Edit hyperparameters.

To ensure data security, do not enter sensitive information, such as plaintext passwords.

For details, see Table 4.

**Table 4** Hyperparameters
Parameter	Description
Name	Hyperparameter name Enter 1 to 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.
Type	Type of the hyperparameter, which can be String, Integer, Float, or Boolean
Default	Default value of the hyperparameter, which is used for training jobs by default
Constraints	Click Restrain. Then, set the range of the default value or enumerated value in the dialog box displayed.
Required	Select Yes or No. If you select No, you can delete the hyperparameter on the training job creation page when using this algorithm to create a training job. If you select Yes, you cannot delete the hyperparameter on the training job creation page when using this algorithm to create a training job.
Description	Description of the hyperparameter Only letters, digits, spaces, hyphens (-), underscores (_), commas (,), and periods (.) are allowed.

Adding Training Constraints

You can add training constraints of the algorithm based on your needs.

Resource Type: Select the required resource types.
Multicard Training: Choose whether to support multi-card training.
Distributed Training: Choose whether to support distributed training.

Runtime Environment Preview

When creating an algorithm, click the arrow on in the lower right corner of the page to know the path of the code directory, boot file, and input and output data in the training container.