Creating a Training Job for Automatic Model Tuning

Context

To use ModelArts hyperparameter search, the AI engine must be either pytorch_1.8.0-cuda_10.2-py_3.7-ubuntu_18.04-x86_64 or tensorflow_2.1.0-cuda_10.1-py_3.7-ubuntu_18.04-x86_64, and the hyperparameter to be optimized must be a float value.

To perform a hyperparameter search without any code modification, follow these steps:

1. Preparations
2. Creating an Algorithm
3. Creating a Training Job
4. Viewing Details About a Hyperparameter Search Job

Preparations

• Create a dataset in ModelArts or upload a training dataset to an OBS directory.
• Upload your training script to an OBS directory. For details about how to develop a training script, see Developing Code for Training Using a Preset Image.
• Print search indicator parameters in the training code.
• Create at least one empty folder in OBS for storing training outputs.
• Make sure your account is not in arrears, as training jobs consume resources.
• Make sure your OBS directory and ModelArts are in the same region.

Creating an Algorithm

Log in to the ModelArts console and create an algorithm by referring to Creating an Algorithm. The image must use the pytorch_1.8.0-cuda_10.2-py_3.7-ubuntu_18.04-x86_64 or tensorflow_2.1.0-cuda_10.1-py_3.7-ubuntu_18.04-x86_64 engine.

To define hyperparameters for optimization, specify the name, type, default value, and constraints in Hyperparameter. For details, see Table 6.

To enable auto search for the algorithm, select autoSearch(S), print search parameters in the code, and configure the following parameters. ModelArts uses a regular expression to obtain search indicator parameters during an auto search, and then performs hyperparameter optimization based on the specified optimization direction.

Figure 1 Enabling auto search

• Search Indicator

  The search indicator represents the value of the objective function, such as loss or accuracy. By optimizing and converging this value in the desired direction, you can find the optimal hyperparameters to enhance model accuracy and convergence speed.

  Table 1 Search indicator parameters

  Parameter	Description
  Name	Enter a search indicator name. This value must be identical to the search indicator parameter in the code.
  Optimization Direction	Select max or min.
  Counter regularization	Enter a regular expression or click Generate Intelligently to generate a regular expression automatically.

• Setting Automatic Search Parameters

  Select hyperparameters from the Hyperparameters configuration. Only float-type hyperparameters are supported. After selecting autoSearch(S), set the value range.

• Search Algorithm Configuration

  ModelArts has three built-in algorithms for hyperparameter search. You can select one or more algorithms as needed. The algorithms and their parameter description are as follows:

  • bayes_opt_search: Bayesian Optimization (SMAC)
  • tpe_search: Tree-structured Parzen Estimator (TPE)
  • anneal_search: Simulated Annealing

After creating the algorithm, use it to create a training job.

Creating a Training Job

Log in to the ModelArts console and create a training job by referring to Creating a Production Training Job.

Viewing Details About a Hyperparameter Search Job

After a training job is completed, you can review the results of the automated hyperparameter search to evaluate the job's performance.

If the training job is a hyperparameter search job, go to the training job details page and click the Auto Search Results tab to view the hyperparameter search results.

Figure 3 Hyperparameter search results