Introduction to Hyperparameter Search

ModelArts training supports hyperparameter search, which can automatically search for optimal hyperparameters for your models.

During model training, many hyperparameters, such as learning_rate and weight_decay, need to be adjusted based on actual requirements. This may cost an experienced algorithm engineer a lot of time and effort on manual optimization. However, the hyperparameter search supported by ModelArts can automatically optimize hyperparameters without the help of algorithm engineers and has higher optimization speed and precision than manual optimization.

ModelArts supports the following hyperparameter search algorithms:

• Bayesian Optimization (SMAC)
• TPE Algorithm
• Simulated Annealing Algorithm