How Do I Create a Training Job When a Dependency Package Is Referenced by the Model to Be Trained?
Store the pip-requirements.txt file in the training code directory.
Any one of the following file names can be used. This section uses pip-requirements.txt as an example.
- pip-requirement.txt
- pip-requirements.txt
- requirement.txt
- requirements.txt
Before the training boot file is executed, the system automatically runs the following command to install the specified Python packages:
pip install -r pip-requirements.txt
- For details about the code directory, see Storing the Installation File in the Code Directory.
- For details about the specifications of pip-requirements.txt, see Installation File Specifications.
Storing the Installation File in the Code Directory
ModelArts allows you to install third-party dependency packages during model training in either of the following ways:
- Method 1 (recommended): , store the required files or installation packages in the code directory.
Figure 1 Creating an algorithm
- Method 2: Before using a common framework to create a training job, store the required files or installation packages in the code directory. (This function will be unavailable soon.)
Figure 2 Using a common framework to create an algorithm
Installation File Specifications
The installation file varies depending on the dependency package type.
- WHL packages
If the training background does not support the download of open source installation packages or use of user-compiled WHL packages, the system cannot automatically download and install the package. In this case, place the WHL package in the code directory, create a file named pip-requirements.txt, and specify the name of the WHL package in the file. The dependency package must be a .whl file.
Take for example, an OBS path specified by Code Dir that contains model files, the .whl file, and the pip-requirements.txt file. The code directory structure would be as follows:
|---OBS path to the model boot file |---model.py #Model boot file |---XXX.whl #Dependency package. If multiple dependencies are required, place multiple dependency packages here. |---pip-requirements.txt #Defined configuration file, which specifies the name of the dependency packageThe following shows the content of the pip-requirements.txt file:
numpy-1.15.4-cp36-cp36m-manylinux1_x86_64.whl tensorflow-1.8.0-cp36-cp36m-manylinux1_x86_64.whl
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.
