Boot File

The code developed locally must be uploaded to the ModelArts backend. In code, it is error-prone to set the path for storing the dependency file.

The following general solution is recommended: Use the OS API to obtain the file path.

Example:

|---project_root                # Root directory for code
   |---BootfileDirectory        # Directory where the boot file is located
     |---bootfile.py            # Boot file
   |---otherfileDirectory       # Directory of other dependency files
     |---otherfile.py           # Other dependency files
    

Do as follows to obtain the path of the dependency file, otherfile_path in this example, in the boot file:

import os 
current_path = os.path.dirname(os.path.realpath(__file__)) # Directory where the boot file is located
project_root = os.path.dirname(current_path) # Root directory of the project, which is the code directory set on the ModelArts training console
otherfile_path = os.path.join(project_root, "otherfileDirectory", "otherfile.py")

Run the following commands to obtain the sizes of the copied folders and the folders to be copied. Then determine whether the folder is copied based on the command output.

import moxing as mox 
mox.file.get_size('obs://bucket_name/obs_file',recursive=True)

get_size indicates the size of the file or folder to be obtained. recursive=True indicates that the type is folder. False indicates that the type is file.

If the command output is consistent, the folder is copied. If the command output is inconsistent, the folder is not copied.

For more MoXing operations, see MoXing Developer Guide.

ModelArts allows you to invoke shell scripts. You can save the shell script to the OBS path where the code directory is located.

For example, the code directory is /bucket-name/code/, the boot file path is /bucket-name/code/test.py, and the shell script path is /bucket-name/code/test.sh.

The backend automatically downloads the code directory to the /home/work/user-job-dir/ directory of the container. You can invoke the shell script in the boot file using the following commands:

import os 
os.system('bash /home/work/user-job-dir/code/test.sh')