Updated on 2024-08-14 GMT+08:00

Advanced Usage

Storage

This object contains capabilities of InputStorage and OutputStorage and can be flexibly used based on your needs.

Parameter

Description

Mandatory

Data Type

name

Name

Yes

str

title

If this parameter is left blank, the value of name is used by default.

No

str

description

Description

No

str

create_dir

Whether to create a directory. The default value is False.

No

bool

with_execution_id

Whether to combine execution_id when a directory is created. The default value is False. This parameter can be set to True only when create_dir is set to True.

No

bool

The following is an example:

  • Implementing InputStorage capabilities
    import modelarts.workflow as wf 
    # Create a Storage object (with_execution_id=False, create_dir=False).
    storage = wf.data.Storage(name="storage_name", title="title_info", description="description_info", with_execution_id=False, create_dir=False) 
    input_data = wf.data.OBSPath(obs_path = storage.join("directory_path")) # Add a slash (/) after a directory, for example, storage.join("/input/data/").
    
    When a workflow is running, if the root path of the storage object is /root/, the obtained path will be /root/directory_path.
  • Implementing OutputStorage capabilities
    import modelarts.workflow as wf 
    # Create a Storage object (with_execution_id=True, create_dir=True).
    storage = wf.data.Storage(name="storage_name", title="title_info", description="description_info", with_execution_id=True, create_dir=True) 
    output_path = wf.data.OBSOutputConfig(obs_path = storage.join("directory_path")) # Only a directory can be created.
    
    When a workflow is running, if the root path of the storage object is set to /root/, the system will automatically create a relative directory and the obtained path will be /root/Execution ID/directory_path.
  • Implementing different capabilities of a Storage object through the join method
    import modelarts.workflow as wf 
    # Create a Storage object. Assume that the root directory of the Storage object is /root/.
    storage = wf.data.Storage(name="storage_name", title="title_info", description="description_info", with_execution_id=False, create_dir=False) 
    input_data1 = wf.data.OBSPath(obs_path = storage) # The obtained path is /root/.
    input_data2 = wf.data.OBSPath(obs_path = storage.join("directory_path")) # The obtained path is /root/directory_path. Ensure that the path exists.
    output_path1 = wf.data.OBSOutputConfig(obs_path = storage.join(directory="directory_path", with_execution_id=False, create_dir=True)) # The system automatically creates a directory /root/directory_path.
    output_path2 = wf.data.OBSOutputConfig(obs_path = storage.join(directory="directory_path", with_execution_id=True, create_dir=True)) # The system automatically creates a directory /root/Execution ID/directory_path.

Chain call is supported for Storage.

The following is an example:
import modelarts.workflow as wf 
# Create a base class Storage object. Assume that the root directory of the Storage object is /root/.
storage = wf.data.Storage(name="storage_name", title="title_info", description="description_info", with_execution_id=False, create_dir=Fals)
input_storage = storage.join("directory_path_1") # The obtained path is /root/directory_path_1.
input_storage_next = input_storage.join("directory_path_2") # The obtained path is /root/directory_path_1/directory_path_2.