Introduction to MoXing Framework

Why mox.file

OBS is an object-based massive storage service. It cannot access files on OBS in the same way as accessing a local UNIX file system. You must read and write files on OBS through network requests. You are advised to use MoXing Framework (mox.file) to perform operations on the OBS directory in ModelArts.

Generally, Python provides the following function to open a local file:

1
2

with open('/tmp/a.txt', 'r') as f:
  print(f.read())

An OBS directory usually starts with obs://, for example, obs://bucket/XXX.txt. You cannot directly use the open function to open an OBS file. The preceding code for opening a local file will report an error.

OBS provides many functions and tools, such as SDK, API, OBS Console, and OBS Browser. ModelArts mox.file provides a set of APIs for accessing OBS. The APIs simulate the operations of a local file system, allowing users to operate OBS files conveniently. For example, you can use the following code to open a file on OBS:

1
2
3

import moxing as mox
with mox.file.File('obs://bucket_name/a.txt', 'r') as f:
  print(f.read())

The following Python code lists a local path:

1
2

import os
os.listdir('/tmp/my_dir/')

To list an OBS path, you can use the following code of mox.file:

1
2

import moxing as mox
mox.file.list_directory('obs://bucket_name/my_dir/')