Help Center> ModelArts> MoXing Developer Guide> Mapping Between mox.file and Local APIs and Switchover
View PDF

Mapping Between mox.file and Local APIs and Switchover

API Mapping

  • Python: local file operation APIs of Python. The APIs can be shifted to the corresponding MoXing file operation APIs (mox.file) by one click.
  • mox.file: file operation APIs of MoXing Framework. The APIs correspond to the Python APIs.
  • tf.gfile: TensorFlow APIs with the same functions as MoXing file operation APIs. In MoXing, file operation APIs cannot be automatically switched to TensorFlow APIs. The following table lists only the APIs with similar functions.
Table 1 API mapping

Python (Local File Operation API)

mox.file (MoXing File Operation API)

tf.gfile (TensorFlow File Operation API)

glob.glob

mox.file.glob

tf.gfile.Glob

os.listdir

mox.file.list_directory(..., recursive=False)

tf.gfile.ListDirectory

os.makedirs

mox.file.make_dirs

tf.gfile.MakeDirs

os.mkdir

mox.file.mk_dir

tf.gfile.MkDir

os.path.exists

mox.file.exists

tf.gfile.Exists

os.path.getsize

mox.file.get_size

-

os.path.isdir

mox.file.is_directory

tf.gfile.IsDirectory

os.remove

mox.file.remove(..., recursive=False)

tf.gfile.Remove

os.rename

mox.file.rename

tf.gfile.Rename

os.scandir

mox.file.scan_dir

-

os.stat

mox.file.stat

tf.gfile.Stat

os.walk

mox.file.walk

tf.gfile.Walk

open

mox.file.File

tf.gfile.FastGFile(tf.gfile.Gfile)

shutil.copyfile

mox.file.copy

tf.gfile.Copy

shutil.copytree

mox.file.copy_parallel

-

shutil.rmtree

mox.file.remove(..., recursive=True)

tf.gfile.DeleteRecursively

One-Click Shift

You can use the following code to map the OS APIs in Table 1 to the mox.file APIs. Write the following code at the beginning of the boot script. When the OS APIs in the first column of the table are called during Python running, the APIs are automatically mapped to the APIs in the second column.

1
2
 
import moxing as mox
mox.file.shift('os', 'mox')

After the shift operation is completed, you can use the os.listdir or open function to operate OBS directories or files. The sample code is as follows:

1
2
3
4
5
6
7
8
 
import os 
import moxing as mox  

mox.file.shift('os', 'mox')  

print(os.listdir('obs://bucket_name')) 
with open('obs://bucket_name/hello_world.txt') as f:
   print(f.read())