Help Center/ ModelArts/ MoXing Developer Guide/ Introduction to MoXing Framework
Updated on 2024-09-30 GMT+08:00
View PDF
Share

Introduction to MoXing Framework

MoXing Framework provides basic common components for MoXing. For example, it facilitates access to Huawei Cloud OBS. Importantly, MoXing Framework is decoupled from specific AI engines and can be seamlessly integrated with all major AI engines (including TensorFlow, MXNet, PyTorch, and MindSpore) supported by ModelArts. MoXing Framework allows you to interact with OBS components using the mox.file APIs described in this section.

MoXing primarily serves to streamline the process of reading and downloading data from OBS buckets. However, it is not suitable for OBS parallel file systems. You are advised to call OBS Python SDKs to develop production service code. For details, see API Overview of OBS SDK for Python.

Why mox.file

Use Python to open a local file.

1
2
 
with open('/tmp/a.txt', 'r') as f:
  print(f.read())

An OBS directory starts with obs://, for example, obs://bucket/XXX.txt. You cannot directly use the open function to open an OBS file. The preceding code for opening a local file will report an error.

OBS provides many functions and tools, such as SDK, API, OBS Console, and OBS Browser. ModelArts mox.file provides a set of APIs for accessing OBS. The APIs simulate the operations of a local file system, allowing users to operate OBS files conveniently. For example, you can use the following code to open a file on OBS:

1
2
3
 
import moxing as mox
with mox.file.File('obs://bucket_name/a.txt', 'r') as f:
  print(f.read())

The following Python code lists a local path:

1
2
 
import os
os.listdir('/tmp/my_dir/')

To list an OBS path, add the following code in mox.file:

1
2
 
import moxing as mox
mox.file.list_directory('obs://bucket_name/my_dir/')