Help Center/ ModelArts/ ModelArts User Guide (Standard)/ ModelArts Standard Preparations/ Creating an OBS Bucket for ModelArts to Store Data
Updated on 2024-12-26 GMT+08:00
View PDF
Share

Creating an OBS Bucket for ModelArts to Store Data

ModelArts does not provide data storage. Instead, it uses Object Storage Service (OBS) to store data, and backs up and takes snapshots for models, achieving secure, reliable, and cost-effective storage.

All the input data, output data, and cache data during AI development can be stored in OBS buckets for reading. Before using ModelArts, create an OBS bucket and folders for storing data.

Figure 1 Interaction between ModelArts and OBS
Table 1 Relationship between ModelArts and OBS

Function

Subtask

Relationship

Standard workflow of ModelArts Standard ExeML

Data labeling

The data labeled on ModelArts is stored in OBS.

Auto training

After a training job is complete, the generated model is stored in OBS.

Model deployment

ModelArts deploys models stored in OBS as real-time services.

ModelArts Standard AI development lifecycle

Data management
  • Datasets are stored in OBS.
  • The dataset labeling information is stored in OBS.
  • Data can be imported from OBS.

Development environment

Data or code files in a notebook instance can be stored in OBS.

Model training

Datasets, algorithms, running scripts, training outputs, and training logs used by training jobs can be stored in OBS.

Inference deployment

After a training job is complete, the generated model can be stored in OBS. You can import the model from OBS when creating a model.

Procedure for Creating an OBS Bucket

  1. Log in to the OBS console and click Create Bucket in the upper right corner of the page to create an OBS bucket.
    Figure 2 Creating a bucket

    The created OBS bucket and ModelArts are in the same region. For example, if ModelArts is located in the CN-Hong Kong region, select CN-Hong Kong when creating an OBS bucket.

    For details about how to obtain the region where the OBS bucket and ModelArts are located, see Check whether the OBS bucket and ModelArts are in the same region.

    Do not enable Default Encryption. ModelArts cannot read the data from encrypted OBS buckets.

  2. In the bucket list, click the bucket name to access its details page.
    Figure 3 Bucket list

  3. In the navigation pane on the left, choose Objects. On the Objects page, click Create Folder to create an OBS folder. For example, create a folder named flowers in the created c-flowers OBS bucket.
    Figure 4 Creating a folder

    After creating a folder in the OBS bucket, you can upload files. For details about how to upload files, see Upload Overview.

FAQs

  • Why can't I find my created OBS bucket after I select an OBS path in ModelArts?
  • How do I check whether ModelArts and an OBS bucket are in the same region?
  • What should I do if "Error: stat:403" is reported when I perform operations on an OBS bucket?

To resolve the preceding problems or other OBS path exceptions, see Incorrect OBS Path on ModelArts.