Configuring Cloud Resources

Resource Requirements

Creating an OBS Bucket

ModelArts requires you to create an OBS bucket for storing the model weight files, training data, training scripts, and training artifacts. For details, see How Do I Create an OBS Bucket for ModelArts to Store Data?

The following shows the folders in the OBS bucket. The planned directories must be the same as the file paths involved in the training script. After a training job is created and the OBS bucket is added to the code directory, all files in the folder will be copied to the training container. If the file size is too large, the copy speed will decrease. You are advised to only retain related files in the directory to avoid storing irrelevant content. For example, for each training job, the {models} folder only stores the models used this time.

|——{OBS bucket}                                         # Custom bucket name, for example, mindspeed-llm
   |──{mindspeed-a2}                               # Custom OBS folder name, for example, mindspeed-llm-a2
       |──{models}                                 # Custom OBS folder for storing various models, for example, models
           |──{Qwen3-8b}                          # Custom OBS folder for storing models, for example, Qwen3-8b
           |──{Qwen3-30b-a3b}                      # Custom OBS folder for storing models, for example, Qwen3-30b-a3b
           |──{Qwen3-32b}                      # Custom OBS folder for storing models, for example, Qwen3-32b
       |──ckpt_convert_qwen3_hf2mcore.sh       # Qwen3-32b weight conversion script (hf2mg)
       |──data_convert_qwen3_instruction.sh    # Qwen3-32b data processing script
       |──tune_qwen3_32b_4K_full_ptd.sh         # Script for executing the Qwen3-32b fine-tuning task
       |──ckpt_convert_qwen3_mcore2hf.sh       # Qwen3-32b weight conversion script (mg2hf)
       |──run_distributed_task.sh                # Script for starting the complete Qwen3-32b fine-tuning task
       |──{dataset}/                               # Custom OBS folder for storing various datasets, for example, dataset.
            |──{alpaca}/                           # OBS folder for storing the training dataset alpaca
            |──{alpaca_gpt4_data}/                # OBS folder for storing the training dataset alpaca_gpt4_data
       |──{output}                                # Custom OBS folder for storing the final training output

You can use tools such as OBS Browser+ and obsutil to access and manage OBS buckets and upload and download files and data.