Storage Mounting

Overview

In ModelArts inference deployment scenarios, file storage mounting is a core capability designed to address LLM storage, data sharing, and loading acceleration. It adapts to high-performance, high-stability, and high-throughput inference service requirements. The core applicable scenarios are as follows:

1. Storage and loading of ultra-large models: LLMs and multimodal models can range in size from tens of gigabytes to terabytes, making it impossible to embed them directly into container images. Model files must be centrally stored using mounted storage so that inference instances can read them directly, preventing bloated images and deployment timeouts.
2. Data sharing across multiple instances: Multiple replicas of the same inference service, or different inference services altogether, often need to share configuration files, inference dependency data, and intermediate results. Mounted storage enables centralized data management and real-time synchronization, eliminating the need for repeated data copying.
3. Inference acceleration and rapid recovery: By utilizing local storage acceleration features, models are cached onto the host machine. When a container restarts due to a failure, it reuses the cached model, eliminating the time consumed by repeatedly pulling the model. This significantly improves service recovery efficiency and fits high-availability service scenarios.
4. Rapid reuse of pre-warmed models: High-frequency models can be warmed up in the resource pool in advance. When deploying an inference service, the pre-warmed model is mounted directly without needing to be reloaded, shortening deployment time and accommodating high-frequency, rapid-launch inference demands.
5. Custom model deployment: Custom models that are not built directly into ModelArts can be integrated into inference services via storage mounting. This flexibly accommodates deployment scenarios for third-party or in-house models.

Storage Types

ModelArts real-time inference services support configuring the storage type within the model source, and it also supports mounting storage during unit configuration.

• Model source storage type

  The storage selected under the model settings on the deployment page is specifically dedicated to storing the model weight files required for running inference. Its primary purpose is to load the AI model, making it an absolute prerequisite for service startup. In most scenarios, it is read-only and will not be modified. For the specific configuration entry point, see Configuring Deployment Settings.

  Figure 1 Model Source > Storage Type
  
  Supported storage types for model sources: OBS buckets, OBS parallel file systems, SFS Turbo, and pre-warmed models.

  Table 1 Recommended storage types for model sources

  Storage Type	Use Case	Constraint
  Object Storage Service – Bucket	Small and medium-sized lightweight models (tens of MB to several GB). Testing environments, demo scenarios, and non-core online services. Low model update frequencies, low concurrency, and minimal access pressure. Scenarios prioritizing the lowest storage cost over loading speed. Simple inference tasks with no requirement for massive small file reading.	Not applicable to large or ultra-large models, high-concurrency inference, or scenarios requiring rapid batch service startup.
  Object Storage Service – Parallel File System	The preferred choice for popular LLM deployment (7B, 13B, 34B, and other general LLMs). Multi-instance, high-concurrency inference, and clustered real-time inference. Models with numerous files, deeply nested directories, or massive amounts of small files. Interconnected training and inference data requiring frequent reads/writes of model dependency files. Scenarios seeking a balance between high performance and low cost, avoiding premium-priced SFS Turbo. Enterprise-grade, official online inference services.	/
  Scalable File Service Turbo (SFS Turbo)	Official production launch of ultra-large parameter models (100B+ parameters). High-stability, low-latency, strong-consistency core services such as finance and government affairs. Scenarios requiring second-level model loading and ultra-fast service restart/recovery. Real-time modification of model configurations or dynamic reading and writing of weights during inference. Extremely high requirements for I/O latency, stability, and fault recovery speed.	Higher usage costs. Can only be used in dedicated resource pools, and the dedicated resource pool must be associated with SFS Turbo.
  Pre-warmed Model	Rapid testing, temporary launches, and emergent scaling. Frequent and repeated deployment of standardized general models. Scenarios prioritizing the fastest startup speed without waiting for model downloads and loading. Large-scale batch deployment of inference services within the same dedicated resource pool.	Can only be used in dedicated resource pools. Models must be warmed up on the nodes in advance. Services can only be scheduled to pre-warmed nodes; the service cannot start if scheduling fails.

• Unit Settings > Mount File Storage

  This extra storage mounted within the inference unit does not store the primary model. Instead, it is dedicated to mounting service-operational supporting data, input/output files, and inference output dumps. Its purpose is to support the model throughout its entire service workflow. Classified as a supplementary service resource, it is optional and operates mostly in read/write mode. For the specific configuration entry point, see Configuring Deployment Settings.

  Figure 2 Unit Settings > Mount File Storage
  

  Supported storage types for file storage in unit settings: OBS buckets, OBS parallel file systems, SFS Turbo, and pre-warmed models.

  Supported storage types for artifact dump in unit settings: OBS parallel file systems.

Object Storage Service – Bucket

• Storage Location: Select an OBS bucket path. Cross-region OBS buckets cannot be selected. You can add up to 15 paths. Encrypt sensitive data before saving it to your OBS bucket.

  It is good practice to create a directory. Avoid using inventory directories or system directories with strict permissions.

  You can choose your own OBS bucket or enter a path. The path must start with obs:// and end with a slash (/), like this: obs://bucketname/path/. For shared buckets from other users, you must enter the path.

  When ModelArts connects with IAM and you use a shared bucket path, the bucket owner must grant you access and read permissions in the bucket's ACL policy.

  Figure 3 Bucket ACL permissions
  

  The owner of the shared bucket must set up an agency in ModelArts to grant OBS permissions to all users. To do this, select all users for authorization and choose OBS for the function permission. For details, see Configuring Agency Authorization for ModelArts with One Click.

• Mount Path: Enter the container mount path, for example, /obs-mount/. It is good practice to create a directory. Avoid using inventory directories or system directories with strict permissions. Avoid using nested directories when configuring multiple mount paths.
• Local Storage Acceleration: Model data is pulled from external storage to the host (usually a CCE cluster node), and the model path on the host is mounted to the container directory specified by the user. If the local cache is still on the current node after the service container is restarted due to a fault, the model pulling process can be skipped. With local storage acceleration, when a service container is restarted due to a fault, the local cache can be reused, implementing fast recovery. The local cache can only be retained if the node where the service container is located is not destroyed and the storage volume is not deleted. If the node is faulty or the storage volume is cleared, the cache becomes invalid, and the model data needs to be pulled again.

  When using local storage acceleration, make sure the data disk has over 1,024 GiB of space in the dedicated resource pool.

Object Storage Service – Parallel File System

• Storage Location: Select a storage path. A cross-region OBS parallel file system cannot be selected. Encrypt sensitive data before saving it to your OBS parallel file system.

  You can choose your own OBS parallel file system or enter a path. The path must start with obs:// and end with a slash (/), like this: obs://bucketname/path/. For shared OBS paths from other users, you must enter the path.

  When ModelArts connects with IAM and you use a shared OBS path, the owner must grant you access and read permissions in the OBS object's ACL policy.

  Figure 4 ACL permissions
  

  The owner of the shared bucket must set up an agency in ModelArts to grant OBS permissions to all users. To do this, select all users for authorization and choose OBS for the function permission. For details, see Configuring Agency Authorization for ModelArts with One Click.

• Mount Path: Enter the container mount path, for example, /obs-mount/.
  • Select a new directory. If you select an existing directory, existing files will be overwritten. OBS mounting allows you to add, view, and modify files in the mount directory but does not allow you to delete files in the mount directory. To delete files, manually delete them in the OBS parallel file system.
  • Mount an empty directory to the container. If the directory is not empty, ensure that the directory does not contain any files that affect container startup. Otherwise, the files will be replaced, and the container cannot start normally. As a result, the workload may not be deployed.
  • The mount path must start with a slash (/) and can contain a maximum of 1,024 characters, including letters, digits, and the following special characters: \_-.
• Local Storage Acceleration: Model data is pulled from external storage to the host (usually a CCE cluster node), and the model path on the host is mounted to the container directory specified by the user. If the local cache is still on the current node after the service container is restarted due to a fault, the model pulling process can be skipped. With local storage acceleration, when a service container is restarted due to a fault, the local cache can be reused, implementing fast recovery. The local cache can only be retained if the node where the service container is located is not destroyed and the storage volume is not deleted. If the node is faulty or the storage volume is cleared, the cache becomes invalid, and the model data needs to be pulled again.

  When using local storage acceleration, make sure the data disk has over 1,024 GiB of space in the dedicated resource pool.

Scalable File Service Turbo (SFS Turbo)

You can mount storage only if you use a dedicated resource pool for deploying the service. This pool must be associated with an SFS Turbo file system. For details, see Associating the Network of a Dedicated Resource Pool File System with SFS Turbo.

Parameters:

• File System/File System Directory: Select an SFS Turbo file system and the directory where the model file is located. A cross-region SFS Turbo file system cannot be selected.
• Mount Path: Enter the mount path of the container, for example, /sfs-turbo-mount/. Select a new directory. If an inventory directory is selected, the inventory files in it will be overwritten. Avoid using nested directories when configuring multiple mount paths.
• Mount Mode: Read/Write and Read-only are supported.
• Local Storage Acceleration: Model data is pulled from external storage to the host (usually a CCE cluster node), and the model path on the host is mounted to the container directory specified by the user. If the local cache is still on the current node after the service container is restarted due to a fault, the model pulling process can be skipped. With local storage acceleration, when a service container is restarted due to a fault, the local cache can be reused, implementing fast recovery. The local cache can only be retained if the node where the service container is located is not destroyed and the storage volume is not deleted. If the node is faulty or the storage volume is cleared, the cache becomes invalid, and the model data needs to be pulled again.

Notes:

• A file system can be mounted only once and to only one path. Each mount path must be unique. You can mount up to eight disks.
• If you need to mount multiple files, do not use the paths that are the same or similar, for example, /obs-mount/ and /obs-mount/tmp/.
• Once you have chosen SFS Turbo, avoid disassociating SFS Turbo. Otherwise, mounting will not be possible. When you mount the backend OBS storage on the SFS Turbo page, make sure to set the client's umask permission to 777 for normal use.

Pre-warmed Model

• Pre-warmed Model: Select a model that has been warmed up in the ModelArts resource pool.
• Mount Path: Specify the mount path inside the container. You are advised to create a directory and do not select an existing directory or a system directory with strict permissions. The mount path must start and end with a slash (/). The path can contain at most 255 characters. The mount path must be the same as that of other model sources.
  

  If the task is assigned to nodes that are not pre-warmed, the deployment will fail due to resource limits.