Notes and Constraints

Compute resources

All compute resource specifications in pay-per-use, yearly/monthly, and package modes, including CPU, GPU, and NPU

All types of purchased compute resources cannot be used across regions.

Compute resources

Package

Packages are used only for public resource pools and cannot be used for dedicated resource pools.

ModelArts Standard notebook instance

A maximum of 10 notebook instances can be created under one account.

No

For more information, see Creating a Notebook Instance.

ModelArts Standard real-time service

A maximum of 20 real-time services can be created under one account.

Yes

Submit a service ticket to increase the quota.

For more information, see Deploying a Model as a Real-Time Service.

ModelArts Standard batch service

A maximum of 1,000 batch services can be created under one account.

No

For more information, see Deploying a Model as a Batch Inference Service.

ModelArts Standard edge service

A maximum of 1,000 edge services can be created under one account.

No

None

ModelArts Standard dedicated resource pool

A maximum of 50 dedicated resource pools can be created under one account.

Yes

Submit a service ticket to increase the quota.

For more information, see Creating a Standard Dedicated Resource Pool.

ModelArts Standard tag

A maximum of 20 tags can be added to a training job, notebook instance, or real-time service.

No

For more information, see How Does ModelArts Use Tags to Manage Resources by Group?

ModelArts Standard dedicated resource pool

• It is good practice to create no more than 30 nodes at a time. Otherwise, the creation may fail due to traffic limiting.

  For more information, see Creating a Standard Dedicated Resource Pool.

• Only pools in the Running status can be resized. The number of instances cannot be decreased to 0.
• A pool's job types can only be modified while it is running.
• A pool's driver can only be upgraded while it is running and there are GPU or Ascend resources in its nodes.
• For a logical resource pool, the driver can be upgraded only after node binding is enabled. To enable node binding, submit a service ticket to contact engineers.

ModelArts Standard notebook instance

• Deleted notebook instances cannot be recovered. After a notebook instance is deleted, the data stored in the mounted directory will be deleted.
• You can only change an image on a stopped notebook instance.
• You can modify a notebook instance's specifications while it is stopped, running, or failed to start.
• The target notebook instance must use EVS for storage. If the original capacity of an EVS disk is 4096 GB, the disk capacity cannot be expanded. A maximum of 100 GB can be added at a time.
• After the notebook instance is stopped, the expanded EVS capacity still takes effect. The EVS billing is based on the expanded capacity. An EVS disk is billed as long as it is used. To stop billing an EVS disk, delete data from the EVS disk and release the disk.
• Images stored in a notebook instance cannot be larger than 35 GB and there cannot be more than 125 image layers. Otherwise, the image cannot be saved.

ModelArts Standard training job

• Training logs are retained for only 30 days. To permanently store logs, enable persistent log saving and set a job log path for dumping when creating a training job. For Ascend training, you need to configure the OBS path for storing training logs by default. You need to manually enable Persistent Log Saving for training jobs using other resources.
• Only dedicated resource pools allow logging in to training containers using Cloud Shell. The training job must be running.
• Algorithms subscribed from AI Gallery cannot be saved as new algorithms.
• Suspension can be detected only for training jobs that run on GPUs.
• The priority can be set for a training job only when a new-version dedicated resource pool is used. The job priority cannot be set for training jobs using a public resource pool or old-version dedicated resource pool.
• Only the PyTorch and MindSpore frameworks can be used for distributed training and debugging. If you want to use MindSpore, each node must be equipped with eight cards.
• When using a custom image to create a training job, ensure that the custom image size is under 15 GB and does not exceed half of the container engine space in the resource pool. An oversized image affects the startup of a training job. The container engine space of a ModelArts public resource pool is 50 GB. By default, the container engine space of a dedicated resource pool is also 50 GB. You can customize the container engine space when creating a dedicated resource pool.
• The uid of the default user of a custom training image must be 1000.

Model Standard inference model

• The maximum size of a model file from OBS is 20 GB. For more information, see Creating an AI Application.
• If the size of your file exceeds the container engine space, a message will be displayed, indicating that the image space is insufficient. The maximum container engine space in a public resource pool is 50 GB, and that for a dedicated resource pool is 50 GB by default. You can set the container engine space for a dedicated resource pool when you create it, which does not increase costs.

  For more information, see Restrictions on the Size of an Image for Importing an AI Application.

• After deploying a model in an ExeML project, it is automatically added to the model list. ExeML-generated models can only be deployed, not downloaded.

ModelArts Standard inference service

• Cloud Shell can only access a container when the associated real-time service is deployed within a dedicated resource pool and running.
• Batch services can only be deployed in public resource pools.
• For models in synchronous request mode, if the prediction request latency exceeds 60 seconds, the request will fail, and there is a possibility that the service may be interrupted. In this case, create an image in asynchronous request mode.

ModelArts Lite Server

• If ModelArts Lite Server runs on BMSs, upgrading or changing the OS kernel or driver can render the driver and kernel incompatible, preventing the OS from starting or making basic functions unavailable.

  To upgrade or change the OS kernel or driver, contact Huawei Cloud technical support.

• ModelArts Lite Server running on ECSs does not support OS reinstallation. Similarly, some BMSs in specific regions have this limitation. To reinstall the OS, switch to a different one.
• When you reinstall or change the OS of ModelArts Lite Server, the EVS system disk ID changes, which may differ from the original ID in your purchase order. This prevents you from expanding the EVS system disk capacity, resulting in an error message: "The order is expired. The capacity cannot be expanded. Renew the order." You are advised to expand the storage capacity by attaching EVS or SFS disks.

ModelArts Lite Cluster

• Only pools in the Running status can be resized. The number of instances cannot be decreased to 0.
• You can specify the container engine space size when creating a resource pool.
• To change the container engine space for new nodes in an existing Lite Cluster resource pool, set the desired size. The container engine space of existing nodes cannot be modified, as this would create inconsistencies in the dockerBaseSize across nodes with the same flavor within the pool, potentially disrupting task execution on different nodes.
• A pool's driver can only be upgraded while it is running and there are GPU or Ascend resources in its nodes.
• For a logical resource pool, the driver can be upgraded only after node binding is enabled. To enable node binding, submit a service ticket to contact engineers.

Interaction between ModelArts and OBS

• ModelArts does not support encrypted OBS buckets. When creating an OBS bucket, do not enable bucket encryption.
• ModelArts does not support cross-region access to OBS buckets. Ensure that OBS and ModelArts are in the same region.