What's New

1

Using Notebook Instances Online Through Code Server

Code Server is an open-source project that allows you to run Visual Studio Code on a remote server and access it entirely through a web browser. You can start Code Server in a notebook instance to develop code.

--

Using Code Server for Code Development

2

Viewing Historical Records of Live Comparison

You can view the historical records of live comparison. You can click any historical dialog to continue to ask questions based on the dialog.

--

Live Comparison

3

Data Preparation Error Codes

If an error occurs when you use the data preparation function, rectify the fault by referring to the error code document.

--

Data Preparation Error Codes

1

Image document optimization

• Enhanced the "ModelArts Preset Images" section with details on naming rules, image capabilities, core prest images, and how to view the image address, helping you quickly understand and use preset images.

• Enhanced the "ModelArts Unified Images" section with an optimized image list and additional details, such as image components, helping you quickly obtain and use unified images.

--

ModelArts Preset Images

ModelArts Unified Images

1

Upgrading JupyterLab

To enhance the usability of notebook, JupyterLab has been upgraded to version 4.4.10. This major leap from the previous version (3.2.3) significantly optimizes performance and startup speed. Key improvements include a rebuilt editor, enhanced debugging and collaboration tools, a more stable ecosystem, and comprehensive refinements to the interactive experience and memory efficiency.

--

Upgrading JupyterLab

1

Training log page optimization

The training job details page is optimized. You can view user logs, system logs, and all logs.

--

Viewing Training Job Logs

1

Distributed Model Training - Creating a Ray Cluster

Users performing reinforcement learning (RL) training tasks typically require a Ray cluster to ensure smooth task execution. The ModelArts environment supports running Ray distributed jobs by starting the Ray cluster before the training begins.

--

Creating a Ray Cluster

1

Independent MaaS documentation

MaaS documentation is separated from ModelArts documentation and independently developed, with enriched content to facilitate search.

--

MaaS Service Documentation

1

Added the NodeTaskHub plug-in to Resource Management (Lite Server)

The NodeTaskHub plug-in has been added to ModelArts Lite Server. This plug-in supports the delivery of tasks such as Ascend software upgrades, pressure tests, fault diagnosis, and system configuration. It enables quick closure of basic O&M tasks (such as driver firmware upgrade and system configuration), reducing the fault locating period from 3 days to 3 hours and significantly improving fault locating efficiency.

--

Installing the AI Plugin for a Lite Server

2

Added Plug-in Square

ModelArts provides the Plug-in Square function to aggregate plug-ins in one place. It offers a wide range of plug-ins, including Device Plugin, kube-prometheus-stack, and NodeLocal DNS Cache, to enhance resource utilization and O&M capabilities and meet diverse service requirements.

--

Overview

3

Added fault detection and fast recovery on the training platform

The reliability of ModelArts training jobs is enhanced by improving detection, fast recovery, and logging capabilities, thereby improving the maintainability of training jobs.

• Added the function of viewing fault recovery and statistics in job details.

• Added alarms for exceptions such as image pull failures and storage mounting failures during job running.

• Added automatic job log dump and fault information such as operator information and memory information.

--

Viewing Training Job Events

1

AI Computing Service on CloudMatrix

The next-generation Ascend AI computing service CloudMatrix384 supports high-speed bus interconnection of up to 384 processing units (PUs), provides high-performance and highly reliable supernode clusters, and supports efficient running of models and algorithms, enabling efficient deployment of models and algorithms.

• Scale-up: A supernode is composed of 384 PUs interconnected with high-speed network switches.

• Scale-out: Switches on the parameter plane support clusters with up to 160,000 PUs.

--

Scaling a Lite Server Supernode

1

Enabling node draining when scaling logical subpools with node binding enabled

Check if node draining is required when scaling logical subpools with node binding enabled. If node draining is enabled, it runs only once for the nodes being scaled.

Commercial use

Viewing Details About a Standard Dedicated Resource Pool

2

Viewing jobs running a single node of a Standard dedicated resource pool

In the Nodes tab of the resource pool details page, you can view the jobs running on a single node.

Commercial use

Viewing Details About a Standard Dedicated Resource Pool

3

Installing plug-ins for dedicated resource pools in Standard & Lite Cluster

ModelArts offers several plug-ins to help you expand resource pool functions as needed.

Commercial use

Standard dedicated resource pool plug-ins

Lite Cluster plug-ins

1

Enabling or disabling the deletion lock for resource pool nodes

To prevent nodes from being deleted or unsubscribed by mistake, you can enable the deletion lock. Once enabled, the nodes cannot be deleted or unsubscribed unless the lock is disabled.

Commercial use

Rectifying a Faulty Node in a Standard Dedicated Resource Pool

Managing Lite Cluster Nodes

2

Enabling or disabling node binding for a dedicated resource pool

If a logical subpool has been created in a resource pool, you can enable node binding to bind dedicated nodes to the logical subpool.

Commercial use

Viewing Details About a Standard Dedicated Resource Pool

3

Using Ascend FaultDiag to diagnose logs in a ModelArts Lite Cluster resource pool

If a logical subpool has been created in a resource pool, you can enable node binding to bind dedicated nodes to the logical subpool.

Commercial use

Using Ascend FaultDiag to Diagnose Logs in the ModelArts Lite Cluster Resource Pool

4

Binding nodes in a standard resource pool to a logical subpool

After node binding is enabled for a Standard dedicated resource pool, nodes in the resource pool can be bound to a specified logical subpool.

Commercial use

Viewing Details About a Standard Dedicated Resource Pool

5

Authorizing O&M on the event center page

To view the faulty nodes reported by the ModelArts O&M platform, log in to the ModelArts console. In the navigation on the left, choose Resource Management > Event Center. The planned events of the faulty nodes are displayed, including the basic information, event type, event status, and event description. You can either redeploy the nodes or authorize Huawei technical support to perform O&M operations.

Commercial use

Rectifying a Faulty Node in a Standard Dedicated Resource Pool

Managing Lite Cluster Nodes

1

Accessing a real-time service through a VPC channel

To connect to a ModelArts VPC endpoint service, you can create a VPC endpoint in a specific VPC within your account.

Commercial use

Accessing a Real-Time Service Through a VPC Channel

2

Batch input of hyperparameters and environment variables for training jobs

When creating or modifying a training job, you can input hyperparameters and environment variables in batches.

Commercial use

Creating a Production Training Job

3

Starting MindSpore using msrun or torchrun

To run the preset Ascend-Powered-Engine training framework, you can configure the environment variable MA_RUN_METHOD to set the boot mode as either msrun or torchrun.

Commercial use

Starting a Preset Image's Boot File

4

ModelArts Notebook scheduled task

You can create scheduled tasks, run notebook files with just one click, and customize the time and frequency for executing code blocks.

Commercial use

Creating a Scheduled Job in JupyterLab

5

Password-free SSH mutual trust between nodes for a training job created using a custom image

If you use a custom image based on the MPI or Horovod framework for distributed training, you can configure password-free SSH mutual trust between training job nodes.

Commercial use

Configuring Password-free SSH Mutual Trust Between Nodes for a Training Job Created Using a Custom Image

6

Training job suspension detection

The environment variable MA_HANG_DETECT_TIME is set to 30 by default, which means a job is considered suspended if its process I/O does not change for 30 minutes. This parameter can be modified.

Commercial use

Detecting Training Job Suspension

1

ModelArts documentation upgrade

The user guide is now organized by product, with separate sections for each. The ModelArts console operation guide is now included in the ModelArts Standard User Guide.

Added user guides for ModelArts Lite Server, ModelArts Lite Cluster.

Commercial use

ModelArts Standard Usage

Lite Server Usage

1

Training fault recovery details

If a training job experiences a fault, such as process-level recovery, POD-level rescheduling, or job-level rescheduling, you can access the job details page to view the fault recovery details. This page displays the start and stop details of the training job.

Commercial use

Training Job Fault Tolerance Check

2

Customizable training specifications

When creating a training job, you can customize dedicated resource pool specifications to improve resource utilization.

Commercial use

Creating a Training Job

1

Billing details

The new topic introduces ModelArts billing modes and items. You can learn how to change the billing mode, renew a subscription, view bills, and stop being charged.

Commercial use

Billing

1

Notebook cache directory capacity alarms

There were no capacity alarms for cache directories in the development environment. Once the capacity usage exceeds the limit, notebook instances will be restarted, and multiple configurations will be reset. As a result, your data and the environment are discarded. Monitoring and alarms of cache directories are now available so the data can be reported to AOM.

Commercial use

Reporting Notebook Cache Directory Capacity

2

Node replacing

You can replace a single node in dedicated resource pools.

Commercial use

Viewing Details About a Resource Pool

3

List of faulty node isolation codes

Isolation codes for faulty nodes and fault locating methods are provided.

Commercial use

Abnormal Status of a Dedicated Resource Pool

4

CIDR Block parameter for creating dedicated resource pools

You can use the default CIDR block or set a custom CIDR block when you create a dedicated resource pool.

Commercial use

Creating a Resource Pool

5

Master Distribution parameter for creating dedicated resource pools

You can set Master Distribution to either Random or Custom when you create a dedicated resource pool.

Commercial use

Creating a Resource Pool

6

IPv6 for creating dedicated resource pools

You can enable IPv6 when you create a network and a dedicated resource pool.

Commercial use

Creating a Resource Pool

7

Storage volume mounting for real-time services

You can enable this function when you create a real-time service. A storage volume will be mounted to a compute node (compute instance) as a local directory.

Commercial use

Deploying as a Real-Time Service

8

IPv6 for real-time services

You can enable IPv6 for a real-time service during creation.

Commercial use

Deploying as a Real-Time Service

9

Runtime User ID configuration

If you set Boot Mode to Preset Image and the framework version to Customize during training job creation, you can customize the user ID used during container runtime.

Commercial use

Creating a Training Job

10

Tips for locating faults for Cloud Shell

Cloud Shell cannot be used if a training job is not in running state or the permission is insufficient. You can locate the fault as prompted.

Commercial use

Using Cloud Shell to Debug a Production Training Job

1

Tool Guide deleted

The original content in the Tool Guide are now provided in "DevEnviron" > "Local IDE" > "Local IDE (PyCharm)" > "Connecting to a Notebook Instance Through PyCharm Toolkit".

Discontinued

PyCharm Toolkit

1

Filtering training jobs by resource pool

You can filter training jobs by resource pool in the training job list.

Commercial use

Rebuilding

Stopping

or Deleting a Training Job

2

Importing large models for ModelArts inference

You can import large models to create AI applications and deploy services in ModelArts.

Commercial use

Using a Large Model to Create an AI Application and Deploy a Real-Time Service

1

Submitting jobs, creating images, and image switching using ModelArts CLI for local development supported

ModelArts CLI, also called ma-cli, is a cross-platform command line tool used to connect to ModelArts and run management commands on ModelArts resources. ma-cli allows you to interact with cloud services through ModelArts notebook and on-premises VMs. You can run ma-cli commands for command autocomplete and authentication, as well as creating images, submitting ModelArts training jobs and DLI Spark jobs, and copying OBS data.

Commercial use

ModelArts CLI Command Reference

2

Notebook instance events added

Added events of a notebook instance's lifecycle.

Commercial use

Viewing Notebook Instance Events

3

Assigning fine-grained permissions for setting training job priorities released

You can assign a priority level to a training job that uses a new-version dedicated resource pool. The default priority levels are 1 or 2. If you have the permission to use the highest priority level, you can assign 1, 2, or 3 to your job.

Commercial use

Priority of a Training Job

4

Added information about using Grafana to view AOM monitoring metrics

Grafana is a tool for data visualization that offers various monitoring views and templates. You can import some common monitoring view templates from ModelArts to Grafana. Then, you can use Grafana to see the ModelArts monitoring metrics that are sent to AOM.

Commercial use

Using Grafana to View AOM Monitoring Metrics

5

Accessing a real-time service through WebSocket supported

For a real-time service that uses WebSocket, the client and the server can exchange data in both directions through a persistent connection if they complete the initial handshake successfully.

Commercial use

Accessing a Real-Time Service Through WebSocket

6

Optimized sub-section titles of "Creating an AI Application" under "Managing AI Applications

The sub-section titles of "Creating an AI Application" show the meta model sources on the page for creating an AI application on the management console.

Commercial use

Creating an AI Application

1

New-version dedicated resource pools released

Compared with old-version dedicated resource pools, new-version dedicated resource pools have made improvements in both product and functions. For example, the original dedicated resource pools for development/training and service deployment have been unified, the network of a dedicated resource pool can be customized to access, the cluster information is more complete, the accelerator card driver of a cluster can be customized to manage, and resource allocation is fine-grained.

Commercial use

Resource Pool Management

1

TMS

ModelArts can work with Tag Management Service (TMS). When creating resource-consuming tasks in ModelArts, for example, training jobs, configure tags for these tasks so that ModelArts can use tags to manage resources by group.

Commercial use

Creating a Notebook Instance

Deploying as a Real-Time Service

2

Viewing notebook instance events released

When a notebook instance starts or is running, you can view its events on the notebook instance details page. Through events, you can obtain details about the running or abnormal status of an instance.

Commercial use

Viewing Notebook Instance Events

3

Data labeling platform discontinued

The data labeling platform has been discontinued. The labeling console of the original team labeling has been integrated into the ModelArts console.

Discontinued

Logging In to the Data Labeling Platform

4

Event viewing optimized in new-version job training

The event display has been optimized so that you can clearly obtain the current training phase and the duration.

Commercial use

Viewing Training Job Events

1

File transferring supported

Through file transferring, local files and folders can be uploaded to OBS, and the files and folders in OBS can be downloaded to a local path.

Open beta testing

Transferring Files (Recommended)

2

Fine-grained permissions for new-version DevEnviron and training jobs

Permission policies and supported actions are added for fine-grained permissions.

Open beta testing

DevEnviron Permissions (Recommended)

Training Job Permissions (Recommended)

1

Viewing the notebook instances of all IAM users under one tenant account

Any IAM user granted with the listAllNotebooks and listUsers permissions can click View all on the notebook page to view the instances of all IAM users in the current IAM project.

Open beta testing

Viewing All Notebook Instances of an IAM Project

2

Detecting training job suspension

A training job may be suspended due to unknown reasons. If the suspension cannot be detected promptly, resources cannot be released, leading to a waste. To minimize resource cost and improve user experience, ModelArts provides suspension detection for training jobs. With this function, suspension can be automatically detected and displayed on the log details page. You can also enable notification so that you can be promptly notified of job suspension.

Open beta testing

Detecting Training Job Suspension

3

Examples for calling real-time service APIs in multiple development languages

ModelArts supports the calling of a real-time service for prediction through APIs. To provide better guidance for you, Python and Java request examples in multiple authentication modes have been provided.

Open beta testing

Accessing Real-Time Services

1

Custom images for creating algorithms and training jobs

In new-version training jobs, custom images can be used to create and run training jobs.

Open beta testing

Creating a Custom Image for Model Training

2

Viewing events of training jobs

During the running of a new-version training job, you can view its events for accurate fault locating.

Open beta testing

Viewing Training Job Events

3

Automatic recovery for training jobs

If a new-version training job failed due to a hardware fault, fault tolerance check and automatic recovery are now available.

Open beta testing

Training Job Fault Tolerance Check

1

Optimized agency authorization

Agency authorization has been optimized for refined permissions management. This allows you to custom authorization for exchanging data between ModelArts and other services.

Open beta testing

Configuring Access Authorization

1

Model Management renamed AI Application Management

To better distinguish between models and AI applications, Model Management has been renamed AI Application Management. A model refers to a model file. An AI application refers to an image that can be deployed, a packed model, or an application obtained through ExeML.

Open beta testing

Introduction to AI Application Management

1

Access key authorization discontinued

To better distinguish between models and AI applications, Model Management has been renamed AI Application Management. A model refers to a model file. An AI application refers to an image that can be deployed, a packed model, or an application obtained through ExeML.

Open beta testing

Configuring Access Authorization

1

Datasets of the new version released

In the new version, creating a dataset is decoupled from creating a labeling job, which is more user-friendly.

Open beta testing

Creating a Dataset

Creating a Labeling Job

2

Training logs of the new version released

On the page for viewing training logs of the new version, common errors leading to a training job failure can be automatically identified, and solutions are provided accordingly.

Open beta testing

Training Log Failure Analysis

1

Hyperparameter search available in training management of the new version

ModelArts hyperparameter search automatically tunes hyperparameters, which surpasses manual tuning in both speed and precision.

Commercial use

Automatic Model Tuning (AutoSearch)

2

Training management of the new version released

Both training jobs and algorithm management of the new version are coupled for better training experience. Training management of the old version is retained.

Open beta testing

Model Training

1

Basic tutorials for assigning permissions for using ModelArts

If you already have a Huawei Cloud account and need to assign the account permissions to multiple users so that they can access ModelArts, you can assign permissions by referring to this case.

Commercial use

Assigning Basic Permissions for Using ModelArts

2

Authorization management APIs

You can call APIs for authorization management.

Commercial use

Authorization Management

1

Development environment update for notebook instances (The old-version notebook has been discontinued. Use the new-version notebook.)

Each development environment supports multiple AI engines. You can use all supported AI engines in the same notebook instance. This version updates the display of the development environment when a notebook instance is created.

Commercial use

Creating a Notebook Instance

1

Data management: table datasets

Newly created table datasets support a wide range of data sources, such as Object Storage Service (OBS), Data Warehouse Service (DWS), Data Lake Insight (DLI), and MapReduce Service (MRS). In this version, DWS table data can be used as the data source to meet the requirements of different application scenarios.

Open beta testing

Table Datasets

1

GitHub code library for notebook (This function of the old-version notebook has been discontinued. For details about how to download the Git code library for the new version, see Using the Git Plug-in.)

When creating notebook instances, you can download the public and private repositories of GitHub and perform operations on the graphical user interface (GUI) using the Git plug-in of JupyterLab.

Commercial use

GitHub Code Library

2

Data management: video dataset management

Video datasets are supported. You can label a video in ModelArts to identify the location and class of each object in the video.

Open beta testing

Video Datasets