Scenario-based Deployment

One-Click Deployment: Rapid Launch with Zero Learning Curve

ModelArts features a rich preset of model assets and provides a one-click deployment capability for them. For users looking to quickly experience the platform's inference deployment capabilities, this mode significantly streamlines the process. You can obtain a fully deployed inference service within minutes without worrying about underlying environment configurations, dependency installations, or service orchestration.

The model assets supporting one-click deployment cover popular third-party models such as the DeepSeek series, GLM series, and Qwen3 series, offering OpenAI-compatible APIs and end-to-end model services.

Figure 1 Quick Start Guide for One-Click Model Deployment

For quick-start tutorials related to one-click deployment, see

Getting Started with Inference: Deploying Qwen3-32B-64k with One Click

For more information about model assets and calls, see Quickly Calling Preset Models.

Custom Deployment of Built-in Models: Flexible Customization, Precise Adaptation

In addition to one-click quick deployment, ModelArts supports deep custom deployment for built-in model assets. You can inject personalized configurations while retaining the benefits of platform automation, achieving seamless integration between the model and service logic.

Customization capabilities include:

• Model parameter tuning: Supports modifying inference engine parameters (such as batch size, max tokens, temperature, etc.) to balance latency and throughput.
• Environment variable injection: Configures service keys, log levels, monitoring endpoints, etc., via custom environment variables.
• Startup script customization: Allows you to inject custom boot commands or preprocessing scripts to perform data preparation prior to model loading or post-processing logic.

For specific deployment guidance, see Deploying a Real-Time Inference Service Using a Single Node. When deploying, select platform assets as the model source. This deployment method is only supported in dedicated resource pools.

Figure 2 Custom deployment based on platform assets

Pulling Models from the vllm-ascend Open-Source Community: Ecosystem Integration, Cutting-Edge Availability

ModelArts actively embraces the open-source ecosystem, supporting close collaboration with the Ascend community. Users can directly pull adapted and optimized LLMs from the vllm-ascend open-source community, enabling rapid deployment of cutting-edge models on Huawei Cloud Ascend compute.

Key features

• Direct community model access: Supports pulling model weights and configurations released by the vllm-ascend community.
• Native Ascend adaptation: Pulled models are pre-optimized for Ascend NPUs through operator optimization and memory management tuning, ensuring efficient execution on Ascend hardware.
• Synchronized version updates: Keeps pace with community iterations, allowing you to conveniently access the latest model versions and performance patches.

Applicable scenarios

• Enterprises seeking to use the latest open-source LLMs (e.g., Llama, Qwen, ChatGLM, etc.).
• Teams needing to leverage Ascend compute for LLM inference but lacking low-level adaptation experience.
• Researchers participating in co-building the open-source ecosystem who want to quickly validate the performance of community models.

Ascend-vLLM Framework Deployment: Ultimate Performance, High-Scale Concurrency

To meet the demanding requirements of LLMs for high concurrency and low latency, ModelArts provides a specialized deployment solution based on the Ascend-vLLM framework. This solution deeply integrates the hardware features of the Ascend NPU with the efficient vLLM inference engine to achieve extreme optimization of inference performance.

Technology highlights

• PagedAttention optimization: Implements efficient attention memory management on the Ascend NPU, significantly reducing memory fragmentation and improving KV cache utilization.
• Continuous batching: Dynamically schedules requests to maximize NPU compute unit utilization and increase throughput.
• Multi-PU parallel inference: Supports strategies such as tensor parallelism and pipeline parallelism, making it easy to deploy 100-billion-parameter LLMs.
• Low latency assurance: Specially optimized for TTFT and end-to-end latency to meet the requirements of real-time interactive scenarios.

Application scenarios

• High-concurrency real-time services (e.g., intelligent customer service, real-time translation, code assistants).
• Inference deployment of 100-billion-parameter LLMs.
• Production environments with extreme demands for inference latency and throughput.

Custom Model and Image Deployment: Complete Control, Private Adaptation

For enterprise users with in-house developed models, special service logic, or strict compliance requirements, ModelArts supports fully custom model and container image deployment. You can independently prepare model files (e.g., .ckpt, .safetensors, .onnx, etc.) and Docker images containing all dependency environments, achieving complete control from code to environment.

Core capabilities

Support for any model format: Not restricted to platform built-in models; supports uploading model files in any format, including models that have undergone private fine-tuning, quantization, or pruning.

Custom Docker images: You can build Docker images containing specific Python versions, C++ dependencies, and custom inference service code (such as Flask/FastAPI wrappers), and push them to SWR (software repository).

Full lifecycle management: Supports custom health checks, startup probes, readiness probes, as well as custom log collection and monitoring metric reporting.

Security and compliance: Complete isolation between model data and runtime environments, meeting industry standards with strict requirements for data sovereignty and code auditing, such as finance and government services.

Application scenarios

In-house developed model deployment: In-house developed non-open-source models or models subjected to deep private training.

Complex service logic integration: Scenarios where inference services need to embed complex service validation, data masking, or multi-model pipeline logic.

Legacy system migration: Smoothly migrating existing inference services from local IDCs or private clouds to the cloud, maintaining code and environment consistency.

Special dependency environments: Scenarios requiring non-standard libraries, specific driver versions, or proprietary software licenses.

For guidance on this deployment solution, see Performing Custom Service Deployment in this document.

Summary: Choosing the Deployment Mode That Fits Your Needs

ModelArts inference services use layered, scenario-driven deployments to make models easier to use and efficiently utilized, speeding up the move of AI applications from labs to real-world settings.