Getting Started with Inference: Deploying Qwen3-32B-64k with One Click

Scenario

Qwen3-32B-64k is a large language model (LLM) launched by Alibaba Qwen. Additionally, this model supports seamless switching between thinking and non-thinking modes. You can switch between the two as needed during a dialog. The model's inference capability significantly outperforms that of QwQ, and its general capability far exceeds that of Qwen2.5-32B-Instruct, achieving SOTA performance among models of the same scale in the industry. This case uses the Ascend-vLLM framework to deploy the Qwen3-32B-64K model with just a few clicks, illustrating the entire process of deploying and calling a ModelArts service.

To facilitate quick deployment of this model, the ModelArts console provides a one-click deployment case. With the extensive open-source models and inference frameworks provided by ModelArts, you can quickly start model deployment by simply filling in a form.

Billing

Qwen3-32B-64k can be deployed in a public resource pool or dedicated resource pool. The billing mode varies depending on the deployment mode. The billing modes for the two deployment modes are as follows:

• Inference service deployment in a public resource pool is billed based on duration. Costs are incurred when the status is Running or Alarm. Stop the service when not in use. For details, see Inference Deployment Billing Items.
• Inference service deployment in a dedicated resource pool. Fees for dedicated resource pools are paid upfront upon purchase. There are no additional charges for service deployment. For details, see Billing Item.

Prerequisites

You have registered a Huawei Cloud account, completed real-name authentication, and been granted related permissions. For details, see 1. Prerequisites: Accounts and Permissions.

Constraints

Inference timeout interval: maximum time to wait for the system to return the first token result after a request is sent. The default value is 300s. If no response is received within this period, the request is automatically terminated. In streaming transmission scenarios, the timeout interval is refreshed each time a response to a request is received. However, the end-to-end response timeout interval of the system is fixed at 3,600 seconds.

Service Deployment

1. On the ModelArts console, choose Getting Started, select Qwen3-32B-64k in the Popular Models area, and click Deploy. The Create Service page is displayed.
   

   The new console in the CN-Hong Kong region is recommended. One-click model deployment is supported only on the new ModelArts console.

   In the upper right corner of the overview page of the old console, click Go to New Version to switch to the new console.

2. On the Create Service page, set parameters according to Table 1 and click OK.

   Table 1 Parameters for deploying a real-time service in one-click mode

   Parameter	Description	Recommended Value
   Models	One-click deployment supports the deployment of preset models.	The default value is Qwen3-32B-64k.
   Name	Name used to identify and manage the real-time service. Enter a name as prompted. The value can contain 1 to 64 characters, including letters, digits, hyphens (-), and underscores (_).	service-test
   Resource Pool Type	One-click deployment of real-time services supports both public resource pools and dedicated resource pools. Public resource pool Public resource pool for deploying the real-time service. The public resource pool supplies shared compute clusters assigned according to job parameters. Each job operates with its own isolated resources. This option offers cost-effective and flexible solutions for tasks like development and testing. Choosing the public resource pool might leave fewer resources available because of its limits. If this happens, join the queue and wait your turn. Dedicated resource pool Dedicated resource pool for deploying the real-time service. The resources provided in a dedicated resource pool are exclusive and more controllable. Use dedicated resource pools for core production services to secure exclusive resources. To select a dedicated resource pool, create one in advance. For details, see Creating a Dedicated Resource Pool. To deploy a model using a dedicated resource pool, the following conditions must be met: The instance specifications of the resource pool must meet the requirements of the model for the PU type and number of PUs. You can click the model name on the Getting Started page on the console and view the inference feature on the model details page. Figure 1 Model inference feature To ensure network connectivity between processing units (PUs) on the same node and across different nodes during multi-node model deployment, nodes with identical specifications must meet the same instance specifications.	Public resource pool
   Instance Specifications	Select the hardware resource configuration for the real-time service instances.	Default value: 2 * Snt9b3 | 48 vCPUs | 384 GiB | ARM
   Auto Stop	Auto-stop timer. Default: 1 hour; maximum: 24 hours. When auto stop is enabled, the system tracks how long the service runs. It will shut down the service if the runtime goes beyond the set limit. After the real-time service is deployed, you can choose Model Inference > Real-Time Inference on the console. Choose More > Configure Auto Stop and reconfigure the auto stop time.	Select this option. Default: 1 hour.

3. If you select Public resource pool, confirm the billing impact in the dialog box and click OK.

   If you select Dedicated resource pool, click OK to start the deployment.

   After the deployment is complete, view the basic information about the service on the ModelArts console by choosing Model Inference > Real-Time Inference.

   Once the service status changes from Deploying to Running, the service is deployed.

Creating an API Key and Binding It to a Real-Time Service

By default, the service deployed in one-click mode uses API key authentication. You need to create your own API key and bind it to the preset service.

1. On the ModelArts console, choose Model Inference > Real-Time Inference. Click the API Key Authorization Management tab.
2. Click Create API Key. In the Create API Key dialog box, enter the API key information, set Authorization Scope to Specified real-time services, and click OK. The CSV file with the same name as the API key is automatically downloaded. Open the CSV file and search for the content corresponding to the api_key column. The content is the API key to be used. The CSV file that stores the API key cannot be downloaded again. Keep the CSV file or the API key secure to prevent loss.
3. Click Bind in the Operation column of the API key.
4. In the Bind Service dialog box, select the service created in Service Deployment and click OK.

An API key is a credential for calling the API of a deployed model. If the API key is obtained by non-service personnel, the non-service personnel can use the API key to call the API, which will cause economic loss to you.

To avoid the preceding risks, keep your API key secure.

Debugging a Service Online

1. On the ModelArts console, choose Model Inference > Real-Time Inference. Click the deployed service to go to the details page.
2. Click the Prediction tab and set the request parameters.
   • Chat API: Add /v1/chat/completions to the end of the existing URL.
   • Headers: Add a request header. Retain the default key Authorization and replace the default API key with the API key saved in Creating an API Key and Binding It to a Real-Time Service. Alternatively, delete the Authorization key pair. The system automatically switches to IAM token authentication without requiring any additional configuration.
   • Set Body to raw and model to qwen3_32b. The following is an example request:

     {
         "model": "qwen3_32b",
         "messages": [
           {"role": "system", "content": "You are a helpful assistant."},
           {"role": "user", "content": "Hello"}
         ]
     }

Calling a Service

Using the API URL and API Key

This guide uses a shared gateway by default. After deployment, the API URL and token information is available on the service details page.

• Public API URL: On the Model Inference > Real-Time Inference page of the ModelArts console, click your deployed service to enter the details page. Copy the Public API URL.
• Model API: The API of the Qwen3-32B model is /v1/chat/completions.
• API Key: Obtain the API key saved in Creating an API Key and Binding It to a Real-Time Service.

Using cURL or Python

The sample code is as follows.

The API URL for calling a real-time service is in the format of Public API URL + Model API.

Replace API_KEY in the sample code with the obtained API key.

Stopping or Deleting a Service

A real-time service created using a public resource pool is billed based on the service duration. If a dedicated resource pool is used for service deployment, dedicated resource pool resources are occupied. If you no longer need to use a service, stop or delete it to avoid unnecessary fees.

Figure 2 Stopping or Deleting a Service

Model Capabilities

"chat_template_kwargs": {
"enable_thinking": false
}

Note: √ indicates supported, × indicates not supported, and / indicates not involved.

FAQ

How Do I Change the Request Timeout of a Real-Time Service Deployed in One-Click Mode?

In the high availability settings of the real-time inference service information, you can modify the request timeout interval of the service. For details, see Modifying Real-Time Inference Service Information.

Helpful Links

For more information about inference deployment, see Deploying and Using Real-Time Inference.