Help Center/ Cognitive Engagement Center/ User Guide/ Tenant Administrator Guide/ Robot Management and Configuration Guide/ Large Language Model Service Zone/ LLM-based Application Services/ Preparation/ Configuring a Model
Updated on 2025-01-23 GMT+08:00
View PDF
Share

Configuring a Model

The LLM Engine provides multiple common models for users to select from based on the actual service and budget.

Prerequisites

You have contacted the system administrator to enable the LLM Engine feature for the tenant.

Procedure

  1. Sign in to the AICC as a tenant space administrator, choose Configuration Center > Chatbot Management > Large Model Service > System Management > Model Management, and bind a model.
  2. On the Large Model tab, click . The Adding a Model page is displayed.

    Figure 1 Adding a model

    Parameter

    Description

    Name

    This parameter is user-defined. It is recommended that this parameter be related to the model type for future use.

    Type

    Select a model based on the actual service.

    Version

    Large language model version which is set as required.

    Using azure_openai as an example

    api_base

    URL for requesting a large model.

    api_key
    • Function: api_key is a key used for identity authentication. It needs to be provided each time a request is sent to the OpenAI API. This is the key to ensuring that only authorized users can access the API service.
    • Obtaining method: To obtain the value of api_key, you need to register with the OpenAI official website and create an account. After you create an account and log in to the system, you can find your API key on the OpenAI user console.
    • Security: api_key is sensitive information and must be kept properly. Do not expose your API key in public code libraries or client codes to prevent unnecessary fees and security risks.

    api_version
    • Function: api_version allows users to specify the version to be used through an API request. This is especially important when the API is updated, because a new version may introduce new functions or change existing functions. If you want to use an API of a specific version or ensure backward compatibility, set api_version.
    • Obtaining method: The API version information can be found in the official documents or release notes of OpenAI. The API version may be displayed in the URL of the API or the specific version information may be provided in the API document.
    • Setting method: In some API client libraries or invocations, you may need to specify the API version in the request configuration. For example, you may set the version number in the request URL or request header, or set api_version in the configuration of some libraries.

    deployment_id
    • Function: deployment_id is used to specify a specific model deployment instance. This parameter is used to select a specific deployment for API invocation when multiple models or versions are deployed.
    • Model management: During model management and O&M, deployment_id allows users to control and reference different model deployments. This is very important for managing multiple model versions or performing A/B testing of models in the PR environment.

  3. Click Save. The large model is created successfully.
Parent topic: Preparation