Using a Connector to Call a Foundation Model

Prerequisites

• You have obtained the API address and key needed to access the external foundation model.
• The external foundation model's request and response formats must adhere to the OpenAI standard; otherwise, errors might arise during streaming calls and response rendering.

Creating a Foundation Model Connector

This section uses the custom foundation model as an example to describe how to create a foundation model connector.

1. Log in to the application designer by referring to Logging In to the Application Designer.
2. In the navigation pane, choose Integrations.
3. Click Connector Instance under Connector.
4. In the navigation pane, choose Foundation Model > Custom Model.
5. In the right pane, click + and set the parameters.
   Figure 1 Setting foundation model interconnection parameters
   

   Table 1 Parameters for creating a custom foundation model connector

   Parameter	Description
   Label	Custom foundation model connector label, which can be modified after being created. Value: 1–64 characters.
   Name	Custom connector name, which uniquely identifies the connector in the system and cannot be changed after the connector is created. The naming requirements are as follows: The value cannot exceed 64 characters, including the prefix namespace. To prevent duplicate data names among different tenants in the platform, each tenant must define a unique namespace when first creating an application. A tenant can create only one namespace. After being created, the namespace cannot be modified. Start with a letter and can contain only letters, digits, and underscores (_). Do not end with an underscore (_).
   Model Name	Model ID of the specific foundation model. When the connector is called, this field is passed to the model field in the foundation model API.
   URL	URL for accessing the foundation model API, provided by the model provider.
   API Key	Key provided by the foundation model provider for authentication. If no authentication is required, you can omit this parameter.
   Context Length	Length of the context of the foundation model. When the connector is called, this field is passed to the num_ctx field in the foundation model API. If the foundation model API does not support this parameter or uses the default value, you can omit this parameter.
   Maximum Token Length	Maximum length of a foundation model reply. When the connector is called, this field is passed to the max_tokens field in the foundation model API. If the foundation model API does not support this parameter or uses the default value, you can omit this parameter.
   Description	Description of the foundation model connector. Value range: 0 to 85 characters.

6. Save the settings.
7. Test the connectivity of the foundation model connector.
   1. On the custom foundation model details page, click Test. On the displayed dialog box, click Test. The foundation model connector test page is displayed.
   2. In the Input Content text box, enter the dialog content (for example, "Hello"), and click Test. Check whether the returned message is correct.
      If the returned message contains the streaming response packet, the foundation model connector is configured successfully. Otherwise, check whether the foundation model connector information is correct.

      Figure 2 Foundation model connector test page
      

Using a Script to Call the Foundation Model Connector

After a connector is created, you can call the foundation model connector in a script to quickly integrate the connector into intelligent applications.

1. Create an empty script by referring to Creating a Blank Script.
2. In the script editor, enter the following code:

   import * as llm from 'llm';
   /*
    * The input parameter is defined by `@action.param()`.
    */
   export class Input {
       @action.param({ type: "String", required: true, description: "the chat message" })
       inputParam: string;
   }
   /*
    * The output parameter is defined by `@action.param()`.
    */
   export class Output {
   }
   
   /*
    * Defin the main service class.
    * 
    * - The service entry function is defined by `@action.method()`.
    * - All dependent objects should be defined via `@useObject([])` (for objects) or `@useBo([])` (for business 
   objects).
    */
   export class LLMConnectorScript {
       @action.method({ input: "Input", output: "Output", description: "invoke LLM connector" })
       run(input: Input): Output {
           let output = new Output();
           // Create a foundation model connector client. The first parameter is the name of the created foundation model connector, and the second parameter is the foundation model type.
           let client = llm.newClient("Namespace__selfHostDeepSeekR1", "custom_llm");
           let message = [{ "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": input.inputParam }];
           let resp = client.chat(message, {
               "temperature": 0.8,
               "top_p": 1.0,
               "presence_penalty": 0.0
           });
           return output;
       }
   }

   Namespace__selfHostDeepSeekR1 is the name of the connector created in Creating a Foundation Model Connector, and custom_llm is the type of the foundation model. Table 2 shows the mapping between foundation model types and foundation model API providers.

   Table 2 Mapping between foundation model types in a script and foundation model API providers

   Foundation Model Provider	Foundation Model Type
   DeepSeek	deepseek
   Qwen	qwen
   MaaS cloud platform	maas
   Pangu	pangu
   Hunyuan	hunyuan
   Doubao	doubao
   ERNIE Bot	qianfan
   chatGLM	glm
   Kimi	kimi
   Ollama	ollama
   Xinference	xinference
   Custom model	custom_llm

3. Click in the upper part of the script editor to save the script.
4. Click to execute the script.
5. On the input parameter tab page at the bottom of the page, enter request parameters and click in the upper right corner of the test window.

   {
       "inputParam": "Hello"
   }

   On the output parameter tab page, you can check the streaming response result of the foundation model.

   Hello! How can I help you?

6. Click in the upper part of the script editor to activate the script.

Using a Flow to Call the Foundation Model Connector

After a connector is created, you can call the foundation model connector in a flow to quickly integrate the connector into intelligent applications.

1. Create a blank flow by referring to Creating a Blank Flow.
2. Create the input and output parameters of the flow.
   The flow calls the foundation model API. Therefore, the flow's input and output parameters must be defined and match the API's parameter types.

   1. On the right of the flow designer, click .
   2. On the Context page, click + next to Variable to create a variable.
   3. Click next to the variable, change the variable name to inputMessages, set the data type to Any, set the description to "Dialog content", and click Save.
   4. Repeat the preceding operations to create the inputConfig variable in Table 3.

      Table 3 Flow variables

      Parameter	Type	Description
      inputMessages (created)	Any	Dialog content
      inputConfig	Any	Foundation model parameter configuration

   5. Select the start diagram element and click to set the input parameters of the flow.
      Figure 3 Setting the input parameters of the flow
      

3. On the left of the flow design page, choose Connector > Connector Instance > Foundation Model, and drag the custom foundation model diagram element to the canvas.

   The custom foundation model connector is a preset API for interconnecting with the custom foundation model connector.

4. Select the custom foundation model connector diagram element and select the connector Namespace__selfHostDeepSeekR1 created in Creating a Foundation Model Connector in the basic information.
   Figure 4 Associating the diagram element with the foundation model connector
   

5. Click and set the action and parameters.
   Figure 5 Configuring the action and parameters
   

6. Connect the start diagram element and the custom foundation model connector diagram element.
   Figure 6 Connecting diagram elements
   

7. Click to save the flow.
8. Click to activate the flow.
9. Set input parameters and click the run button.

   {
       "inputMessages": [
           {
               "role": "user",
               "content": "Who are you"
           }
       ],
       "inputConfig": {
           "temperature": 0.8,
           "top_p": 1.0,
           "presence_penalty": 0.0
       }
   }

10. Click in the upper part of the editor to activate the flow.