Function Calling

Overview

Tool calling (also known as function calling or FC) links foundation models with external tools and APIs. It acts as a bridge between natural language and information systems by converting user requests into precise tool or API commands, effectively addressing their needs.

Working principle: Developers describe the functions and definitions of the tool to the model through natural language. The model determines whether to call the tool during the dialog. When the model needs to be called, the model returns the tool functions and input parameters that meet the requirements. The developer calls the tool and fills the result back to the model. The model summarizes or continues to plan subtasks based on the result.

Supported Models

For models that currently support function calling, see the Capability row in Text Generation.

Specifications

Tool Choice Modes

Tool choice supports two modes: auto for automatic identification and named for specific function calls. It does not support the required mode for mandatory function calls.

"tool_choice": "auto"

"tool_choice": {"type": "function", "function": {"name": "get_weather"}

"tool_choice": "required"

Supported Processing Modes

Feature Compatibility

The table below lists the compatibility between function calling and the existing key features.

Basic Usage Process

1. Define tools.

   FC provides available tools to the model via the tools field in JSON format. The tools field includes information such as tool name, description, and parameter definitions.

   Define the Tool Function

   def get_weather(location: str, unit: str):
       return f"Getting the weather for {location} in {unit}..."

   • Assume a tool function named get_weather is defined in the code to retrieve weather information for a specified location.
     • location: The location for the weather query. This parameter is mandatory.
     • unit: The temperature unit for the returned result. This parameter is optional.
   • Note: This example simulates a weather query scenario. In real applications, you should call an actual weather API.

   Define Tools

   "tools": [{
       "type": "function",
       "function": {
           "name": "get_weather",
           "description": "Get the current weather in a given location",
           "parameters": {
               "type": "object",
               "properties": {
                   "location": {"type": "string", "description": "City and state, e.g., 'San Francisco, CA'"},
                   "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
               },
               "required": ["location"]
           }
       }
   }]

   • The tools field is a list, where each element represents a callable tool function. In this example, a tool named get_weather is defined.
   • type: The type of the tool. This is a fixed value function, indicating a callable tool function.
   • function: The function object that defines the tool's details, including name, description, and parameters.
     • name: The function name, here it is get_weather.
     • description: A description of the function, explaining its purpose.
     • parameters: The required parameters for the function, defined as an object containing location and unit.
       • location: Location information for the weather query. Type: string.
       • unit: Temperature unit. Type: string (enum). Options: celsius and fahrenheit.
       • required: Specifies required parameters. In this case, location is required.
2. Send a request.

   Include questions and required tools in your request. The model will detect and provide the necessary tools and their parameters based on these queries.

   from openai import OpenAI
   import httpx
   
   import json
   client = OpenAI(base_url="https://api-ap-southeast-1.modelarts-maas.com/openai/v1", api_key="MAAS_API_KEY", http_client=httpx.Client(verify=False))  # Replace MAAS_API_KEY with the obtained API key.
   
   messages = [{"role": "user", "content": "What's the weather like in San Francisco?"}]
   tools = [
       {
           //For details, see the tools defined in Step 1.
       }
   ]
   
   response = client.chat.completions.create(
       
       model="deepseek-v3.2",
       messages=messages ,
       tools=tools,
       tool_choice="auto"
   )

3. Call external tools.

   Use the details about tools and parameters provided by the model to call the relevant external tool or API and retrieve its output.

   # Extract the tool orchestration parameters provided by the model.
   tool_call = response.choices[0].message.tool_calls[0].function
   # Tool names
   tool_name = tool_call.function.name
   # Execute the tools based on the tool names.
   if tool_name == "get_weather":
       # Extracted user parameters
       arguments = json.loads(tool_call.function.arguments)
       # Call the tools.
       tool_result = get_weather(**arguments)

   • Obtain the list of tools called by the model from tool_calls returned by the model.
   • Execute the tool based on the tool name. If the tool name is get_weather, extract user parameters and call the get_weather function to obtain the tool execution result.
4. Fill in execution results and generate final response.
   Return the tool execution result to the model as a message with role=tool. The model will then generate the final response based on this result.

   messages.append(completion.choices[0].message)
   messages.append({
       "role": "tool",
       "tool_call_id": tool_call.id,
       "content": tool_result
       "name": tool_name
   })
   # Call the model to generate the final response.
   final_response = client.chat.completions.create(
       model="deepseek-v3.2",
       messages=messages ,
       tools=tools,
       tool_choice="auto"
   )
   print(final_response.choices[0].message.content)

Complete Code Example

from openai import OpenAI
import httpx
import json
    api_key = "MAAS_API_KEY"  # Replace MAAS_API_KEY with the obtained API key.
client = OpenAI(base_url="https://api-ap-southeast-1.modelarts-maas.com/openai/v1", api_key=api_key, http_client=httpx.Client(verify=False))
def get_weather(location: str, unit: str):
    return f"Getting the weather for {location} in {unit}..."
tool_functions = {"get_weather": get_weather}
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City and state, e.g., 'San Francisco, CA'"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location", "unit"]
        }
    }
}]
response = client.chat.completions.create(
    model="deepseek-v3.2",
    messages=[{"role": "user", "content": "What's the weather like in San Francisco?"}],
    tools=tools,
    tool_choice="auto"
)
tool_call = response.choices[0].message.tool_calls[0].function
print(f"Function called: {tool_call.name}")
print(f"Arguments: {tool_call.arguments}")
print(f"Result: {get_weather(**json.loads(tool_call.arguments))}")

Example output:

Function called: get_weather
Arguments: {"location": "San Francisco, CA", "unit": "fahrenheit"}
Result: Getting the weather for San Francisco, CA in fahrenheit...

Prompt Best Practices

The performance of function calling largely depends on the reasoning capabilities of the underlying model. To achieve efficient tool invocation, you should define prompts with clear and concise task descriptions, paired with well-structured tool function definitions. When using FC, follow these principles:

1. Keep task definitions simple and direct. Avoid including irrelevant or excessive information that may distract the model.
2. Avoid using the model for tasks that can be handled by code, as model reasoning is inherently probabilistic.

The table below lists common mistakes and recommended fixes.

{
    "type": "function",
    "function": {
        "name": "func1",
        "description": "Utility function"
    }
}

{
    "type": "function",
    "function": {
        "name": "CreateTask",
        "description": "Creates a task for the user and returns the task ID when a new work item is needed."
    }
}

{
    "time": {
        "type": "object",
        "description": "City",
        "properties": {
            "city": {
                "description": "City"
            }
        }
    }
}

{
    "time": {
        "type": "string",
        "description": "City"
    }
}

{
    "time": {
        "type": "object",
        "description": "City",
        "properties": {
            "city": {
                "description": "Always pass Hangzhou"
            }
        }
    }
}

You are communicating with user Marvin. You need to first query the user ID, then use it to create a task...

You are communicating with user Marvin (ID=123). You can use the user ID to create a task...

You can use the ID to find the user and get the task ID.

Each user has a unique user ID; each task has a task ID. You can use the user ID to retrieve user information and obtain all associated task IDs.

Query the weather in Beijing.

Query the weather in Beijing on July 30, 2025.

Exception Handling

JSON Format Error Tolerance Mechanism

If the JSON format is slightly invalid, use the json-repair library for fault tolerance.

import json_repair
 
invalid_json = '{"location": "Beijing", "unit": "°C"}'
valid_json = json_repair.loads(invalid_json)

Tool Call Exception

If the model fails to call the tool function properly because of the prompt or function definition, improve both using best practices.

Model Return Error

FC relies on the model's capabilities. Because of model hallucination, its output might not match expectations, causing the tool to fail.

If the call failure rate is low in a certain scenario, you can set the retry mechanism to harden the reliability.

FAQs

• When FC is used, only one function is called. Why are multiple function calls returned?

  The model calculates how many times a function is called. This often happens when prompts or tool functions are not clear. To fix this, improve your prompts and tool definitions using recommended practices.

• When FC is used, why cannot tool_calls be parsed after max_tokens is configured?

  If the value of max_tokens is too small, the model output may be truncated and the tool_calls content cannot be properly parsed. Therefore, you need to set max_tokens to a proper value.

• How do I determine if a model needs to call a function?

  The model automatically evaluates whether a tool call is required based on the user's query, the tool definitions provided, and the tool_choice parameter. If the response includes a tool_calls field, a tool call is required.

• Why does the model sometimes output special tokens that prevent tool_calls from being parsed when an output format is specified in the system prompt?

  This usually occurs when the format defined in the system prompt conflicts with the model's required formatting specifications. You are advised to refer to best practices to optimize your prompts.