Tool calls

Let the model call your functions: declare tools with JSON Schema, run the calls it returns, and send the results back.

Tool calling lets the model ask your code to run a function. You describe the tools, the model returns calls, and you send back the results.

Tool calls connect the model to your code: weather lookups, database queries, sending an email. The model does not run anything itself. It returns the name of a function and its arguments, your code runs the function, and you send the result back so the model can write the final answer. The format is the OpenAI tools format, so existing tool definitions work unchanged. Models that support tools are marked on Models & pricing.

The round trip#

  1. Send the conversation with a tools list.
  2. The model answers with finish_reason: "tool_calls" and one or more entries in message.tool_calls.
  3. Run each call. Append the assistant message, then one tool message per call with the matching tool_call_id.
  4. Send the conversation again. The model now answers in text, or asks for more calls.

Python

import json
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MODELLANE_API_KEY"],
    base_url="https://usemodellane.com/v1",
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name, e.g. Lisbon"},
                },
                "required": ["city"],
            },
        },
    }
]


def get_weather(city):
    # Replace with a real lookup.
    return {"city": city, "temperature_c": 21, "sky": "clear"}


messages = [{"role": "user", "content": "What's the weather in Lisbon and Oslo?"}]

response = client.chat.completions.create(
    model="lane-1", messages=messages, tools=tools
)
message = response.choices[0].message

while message.tool_calls:
    messages.append(message)
    for call in message.tool_calls:
        args = json.loads(call.function.arguments)
        result = get_weather(**args)
        messages.append(
            {"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)}
        )
    response = client.chat.completions.create(
        model="lane-1", messages=messages, tools=tools
    )
    message = response.choices[0].message

print(message.content)

Node.js

import OpenAI from "openai"

const client = new OpenAI({
  apiKey: process.env.MODELLANE_API_KEY,
  baseURL: "https://usemodellane.com/v1",
})

const tools = [
  {
    type: "function",
    function: {
      name: "get_weather",
      description: "Get the current weather for a city.",
      parameters: {
        type: "object",
        properties: {
          city: { type: "string", description: "City name, e.g. Lisbon" },
        },
        required: ["city"],
      },
    },
  },
]

// Replace with a real lookup.
const getWeather = ({ city }) => ({ city, temperature_c: 21, sky: "clear" })

const messages = [{ role: "user", content: "What's the weather in Lisbon and Oslo?" }]

let response = await client.chat.completions.create({ model: "lane-1", messages, tools })
let message = response.choices[0].message

while (message.tool_calls?.length) {
  messages.push(message)
  for (const call of message.tool_calls) {
    const result = getWeather(JSON.parse(call.function.arguments))
    messages.push({ role: "tool", tool_call_id: call.id, content: JSON.stringify(result) })
  }
  response = await client.chat.completions.create({ model: "lane-1", messages, tools })
  message = response.choices[0].message
}

console.log(message.content)

The first response looks like this:

JSON

{
  "role": "assistant",
  "content": null,
  "tool_calls": [
    {"id": "call_1", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"Lisbon\"}"}},
    {"id": "call_2", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"Oslo\"}"}}
  ]
}

Parallel calls#

A single response can hold several calls, as in the example above. They are independent: run them in any order, or at the same time, and send one tool message for each before the next request. Every call needs its own result; a missing tool_call_id makes the next request fail.

Choosing when tools are used#

tool_choice controls whether the model must, may or must not call a tool:

ParamValue
autoDefault when tools are sent. The model decides
noneThe model answers in text and calls no tool
requiredThe model must call at least one tool
{"type": "function", "function": {"name": "get_weather"}}The model must call this function

Streaming tool calls#

With stream: true, calls arrive in pieces under delta.tool_calls. The first piece of each call has its index, id and function name; later pieces with the same index add text to function.arguments. Concatenate the pieces per index and parse the arguments when the chunk with finish_reason: "tool_calls" arrives. The SDK helpers that collect a streamed message do this for you.

Tips#

  • Describe tools well. The model chooses tools from their name, description and parameter descriptions. Say when to use a tool, not only what it does.
  • Validate arguments. arguments is a JSON string written by the model. Parse it, check it against your schema and handle bad values before you run anything.
  • Keep the list short. Send only the tools that fit the current task. Every tool definition is part of the prompt and is billed as input.
  • Return compact results. Tool results are prompt tokens in the next request. Send the fields the model needs, not whole API responses.
  • Guard side effects. For actions that spend money or change data, ask the user to confirm before your code runs the call.