Tool Calling

Tool calling (aka function calling) allows the AI model to use tools in your application to perform actions or retrieve context to answer a user's question.

How tool calling works

Here's an example play-by-play of how tool calling works:

User Input: The user asks a question or makes a request that requires additional information or action.
Model Response: The AI model processes the input and determines that it needs to call a tool to fulfill the request.
Tool Call: The model generates a tool call, which includes the name of the tool and any necessary parameters.
Tool Execution The FreeToken client will execute a closure with the tool call information including the name of the tool and parameters passed from the AI.
Tool Response: Your application will return a response from the tool call in plain text format that will be passed back to the AI model as a tool message.
Response: The AI model processes the tool response and generates a response, sometimes calling more tools if needed, until it can provide a complete answer to the user's request.

Built-in Tool Calls

FreeToken provides a few built-in tool calls that will be automatically processed by the client. These tool calls will be transparent to your host application, meaning you don't need to implement any additional logic to handle them. The client will automatically execute these tool calls and return the results as tool messages.

web_search: If you have enabled the web search tool in the Agent, the AI model will automatically call it when it needs to search the web for information. The client will handle the web search and return the results as a tool message.
article_lookup: If you have RAG enabled for the Agent, the AI model will automatically call this tool to look up Documents in the knowledge base. The client will handle the article lookup and return the results as a tool message.

Custom Tool Calls

You can define your own tool calls in your application. To do this, first you'll need to define your JSON Schema for the tool call. This schema will define the parameters that the tool call accepts and their types.

Example JSON Schema for a tool call:

{
    "type": "function",
    "name": "get_weather",
    "description": "Retrieves current weather for the given location.",
    "parameters": {
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "City and country e.g. San Francisco, California, USA"
            },
            "units": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"],
                "description": "Units the temperature will be returned in."
            }
        },
        "required": ["location", "units"],
        "additionalProperties": false
    }
}

The AI model reads this schema and will generate tool calls that match the schema when it needs to call the tool.

Once you have defined your tool call schema, you can register it with the FreeToken client. The client will then handle the execution of the tool call when the AI model generates it.

// ... Configuration and device session registration completed ...

let getWeatherToolCall = """
{
  "type": "function",
  "function": {
    "name": "get_current_weather",
    "description": "Get the current weather for a location",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {
          "type": "string",
          "description": "The location to get the weather for, e.g. San Francisco, CA"
        },
        "format": {
          "type": "string",
          "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'",
          "enum": [
            "celsius",
            "fahrenheit"
          ]
        }
      },
      "required": [
        "location",
        "format"
      ]
    }
  }
}
"""

await client.addToolDefinition(
  name: "get_current_weather",
  definitionJSON: getWeatherToolCall
)

When the AI model runs next, it will be provided all the tool calls that have been registered with the client. If the model generates a tool call that matches one of the registered tool calls, the client call call the toolCallback closure in the runMessageThread.


await FreeToken.shared.runMessageThread(
  id: "msg-thr-id",
  success: { resultMessage in
    // Successfully ran the message thread on the local device
    print("Message thread result: \(resultMessage.content)")
}, error: { error in
    // Something went wrong while running the message thread
}, toolCallback: { toolCalls in
  // Iterate through the tool calls and concatenate the responses
  return toolCalls.map { toolCall in
    // Handle each tool call based on its name
    let response: String
    if toolCall.name == "get_current_weather" {
      // Execute the tool call with the parameters provided by the AI model
        let location = toolCall.arguments["location"] ?? "Unknown location"
        let format = toolCall.arguments["format"] ?? "celsius"
      
      // Call your weather API or perform the action to get the weather
      response = "The current weather in \(location) is 20 degrees \(format)."
    } else {
      // Handle other tool calls or return an error if the tool call is not recognized
      response = "" 
    }
    
    return response
  }.joined(separator: "\n\n")
})

Supressing Tool Call Access at the Message Thread Level

You may want to define all tool calls, but not allow them in specific message threads. For example, you may certain threads that are for specific tasks that do not require tool calls, or you may want to prevent the AI model from using tools in certain contexts.

To do this you specify toolAccess when creating the message thread. Tool access will default to .allowAll which allows the AI model to use all registered tool calls. You can set it to .denyAll to prevent any tool calls from being used in that thread. It will process tool access in order of the Array. So [.denyAll, .allow("get_current_weather")] will remove all tool calls except for get_current_weather.

This will also work with built-in tool calls like web_search and article_lookup.

import FreeToken
// ... Configuration and device session registration completed ...

await client.createMessageThread(
  toolAccess: [.denyAll, .allow("get_current_weather")],
  success: { thread in
    // Successfully created the message thread with restricted tool access
    print("Message thread created with ID: \(thread.id)")
}, error: { error in
    // Something went wrong while creating the message thread
    print("Error creating message thread: \(error.localizedDescription)")
})

Suppressing Tool Call Access when running a Message Thread

You can also suppress tool calls when running a message thread by specifying toolAccess in the runMessageThread method. This will further override the tool access defined in the message thread itself. This will not prevent the AI model from generating the suppressed tool calls as a response, bit will prevent them from being sent to the toolCallback closure.

Note: You cannot add additional tool calls to a message thread that has already been created. You can only restrict access to existing tool calls. You cannot add new tool calls because they are defined in the system message which is resident in the memory of the AI model at the time you run the message thread.


await client.runMessageThread(
  id: "msg-thr-id",
  toolAccess: [.denyAll],
  success: { resultMessage in
    // Successfully ran the message thread with restricted tool access
    print("Message thread result: \(resultMessage.content)")
}, error: { error in
    // Something went wrong while running the message thread
    print("Error running message thread: \(error.localizedDescription)")
}, toolCallback: { toolCalls in
    // This will not include any tool calls that were denied access
    return ""
})