Implementing Tool Calling

Implementing Tool Calling

Modern agent systems rely heavily on tool usage.

Examples of tools include:

• web search APIs
• database queries
• code execution
• file system operations
• external service APIs

Example workflow:

User Question
      ↓
Agent reasoning
      ↓
Tool call
      ↓
Observation
      ↓
Final answer

Without tools, language models can only rely on their training data.

Tool calling allows agents to interact with the external world.

Tool Calling Architecture

A typical tool calling system has three components:

Conceptually:

Agent Reasoning
      ↓
Tool Selection
      ↓
Tool Dispatcher
      ↓
Tool Execution
      ↓
Observation

The observation is then fed back into the agent loop.

Schema-Based Tool Execution

Tools should be described using structured schemas.

Schemas define:

• tool name
• description
• required inputs
• output format

Example schema:

{
  "name": "web_search",
  "description": "Search the web for information",
  "parameters": {
    "query": "string"
  }
}

This schema allows the language model to understand how to call the tool.

Why Schemas Are Important

Schemas provide several advantages.

Structured Input

The model knows exactly which arguments to provide.

Validation

Inputs can be validated before execution.

Tool Discovery

Agents can examine available tools dynamically.

Example Tool Schema

Example tool registry:

[
  {
    "name": "web_search",
    "parameters": {
      "query": "string"
    }
  },
  {
    "name": "calculator",
    "parameters": {
      "expression": "string"
    }
  }
]

This registry describes the tools available to the agent.

Parsing Tool Calls

The language model typically generates structured tool calls.

Example model output:

{
  "tool": "web_search",
  "args": {
    "query": "H100 GPU architecture"
  }
}

The runtime must parse this output and dispatch the tool.

Tool Dispatching

Tool dispatching determines which function to execute.

Conceptually:

Tool Call
   ↓
Dispatcher
   ↓
Correct Tool Function

Example mapping:

The dispatcher routes the call to the correct implementation.

Example Python Tool System

tools = {
    "web_search": web_search,
    "calculator": calculator
}

def dispatch_tool(call):

    tool_name = call["tool"]
    args = call["args"]

    if tool_name not in tools:
        raise ValueError("Unknown tool")

    return tools[tool_name](**args)

This dispatcher selects the correct tool.

Example Rust Tool System

type ToolFn = fn(HashMap<String, String>) -> String;

fn dispatch_tool(name: &str, args: HashMap<String, String>) -> String {

    match name {

        "web_search" => web_search(args),
        "calculator" => calculator(args),

        _ => panic!("Unknown tool")
    }
}

Rust’s pattern matching makes tool dispatch straightforward.

Integrating Tool Calls Into the Agent Loop

The runtime must integrate tool calls into the agent loop.

Example flow:

Reasoning step
      ↓
Model outputs tool call
      ↓
Runtime dispatches tool
      ↓
Tool returns observation
      ↓
State updated

The observation becomes part of the agent’s context.

Updated Agent Loop

After adding tools, the loop looks like this:

Goal
 ↓
Reason
 ↓
Select Tool
 ↓
Execute Tool
 ↓
Observation
 ↓
Update State
 ↓
Repeat

This cycle allows the agent to gather information and solve complex tasks.

Example Integrated Loop

thought = llm.generate(prompt)

action = parse_action(thought)

if action["type"] == "tool":

    observation = dispatch_tool(action)

    state["observations"].append(observation)

let thought = llm.generate(&prompt);

let action = parse_action(&thought);

if action.kind == ActionKind::Tool {

    let observation = dispatch_tool(&action.name, action.args);

    state.observations.push(observation);
}

The observation is then fed into the next reasoning step.

Handling Tool Errors

Tools may fail for various reasons:

• network issues
• invalid inputs
• API failures

The runtime should handle errors gracefully.

Example strategy:

Tool failure
      ↓
Retry
      ↓
Fallback tool
      ↓
Return error observation

The agent can then decide how to proceed.

Tool Observations

Every tool call produces an observation.

Example observation:

Search results:
AMD Instinct MI250X powers the Frontier supercomputer.

Observations become part of the agent state.

Example state update:

Observation → added to context

This allows the agent to reason about new information.

Tool Calling in Modern AI Systems

Most modern agent frameworks support schema-based tool calling.

Examples include:

Although the APIs differ, the underlying architecture is similar.

The Runtime So Far

After this article, our minimal runtime now includes:

Agent Runtime
 ├─ State Machine
 ├─ Agent Loop
 ├─ Tool Registry
 ├─ Tool Dispatcher
 └─ Observation Handling

This system already resembles the core architecture used by many real agent frameworks.

Looking Ahead

In the next article we will add time-travel debugging, a technique that allows developers to replay and analyze agent execution.

This feature is extremely useful when debugging complex agent workflows.

→ Continue to 11.4 — Time-Travel Debugging