Tool Calling Guide

TL;DR

• Tool calling is how LLMs take action - they decide to invoke external capabilities (APIs, databases, code execution, search) instead of generating text alone.

• Tools extend the model beyond its training data - real-time information, private data, computations, and side effects become available through tool invocation.

• The agent loop is the core pattern - the model reasons about which tool to use, your application executes it, and the result feeds back into the model's context for the next decision.

• Tool design is API design - clear descriptions, typed parameters, idempotent operations, and graceful error messages determine agent reliability.

• Production tool calling requires guardrails - permission checks, rate limits, argument validation, iteration caps, and audit logging are non-negotiable.

Why This Matters

An LLM without tools is a text generator. It cannot check your database, send an email, run code, fetch live data, or interact with any system. Tool calling transforms it from a chatbot into an agent - a system that perceives (via tools), reasons (via the LLM), and acts (via tool execution).

Every production AI agent - customer support bots that look up orders, coding assistants that run tests, research agents that search the web - is built on tool calling. The quality of your tools (their descriptions, parameter design, error handling) determines agent reliability more than model choice.

Tool calling is also the fastest-growing area of LLM engineering. New protocols (MCP), frameworks (LangGraph, CrewAI), and model capabilities (parallel tool calls, computer use, browser use) are expanding what agents can do. Understanding the fundamentals lets you evaluate and adopt these advances.

The Problem Tool Calling Solves

LLMs have three fundamental limitations that tools address:

1. No real-time data - Models have a knowledge cutoff. They cannot tell you today's stock price, current weather, or the status of your order. Tools connect them to live data sources.

2. No computation - LLMs are bad at precise math, data processing, and algorithmic tasks. A code execution tool offloads computation to an interpreter that gets exact answers.

3. No side effects - Models cannot send emails, create database records, or trigger workflows. Tool calling gives them the ability to act on the world.

Without tools, you work around these limitations with RAG (for data), prompt engineering (for format), and hope (for accuracy). With tools, the model dynamically selects the right capability for each step of a task.

How We Got Here

Tool calling moved from prompt hacks to first-class model APIs and open protocols:

Diagram: From free-form answers to tool use

flowchart LR
    A[Prompted JSON] --> B[Native function calling]
    B --> C[Parallel tool calls]
    C --> D[Agent tool loops]
    D --> E[MCP + sandboxes]

Major components and how control or data moves between them.

Public references: OpenAI function calling, Anthropic tool use, ReAct, and the Model Context Protocol.

What Is Tool Calling?

Tool calling is the pattern where an LLM decides to invoke an external function, API, or service to accomplish part of a task. The model acts as a router and planner; your application provides the capabilities and executes them.

A tool has three components:

Tool = Definition (schema) + Implementation (code) + Description (when to use)

• Definition - JSON Schema describing the tool's name, parameters, and types. See function calling for the technical format.

• Implementation - Your code that actually executes the operation (API call, database query, script).

• Description - Natural language explaining when and why the model should use this tool. This is the most important part for reliability.

See function calling for the wire format; this guide focuses on designing and operating the tools themselves.

How Tool Calling Works

The Agent Loop (ReAct Pattern)

The most common tool calling pattern follows ReAct (Reason + Act):

Diagram: ReAct tool-calling loop

flowchart TD
    M[User message + tools] --> L[LLM decides]
    L -->|final answer| Out[Respond]
    L -->|tool call| V[Validate args + auth]
    V --> X[Execute tool]
    X --> O[Observation → context]
    O --> L

Major components and how control or data moves between them.

Each iteration: the model reasons about what to do next, selects a tool, your code executes it, and the result returns to the model.

Tool Categories

Parallel vs Sequential Tool Calls

Diagram: Parallel vs sequential tool calls

flowchart LR
    subgraph parallel [Independent]
        T1[search]
        T2[get_weather]
    end
    subgraph sequential [Dependent]
        A[get_order] --> B[issue_refund]
    end

Major components and how control or data moves between them.

Parallel calls reduce latency when tools are independent. Sequential calls are required when later tools depend on earlier results.

Architecture

Production tool calling is a thin control plane around the model:

Diagram: Tool registry and execution path

flowchart TB
    UI[App / API] --> Orch[Orchestrator]
    Orch --> LLM[LLM + tool schemas]
    Orch --> Reg[Tool registry]
    Orch --> Guard[Auth / validation / rate limits]
    Reg --> Impl[Tool implementations]
    Impl --> Ext[APIs, DBs, sandboxes, MCP servers]
    Orch --> Log[Audit + traces]

Major components and how control or data moves between them.

Step-by-Step Flow

Building a tool-augmented agent from scratch:

1. Identify required capabilities - What actions must the agent perform? List every external interaction.

2. Design tool interfaces - One tool per capability. Define clear schemas with typed parameters and descriptions.

3. Implement tool functions - Write the actual code with error handling, timeouts, and idempotency where possible.

4. Register tools - Add to a tool registry accessible by the orchestrator.

5. Write the system prompt - Explain the agent's role, available tools, and decision-making guidelines.

6. Build the agent loop - Send message → check for tool calls → execute → return results → repeat.

7. Add safety layers - Validation, permissions, rate limits, iteration caps, audit logging.

8. Test with golden scenarios - Multi-step tasks covering tool selection, error recovery, and edge cases.

9. Deploy with monitoring - Log every tool call, track success rates, alert on anomalies.

Real Production Example

A data analysis agent with SQL query, Python execution, and chart generation tools:

import json
import sqlite3
from openai import OpenAI

client = OpenAI()

def create_data_agent(db_path: str):
    conn = sqlite3.connect(db_path)

tools = [
        {
            "type": "function",
            "function": {
                "name": "run_sql",
                "description": "Execute a read-only SQL query against the analytics database. Use for data retrieval, aggregation, and filtering.

Only SELECT statements allowed.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "query": {"type": "string", "description": "SQL SELECT query"},
                        "explanation": {"type": "string", "description": "Brief explanation of what this query retrieves"},
                    },
                    "required": ["query", "explanation"],
                },
            },
        },
        {
            "type": "function",
            "function": {
                "name": "run_python",
                "description": "Execute Python code for data analysis, calculations, or chart generation.

Has access to pandas, matplotlib, and numpy.

Use when SQL is insufficient.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "code": {"type": "string", "description": "Python code to execute"},
                        "explanation": {"type": "string", "description": "What this code computes and why"},
                    },
                    "required": ["code", "explanation"],
                },
            },
        },
    ]

def execute_tool(name: str, arguments: dict) -> str:
        if name == "run_sql":
            query = arguments["query"].strip()
            if not query.upper().startswith("SELECT"):
                return json.dumps({"error": "Only SELECT queries are allowed"})
            try:
                cursor = conn.execute(query)
                columns = [desc[0] for desc in cursor.description]
                rows = cursor.fetchmany(100)
                return json.dumps({"columns": columns, "rows": rows, "row_count": len(rows)})
            except Exception as e:
                return json.dumps({"error": f"SQL error: {str(e)}"})

elif name == "run_python":
            # Production: use a sandboxed executor (E2B, Docker, Modal)
            import io, contextlib
            output = io.StringIO()
            try:
                with contextlib.redirect_stdout(output):
                    exec(arguments["code"], {"__builtins__": {}}, {})
                return json.dumps({"output": output.getvalue()[:5000]})
            except Exception as e:
                return json.dumps({"error": f"Execution error: {str(e)}"})

        return json.dumps({"error": f"Unknown tool: {name}"})

    def analyze(question: str) -> str:
        messages = [
            {"role": "system", "content": "You are a data analyst. Use SQL for data retrieval and Python for complex analysis. Always explain your findings."},
            {"role": "user", "content": question},
        ]

        for _ in range(8):
            response = client.chat.completions.create(
                model="gpt-4o", messages=messages, tools=tools, temperature=0.1,
            )
            msg = response.choices[0].message
            messages.append(msg)

            if not msg.tool_calls:
                return msg.content

            for tc in msg.tool_calls:
                args = json.loads(tc.function.arguments)
                result = execute_tool(tc.function.name, args)
                messages.append({"role": "tool", "tool_call_id": tc.id, "content": result})

        return "Analysis could not be completed within the iteration limit."

    return analyze

Design Decisions

Comparisons

Tool calling vs function calling

Function calling is how the model emits a call; tool calling is what you expose and how you run it safely.

Tool calling vs MCP

Tool calling vs agents

A single structured tool call is not an agent. Agents use tool calling inside a bounded control loop - often with planning and guardrails.

Computer use vs browser use

Provider-native computer use and browser use toolsets are still tool calling: the model emits structured actions, your (or the vendor’s) runtime executes them, and results return into the loop.

As of 2026-08-19, Anthropic’s computer use toolset is generally available on the Claude API as computer_toolset_20260801 (no beta header; batch actions; zoom on by default). The companion browser_toolset_20260801 adds viewport-scoped control. Both are documented for Claude Fable 5, Mythos 5, Opus 5, Sonnet 5, and Opus 4.8. Treat GA as a schema and supportability milestone—still require HITL for destructive clicks, domain allowlists, and session isolation.

Prefer browser use when the task is web-only and you can host the browser. Prefer computer use when the agent must operate outside a browser. Prefer narrow typed tools (search, ticket APIs, code execution sandboxes) when you do not need pixel-level UI control.

Common Mistakes

1. Tools that are too broad - A single run_command tool that accepts arbitrary shell commands is a security disaster. Design narrow, specific tools with typed parameters.

2. Missing error context - Returning "error" without details prevents the model from self-correcting. Return structured errors: {"error": "Column 'revenue' not found. Available columns: id, name, amount, date"}.

3. No timeout on tool execution - A slow API call blocks the entire agent loop. Set timeouts (5–30s per tool) and return timeout errors to the model.

4. Allowing destructive operations without confirmation - Tools that delete, send, or modify should require explicit user confirmation or elevated permissions.

5. Not logging tool calls - Without audit logs, debugging agent failures is guesswork. Log every tool call with inputs, outputs, latency, and the conversation context that triggered it.

6. Overloading the system prompt with tool instructions - Tool descriptions belong in the tool schema, not the system prompt. The system prompt should describe the agent's role; tool descriptions describe capabilities.

Where It Breaks Down

• Tool selection errors - The model calls the wrong tool or hallucinates parameters. More tools = more confusion. Keep the tool set focused.

• Error cascades - One failed tool call leads to fabricated data in subsequent steps. Implement circuit breakers: after N consecutive failures, stop the loop and report to the user.

• Context bloat - Each tool call adds arguments + results to the conversation.

A 10-step agent loop can consume 20K+ tokens. Summarize old tool results.

• Non-idempotent actions - Retrying a failed send_email tool sends duplicate emails. Design action tools to be idempotent or include deduplication keys.

• Latency compounding - Each loop iteration adds 2–5 seconds. A 5-step task takes 10–25 seconds. Users need progress indicators and streaming status updates.

Decision tree: tools vs plain generation

Decision tree: When to expose tools

flowchart TD
    A[Does the answer need live data or side effects?] -->|No| B[Plain generation]
    A -->|Yes| C{Reads only?}
    C -->|Yes| D[Typed read tools + timeouts]
    C -->|No| E[Writes / sends / spends]
    E --> F[HITL + idempotency keys]
    D --> G[Narrow schemas; log every call]
    F --> G

Prefer retrieval-augmented answers over tools when you only need documents - tools are for actions and live systems.

When NOT to Use Tool Calling

Skip tools (or keep them read-only) when:

1. A single LLM answer is enough - classification, short copy, pure generation.
2. You can call the API yourself - if the path is fixed, a workflow beats an LLM selecting tools.
3. You cannot sandbox or authorize writes - never expose shell/DB write tools without policy.
4. Latency must be sub-second - each tool round-trip compounds.
5. The tool set is huge and uncurated - selection quality collapses past ~15–20 tools.

Prefer RAG for knowledge lookup without side effects, and workflows when the step graph is known (Workflows vs Agents).

Running in Production

Best Practice

✅ Best Practices - Narrow typed tools, structured errors, timeouts, permission checks, iteration caps, and golden scenarios for tool selection and recovery.

Warning

Code execution tools are the highest-risk tool category. Always run in an isolated sandbox with no network access, no filesystem access beyond a temp directory, and strict resource limits. Never execute model-generated code on your application server.

Related Guides

• Models: GPT-5 · Claude Sonnet · Claude Fable · Claude Opus · Gemini 3.1 Pro — tool-use capable foundation models.

• Companies: OpenAI · Anthropic — native tool/function calling APIs.

• LangChain / LangGraph - Agent frameworks with tool binding and execution loops - Best AI Agent Frameworks.

• CrewAI / AutoGen - Multi-agent tool orchestration - LangGraph vs CrewAI.

• E2B - Cloud sandboxes for safe code execution tools.

• Anthropic Tool Use / OpenAI function calling - Native model APIs.

• MCP - Model Context Protocol for standardized tool integration.

• Function Calling - The structured mechanism underlying tool calling.

• Model Context Protocol - Open standard for connecting LLMs to tools and data sources.

• AI Agents - Agents wrap tool calling in planning, memory, and multi-step orchestration.

• Agent Evaluation - Score tool selection, arguments, recovery, and trajectories.

• Agent Architectures - ReAct, Plan-and-Execute, and related loop patterns.

• Guardrails - Tool rails before execution.

• Human-in-the-Loop - Approval gates for irreversible tools.

• Prompt Engineering - Tool descriptions and agent system prompts.

• AI Security - Prompt injection can trigger unauthorized tool execution.

If you understood this topic, read next:

Diagram: Learning path for tool calling

flowchart LR
    A[Functions] --> B[Tools]
    B --> C[Agents]
    C --> D[Guardrails]
    D --> E[HITL]
    E --> F[Model Context Protocol]

Prerequisites: Function Calling · Prompt Engineering

Next topics: Model Context Protocol · AI Agents · Guardrails

Estimated time: 50 min · Difficulty: Intermediate

Key Takeaways

• Tool calling transforms LLMs from text generators into systems that can query, compute, and act.
• Tool design is API design - clear descriptions and typed parameters determine reliability.
• The agent loop (reason → act → observe → repeat) is the core pattern for multi-step tool use.
• Always validate, authorize, timeout, and log every tool execution.
• Keep tool sets focused (3–15), return structured errors, and cap iteration count.
• Function calling is the mechanism; MCP is a transport/discovery protocol; agents add the control loop.
• Gate irreversible tools with HITL and guardrails.

FAQs

What is the difference between tool calling and function calling?

Function calling is the technical API mechanism (structured JSON output). Tool calling is the broader engineering pattern - designing, implementing, and orchestrating external capabilities that LLMs invoke. Function calling is how; tool calling is what and why.

How many tools should an agent have?

Start with 3–5 focused tools. Add more only when the model consistently fails to accomplish tasks due to missing capabilities. Beyond 15–20 tools, selection accuracy degrades noticeably.

Can tools call other tools?

Your orchestrator can chain tools, but the model should not directly invoke tools from within tool implementations. Keep tools atomic; let the agent loop handle composition.

How do I handle tools that require authentication?

Store credentials in your application, not in tool definitions. The tool implementation accesses credentials based on the authenticated user context. Never pass API keys or tokens through the model.

What is the Assistants API vs rolling my own?

OpenAI Assistants API provides managed tool execution, file search, and code interpreter with thread management. Rolling your own gives full control over the loop, tool set, and error handling. Use Assistants for prototyping; build custom for production control.

How do I test tool calling agents?

Define scenarios: input message → expected tool sequence → expected final output. Run against a golden set. Mock tool implementations for unit tests; use real tools for integration tests.

Should I let the model retry failed tool calls?

Yes, with limits. Return structured errors to the model - it often self-corrects (wrong SQL column, invalid parameter). Cap retries at 2–3 per tool call to prevent infinite loops.

How does tool calling relate to MCP?

MCP standardizes how tools are defined, discovered, and invoked across applications. Tool calling is the general pattern; MCP is a specific protocol implementation. See MCP guide.

What is the difference between computer use and browser use?

Computer use drives a full desktop UI (typically via screenshots and pointer/keyboard actions). Browser use drives a browser you host, often with accessibility-tree and element references. Prefer browser use for web-only tasks; prefer computer use when the agent must leave the browser; prefer narrow typed APIs when you do not need UI control.

Can I use tool calling with open-source models?

Yes. Llama 3, Mistral, and Qwen support tool calling via fine-tuning or prompt-based formats. Quality is lower than GPT-4o or Claude for complex tool selection. Test thoroughly on your tool set.

How do I prevent the agent from calling dangerous tools?

Implement a permission layer between the model and tool execution. Map user roles to allowed tools. Block destructive tools by default. Require explicit user confirmation for irreversible actions.

References

• ReAct: Synergizing Reasoning and Acting (Yao et al., 2022)
• OpenAI Function Calling
• Anthropic Tool Use
• Anthropic Computer Use (GA)
• Anthropic Browser Use

Further Reading

• Model Context Protocol
• LangGraph Documentation
• LangChain Agents Documentation