Function Calling Guide

TL;DR

• A language model cannot independently read your current database, inspect live weather, charge a card, or send an email. Function calling gives it a structured way to request such capabilities from application code.
• The model generates a tool name and arguments during inference. It does not execute Python, contact the API, or inherit the caller's permissions.
• Tool schemas improve syntax and selection, but they do not prove factual correctness, business validity, authorization, or safety. Validate all arguments again in trusted code.
• Training teaches a model the general pattern of selecting tools and producing arguments. At inference time, your prompt and schemas advertise the tools available for that request.
• Function calling is a mechanism, not an agent. A single request may produce one tool call; an agent adds a loop, state, planning, stopping rules, and recovery.
• MCP standardizes how clients discover and invoke remote tools and resources. Function calling is the model-facing intent format that can sit above MCP, direct SDK calls, or internal functions.
• Production implementations need an allowlist, typed schemas, permission checks, bounded timeouts, idempotency, sanitized errors, output limits, tracing, and evaluations.

On this page

• Why This Matters
• The Problem Function Calling Solves
• How We Got Here
• What Is Function Calling?
• How Function Calling Works
• Architecture
• Step-by-Step Flow
• Real Production Example
• Design Decisions
• Comparisons
• Common Mistakes
• Where It Breaks Down
• When NOT to Use Function Calling
• Running in Production
• Related Guides
• Interview Questions
• Key Takeaways
• FAQs
• References
• Further Reading

Why This Matters

Plain language generation is useful for explanations and drafting, but production applications often need current facts or side effects. Model weights are a compressed snapshot of training data. They do not contain today's inventory, the authenticated user's account balance, or a reliable connection to an operational system. Even when a model memorized a fact during training, that fact may be stale.

Function calling separates two responsibilities:

1. Probabilistic intent generation: the model maps conversation context to a structured proposal such as lookup_order(order_id="ORD-123").
2. Deterministic application control: trusted code validates the proposal, applies identity and policy, invokes the implementation, and records the outcome.

That boundary makes LLM integration compatible with ordinary software engineering. The tool implementation can use database transactions, service accounts, retries, circuit breakers, audit logs, and access-control systems. The model remains a decision-support component, not a privileged runtime.

This separation also improves testing. Teams can evaluate whether the model selected the right function and arguments independently from whether the downstream service succeeded. A payment outage is an execution failure; choosing issue_refund for a shipping-status question is a model-routing failure. Combining both into one “agent failed” metric obscures the remedy.

The Problem Function Calling Solves

LLMs produce token sequences. Operational systems consume typed requests. Before native function calling, developers prompted models to embed JSON inside prose and then extracted it with regular expressions or delimiter rules:

Instruction: Reply with JSON when a lookup is needed.
Output: Sure — I will check that now.
        ```json
        {"action":"lookup_order","id":"ORD-123"}
        ```

This approach creates several ambiguities. The application must distinguish an action request from JSON merely quoted in user content. Markdown fences and commentary break parsers. Missing fields and invented action names require ad hoc recovery. Most importantly, parsing JSON does not establish that the requested operation is permitted.

Native function calling provides a dedicated response field for one or more tool intents. Providers often constrain decoding or validate the generated shape so that arguments conform more closely to JSON Schema. The application no longer needs to infer whether prose represents an action.

Function calling solves the interface problem between natural-language intent and typed application APIs. It does not solve:

• whether the user's claim is true;
• whether required information is missing;
• whether the selected operation is appropriate;
• whether the caller may perform it;
• whether the downstream system is available;
• whether the final natural-language answer accurately reflects the result.

Those remain application and evaluation responsibilities.

How We Got Here

Early LLM applications relied on prompt conventions: “return only JSON,” ReAct-style Thought/Action/Observation text, or stop-token parsers. Frameworks wrapped those conventions in output parsers, but small model or prompt changes could alter formatting. API providers then introduced native function or tool fields, allowing models to be trained and served with explicit tool-call representations.

Later generations added parallel calls, stricter schema-constrained generation, and provider-specific controls for forcing or disabling tools. Agent frameworks built loops around these primitives. MCP subsequently addressed a different layer: standard discovery and transport between AI clients and external capability servers.

Diagram: Evolution of function calling

timeline
    title From prompt conventions to interoperable tools
    section Prompt era
      Text actions : ReAct-style action strings
      JSON prompting : Output parsers and retries
    section Native APIs
      Function fields : Typed name and arguments
      Strict schemas : Constrained structured generation
      Parallel calls : Multiple intents per response
    section Orchestration
      Agent loops : State, retries, stopping rules
      MCP : Standard capability discovery and transport

The evolution moved structure out of fragile prose while leaving execution and policy in application code.

Training versus inference

Tool use spans two distinct phases that are often conflated.

During training or post-training, examples teach the model relationships among user requests, tool descriptions, argument structures, tool results, and final answers. Preference optimization may reward correct tool selection and penalize invented arguments. Some providers also train special tokens or output heads for tool calls. Application developers usually do not control this phase unless they fine-tune a model.

During inference, the application sends the current conversation and a list of available tool schemas. The model conditions on those definitions and generates either normal content or a structured call. No tool is learned permanently from one request. Removing a schema removes that advertised capability for the request, although the model may still discuss it in prose.

A schema is therefore both a machine-readable contract and inference context. Names, descriptions, enums, and examples influence model behavior much like a prompt. Yet the schema does not grant access: the execution layer decides what the caller can actually do.

What Is Function Calling?

Function calling is an inference capability in which an LLM emits a structured intent containing:

• a tool or function identifier;
• serialized arguments;
• usually a provider-generated call ID used to correlate the result;
• optionally multiple calls in one assistant turn.

The term “function” is historical. The implementation may be a local function, SQL repository method, HTTP API, queue publication, workflow start, or MCP tool. The model sees only the advertised contract.

A call has three correctness layers:

1. Syntactic correctness: arguments parse and conform to the declared schema.
2. Semantic correctness: values make sense for the requested task; for example, the refund amount does not exceed the captured payment.
3. Policy correctness: the authenticated principal may perform the operation on the target resource.

Provider-side structured generation can strengthen the first layer. Only your domain and authorization services can establish the other two.

Intent is not execution

If the model emits:

{
  "name": "create_refund",
  "arguments": { "order_id": "ORD-123", "amount_cents": 2500 }
}

nothing has been refunded. The output is equivalent to an untrusted client request. The orchestrator must bind the real user identity from the authenticated session—not from model arguments—then validate the order, permissions, amount, idempotency key, and operational limits.

This distinction is the central engineering property of function calling. Any framework that makes execution look automatic still implements this boundary somewhere. Locate it before deploying.

How Function Calling Works

At request time, the application chooses a bounded set of tools and serializes their schemas into the model request. The model predicts a response based on the user message, prior messages, system instructions, and those schemas. Depending on the API, tool_choice can permit automatic selection, require some tool, force one named tool, or prohibit tools.

When the response contains tool calls, the application:

1. parses each argument payload;
2. resolves the name through a server-side allowlist;
3. validates the payload with a typed model;
4. applies business rules and authorization using trusted identity;
5. executes under timeout and resource limits;
6. converts the result or sanitized error into a tool-result message;
7. asks the model to synthesize a response, or continues a bounded loop.

Diagram: Model and application responsibilities

sequenceDiagram
    actor U as User
    participant A as Application
    participant M as LLM API
    participant P as Policy
    participant T as Tool service
    U->>A: Authenticated request
    A->>M: Messages plus allowed schemas
    M-->>A: Structured tool intent
    A->>A: Parse and validate
    A->>P: Check principal and resource
    P-->>A: Allow or deny
    alt allowed
        A->>T: Execute with timeout
        T-->>A: Typed result
    else denied
        A->>A: Create sanitized denial
    end
    A->>M: Tool result
    M-->>A: User-facing answer
    A-->>U: Response

The LLM proposes an action, while the application retains identity, policy, execution, and audit authority.

Schemas guide generation; validators enforce contracts

Use narrow schemas. Prefer enums over open strings, integer minor currency units over floats, explicit bounds, and separate tools for materially different permission levels. Avoid accepting raw SQL, arbitrary URLs, shell commands, filesystem paths, or generic execute(action, payload) structures.

Descriptions should state when to use the tool, when not to use it, and what missing information must be requested. They should not contain secrets or internal implementation details because schemas are sent to the model provider and consume context tokens.

Parallel calls are useful for independent reads, such as fetching weather for three cities. Do not execute calls concurrently when order matters, when they mutate the same resource, or when one result determines authorization for another. The model's array order is not a transaction plan.

Architecture

A production function-calling system is an orchestration pipeline, not a direct connection from model to database.

Diagram: Production function-calling architecture

flowchart LR
    C[Client] --> G[API gateway]
    G --> O[Orchestrator]
    O --> R[Tool registry]
    O --> L[LLM provider]
    L --> O
    O --> V[Schema validator]
    V --> P[Policy engine]
    P --> X[Execution adapter]
    X --> S[(Systems of record)]
    X --> Q[Job queue]
    O --> B[Trace and audit]
    X --> B

The orchestrator mediates every transition from probabilistic intent to privileged execution.

Tool registry

The registry maps public tool names to schemas, validators, implementations, required permissions, timeout budgets, mutability classes, and result serializers. The name supplied by the model must be resolved through this registry; never dynamically import or reflect arbitrary names.

Orchestrator

The orchestrator owns conversation state, tool exposure, iteration limits, correlation IDs, execution scheduling, and result messages. It should expose only tools relevant to the current user and task. Dynamic tool filtering reduces context cost and accidental selection but must not replace runtime authorization.

Policy layer

Policy decisions use authenticated principal, tenant, resource ownership, action, and environment. Do not let the model provide user_id, role, or tenant_id as authoritative identity. Those values come from signed sessions, service credentials, or trusted request context.

Execution adapters

Adapters translate validated arguments into calls to systems of record. They implement deadlines, retries where safe, idempotency, circuit breaking, concurrency controls, and output normalization. Read tools and mutation tools should have different policies and observability.

Result boundary

Tool output is also untrusted from the model's perspective. Web pages, emails, and retrieved documents may contain prompt injection. Limit fields and size, mark data as untrusted, and avoid returning credentials, stack traces, or unnecessary personal data.

Step-by-Step Flow

1. Authenticate the request. Resolve the principal and tenant before invoking the model.
2. Classify the operation context. Determine whether this route permits reads, writes, or only text.
3. Select tool definitions. Advertise the smallest relevant set based on product state and coarse permissions.
4. Call the model. Include stable system instructions, conversation context, schemas, and an overall deadline.
5. Inspect the finish state. Handle text, refusal, truncation, malformed responses, and tool calls explicitly.
6. Resolve each tool name. Reject unknown names through a fixed registry.
7. Parse and validate arguments. Apply Pydantic or equivalent typed validation, bounds, and normalization.
8. Apply business validation. Check state transitions, ownership, limits, and dependencies against current data.
9. Authorize. Evaluate the trusted principal against the requested action and resource.
10. Execute with controls. Use per-tool timeout, cancellation, idempotency, and bounded concurrency.
11. Sanitize the result. Return only fields needed for synthesis, with stable error codes.
12. Append correlated tool results. Preserve each call ID exactly as required by the provider.
13. Synthesize or continue. Permit another model turn only within the iteration and time budget.
14. Record telemetry. Trace selection, validation, policy, execution, synthesis, latency, and outcome.
15. Return a clear user result. Distinguish completed, denied, failed, and queued actions.

Diagram: Tool-call lifecycle

stateDiagram-v2
    [*] --> Proposed
    Proposed --> Rejected: unknown or malformed
    Proposed --> Validated: schema valid
    Validated --> Denied: policy denies
    Validated --> Running: policy allows
    Running --> TimedOut: deadline reached
    Running --> Failed: dependency error
    Running --> Completed: result returned
    Completed --> Synthesized
    Denied --> Synthesized
    Failed --> Synthesized
    TimedOut --> Synthesized
    Rejected --> Synthesized
    Synthesized --> [*]

Explicit lifecycle states prevent a proposed call from being mistaken for a completed action.

Real Production Example

The following Python example uses the OpenAI tools API shape, Pydantic validation, trusted authorization context, per-tool timeouts, an allowlisted registry, and sanitized errors. It models an order-support endpoint. The authenticated principal is passed separately and is never accepted from model arguments.

import asyncio
import json
from dataclasses import dataclass
from typing import Any, Awaitable, Callable, Literal

from openai import AsyncOpenAI
from pydantic import BaseModel, ConfigDict, Field, ValidationError

client = AsyncOpenAI()


@dataclass(frozen=True)
class Principal:
    user_id: str
    tenant_id: str
    permissions: frozenset[str]


class LookupOrderArgs(BaseModel):
    model_config = ConfigDict(extra="forbid")
    order_id: str = Field(pattern=r"^ORD-[A-Z0-9]{6,20}$")


class RefundArgs(BaseModel):
    model_config = ConfigDict(extra="forbid")
    order_id: str = Field(pattern=r"^ORD-[A-Z0-9]{6,20}$")
    amount_cents: int = Field(gt=0, le=100_000)
    reason: Literal["duplicate", "defective", "not_received", "other"]
    idempotency_key: str = Field(min_length=16, max_length=80)


async def lookup_order(args: LookupOrderArgs, principal: Principal) -> dict[str, Any]:
    # Repository query must scope by tenant_id; never trust a tenant from model output.
    order = await orders.get(args.order_id, tenant_id=principal.tenant_id)
    if order is None:
        return {"status": "not_found"}
    if order.customer_user_id != principal.user_id and "orders:read:any" not in principal.permissions:
        raise PermissionError("order_read_denied")
    return {
        "order_id": order.id,
        "status": order.status,
        "captured_cents": order.captured_cents,
        "refunded_cents": order.refunded_cents,
    }


async def create_refund(args: RefundArgs, principal: Principal) -> dict[str, Any]:
    if "refunds:create" not in principal.permissions:
        raise PermissionError("refund_create_denied")

    order = await orders.get_for_update(args.order_id, tenant_id=principal.tenant_id)
    if order is None:
        return {"status": "not_found"}
    refundable = order.captured_cents - order.refunded_cents
    if args.amount_cents > refundable:
        return {"status": "rejected", "code": "amount_exceeds_refundable"}

    refund = await payments.refund(
        payment_id=order.payment_id,
        amount_cents=args.amount_cents,
        reason=args.reason,
        idempotency_key=args.idempotency_key,
    )
    return {"status": refund.status, "refund_id": refund.id}


@dataclass(frozen=True)
class ToolSpec:
    args_model: type[BaseModel]
    handler: Callable[[Any, Principal], Awaitable[dict[str, Any]]]
    timeout_seconds: float


REGISTRY = {
    "lookup_order": ToolSpec(LookupOrderArgs, lookup_order, 2.0),
    "create_refund": ToolSpec(RefundArgs, create_refund, 5.0),
}

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": name,
            "description": (
                "Read an order visible to the current user."
                if name == "lookup_order"
                else "Create a refund only after the user confirms amount and reason."
            ),
            "strict": True,
            "parameters": spec.args_model.model_json_schema(),
        },
    }
    for name, spec in REGISTRY.items()
]


def safe_error(code: str) -> dict[str, str]:
    return {"status": "error", "code": code}


async def execute_tool(call: Any, principal: Principal) -> dict[str, Any]:
    spec = REGISTRY.get(call.function.name)
    if spec is None:
        return safe_error("unknown_tool")
    try:
        raw_args = json.loads(call.function.arguments)
        args = spec.args_model.model_validate(raw_args)
        return await asyncio.wait_for(
            spec.handler(args, principal),
            timeout=spec.timeout_seconds,
        )
    except (json.JSONDecodeError, ValidationError):
        return safe_error("invalid_arguments")
    except PermissionError:
        return safe_error("permission_denied")
    except TimeoutError:
        return safe_error("tool_timeout")
    except Exception:
        # Log exception with trace ID server-side; do not expose internals to the model.
        return safe_error("dependency_failure")


async def answer(message: str, principal: Principal) -> str:
    messages: list[Any] = [
        {
            "role": "system",
            "content": (
                "You support orders. Never invent IDs or claim an action succeeded "
                "unless its tool result says completed. Ask for confirmation before refunds."
            ),
        },
        {"role": "user", "content": message},
    ]

    for _ in range(3):
        response = await asyncio.wait_for(
            client.chat.completions.create(
                model="gpt-4.1",
                messages=messages,
                tools=TOOLS,
                tool_choice="auto",
                temperature=0,
            ),
            timeout=12.0,
        )
        assistant = response.choices[0].message
        messages.append(assistant)
        if not assistant.tool_calls:
            return assistant.content or "No response was generated."

        # Mutating calls should normally run serially; independent reads may be gathered.
        for call in assistant.tool_calls:
            result = await execute_tool(call, principal)
            messages.append(
                {
                    "role": "tool",
                    "tool_call_id": call.id,
                    "content": json.dumps(result),
                }
            )

    return "The request exceeded the allowed tool-call steps."

The example deliberately performs validation in two places. Pydantic enforces shape, types, extra-field rejection, and simple bounds. The handlers enforce current business state and permissions. Strict provider schemas reduce malformed outputs but do not eliminate either server-side layer.

In a real service, also attach a request deadline, trace ID, model snapshot, prompt version, schema version, and hashed or redacted argument fields. Refund confirmation should be represented as trusted application state or a signed approval token, not inferred solely from conversation prose.

Design Decisions

Automatic, required, forced, or disabled tools

Use automatic selection for mixed conversational and lookup requests. Require a tool when the endpoint's contract demands grounded data, such as a price quote. Force a named tool when deterministic routing has already chosen the operation and the model is only extracting arguments. Disable tools for routes that must not perform external actions.

One broad tool or several narrow tools

Narrow tools improve policy mapping and reduce ambiguous arguments. A generic manage_order(action, payload) is compact but creates a large, weakly typed privilege surface. Separate read, refund, cancel, and address-change operations when their permissions and risks differ.

Model recovery or deterministic failure

Returning a stable validation error can let the model correct a missing field. Do not allow autonomous retries for authorization denial, destructive calls, non-idempotent operations, or ambiguous payment outcomes. Those conditions require a user-visible stop or reconciliation workflow.

Sequential or parallel execution

Parallelize independent, side-effect-free reads when latency matters. Execute mutations serially unless the business transaction explicitly supports concurrency. Preserve a global concurrency limit even for reads; a model can request many calls in one turn.

Direct execution or queued workflow

Use synchronous execution for fast operations that fit within the request deadline. For long-running jobs, validate and authorize synchronously, enqueue a durable workflow, and return a job ID. Do not hold an LLM request open for minutes.

Stable schema evolution

Version tool behavior like an API. Add optional fields compatibly; create a new tool name or version for incompatible semantics. Record the schema version used for every call so incidents and evaluations can reproduce behavior.

When should I use this?

Comparisons

Function calling versus agents

A function call can be single-shot: classify the request, generate arguments, execute once, and return. An agent adds a feedback loop in which observations influence subsequent choices. Function calling is often inside an agent, but using tools does not make a system autonomous. Prefer a single call or deterministic workflow when the path is known; every extra loop increases latency, cost, and failure surface.

Function calling versus MCP

MCP and function calling operate at different layers. An MCP client discovers tool definitions from a server and invokes them using protocol messages. The client may expose those definitions to a model through a provider's function-calling API. The model still emits intent; the host still applies consent and policy; the MCP server still validates the request. MCP reduces integration coupling but does not remove security boundaries.

Function calling versus structured outputs

Both use schemas, but their semantics differ. A structured output such as {"sentiment":"negative"} is usually the final application result. A function call such as issue_refund(...) requests execution. Treating all schema-constrained JSON as executable is dangerous because it erases the action boundary.

Common Mistakes

1. Saying the model “calls the API.” The model emits tokens representing a request. Your runtime calls the API.
2. Trusting strict schemas as authorization. Structural validity says nothing about caller rights or resource state.
3. Putting identity in arguments. A model-generated tenant_id or role is not authentication context.
4. Exposing every tool on every request. Large catalogs consume context, reduce selection accuracy, and increase attack surface.
5. Using vague overlapping descriptions. Similar tools need explicit positive and negative selection criteria.
6. Allowing arbitrary code, SQL, URLs, or paths. These turn language ambiguity and prompt injection into direct security exposure.
7. Retrying mutations blindly. A timeout may occur after a side effect committed. Use idempotency and reconciliation.
8. Returning raw dependency errors. Stack traces, table names, and credentials can leak through the model to users.
9. Treating tool output as trusted instructions. Retrieved content can carry prompt injection; constrain and label it as data.
10. Claiming success from the proposed call. Only the execution result can establish completion.
11. Omitting loop and token budgets. Repeated calls can create runaway cost and load.
12. Testing only happy paths. Include denials, malformed values, stale state, injections, timeouts, duplicate requests, and partial outages.

Where It Breaks Down

Function calling remains probabilistic at the selection and argument-generation layers. Models may choose a semantically adjacent tool, omit required context, invent IDs, or overuse a tool when direct text would suffice. More schemas and longer descriptions do not monotonically improve reliability; they consume context and can introduce conflicting cues.

Failures also occur after correct selection. Downstream data changes between lookup and mutation. An operation completes but the response times out. Parallel calls race. Tool output exceeds the context window. A malicious document tells the model to call an unrelated privileged tool. A provider model update changes selection rates.

Schema constraints are limited by provider support. Some JSON Schema keywords may be ignored or rejected. Recursive and highly polymorphic structures are difficult for both models and APIs. Keep the model-facing contract simpler than the internal domain model.

Multi-step loops amplify compounding error. If each decision is 95% correct, five independent decisions have only about a 77% chance of all being correct. Actual errors are correlated, so the simple calculation is not a reliability prediction, but it illustrates why bounded deterministic workflows outperform open-ended loops for known procedures.

When NOT to Use Function Calling

Do not use function calling when:

• the answer is entirely contained in trusted request context and no typed extraction is required;
• a deterministic parser, form, button, or API endpoint can capture the same intent more reliably;
• the operation is a fixed business workflow that should not depend on probabilistic routing;
• the action is irreversible and cannot be protected by explicit confirmation, authorization, and idempotency;
• legal or safety requirements demand deterministic decisions rather than model judgment;
• required tools would expose arbitrary code execution or unrestricted network access without isolation;
• latency cannot accommodate model inference plus tool execution;
• a small classifier or rules engine meets the accuracy and cost target.

Running in Production

Best Practice

Maintain a typed registry that binds each public tool name to its schema version, implementation, mutability class, permission, timeout, result limit, and owner. Generate provider schemas from the same typed definitions used for runtime validation where practical.

Warning

Prompt injection can arrive through user text or tool results. Never let model instructions override server-side authorization, network policy, tenant isolation, confirmation state, or an execution allowlist.

Important

Define success from the system of record, not from model language. “I refunded your order” is valid only after a correlated execution result confirms the refund or durable workflow state.

Decision Trade-off

Dynamically exposing fewer tools improves cost and selection accuracy, but it adds routing logic and can hide a needed capability. Measure false exposure and false omission rates on representative requests.

Reliability controls

Use an overall request deadline plus shorter per-tool deadlines. Retry only transient, idempotent operations with exponential backoff and jitter. For ambiguous mutation outcomes, query the system of record using the idempotency key rather than issuing the mutation again. Add circuit breakers and bulkheads so a failing dependency does not consume all application workers.

Bound maximum tool calls, model turns, parallel calls, argument size, and result size. Cancellation should propagate when the client disconnects or the overall budget expires. Long jobs belong in durable queues with status APIs.

Security controls

Apply least privilege at three layers: advertise only relevant tools, authorize every execution, and use restricted downstream credentials. Separate read and write service identities. Egress-control tool workers, sandbox code execution, redact secrets, and audit mutations. See Guardrails for policy enforcement and AI Security for prompt injection and confused-deputy threats.

Human confirmation must be bound to an exact action payload. “Yes” in conversation should not authorize a later modified amount or recipient. Store a signed approval record containing principal, tool, normalized arguments, expiry, and nonce.

Observability

Trace model request, tool selection, validation, authorization, execution, and synthesis as separate spans. Useful fields include model and prompt version, schema version, tool name, call ID, outcome code, latency, retry count, token usage, and tenant-safe correlation identifiers. Redact or hash sensitive arguments.

Monitor:

• tool selection precision and recall;
• argument validation failure rate;
• authorization denials and unknown-tool attempts;
• execution latency and timeout rate by dependency;
• duplicate and idempotency-conflict rate;
• average calls and model turns per request;
• claims of completion without successful execution;
• cost and token use by route.

Evaluation and rollout

Build a golden set containing expected tool, expected normalized arguments, expected no-tool cases, and permitted alternatives. Add adversarial cases: quoted tool JSON, cross-tenant IDs, prompt injection, missing confirmation, extreme values, and unsupported requests. Run evaluations against pinned model snapshots and again before provider upgrades.

Shadow new schemas or models without executing mutations. Compare intent traces, then canary read-only traffic. For write tools, use sandbox systems or dry-run adapters before production. Roll back schema and prompt changes independently.

Related Guides

• Models: GPT-5 · Claude Sonnet · Claude Fable · Gemini 3.1 Pro — models with production function-calling APIs.

• Companies: OpenAI · Anthropic · Google — schema-constrained tool intent providers.

• Tool Calling broadens the concept beyond function-shaped interfaces.

• Structured Outputs covers schema-constrained final responses.

• Prompt Engineering explains how instructions and descriptions shape inference behavior.

• AI Agents adds stateful, bounded tool-use loops.

• Model Context Protocol standardizes discovery and transport for tools and resources.

• Guardrails covers policy and validation layers.

• AI Security covers prompt injection, confused deputies, and privilege boundaries.

Interview Questions

What is the most important boundary in function calling?

The model generates a structured proposal; trusted application code executes it. Validation, identity, authorization, transaction semantics, timeouts, and auditing belong outside the model.

Does JSON Schema make a tool call safe?

No. It can enforce shape and some bounds, but it cannot establish ownership, business invariants, current state, or permission. Server-side typed and semantic validation remain mandatory.

How is function calling different from an agent?

Function calling is an inference output mechanism. An agent is an orchestration architecture that repeatedly chooses actions, observes results, stores state, and applies stopping rules. An agent may use function calling internally.

How would you prevent duplicate side effects after a timeout?

Require an idempotency key, persist operation state, and reconcile against the downstream system before retrying. A timeout is an unknown outcome, not proof that execution failed.

Why should authenticated identity not be a tool argument?

The model can invent or alter arguments. Identity must come from a trusted session or service credential and be passed out of band to policy and execution code.

When should parallel tool calls be disabled?

Disable them for dependent operations, mutations of shared state, calls requiring ordered policy checks, or dependencies without adequate concurrency limits.

How do MCP and function calling compose?

An MCP client discovers schemas and invokes an MCP server. The host can expose those schemas to a model through function calling, then translate approved model intents into MCP calls.

What metrics separate model and system quality?

Measure tool-selection and argument accuracy for the model; validation, authorization, dependency success, latency, and idempotency for execution; and groundedness of the final answer for synthesis.

Key Takeaways

• Function calling exists because model inference alone cannot access live systems or safely create side effects.
• Models generate structured tool intents; applications execute tools.
• Training teaches general tool-use behavior, while inference supplies the tools available now.
• Typed schemas improve reliability but do not replace business validation or authorization.
• Single-shot function calling is not an agent; agents add loops, state, and stopping rules.
• MCP standardizes tool integration but preserves the same model, host, and server trust boundaries.
• Production safety depends on allowlists, least privilege, deadlines, idempotency, sanitized results, traces, and adversarial evaluations.
• Prefer deterministic workflows whenever the operation path is known.

FAQs

Does an LLM actually run the function?

No. It emits a structured name and arguments. The host application decides whether and how to execute the corresponding implementation.

Can strict structured generation eliminate validation?

No. It reduces syntax errors. Your service must still validate with the authoritative schema and enforce semantic constraints, permissions, and current resource state.

How many tools should be sent to the model?

There is no universal limit. Use the smallest relevant set and measure selection performance. Similarity among tools and schema token size often matter more than raw count.

Should tool descriptions include examples?

Use concise examples when they clarify ambiguous arguments, but keep schemas small and avoid sensitive data. Negative guidance—when not to call a tool—can be equally useful.

Can function calling be used without an agent framework?

Yes. A direct model request, one validation-and-execution step, and one synthesis request are often easier to operate than a framework-managed loop.

What should a tool return on failure?

Return a stable, sanitized code and minimal recovery information, such as permission_denied or dependency_unavailable. Log detailed diagnostics only in protected telemetry.

How should long-running tools work?

Start a durable job after authorization, return a job identifier, and expose status through a separate read operation. Do not keep an inference request open indefinitely.

Is forcing a tool more reliable?

It removes tool-selection uncertainty but not argument or execution errors. Force a tool only when deterministic application logic has already established that it is the correct operation.

Can tool results contain prompt injection?

Yes. Search results, documents, emails, and web pages are untrusted content. Limit returned fields, label them as data, and keep policy outside model instructions.

Should write tools be exposed directly?

Only with narrow schemas, explicit authorization, confirmation where appropriate, idempotency, audit logging, and safe failure handling. For high-risk operations, use a deterministic approval workflow.

References

• OpenAI Function Calling Guide
• OpenAI Structured Outputs Guide
• JSON Schema Specification
• Pydantic Models Documentation
• Anthropic Tool Use Documentation
• Google Gemini Function Calling Documentation
• Model Context Protocol Specification
• OWASP Top 10 for Large Language Model Applications

Further Reading

• OpenAI API Error Handling
• Pydantic JSON Schema Documentation
• OAuth 2.0 Security Best Current Practice
• The Idempotency-Key HTTP Header Field
• NIST AI Risk Management Framework