AI Agents Guide

TL;DR

• An AI agent is an LLM-driven control loop that observes state, reasons about a goal, selects actions (often tool calls), and repeats until the task is done or a stop condition is met.

• Agents differ from chatbots because they act autonomously across multiple steps - they don't just respond; they plan, execute, observe results, and adapt.

• The core components are always the same: a language model, a tool registry, a state store, and an orchestration loop that decides when to think, act, or stop.

• Reliability comes from constraints, not model intelligence alone - bounded iterations, typed tool schemas, human approval gates, and observability at every step.

• Most production systems are hybrids - deterministic workflows for predictable paths, agent loops for dynamic reasoning. Pure autonomy is rarely the right default.

Why This Matters

The first wave of LLM applications was chat: ask a question, get an answer. The second wave is action: resolve a support ticket, triage an incident, refactor a codebase, reconcile expenses, or research a market segment - tasks that require multiple steps, external data, and decisions along the way.

If you're building software that needs to do things rather than just say things, you're building an agent - whether you call it that or not. Customer support bots that look up orders and issue refunds. Dev tools that read files, run tests, and open pull requests. Research assistants that search, summarize, and cross-reference sources. These all share the same underlying pattern.

Understanding agents matters because the failure modes are different from chat. A chatbot that hallucinates is annoying. An agent that hallucinates a SQL query and deletes rows is a production incident. The engineering discipline - tool design, state management, guardrails, evaluation - is what separates demos from systems people trust.

The Problem AI Agents Solve

Single-shot LLM calls fail when:

1. The task requires external data the model wasn't trained on or can't know at inference time (live APIs, private databases, current events).

2. The task decomposes into multiple steps where each step depends on the outcome of the previous one.

3. The correct action isn't known upfront - the system must explore, backtrack, or try alternatives.

4. Side effects must happen in the real world - sending emails, updating records, deploying code - not just generating text.

Prompting alone handles none of this reliably. Fine-tuning teaches behavior but doesn't give live access to tools. RAG injects knowledge but doesn't execute actions. Agents combine reasoning with tool execution in a loop, closing the gap between "the model knows what to do" and "the system actually did it."

What Is an AI Agent?

An AI agent is a software system where a large language model serves as the decision-making core. Given a goal and current state, the agent decides what to do next - typically by generating structured output that maps to a tool call, a plan update, or a final response.

Formally, an agent implements a policy π(state) → action in a loop:

def agent_loop(goal, tools, max_steps=10):
    state = {"goal": goal, "messages": [], "observations": []}

    for step in range(max_steps):
        action = llm.decide(state, tools)

        if action.type == "final_answer":
            return action.content

        if action.type == "tool_call":
            result = tools.execute(action.name, action.args)
            state["observations"].append(result)
            state["messages"].append({"role": "tool", "content": result})
        else:
            raise ValueError(f"Unknown action: {action.type}")

    raise RuntimeError("Agent exceeded max steps without completing goal")

This is deliberately simple. Every framework - LangChain, LangGraph, AutoGen, CrewAI - adds structure around this loop: typed state, checkpointing, parallel branches, human-in-the-loop interrupts, and retry logic.

Agents are not magic. They are control systems with an LLM as the controller. Treat them like any distributed system: define interfaces, bound execution, log everything, and test failure paths.

How AI Agents Work

An agent run proceeds through four phases that repeat until termination:

1. Perception

The agent receives input: user message, system prompt, conversation history, retrieved documents, tool outputs from prior steps, and structured state (task ID, permissions, budget remaining).

2. Reasoning

The LLM processes context and decides the next move. Depending on architecture, this may be implicit (direct tool selection) or explicit (chain-of-thought before action). Reasoning models like o1 and DeepSeek-R1 externalize this as visible thinking tokens.

3. Action

The agent executes a tool call, updates internal state, or emits a final response. Tool calls should be structured (JSON schema) so your runtime can validate arguments before execution.

4. Observation

Tool results flow back into state. The loop continues until the agent produces a final answer, hits a step limit, encounters an unrecoverable error, or a human approves/rejects an action.

ReAct interleaves reasoning traces and tool actions: the model thinks about what to do, calls a tool, observes the result, and repeats until it can answer.

Source: Google Research

Architecture

A production agent system has distinct layers:

The orchestrator is the critical piece. Raw while-loops work for prototypes; production needs checkpointing (resume after crash), idempotent tool design, and explicit termination conditions.

Step-by-Step Flow

Consider an agent tasked with: "What's our Q2 revenue for the EMEA region, and how does it compare to Q1?"

1. Parse goal - Orchestrator initializes state with user query and available tools: sql_query, chart_generator, send_slack_message.

2. Plan (optional) - Agent identifies subtasks: fetch Q2 EMEA revenue, fetch Q1 EMEA revenue, compute delta, format response.

3. First tool call - Agent calls sql_query with a generated SELECT statement filtered by region and quarter.

4. Observe - Database returns {q2_revenue: 4200000}.

Result appended to state.

5. Second tool call - Agent calls sql_query for Q1 with corrected date range based on schema observation.

6. Observe - Returns {q1_revenue: 3800000}.

7. Reason - Agent computes 10.5% growth, decides no chart needed for this simple comparison.

8. Final answer - Agent synthesizes: "Q2 EMEA revenue was $4.2M, up 10.5% from Q1's $3.8M."

9. Log trace - Full step sequence stored for audit and eval.

If step 3 returned a SQL error, a well-designed agent would read the error, inspect schema via another tool, rewrite the query, and retry - not fail silently or hallucinate numbers.

Real Production Example

Here's a minimal but production-shaped ReAct agent using LangGraph:

from typing import Annotated, TypedDict
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from langgraph.graph import StateGraph, END
from langgraph.prebuilt import ToolNode
from langgraph.graph.message import add_messages
import operator

# --- Tools ---
@tool
def search_orders(customer_id: str) -> str:
    """Look up recent orders for a customer ID."""
    orders = db.query("SELECT id, status, total FROM orders WHERE customer_id = ?", customer_id)
    return str(orders[:5])

@tool
def issue_refund(order_id: str, reason: str) -> str:
    """Issue a refund for an order. Requires order_id and reason."""
    if not auth.can_refund(current_user, order_id):
        return "ERROR: Insufficient permissions"
    refund_id = payments.refund(order_id, reason=reason)
    return f"Refund {refund_id} issued for order {order_id}"

tools = [search_orders, issue_refund]
tool_node = ToolNode(tools)

# --- State ---
class AgentState(TypedDict):
    messages: Annotated[list, add_messages]
    step_count: Annotated[int, operator.add]

# --- Nodes ---
llm = ChatOpenAI(model="gpt-4o", temperature=0).bind_tools(tools)

def agent_node(state: AgentState):
    response = llm.invoke(state["messages"])
    return {"messages": [response], "step_count": 1}

def should_continue(state: AgentState) -> str:
    last = state["messages"][-1]
    if state.get("step_count", 0) >= 8:
        return "end"
    if last.tool_calls:
        return "tools"
    return "end"

# --- Graph ---
graph = StateGraph(AgentState)
graph.add_node("agent", agent_node)
graph.add_node("tools", tool_node)
graph.set_entry_point("agent")
graph.add_conditional_edges("agent", should_continue, {"tools": "tools", "end": END})
graph.add_edge("tools", "agent")

agent = graph.compile()

result = agent.invoke({
    "messages": [
        {"role": "system", "content": "You are a support agent. Verify order status before issuing refunds."},
        {"role": "user", "content": "Customer C-8842 says order O-991 was charged twice. Help them."},
    ],
    "step_count": 0,
})

Key production details in this example: typed tools with docstrings (used for schema generation), permission checks inside tools (not trusting the LLM), step limits to prevent runaway loops, and a system prompt that encodes business rules.

Design Decisions

⚠ Common Mistakes

1. Giving agents too many tools - A registry of 40 tools overwhelms the model. Curate 5–10 relevant tools per task type, or use a router agent that selects a tool subset first.

2. No step limits - Agents can loop indefinitely, burning tokens and API budget. Always set max_iterations and alert when hit.

3. Trusting LLM-generated SQL/code without sandboxing - Execute against read-only replicas, use parameterized queries, and never give write access by default.

4. Poor tool descriptions - The model chooses tools based on names and docstrings. Vague descriptions cause wrong tool selection more often than model capability limits.

5. No observability - Without step-level traces, debugging a failed 7-step agent run is guesswork. Log every LLM input/output and tool result.

6. Treating agents as chatbots with tools bolted on - A single tool call isn't an agent. The value is in multi-step reasoning with observation feedback.

Where It Breaks Down

• Long-horizon tasks - Agents lose coherence over 15+ steps. Decompose into sub-agents or use Plan-and-Execute architectures (Agent Architectures).

• Ambiguous goals - "Make the codebase better" has no termination condition. Agents need concrete, verifiable objectives.

• Adversarial inputs - Prompt injection via tool outputs (e.g., a webpage telling the agent to ignore instructions) is a real attack surface. Sanitize external content.

• Non-deterministic cost - A simple question might take 1 step or 8. Budget caps and model routing are essential for predictable billing.

• Evaluation difficulty - End-to-end agent quality is hard to measure with single metrics.

You need task-specific success criteria and step-level checks.

Running in Production

Best Practice

✅ Best Practices - Instrument every stage, version embedding models, enforce access control at retrieval time, and evaluate on a fixed golden set before shipping changes.

Important

Never give an agent write access to production systems without human approval gates until you've measured task success rate above 95% on a representative eval set - and even then, keep audit trails.

Ecosystem

• Orchestration: LangGraph, LangChain, AutoGen, CrewAI, OpenAI Agents SDK - frameworks for building agent loops with varying levels of structure.

• Tool protocols: Model Context Protocol (MCP) - standard for connecting agents to external data and tools.

• Memory: Mem0, Zep, LangGraph checkpointing - persistent memory layers for cross-session agents.

• Observability: LangSmith, Langfuse, Braintrust, Arize - trace agent runs, evaluate quality, detect regressions.

• Deployment: Modal, Fly.io, AWS Lambda (for short tasks), dedicated worker pools for long-running agents.

Related Technologies

• Tool Calling: The mechanism agents use to invoke external capabilities. Foundation for any agent system.

• Function Calling: Structured JSON output format that LLMs use to specify tool invocations.

• Agent Architectures: ReAct, Plan-and-Execute, Reflexion - patterns that structure the agent loop.

• Agent Planning: How agents decompose goals and replan when execution fails.

• Agent Memory: Short-term and long-term memory patterns for stateful agents.

• Workflows vs Agents: When to use deterministic workflows vs autonomous loops.

• RAG: Retrieval grounds agents in private data - often combined with tool use for knowledge-heavy tasks.

• Guardrails: Safety constraints essential for agents that take real-world actions.

Learning Path

Prerequisites: Large Language Models · Tool Calling · Prompt Engineering

Next topics: Agent Architectures · Agent Planning · Workflows vs Agents

Estimated time: 50 min · Difficulty: Intermediate

FAQs

What is the difference between an AI agent and a chatbot?

A chatbot responds to messages in a single turn (or multi-turn conversation without external actions). An agent autonomously executes a loop of reasoning and tool calls until a goal is achieved. If your system calls APIs, queries databases, or modifies state, it's an agent.

Do I need a framework to build agents?

No. A while-loop with an LLM and tool executor is a valid agent. Frameworks like LangGraph add checkpointing, parallelism, and human-in-the-loop - valuable in production but not required to start.

How many tools should an agent have?

Start with 3–5 tools focused on one task domain. Research shows tool selection accuracy degrades as registry size grows. Use dynamic tool loading or a router if you need more.

What's a reasonable step limit?

8–15 steps for most tasks. Customer support flows rarely need more than 10. Research tasks may need 20 with sub-agent delegation. Always enforce a hard cap.

Can I use open-source models for agents?

Yes. Llama 3, Mistral, and Qwen support tool calling. Smaller models struggle with complex multi-step reasoning - use them for simple tool selection, capable models for planning.

How do agents relate to RAG?

RAG retrieves knowledge; agents take actions. They're complementary - an agent might call a RAG retrieval tool as one step in a larger workflow. See Agentic RAG for the combined pattern.

What is human-in-the-loop for agents?

Pausing the agent loop before irreversible actions (refunds, deployments, emails) for human approval. LangGraph supports interrupt nodes natively. Use this for any write action until trust is established.

How do I test agents?

Create task scenarios with expected outcomes. Assert: correct final answer, correct tools called (in reasonable order), no forbidden tools used, and termination within step limit. Run in CI on every change.

Why do agents sometimes ignore tool results?

Context window pressure, ambiguous tool output format, or conflicting system instructions. Structure tool outputs consistently (JSON with clear fields), and instruct the agent to read observations before proceeding.

Are agents safe for customer-facing use?

With guardrails, yes - but treat write actions as privileged operations. Read-only agents (search, summarize, answer) are lower risk. Progressive autonomy: start read-only, add writes with approval, automate only after eval validation.

What's the difference between agentic AI and AI agents?

Agentic AI describes the paradigm - systems that act autonomously. AI agents are the concrete implementation. The terms are often used interchangeably; Agentic AI covers the broader shift in how we build AI software.

How do I reduce agent latency?

Parallel tool calls where independent, use faster models for intermediate steps, cache frequent lookups, stream status updates, and pre-fetch likely-needed data based on intent classification.

References

• ReAct: Synergizing Reasoning and Acting (Yao et al., 2022)
• LangChain Agents Documentation
• OpenAI Agents Guide

Further Reading

• Anthropic Tool Use Documentation
• LangGraph Documentation
• Microsoft AutoGen

Summary

• An AI agent is an LLM-driven control loop: observe, reason, act, repeat until done. - Agents solve multi-step tasks that require external data and real-world actions - things single LLM calls cannot do. - Production reliability comes from bounded execution, typed tools, permission checks, and step-level observability. - Start with a small tool set, a hard step limit, and read-only tools. Add autonomy incrementally as eval metrics justify it.

• Most production systems combine deterministic workflows with agent loops - pure autonomy is rarely the right choice. - The ecosystem is mature: LangGraph for orchestration, MCP for tools, LangSmith for tracing. Don't build from scratch unless you have specific requirements.