Multi-Agent Systems Guide

TL;DR

• Multi-agent systems use specialized agents that collaborate - each agent has a role, tools, and scope; a coordinator orchestrates their work toward a shared goal.

• Specialization beats generalization - a researcher agent, writer agent, and reviewer agent outperform one agent trying to do everything.

• The orchestrator is the hard part - not the individual agents.

Delegation, state handoffs, conflict resolution, and termination logic live in the coordinator.

• Communication must be structured - free-form agent chat devolves into loops. Use typed messages, shared state, and explicit handoff protocols.

• Start with two agents, not ten - complexity scales nonlinearly. Add agents only when a single agent demonstrably fails on a subtask.

Why This Matters

Single agents hit capability ceilings. A coding agent that's also a security reviewer, test writer, and deployment manager accumulates too many tools, too much context, and too many responsibilities. Error rates climb. Costs spike. Debugging becomes impossible.

Multi-agent systems mirror how engineering teams work: specialists collaborate through defined interfaces. A tech lead decomposes work, assigns it to engineers with relevant expertise, reviews output, and integrates results. The same pattern applies to AI agents - with the added challenge that your "team members" are non-deterministic LLM calls.

This matters because the industry is moving toward multi-agent architectures for complex tasks: software engineering (Cursor, Devin), research (multi-step analysis), customer operations (triage → resolution → follow-up), and data analysis (query → visualize → interpret). Understanding orchestration patterns prevents the most common failure: a pile of agents that talk past each other until the step limit kills the run.

The Problem Multi-Agent Systems Solve

Monolithic agents fail when:

1. Tool registries exceed 10–15 tools - selection accuracy drops sharply.

2. Tasks require conflicting personas - creative writing vs. strict fact-checking need different system prompts and temperatures.

3. Parallel work is possible - research and code generation can happen simultaneously, but a single loop is sequential.

4. Quality gates need separation - the agent that writes code shouldn't be the one that approves it.

5. Context exceeds window limits - sub-agents work with focused context instead of one agent carrying everything.

Multi-agent systems decompose both the task and the cognitive load. Each agent operates in a smaller, more manageable scope with higher success rates per step.

How We Got Here

Multi-agent ideas predate LLMs. What changed is that the "agents" are now language-model policies with tools - orchestration patterns from classic MAS research meet production LLM runtimes:

Diagram: From single agent to coordinated systems

flowchart LR
    A[Classic MAS / blackboard] --> B[Microservices teams]
    B --> C[Single LLM + tools]
    C --> D[Role-specialized agents]
    D --> E[Supervisor + handoffs]
    E --> F[HITL + durable multi-agent]

Major components and how control or data moves between them.

Public references that shaped practice include Anthropic — Building effective agents (prefer simple composable patterns), LangGraph supervisor and Command APIs, Microsoft AutoGen group chat, and CrewAI role/process modes. Product surfaces such as GitHub Copilot and Cursor show specialization in the wild (edit vs review vs terminal) without requiring invented internals.

What Is a Multi-Agent System?

A multi-agent system (MAS) is an architecture where multiple LLM-powered agents - each with defined roles, tools, prompts, and permissions - collaborate under orchestration to achieve a goal none could efficiently accomplish alone.

Core components:

Multi-agent is not the same as multi-step. A single ReAct agent taking 8 steps is not a multi-agent system. Multiple distinct agent instances with separate system prompts and tool sets collaborating on a shared goal - that's multi-agent.

How Multi-Agent Systems Work

Pattern 1: Manager-Worker (Hierarchical)

A manager agent receives the goal, creates a plan, delegates subtasks to worker agents, reviews their output, and synthesizes the final result.

Best for: tasks with clear subtask boundaries, quality review requirements, and varied expertise needs.

Pattern 2: Sequential Pipeline

Agents pass output downstream - each agent transforms the previous agent's work.

Best for: ETL-like workflows, document processing chains, predictable transformation steps.

Pattern 3: Collaborative (Group Chat / Swarm)

Agents discuss in a shared conversation, contributing from their expertise until consensus or a moderator terminates.

Best for: brainstorming, code review, adversarial validation (proposer + critic). Risk: conversation loops - always need a moderator with step limits.

Pattern 4: Handoff (Peer-to-Peer)

Agents transfer control directly - when one agent reaches its scope limit, it hands off to another with context. See Multi-Agent Handoffs.

Best for: customer support, incident response, any domain with escalation paths.

Orchestration lifecycle

Diagram: State lifecycle

stateDiagram-v2
    [*] --> Supervisor
    Supervisor --> Worker: assign / handoff
    Worker --> Supervisor: result
    Supervisor --> ParallelWorkers: fan-out
    ParallelWorkers --> Supervisor: join
    Supervisor --> Review: quality gate
    Review --> Worker: reject / rework
    Review --> Done: approve
    Supervisor --> Interrupt: gated write
    Interrupt --> Supervisor: human decision
    Done --> [*]

Valid states and transitions for this control-plane pattern.

Lifecycle notes: every routing decision should be traced. Interrupt edges need a durable checkpoint so HITL approvals can arrive later without restarting the run.

Architecture

Production multi-agent architecture:

Diagram: Production agent architecture

flowchart TB
    UI[Interface / API] --> Orch[Supervisor / orchestrator]
    Orch --> W1[Specialist A]
    Orch --> W2[Specialist B]
    Orch --> W3[Specialist C]
    Orch --> State[Shared state / checkpoints]
    Orch --> Guard[Per-agent RBAC]
    Orch --> Obs[Traces / evals]
    W1 --> ToolsA[Tool subset A]
    W2 --> ToolsB[Tool subset B]
    W3 --> ToolsC[Tool subset C]

The orchestrator owns control flow; tools, memory, guardrails, and observability surround the LLM policy.

Figure: Specialized agents communicate, delegate, and coordinate under an orchestrator - the system shape of multi-agent design.
Source: LLM-based Multi-Agents survey (arXiv:2402.01680)

Critical design choices:

• Shared state over message passing for most patterns - easier to debug, less information loss.

• Orchestrator owns termination - workers should not decide when the overall task is done.

• Per-agent tool isolation - code agent gets file tools; research agent gets search tools. Not both.

Step-by-Step Flow

Task: "Research competitor pricing and write a summary report with recommendations."

1. Manager receives goal - Analyzes requirements: research (web search), analysis (compare data), writing (format report), review (fact-check).

2. Manager creates delegation plan:

   • Research Agent: find pricing for competitors A, B, C
   • Analysis Agent: compare features vs. price, identify gaps
   • Writer Agent: produce executive summary
   • Review Agent: verify claims against research data

3. Research Agent executes - Uses web_search and scrape_page tools. Writes findings to shared state under research_results.

4. Analysis Agent executes - Reads research_results from shared state. Produces comparison matrix and gap analysis. Writes to analysis.

5. Writer Agent executes - Reads research_results and analysis. Produces draft report. Writes to draft_report.

6. Review Agent executes - Reads all prior outputs. Flags unsupported claims. Returns {approved: false, issues: ["Competitor B pricing unverified"]}.

7. Manager handles rejection - Re-delegates to Research Agent: "Verify Competitor B pricing from primary source."

8. Research Agent re-runs - Updates research_results. Review Agent re-reviews. Returns {approved: true}.

9. Manager synthesizes - Delivers final report to user with agent attribution metadata.

Real Production Example

LangGraph multi-agent with supervisor pattern:

from typing import Annotated, Literal
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, AIMessage
from langgraph.graph import StateGraph, END, MessagesState
from langgraph.types import Command

# --- Specialized agents ---
research_llm = ChatOpenAI(model="gpt-4o-mini").bind_tools([web_search, read_url])
coder_llm = ChatOpenAI(model="gpt-4o-mini").bind_tools([read_file, write_file, run_tests])
writer_llm = ChatOpenAI(model="gpt-4o", temperature=0.3)

def research_agent(state: MessagesState) -> Command[Literal["supervisor"]]:
    result = research_llm.invoke([
        {"role": "system", "content": "You are a research agent. Find and summarize relevant information. Be thorough but concise."},
        *state["messages"],
    ])
    return Command(
        goto="supervisor",
        update={"messages": [AIMessage(content=f"[Research Agent]: {result.content}", name="researcher")]},
    )

def coder_agent(state: MessagesState) -> Command[Literal["supervisor"]]:
    result = coder_llm.invoke([
        {"role": "system", "content": "You are a coding agent. Write clean, tested code. Run tests before reporting done."},
        *state["messages"],
    ])
    return Command(
        goto="supervisor",
        update={"messages": [AIMessage(content=f"[Code Agent]: {result.content}", name="coder")]},
    )

def writer_agent(state: MessagesState) -> Command[Literal["supervisor"]]:
    result = writer_llm.invoke([
        {"role": "system", "content": "You are a technical writer. Synthesize findings into clear documentation."},
        *state["messages"],
    ])
    return Command(
        goto="supervisor",
        update={"messages": [AIMessage(content=f"[Writer Agent]: {result.content}", name="writer")]},
    )

# --- Supervisor ---
members = ["researcher", "coder", "writer"]
supervisor_llm = ChatOpenAI(model="gpt-4o", temperature=0)

def supervisor(state: MessagesState) -> Command[Literal["researcher", "coder", "writer", "__end__"]]:
    if len(state["messages"]) > 20:
        return Command(goto="__end__")

    response = supervisor_llm.with_structured_output(RouteDecision).invoke([
        {"role": "system", "content": f"""You are a supervisor managing: {members}.
        Route tasks to the appropriate agent. Say FINISH when the task is complete."""},
        *state["messages"],
    ])

    if response.next == "FINISH":
        return Command(goto="__end__")
    return Command(goto=response.next)

class RouteDecision(BaseModel):
    next: Literal["researcher", "coder", "writer", "FINISH"]

# --- Graph ---
graph = StateGraph(MessagesState)
graph.add_node("supervisor", supervisor)
graph.add_node("researcher", research_agent)
graph.add_node("coder", coder_agent)
graph.add_node("writer", writer_agent)
graph.set_entry_point("supervisor")
# Each worker returns to supervisor via Command

multi_agent = graph.compile()

Key patterns: supervisor owns routing and termination, workers have isolated tools and prompts, shared message state for context, hard message limit prevents runaway loops.

Design Decisions

Comparisons

Multi-agent vs single agent

Supervisor vs handoffs

Swarm / group chat vs hierarchical

Common Mistakes

1. Too many agents - Five agents for a two-step task adds coordination overhead without benefit. Match agent count to genuine specialization needs.

2. Unbounded group chat - Agents agreeing with each other in loops. Always use a moderator/supervisor with step limits.

3. Shared tool registries - Giving every agent all tools defeats specialization. Curate tools per agent.

4. Lost context on handoff - Worker agents need sufficient context from shared state, not just "continue the task."

5. No termination authority - If workers can declare the task done, you get premature completion. Only the orchestrator terminates.

6. Identical system prompts - Agents need distinct personas, constraints, and expertise descriptions to produce differentiated output.

7. Ignoring cost multiplication - Three agents × five steps each = 15 LLM calls. Budget accordingly.

Where It Breaks Down

• Ambiguous task boundaries - When it's unclear which agent should handle a subtask, supervisors route incorrectly. Define clear agent scopes in system prompts.

• Conflicting agent outputs - Research agent says X, analysis agent says not-X. Need conflict resolution rules or a tiebreaker agent.

• Latency compounding - Sequential multi-agent runs are slow. Parallelize where possible; use async execution.

• Debugging complexity - A failed 4-agent run requires tracing 4 separate LLM call chains. Invest heavily in observability.

• Overhead on simple tasks - "What's 2+2?" doesn't need a multi-agent system. Route by task complexity.

Decision tree: single agent vs multi-agent

Decision tree: When to split into multiple agents

flowchart TD
    A[One agent with 3-7 tools enough?] -->|Yes| B[Stay single-agent]
    A -->|No| C{Clear specialist roles?}
    C -->|No| B
    C -->|Yes| D[Multi-agent + handoffs]
    D --> E[Pick supervisor or peer handoffs]
    E --> F[Add HITL on writes]

Split only when specialization improves eval metrics. Prefer a supervisor when one owner must terminate the run; use peer handoffs when specialists return to a shared case.

When NOT to Use Multi-Agent Systems

Skip multi-agent when:

1. A single agent already clears evals - splitting adds cost without quality gain.
2. The task is one persona / one tool set - "look up order and summarize" does not need a crew.
3. You cannot afford coordination overhead - latency and token budgets are tight.
4. You lack shared state and traces - without checkpoints and per-agent spans, failures are un-debuggable.
5. Roles are vague - "helper" and "assistant" agents that share the same tools will talk past each other.

Prefer a single AI agent, a workflow, or a thin router that selects tools - not agents - until specialization is proven.

Running in Production

Best Practice

✅ Best Practices - Cap agent count and hops, isolate tools per role, checkpoint shared state, gate irreversible writes with HITL, and evaluate routing + end-to-end task success before widening autonomy.

Important

The supervisor/orchestrator is a single point of failure. If it misroutes or terminates early, the entire task fails. Invest in supervisor prompt quality and eval coverage.

Related Guides

• Orchestration: LangGraph (supervisor, Command, Send), CrewAI, AutoGen, OpenAI Agents SDK - compare in Best AI Agent Frameworks.

• Handoffs & durability: Multi-Agent Handoffs, Durable Execution, Human-in-the-Loop.

• Design guidance: Anthropic — Building effective agents.

• Head-to-heads: LangGraph vs CrewAI · OpenAI Agents SDK vs LangGraph

• AI Agents: Single-agent fundamentals that multi-agent systems compose.

• Multi-Agent Handoffs: Typed peer transfer and ownership.

• Human-in-the-Loop: Approval gates on privileged worker actions.

• Durable Execution: Checkpoint shared state across agents and interrupts.

• Agent Architectures: Each worker agent uses an architecture (ReAct, Plan-and-Execute).

• Agent Planning: The orchestrator's core job is planning and delegation.

• Agent Memory: Shared state and cross-agent memory coordination.

• Workflows vs Agents: Multi-agent pipelines blur into workflows - know the boundary.

• Tool Calling: Per-agent tool curation is critical for multi-agent success.

If you understood this topic, read next:

Diagram: Learning path for multi-agent design

flowchart LR
    A[Multi-agent] --> B[Handoffs]
    B --> C[HITL]
    C --> D[Durable]
    D --> E[Memory]
    E --> F[Workflows]

Prerequisites: AI Agents · Agent Architectures · Agent Planning

Next topics: Multi-Agent Handoffs · Human-in-the-Loop · Durable Execution

Estimated time: 55 min · Difficulty: Advanced

Key Takeaways

• Multi-agent systems use specialized agents under an orchestrator - specialization beats one overloaded loop.
• The supervisor owns routing, review, and termination; workers should not declare global done.
• Prefer shared state and typed handoffs over unbounded group chat.
• Start with 2–3 agents; add roles only when evals show a specific subtask failing.
• Isolate tools and prompts per agent; shared registries defeat the point.
• Cost and latency multiply - parallelize independent work and use cheaper models for workers.
• Production needs HITL, durable checkpoints, and cross-agent traces - compare stacks in Best AI Agent Frameworks.

FAQs

When should I use multi-agent vs. a single agent?

When a single agent's tool registry exceeds ~10 tools, task success rate drops on specific subtasks, or you need separation of concerns (writer vs. reviewer). Start single; split when eval data justifies it.

How many agents is too many?

More than 5 for most tasks. Each agent adds coordination overhead, cost, and failure points. If you need more, consider hierarchical structure (manager of managers).

What's the supervisor pattern?

A central orchestrator agent that receives the goal, routes subtasks to worker agents, reviews their output, and decides when the task is complete. The most common production pattern.

How do agents share information?

Three approaches: (1) shared state store (dict/DB) - best for structured data; (2) shared message history - best for conversational context; (3) explicit handoff payloads - typed objects passed on transfer.

Can agents run in parallel?

Yes, when subtasks are independent. LangGraph's Send API dispatches parallel worker nodes. The orchestrator collects results before proceeding.

How do I prevent infinite agent loops?

Hard step limits on the orchestrator, max messages in shared conversation, detection of repeated routing decisions, and timeout on total execution time.

Is multi-agent the same as CrewAI/AutoGen?

CrewAI and AutoGen are frameworks for building multi-agent systems. Multi-agent is the architectural pattern; these are implementations.

How do I test multi-agent systems?

Test each agent in isolation first (unit tests with fixed inputs). Then integration tests with the full orchestrator. Assert routing decisions, output quality per agent, and end-to-end task success.

What model should the supervisor use?

Your most capable model - routing errors cascade through the entire system. Workers can use cheaper models for their specialized tasks.

How do I handle agent disagreements?

Three strategies: (1) tiebreaker agent (reviewer/supervisor decides); (2) voting (majority wins); (3) escalate to human. Define the strategy upfront, not at runtime.

Does multi-agent replace RAG?

No. Individual agents often use RAG tools for knowledge retrieval. Multi-agent is about task decomposition; RAG is about knowledge access. See Agentic RAG.

What's the cost difference vs. single agent?

Typically 2–4× for a 3-agent system, depending on parallelism and step counts. The supervisor adds ~1 LLM call per routing decision. Measure before assuming multi-agent is worth the cost.

References

• Anthropic — Building effective agents
• LangGraph Documentation
• Microsoft AutoGen
• A Survey on Large Language Model based Autonomous Agents (arXiv:2406.00515)

Further Reading

• CrewAI Documentation
• OpenAI Agents Guide
• Best AI Agent Frameworks
• LangGraph vs CrewAI