Multi-Agent Systems Guide

TL;DR

• Multi-agent systems use specialized agents that collaborate - each agent has a role, tools, and scope; a coordinator orchestrates their work toward a shared goal.

• Specialization beats generalization - a researcher agent, writer agent, and reviewer agent outperform one agent trying to do everything.

• The orchestrator is the hard part - not the individual agents.

Delegation, state handoffs, conflict resolution, and termination logic live in the coordinator.

• Communication must be structured - free-form agent chat devolves into loops. Use typed messages, shared state, and explicit handoff protocols.

• Start with two agents, not ten - complexity scales nonlinearly. Add agents only when a single agent demonstrably fails on a subtask.

Why This Matters

Single agents hit capability ceilings. A coding agent that's also a security reviewer, test writer, and deployment manager accumulates too many tools, too much context, and too many responsibilities. Error rates climb. Costs spike. Debugging becomes impossible.

Multi-agent systems mirror how engineering teams work: specialists collaborate through defined interfaces. A tech lead decomposes work, assigns it to engineers with relevant expertise, reviews output, and integrates results. The same pattern applies to AI agents - with the added challenge that your "team members" are non-deterministic LLM calls.

This matters because the industry is moving toward multi-agent architectures for complex tasks: software engineering (Cursor, Devin), research (multi-step analysis), customer operations (triage → resolution → follow-up), and data analysis (query → visualize → interpret). Understanding orchestration patterns prevents the most common failure: a pile of agents that talk past each other until the step limit kills the run.

The Problem Multi-Agent Systems Solve

Monolithic agents fail when:

1. Tool registries exceed 10–15 tools - selection accuracy drops sharply.

2. Tasks require conflicting personas - creative writing vs. strict fact-checking need different system prompts and temperatures.

3. Parallel work is possible - research and code generation can happen simultaneously, but a single loop is sequential.

4. Quality gates need separation - the agent that writes code shouldn't be the one that approves it.

5. Context exceeds window limits - sub-agents work with focused context instead of one agent carrying everything.

Multi-agent systems decompose both the task and the cognitive load. Each agent operates in a smaller, more manageable scope with higher success rates per step.

What Is a Multi-Agent System?

A multi-agent system (MAS) is an architecture where multiple LLM-powered agents - each with defined roles, tools, prompts, and permissions - collaborate under orchestration to achieve a goal none could efficiently accomplish alone.

Core components:

Multi-agent is not the same as multi-step. A single ReAct agent taking 8 steps is not a multi-agent system. Multiple distinct agent instances with separate system prompts and tool sets collaborating on a shared goal - that's multi-agent.

How Multi-Agent Systems Work

Pattern 1: Manager-Worker (Hierarchical)

A manager agent receives the goal, creates a plan, delegates subtasks to worker agents, reviews their output, and synthesizes the final result.

Multi-agent systems assign specialized roles to multiple LLM agents that communicate, delegate, and coordinate to solve complex tasks.

Best for: tasks with clear subtask boundaries, quality review requirements, and varied expertise needs.

Pattern 2: Sequential Pipeline

Agents pass output downstream - each agent transforms the previous agent's work.

Agents exchange messages through defined communication channels, enabling decomposition of workflows across planner, executor, and critic roles.

Best for: ETL-like workflows, document processing chains, predictable transformation steps.

Pattern 3: Collaborative (Group Chat)

Agents discuss in a shared conversation, contributing from their expertise until consensus or a moderator terminates.

Planning agents break goals into sub-tasks, select tools, and revise plans based on intermediate observations.

Best for: brainstorming, code review, adversarial validation (proposer + critic). Risk: conversation loops - always need a moderator with step limits.

Pattern 4: Handoff (Peer-to-Peer)

Agents transfer control directly - when one agent reaches its scope limit, it hands off to another with context.

Agents exchange messages through defined communication channels, enabling decomposition of workflows across planner, executor, and critic roles.

Best for: customer support, incident response, any domain with escalation paths.

Architecture

Production multi-agent architecture:

Planning agents break goals into sub-tasks, select tools, and revise plans based on intermediate observations.

Source: Research paper

Critical design choices:

• Shared state over message passing for most patterns - easier to debug, less information loss.

• Orchestrator owns termination - workers should not decide when the overall task is done.

• Per-agent tool isolation - code agent gets file tools; research agent gets search tools. Not both.

Step-by-Step Flow

Task: "Research competitor pricing and write a summary report with recommendations."

1. Manager receives goal - Analyzes requirements: research (web search), analysis (compare data), writing (format report), review (fact-check).

2. Manager creates delegation plan:

   • Research Agent: find pricing for competitors A, B, C
   • Analysis Agent: compare features vs. price, identify gaps
   • Writer Agent: produce executive summary
   • Review Agent: verify claims against research data

3. Research Agent executes - Uses web_search and scrape_page tools. Writes findings to shared state under research_results.

4. Analysis Agent executes - Reads research_results from shared state. Produces comparison matrix and gap analysis. Writes to analysis.

5. Writer Agent executes - Reads research_results and analysis. Produces draft report. Writes to draft_report.

6. Review Agent executes - Reads all prior outputs. Flags unsupported claims. Returns {approved: false, issues: ["Competitor B pricing unverified"]}.

7. Manager handles rejection - Re-delegates to Research Agent: "Verify Competitor B pricing from primary source."

8. Research Agent re-runs - Updates research_results. Review Agent re-reviews. Returns {approved: true}.

9. Manager synthesizes - Delivers final report to user with agent attribution metadata.

Real Production Example

LangGraph multi-agent with supervisor pattern:

from typing import Annotated, Literal
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, AIMessage
from langgraph.graph import StateGraph, END, MessagesState
from langgraph.types import Command

# --- Specialized agents ---
research_llm = ChatOpenAI(model="gpt-4o-mini").bind_tools([web_search, read_url])
coder_llm = ChatOpenAI(model="gpt-4o-mini").bind_tools([read_file, write_file, run_tests])
writer_llm = ChatOpenAI(model="gpt-4o", temperature=0.3)

def research_agent(state: MessagesState) -> Command[Literal["supervisor"]]:
    result = research_llm.invoke([
        {"role": "system", "content": "You are a research agent. Find and summarize relevant information. Be thorough but concise."},
        *state["messages"],
    ])
    return Command(
        goto="supervisor",
        update={"messages": [AIMessage(content=f"[Research Agent]: {result.content}", name="researcher")]},
    )

def coder_agent(state: MessagesState) -> Command[Literal["supervisor"]]:
    result = coder_llm.invoke([
        {"role": "system", "content": "You are a coding agent. Write clean, tested code. Run tests before reporting done."},
        *state["messages"],
    ])
    return Command(
        goto="supervisor",
        update={"messages": [AIMessage(content=f"[Code Agent]: {result.content}", name="coder")]},
    )

def writer_agent(state: MessagesState) -> Command[Literal["supervisor"]]:
    result = writer_llm.invoke([
        {"role": "system", "content": "You are a technical writer. Synthesize findings into clear documentation."},
        *state["messages"],
    ])
    return Command(
        goto="supervisor",
        update={"messages": [AIMessage(content=f"[Writer Agent]: {result.content}", name="writer")]},
    )

# --- Supervisor ---
members = ["researcher", "coder", "writer"]
supervisor_llm = ChatOpenAI(model="gpt-4o", temperature=0)

def supervisor(state: MessagesState) -> Command[Literal["researcher", "coder", "writer", "__end__"]]:
    if len(state["messages"]) > 20:
        return Command(goto="__end__")

    response = supervisor_llm.with_structured_output(RouteDecision).invoke([
        {"role": "system", "content": f"""You are a supervisor managing: {members}.
        Route tasks to the appropriate agent. Say FINISH when the task is complete."""},
        *state["messages"],
    ])

    if response.next == "FINISH":
        return Command(goto="__end__")
    return Command(goto=response.next)

class RouteDecision(BaseModel):
    next: Literal["researcher", "coder", "writer", "FINISH"]

# --- Graph ---
graph = StateGraph(MessagesState)
graph.add_node("supervisor", supervisor)
graph.add_node("researcher", research_agent)
graph.add_node("coder", coder_agent)
graph.add_node("writer", writer_agent)
graph.set_entry_point("supervisor")
# Each worker returns to supervisor via Command

multi_agent = graph.compile()

Key patterns: supervisor owns routing and termination, workers have isolated tools and prompts, shared message state for context, hard message limit prevents runaway loops.

Design Decisions

⚠ Common Mistakes

1. Too many agents - Five agents for a two-step task adds coordination overhead without benefit. Match agent count to genuine specialization needs.

2. Unbounded group chat - Agents agreeing with each other in loops. Always use a moderator/supervisor with step limits.

3. Shared tool registries - Giving every agent all tools defeats specialization. Curate tools per agent.

4. Lost context on handoff - Worker agents need sufficient context from shared state, not just "continue the task."

5. No termination authority - If workers can declare the task done, you get premature completion. Only the orchestrator terminates.

6. Identical system prompts - Agents need distinct personas, constraints, and expertise descriptions to produce differentiated output.

7. Ignoring cost multiplication - Three agents × five steps each = 15 LLM calls. Budget accordingly.

Where It Breaks Down

• Ambiguous task boundaries - When it's unclear which agent should handle a subtask, supervisors route incorrectly. Define clear agent scopes in system prompts.

• Conflicting agent outputs - Research agent says X, analysis agent says not-X. Need conflict resolution rules or a tiebreaker agent.

• Latency compounding - Sequential multi-agent runs are slow. Parallelize where possible; use async execution.

• Debugging complexity - A failed 4-agent run requires tracing 4 separate LLM call chains. Invest heavily in observability.

• Overhead on simple tasks - "What's 2+2?" doesn't need a multi-agent system. Route by task complexity.

Running in Production

Best Practice

✅ Best Practices - Instrument every stage, version embedding models, enforce access control at retrieval time, and evaluate on a fixed golden set before shipping changes.

Important

The supervisor/orchestrator is a single point of failure. If it misroutes or terminates early, the entire task fails. Invest in supervisor prompt quality and eval coverage.

Ecosystem

• LangGraph: Supervisor pattern, Command API for handoffs, parallel Send, subgraph agents.

• AutoGen: Group chat, sequential chats, nested chats with Microsoft backing.

• CrewAI: Role-based agents with task assignment and sequential/hierarchical process modes.

• OpenAI Agents SDK: Handoff-based multi-agent with built-in guardrails.

• MetaGPT: Software engineering multi-agent (PM, architect, engineer, QA) - opinionated but illustrative.

Related Technologies

• AI Agents: Single-agent fundamentals that multi-agent systems compose.

• Agent Architectures: Each worker agent uses an architecture (ReAct, Plan-and-Execute).

• Agent Planning: The orchestrator's core job is planning and delegation.

• Agent Memory: Shared state and cross-agent memory coordination.

• Workflows vs Agents: Multi-agent pipelines blur into workflows - know the boundary.

• Tool Calling: Per-agent tool curation is critical for multi-agent success.

• Observability: Multi-agent debugging requires step-level tracing across agents.

Learning Path

Prerequisites: AI Agents · Agent Architectures · Agent Planning

Next topics: Workflows vs Agents · Agentic RAG · Agent Memory

Estimated time: 55 min · Difficulty: Advanced

FAQs

When should I use multi-agent vs. a single agent?

When a single agent's tool registry exceeds ~10 tools, task success rate drops on specific subtasks, or you need separation of concerns (writer vs. reviewer). Start single; split when eval data justifies it.

How many agents is too many?

More than 5 for most tasks. Each agent adds coordination overhead, cost, and failure points. If you need more, consider hierarchical structure (manager of managers).

What's the supervisor pattern?

A central orchestrator agent that receives the goal, routes subtasks to worker agents, reviews their output, and decides when the task is complete. The most common production pattern.

How do agents share information?

Three approaches: (1) shared state store (dict/DB) - best for structured data; (2) shared message history - best for conversational context; (3) explicit handoff payloads - typed objects passed on transfer.

Can agents run in parallel?

Yes, when subtasks are independent. LangGraph's Send API dispatches parallel worker nodes. The orchestrator collects results before proceeding.

How do I prevent infinite agent loops?

Hard step limits on the orchestrator, max messages in shared conversation, detection of repeated routing decisions, and timeout on total execution time.

Is multi-agent the same as CrewAI/AutoGen?

CrewAI and AutoGen are frameworks for building multi-agent systems. Multi-agent is the architectural pattern; these are implementations.

How do I test multi-agent systems?

Test each agent in isolation first (unit tests with fixed inputs). Then integration tests with the full orchestrator. Assert routing decisions, output quality per agent, and end-to-end task success.

What model should the supervisor use?

Your most capable model - routing errors cascade through the entire system. Workers can use cheaper models for their specialized tasks.

How do I handle agent disagreements?

Three strategies: (1) tiebreaker agent (reviewer/supervisor decides); (2) voting (majority wins); (3) escalate to human. Define the strategy upfront, not at runtime.

Does multi-agent replace RAG?

No. Individual agents often use RAG tools for knowledge retrieval. Multi-agent is about task decomposition; RAG is about knowledge access. See Agentic RAG.

What's the cost difference vs. single agent?

Typically 2–4× for a 3-agent system, depending on parallelism and step counts. The supervisor adds ~1 LLM call per routing decision. Measure before assuming multi-agent is worth the cost.

References

• ReAct: Synergizing Reasoning and Acting (Yao et al., 2022)
• LangChain Agents Documentation
• OpenAI Agents Guide

Further Reading

• Anthropic Tool Use Documentation
• LangGraph Documentation
• Microsoft AutoGen

Summary

• Multi-agent systems use specialized agents collaborating under an orchestrator - mirroring engineering team structure. - Specialization improves per-step success rates by reducing tools, context, and cognitive load per agent. - The supervisor/orchestrator pattern is the most reliable production architecture - it owns routing, review, and termination. - Communication via shared state is more debuggable than free-form agent chat.

• Start with 2–3 agents, not 10. Add agents only when eval data shows a specific subtask failing. - Per-agent tool isolation and distinct system prompts are non-negotiable for effective specialization. - Multi-agent multiplies cost and latency - parallelize independent work and use cheap models for workers. - Invest in observability across agents - debugging multi-agent failures without traces is guesswork.