Planning Guide

TL;DR

• Agent planning is the process of decomposing a high-level goal into ordered, executable steps that tools and sub-agents can carry out.

• Good plans are verifiable - each step has a clear success criterion, not vague instructions like "handle the issue."

• Replanning is not optional - tools fail, data surprises happen, and the first plan is often wrong. Production agents must detect failure and revise.

• Separate planning from execution - use a capable model to plan, a cheaper model (or deterministic code) to execute individual steps.

• Planning depth should match task complexity - over-planning adds latency; under-planning causes agents to wander.

Why This Matters

An agent without planning is a reactive loop - it responds to the immediate next action without a global view of the goal. That works for "look up this order ID." It fails for "prepare the quarterly board report" or "migrate this service to the new API."

Planning is what transforms an LLM from a clever autocomplete into a task executor. It's also where most of the engineering effort goes: writing planner prompts, defining step schemas, building replan triggers, and verifying completion.

If you've seen an agent that calls the same tool three times, skips a critical step, or declares success prematurely - that's a planning failure, not a model failure. Fixing planning (better decomposition, explicit verification, replan on error) often improves task success rates more than upgrading the model.

The Problem Agent Planning Solves

Complex tasks share common properties:

1. They require sequencing - step B depends on output from step A.

2. They involve conditional logic - "if refund eligible, issue refund; else escalate."

3. They span multiple capabilities - search, compute, write, notify.

4. They have partial failure modes - one step fails but the goal is still achievable via an alternate path.

A single LLM call can't reliably handle all of this. Even a ReAct loop without explicit planning rediscovers the task structure on every step, wasting tokens and increasing error rates.

Planning provides:

• A roadmap the agent (and humans) can inspect.

• Checkpoints where progress is verified before continuing.

• Recovery points where replanning begins after failure.

• Cost control by separating expensive planning from cheap execution.

How We Got Here

Planning for LLM agents reused classical AI planning ideas, then adapted them to probabilistic controllers:

Diagram: From reactive loops to explicit plans

flowchart LR
    A[Classical planners] --> B[Chain-of-thought]
    B --> C[ReAct implicit plans]
    C --> D[Plan-and-Execute]
    D --> E[Replan + verify loops]
    E --> F[HITL + durable plans]

Major components and how control or data moves between them.

Influential public work: ReAct (Yao et al.), LangChain/LangGraph Plan-and-Execute, and Anthropic — Building effective agents.

What Is Agent Planning?

Agent planning is the cognitive process - implemented via LLM calls, structured output, or classical planners - that transforms a goal G into a sequence of steps S = [s₁, s₂, ..., sₙ], where each step is actionable given available tools and current state.

Plans exist on a spectrum of formality:

Most production LLM agents use semi-structured plans - a ordered list of natural language steps with optional dependencies - because they're flexible enough for varied tasks and inspectable enough for debugging.

class TaskPlan(BaseModel):
    goal: str
    steps: list[PlanStep]

class PlanStep(BaseModel):
    id: int
    description: str
    expected_output: str
    depends_on: list[int] = []
    tool_hint: str | None = None  # optional tool suggestion

How Agent Planning Works

Phase 1: Goal Analysis

The planner receives the user goal, available tools, constraints (permissions, budget, deadline), and relevant context (memory, retrieved docs). It analyzes what's being asked and what "done" looks like.

Critical prompt element: define success criteria upfront.

Goal: Reduce API error rate for /checkout endpoint
Success criteria:
  - Identify root cause with evidence
  - Propose or implement fix
  - Verify error rate decreased in staging

Without success criteria, agents stop when they feel done, not when the task is done.

Phase 2: Decomposition

The planner breaks the goal into steps. Effective decomposition follows these rules:

• One capability per step - "Query error logs for /checkout" not "Investigate and fix errors."

• Observable outputs - each step produces verifiable data.

• Minimal dependency chains - parallelize independent steps where possible.

• Include verification steps - "Confirm fix in staging" as an explicit final step.

Phase 3: Execution Monitoring

As steps execute, the orchestrator tracks:

• Step status: pending, running, completed, failed, skipped
• Actual output vs. expected output
• Accumulated context for downstream steps

Phase 4: Replanning

Replanning triggers:

Plan → execute → verify → replan

Diagram: State lifecycle

stateDiagram-v2
    [*] --> AnalyzeGoal
    AnalyzeGoal --> CreatePlan
    CreatePlan --> ExecuteStep
    ExecuteStep --> VerifyStep
    VerifyStep --> ExecuteStep: pass / more steps
    VerifyStep --> Replan: fail
    VerifyStep --> GoalCheck: plan done
    Replan --> ExecuteStep: revised plan
    Replan --> Failed: replan cap
    GoalCheck --> Done: criteria met
    GoalCheck --> Replan: criteria miss
    Done --> [*]
    Failed --> [*]

Valid states and transitions for this control-plane pattern.

Architecture

A planning agent system has distinct components:

Diagram: Plan, execute, and replan flow

flowchart TB
    G[Goal + constraints] --> GA[Goal Analyzer]
    GA --> P[Planner]
    P --> SE[Step Executor]
    SE --> SV[Step Verifier]
    SV -->|ok| SE
    SV -->|fail| RP[Replanner]
    RP --> SE
    SV -->|done| GC[Goal Checker]
    GC --> Out[Final response]

Major components and how control or data moves between them.

• Goal Analyzer - Extracts intent, constraints, and success criteria (lightweight classifier or LLM call).
• Planner - Generates initial step list (capable model, structured output).
• Step Executor - Runs one step with tool access (cheaper model or specialized sub-agent).
• Step Verifier - Checks step output against expected criteria (rules, LLM-as-judge, or schema validation).
• Replanner - Revises remaining steps given failure context (capable model).
• Goal Checker - Final verification that success criteria are met before responding.

Figure: Explicit planning decomposes a goal into ordered steps before (and during) execution - the Plan-and-Execute family.
Source: Survey on LLM-based autonomous agents (arXiv:2406.00515)

Step-by-Step Flow

Task: "Analyze why signup conversion dropped 15% last week and recommend actions."

1. Goal analysis - Planner identifies: metric investigation, funnel analysis, hypothesis generation, recommendation. Success = data-backed root cause + actionable recommendations.

2. Initial plan generated:

   • Step 1: Pull conversion rates for current vs. prior week
   • Step 2: Segment by traffic source, device, geography
   • Step 3: Check for deployment or config changes in the period
   • Step 4: Analyze funnel drop-off by step
   • Step 5: Synthesize findings and recommend actions

3. Execute Step 1 - query_analytics tool returns: overall conversion 2.1% → 1.78% (-15.2%).

4. Verify Step 1 - Output contains numeric comparison ✓

5. Execute Step 2 - Segmentation shows mobile Safari traffic dropped 40%, other segments stable.

6. Execute Step 3 - Deployment log shows CSS change to signup form on mobile, dated day before drop started.

7. Replanner invoked? No - plan still valid. Step 4 confirms funnel drop at "submit email" on mobile Safari.

8. Execute Step 5 - Synthesize: "Mobile Safari signup form regression from CSS deploy X. Recommend: revert commit Y, add mobile E2E test."

9. Goal checker - Root cause identified with evidence ✓ Recommendations provided ✓

10. Final response delivered with plan trace attached for audit.

Real Production Example

LangGraph Plan-and-Execute with structured planning and conditional replanning:

from typing import Annotated, List, Optional
from pydantic import BaseModel, Field
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, END
from typing_extensions import TypedDict
import operator

class Step(BaseModel):
    description: str
    verification: str = Field(description="How to verify this step succeeded")

class Plan(BaseModel):
    steps: List[Step]
    success_criteria: str

class PlanState(TypedDict):
    input: str
    plan: List[Step]
    current_step: int
    step_results: Annotated[list, operator.add]
    replan_count: int
    final_response: str

planner_llm = ChatOpenAI(model="gpt-4o", temperature=0)
executor_llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

PLANNER_PROMPT = """Decompose the task into 3-7 executable steps.
Each step must have a clear verification criterion.
Include a success_criteria field describing when the overall task is complete."""

def create_plan(state: PlanState):
    plan = planner_llm.with_structured_output(Plan).invoke([
        {"role": "system", "content": PLANNER_PROMPT},
        {"role": "user", "content": state["input"]},
    ])
    return {"plan": plan.steps, "current_step": 0, "replan_count": 0}

def execute_current_step(state: PlanState):
    step = state["plan"][state["current_step"]]
    prior = "\n".join([f"Step {i+1}: {r}" for i, r in enumerate(state["step_results"])])

    result = executor_llm.bind_tools(TOOLS).invoke([
        {"role": "system", "content": "Execute exactly one plan step. Use tools as needed."},
        {"role": "user", "content": f"Task: {state['input']}\nStep: {step.description}\nVerify by: {step.verification}\nPrior results:\n{prior}"},
    ])
    # Tool execution loop omitted - assume result.content holds observation
    return {"step_results": [result.content], "current_step": state["current_step"] + 1}

def verify_step(state: PlanState) -> str:
    step_idx = state["current_step"] - 1
    step = state["plan"][step_idx]
    result = state["step_results"][-1]

    if "ERROR" in result or "FAILED" in result:
        return "replan"
    if step_idx + 1 >= len(state["plan"]):
        return "finish"
    return "continue"

def replan(state: PlanState):
    if state["replan_count"] >= 2:
        return {"final_response": "Unable to complete task after multiple replans."}

new_plan = planner_llm.with_structured_output(Plan).invoke([
        {"role": "user", "content": f"""
            Original task: {state['input']}
            Completed steps: {state['step_results']}
            Failed at step {state['current_step']}: {state['plan'][state['current_step']-1].description}
            Create a revised plan for remaining work."""},
    ])
    return {
        "plan": new_plan.steps,
        "current_step": 0,
        "step_results": [],
        "replan_count": state["replan_count"] + 1,
    }

def finish(state: PlanState):
    response = planner_llm.invoke([
        {"role": "user", "content": f"Synthesize final answer.\nTask: {state['input']}\nResults: {state['step_results']}"},
    ])
    return {"final_response": response.content}

# Build graph: create_plan → execute → verify → (continue|replan|finish)

This pattern encodes the core production requirements: structured plans, per-step verification, bounded replanning, and final synthesis.

Design Decisions

Comparisons

Plan-and-Execute vs ReAct

Use ReAct for short reactive tasks; Plan-and-Execute when humans or compliance need to see the plan - see Agent Architectures.

Planning vs workflows

Heavy planning often becomes a workflow. Prefer Workflows vs Agents when the graph is stable; keep LLM planning for branching that depends on observations.

Common Mistakes

1. Vague steps - "Investigate the issue" is not a step. "Query CloudWatch for ERROR logs on /checkout in the last 24h" is.

2. No success criteria - Without defining "done," agents exit early or loop forever.

3. Planning with the execution model - Using GPT-4o for every step execution when a mini model suffices doubles cost.

4. Replanning without context - Replanner must see completed steps, failure reason, and original goal - not just "step 3 failed."

5. Skipping verification - Assuming tool success means step success. A SQL query that returns empty results isn't an error - but it might mean the step failed its intent.

6. Over-decomposition - 20 micro-steps add latency and compound error rates. 5–7 well-scoped steps is the sweet spot for most tasks.

7. Static plans for dynamic environments - Real-time data, user interactions, and external events require replanning capability.

Where It Breaks Down

• Non-decomposable goals - Creative tasks ("write a novel") don't decompose cleanly into verifiable steps. Planning adds overhead without benefit.

• Tight feedback loops - Interactive debugging where each step depends on human input resists upfront planning. Use ReAct instead.

• Plan drift - Long plans where early context becomes irrelevant to later steps. Periodic re-planning (every 3–5 steps) helps.

• Verification brittleness - LLM-as-judge verifiers inherit model biases. Combine with rule-based checks.

• Combinatorial explosion - Conditional plans with many branches are better as decision trees or workflows, not flat step lists.

Decision tree: plan first or react?

Decision tree: When to add explicit planning

flowchart TD
    A[Task has 5+ dependent steps?] -->|No| B[ReAct may be enough]
    A -->|Yes| C{Need a reviewable step list?}
    C -->|Yes| D[Plan-and-Execute]
    C -->|No| B
    D --> E{Steps fail often?}
    E -->|Yes| F[Add verifier + replanner]
    E -->|No| G[Execute plan with step limits]
    F --> G

Explicit plans help long-horizon tasks and audits - they add cost when a short ReAct loop already succeeds.

When NOT to Use Agent Planning

Skip explicit planning when:

1. The task is 1–3 tool calls - ReAct thought traces are enough.
2. The path is fully known - use a workflow/DAG (Workflows vs Agents).
3. The goal is creative/non-decomposable - "write a novel" resists verifiable steps.
4. Every step needs human input - interactive debugging prefers ReAct + HITL, not a long upfront plan.
5. You cannot define success criteria - without "done," planning adds ceremony without termination.

Running in Production

Best Practice

✅ Best Practices - Stream the plan early, verify every step, bound replan attempts, and score plan quality separately from execution on a golden task set.

Important

Always cap replan attempts. Unbounded replanning is an infinite loop with a credit card attached.

Related Guides

• LangGraph: Plan-and-Execute tutorials, Send API for parallel step execution, checkpointing for long plans - see Best AI Agent Frameworks.

• LangChain: LCEL for custom planning chains.

• AutoGen / CrewAI: Manager or hierarchical task assignment - compare LangGraph vs CrewAI.

• DSPy: Programmatic prompt optimization for planner modules.

• AI Agents: Planning is a core capability within agent systems.

• Agent Architectures: Plan-and-Execute is the primary architecture for explicit planning.

• Workflows vs Agents: Heavy planning blurs into workflows - know where to draw the line.

• Multi-Agent Systems / Multi-Agent Handoffs: Complex plans delegate steps to specialists.

• Agent Memory: Past plans and outcomes inform future planning.

• Durable Execution: Persist plan progress across crashes and approvals.

• Structured Outputs: Plans should be structured (Pydantic/JSON schema) for reliable parsing.

• LLM Evaluation: Plan quality and goal verification require eval frameworks.

If you understood this topic, read next:

Diagram: Learning path for agent planning

flowchart LR
    A[Agents] --> B[Architectures]
    B --> C[Planning]
    C --> D[Memory]
    D --> E[Durable]
    E --> F[Handoffs]

Prerequisites: AI Agents · Agent Architectures · Prompt Engineering

Next topics: Multi-Agent Systems · Workflows vs Agents · Durable Execution

Estimated time: 50 min · Difficulty: Advanced

Key Takeaways

• Agent planning decomposes goals into verifiable steps with explicit success criteria.
• Separate planning (capable model) from execution (cheap model or code) for cost and reliability.
• Replanning is essential - cap attempts and give the replanner full failure context.
• Good steps are specific, observable, and scoped to one capability.
• Verify each step and the overall goal before responding to the user.
• 3–7 steps is the sweet spot; over-decomposition adds latency and error rates.
• Prefer workflows for known paths; reserve Plan-and-Execute for open-ended branching.

FAQs

Do all agents need explicit planning?

No. Simple tasks (1–3 tool calls) work fine with ReAct's implicit planning via thought traces. Explicit planning pays off for tasks with 4+ steps or conditional logic.

How many steps should a plan have?

3–7 for most tasks. Fewer than 3 suggests the task might not need planning. More than 10 suggests you need hierarchical decomposition with sub-agents.

What's the difference between planning and chain-of-thought?

Chain-of-thought is inline reasoning within a single response. Planning produces a persistent, inspectable step list that an orchestrator executes over multiple LLM calls.

How do I verify a step succeeded?

Three approaches: (1) rule-based - check output format, non-empty results, expected fields; (2) LLM-as-judge - "Did this step achieve: {verification criterion}?"; (3) tool-level - HTTP 200, SQL rows returned. Combine rules + LLM for best results.

When should the replanner rewrite the entire plan vs. remaining steps?

Rewrite remaining steps when the failure is local (bad query, wrong parameter). Full replan when a core assumption is invalidated (wrong root cause hypothesis, wrong target system).

Can planning be deterministic?

Partially. Template-based plans for recurring tasks (onboarding, incident triage) can be mostly deterministic with LLM filling in parameters. Novel tasks need LLM-generated plans.

How does planning interact with human-in-the-loop?

Show the plan for approval before execution on high-stakes tasks. Allow humans to edit steps. Pause between steps for review if needed. LangGraph interrupt nodes support this.

What models are best for planning?

Use your most capable available model (GPT-4o, Claude Sonnet, Gemini Pro). Planning errors cascade through every subsequent step - this is not where you save on model cost.

How do I handle parallel steps?

Define steps with explicit dependency lists. Steps with no unmet dependencies can execute concurrently using LangGraph's Send API or async tool execution.

Can I use classical planners (STRIPS, PDDL) with LLMs?

Yes, in hybrid systems. LLM translates natural language goal to PDDL, classical planner generates optimal plan, LLM executes steps. Works well in robotics and structured domains; less common in knowledge work.

How do I evaluate planning quality independently from execution?

Create a test set with expert plans. Measure plan similarity (step overlap, ordering), step granularity, and whether the plan would succeed if executed perfectly (plan correctness).

What happens when the plan is right but execution fails?

That's an execution problem, not a planning problem. Fix tool reliability, executor prompts, or step verification. Don't replan if the plan itself is correct.

References

• ReAct: Synergizing Reasoning and Acting (Yao et al., 2022)
• LangGraph Documentation
• Anthropic — Building effective agents

Further Reading

• LangChain Agents Documentation
• OpenAI Agents Guide
• Microsoft AutoGen