Planning Guide

TL;DR

• Agent planning is the process of decomposing a high-level goal into ordered, executable steps that tools and sub-agents can carry out.

• Good plans are verifiable - each step has a clear success criterion, not vague instructions like "handle the issue."

• Replanning is not optional - tools fail, data surprises happen, and the first plan is often wrong. Production agents must detect failure and revise.

• Separate planning from execution - use a capable model to plan, a cheaper model (or deterministic code) to execute individual steps.

• Planning depth should match task complexity - over-planning adds latency; under-planning causes agents to wander.

Why This Matters

An agent without planning is a reactive loop - it responds to the immediate next action without a global view of the goal. That works for "look up this order ID." It fails for "prepare the quarterly board report" or "migrate this service to the new API."

Planning is what transforms an LLM from a clever autocomplete into a task executor. It's also where most of the engineering effort goes: writing planner prompts, defining step schemas, building replan triggers, and verifying completion.

If you've seen an agent that calls the same tool three times, skips a critical step, or declares success prematurely - that's a planning failure, not a model failure. Fixing planning (better decomposition, explicit verification, replan on error) often improves task success rates more than upgrading the model.

The Problem Agent Planning Solves

Complex tasks share common properties:

1. They require sequencing - step B depends on output from step A.

2. They involve conditional logic - "if refund eligible, issue refund; else escalate."

3. They span multiple capabilities - search, compute, write, notify.

4. They have partial failure modes - one step fails but the goal is still achievable via an alternate path.

A single LLM call can't reliably handle all of this. Even a ReAct loop without explicit planning rediscovers the task structure on every step, wasting tokens and increasing error rates.

Planning provides:

• A roadmap the agent (and humans) can inspect.

• Checkpoints where progress is verified before continuing.

• Recovery points where replanning begins after failure.

• Cost control by separating expensive planning from cheap execution.

What Is Agent Planning?

Agent planning is the cognitive process - implemented via LLM calls, structured output, or classical planners - that transforms a goal G into a sequence of steps S = [s₁, s₂, ..., sₙ], where each step is actionable given available tools and current state.

Plans exist on a spectrum of formality:

Most production LLM agents use semi-structured plans - a ordered list of natural language steps with optional dependencies - because they're flexible enough for varied tasks and inspectable enough for debugging.

class TaskPlan(BaseModel):
    goal: str
    steps: list[PlanStep]

class PlanStep(BaseModel):
    id: int
    description: str
    expected_output: str
    depends_on: list[int] = []
    tool_hint: str | None = None  # optional tool suggestion

How Agent Planning Works

Phase 1: Goal Analysis

The planner receives the user goal, available tools, constraints (permissions, budget, deadline), and relevant context (memory, retrieved docs). It analyzes what's being asked and what "done" looks like.

Critical prompt element: define success criteria upfront.

Goal: Reduce API error rate for /checkout endpoint
Success criteria:
  - Identify root cause with evidence
  - Propose or implement fix
  - Verify error rate decreased in staging

Without success criteria, agents stop when they feel done, not when the task is done.

Phase 2: Decomposition

The planner breaks the goal into steps. Effective decomposition follows these rules:

• One capability per step - "Query error logs for /checkout" not "Investigate and fix errors."

• Observable outputs - each step produces verifiable data.

• Minimal dependency chains - parallelize independent steps where possible.

• Include verification steps - "Confirm fix in staging" as an explicit final step.

Phase 3: Execution Monitoring

As steps execute, the orchestrator tracks:

• Step status: pending, running, completed, failed, skipped
• Actual output vs. expected output
• Accumulated context for downstream steps

Phase 4: Replanning

Replanning triggers:

Planning agents break goals into sub-tasks, select tools, and revise plans based on intermediate observations.

Architecture

A planning agent system has five components:

ReAct interleaves reasoning traces and tool actions: the model thinks about what to do, calls a tool, observes the result, and repeats until it can answer.

Source: Google Research

• Goal Analyzer - Extracts intent, constraints, and success criteria (can be a lightweight classifier or LLM call).

• Planner - Generates initial step list (capable model, structured output).

• Step Executor - Runs one step, typically with tool access (cheaper model or specialized sub-agent).

• Step Verifier - Checks step output against expected criteria (rules, LLM-as-judge, or schema validation).

• Replanner - Revises remaining steps given failure context (capable model).

• Goal Checker - Final verification that success criteria are met before responding to user.

Step-by-Step Flow

Task: "Analyze why signup conversion dropped 15% last week and recommend actions."

1. Goal analysis - Planner identifies: metric investigation, funnel analysis, hypothesis generation, recommendation. Success = data-backed root cause + actionable recommendations.

2. Initial plan generated:

   • Step 1: Pull conversion rates for current vs. prior week
   • Step 2: Segment by traffic source, device, geography
   • Step 3: Check for deployment or config changes in the period
   • Step 4: Analyze funnel drop-off by step
   • Step 5: Synthesize findings and recommend actions

3. Execute Step 1 - query_analytics tool returns: overall conversion 2.1% → 1.78% (-15.2%).

4. Verify Step 1 - Output contains numeric comparison ✓

5. Execute Step 2 - Segmentation shows mobile Safari traffic dropped 40%, other segments stable.

6. Execute Step 3 - Deployment log shows CSS change to signup form on mobile, dated day before drop started.

7. Replanner invoked? No - plan still valid. Step 4 confirms funnel drop at "submit email" on mobile Safari.

8. Execute Step 5 - Synthesize: "Mobile Safari signup form regression from CSS deploy X. Recommend: revert commit Y, add mobile E2E test."

9. Goal checker - Root cause identified with evidence ✓ Recommendations provided ✓

10. Final response delivered with plan trace attached for audit.

Real Production Example

LangGraph Plan-and-Execute with structured planning and conditional replanning:

from typing import Annotated, List, Optional
from pydantic import BaseModel, Field
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, END
from typing_extensions import TypedDict
import operator

class Step(BaseModel):
    description: str
    verification: str = Field(description="How to verify this step succeeded")

class Plan(BaseModel):
    steps: List[Step]
    success_criteria: str

class PlanState(TypedDict):
    input: str
    plan: List[Step]
    current_step: int
    step_results: Annotated[list, operator.add]
    replan_count: int
    final_response: str

planner_llm = ChatOpenAI(model="gpt-4o", temperature=0)
executor_llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

PLANNER_PROMPT = """Decompose the task into 3-7 executable steps.
Each step must have a clear verification criterion.
Include a success_criteria field describing when the overall task is complete."""

def create_plan(state: PlanState):
    plan = planner_llm.with_structured_output(Plan).invoke([
        {"role": "system", "content": PLANNER_PROMPT},
        {"role": "user", "content": state["input"]},
    ])
    return {"plan": plan.steps, "current_step": 0, "replan_count": 0}

def execute_current_step(state: PlanState):
    step = state["plan"][state["current_step"]]
    prior = "\n".join([f"Step {i+1}: {r}" for i, r in enumerate(state["step_results"])])

    result = executor_llm.bind_tools(TOOLS).invoke([
        {"role": "system", "content": "Execute exactly one plan step. Use tools as needed."},
        {"role": "user", "content": f"Task: {state['input']}\nStep: {step.description}\nVerify by: {step.verification}\nPrior results:\n{prior}"},
    ])
    # Tool execution loop omitted - assume result.content holds observation
    return {"step_results": [result.content], "current_step": state["current_step"] + 1}

def verify_step(state: PlanState) -> str:
    step_idx = state["current_step"] - 1
    step = state["plan"][step_idx]
    result = state["step_results"][-1]

    if "ERROR" in result or "FAILED" in result:
        return "replan"
    if step_idx + 1 >= len(state["plan"]):
        return "finish"
    return "continue"

def replan(state: PlanState):
    if state["replan_count"] >= 2:
        return {"final_response": "Unable to complete task after multiple replans."}

new_plan = planner_llm.with_structured_output(Plan).invoke([
        {"role": "user", "content": f"""
            Original task: {state['input']}
            Completed steps: {state['step_results']}
            Failed at step {state['current_step']}: {state['plan'][state['current_step']-1].description}
            Create a revised plan for remaining work."""},
    ])
    return {
        "plan": new_plan.steps,
        "current_step": 0,
        "step_results": [],
        "replan_count": state["replan_count"] + 1,
    }

def finish(state: PlanState):
    response = planner_llm.invoke([
        {"role": "user", "content": f"Synthesize final answer.\nTask: {state['input']}\nResults: {state['step_results']}"},
    ])
    return {"final_response": response.content}

# Build graph: create_plan → execute → verify → (continue|replan|finish)

This pattern encodes the core production requirements: structured plans, per-step verification, bounded replanning, and final synthesis.

Design Decisions

⚠ Common Mistakes

1. Vague steps - "Investigate the issue" is not a step. "Query CloudWatch for ERROR logs on /checkout in the last 24h" is.

2. No success criteria - Without defining "done," agents exit early or loop forever.

3. Planning with the execution model - Using GPT-4o for every step execution when a mini model suffices doubles cost.

4. Replanning without context - Replanner must see completed steps, failure reason, and original goal - not just "step 3 failed."

5. Skipping verification - Assuming tool success means step success. A SQL query that returns empty results isn't an error - but it might mean the step failed its intent.

6. Over-decomposition - 20 micro-steps add latency and compound error rates. 5–7 well-scoped steps is the sweet spot for most tasks.

7. Static plans for dynamic environments - Real-time data, user interactions, and external events require replanning capability.

Where It Breaks Down

• Non-decomposable goals - Creative tasks ("write a novel") don't decompose cleanly into verifiable steps. Planning adds overhead without benefit.

• Tight feedback loops - Interactive debugging where each step depends on human input resists upfront planning. Use ReAct instead.

• Plan drift - Long plans where early context becomes irrelevant to later steps. Periodic re-planning (every 3–5 steps) helps.

• Verification brittleness - LLM-as-judge verifiers inherit model biases. Combine with rule-based checks.

• Combinatorial explosion - Conditional plans with many branches are better as decision trees or workflows, not flat step lists.

Running in Production

Best Practice

✅ Best Practices - Instrument every stage, version embedding models, enforce access control at retrieval time, and evaluate on a fixed golden set before shipping changes.

Important

Always cap replan attempts. Unbounded replanning is an infinite loop with a credit card attached.

Ecosystem

• LangGraph: Plan-and-Execute tutorials, Send API for parallel step execution, checkpointing for long plans.

• LangChain: PlanAndExecute agent (legacy), LCEL for custom planning chains.

• AutoGen: Group chat planning with manager agent assigning tasks.

• CrewAI: Task lists with sequential and hierarchical execution.

• DSPy: Programmatic prompt optimization for planner modules.

Related Technologies

• AI Agents: Planning is a core capability within agent systems.

• Agent Architectures: Plan-and-Execute is the primary architecture for explicit planning.

• Workflows vs Agents: Heavy planning blurs into workflows - know where to draw the line.

• Multi-Agent Systems: Complex plans delegate steps to specialized sub-agents.

• Agent Memory: Past plans and outcomes inform future planning.

• Structured Outputs: Plans should be structured (Pydantic/JSON schema) for reliable parsing.

• LLM Evaluation: Plan quality and goal verification require eval frameworks.

Learning Path

Prerequisites: AI Agents · Agent Architectures · Prompt Engineering

Next topics: Multi-Agent Systems · Workflows vs Agents · Agent Memory

Estimated time: 50 min · Difficulty: Advanced

FAQs

Do all agents need explicit planning?

No. Simple tasks (1–3 tool calls) work fine with ReAct's implicit planning via thought traces. Explicit planning pays off for tasks with 4+ steps or conditional logic.

How many steps should a plan have?

3–7 for most tasks. Fewer than 3 suggests the task might not need planning. More than 10 suggests you need hierarchical decomposition with sub-agents.

What's the difference between planning and chain-of-thought?

Chain-of-thought is inline reasoning within a single response. Planning produces a persistent, inspectable step list that an orchestrator executes over multiple LLM calls.

How do I verify a step succeeded?

Three approaches: (1) rule-based - check output format, non-empty results, expected fields; (2) LLM-as-judge - "Did this step achieve: {verification criterion}?"; (3) tool-level - HTTP 200, SQL rows returned. Combine rules + LLM for best results.

When should the replanner rewrite the entire plan vs. remaining steps?

Rewrite remaining steps when the failure is local (bad query, wrong parameter). Full replan when a core assumption is invalidated (wrong root cause hypothesis, wrong target system).

Can planning be deterministic?

Partially. Template-based plans for recurring tasks (onboarding, incident triage) can be mostly deterministic with LLM filling in parameters. Novel tasks need LLM-generated plans.

How does planning interact with human-in-the-loop?

Show the plan for approval before execution on high-stakes tasks. Allow humans to edit steps. Pause between steps for review if needed. LangGraph interrupt nodes support this.

What models are best for planning?

Use your most capable available model (GPT-4o, Claude Sonnet, Gemini Pro). Planning errors cascade through every subsequent step - this is not where you save on model cost.

How do I handle parallel steps?

Define steps with explicit dependency lists. Steps with no unmet dependencies can execute concurrently using LangGraph's Send API or async tool execution.

Can I use classical planners (STRIPS, PDDL) with LLMs?

Yes, in hybrid systems. LLM translates natural language goal to PDDL, classical planner generates optimal plan, LLM executes steps. Works well in robotics and structured domains; less common in knowledge work.

How do I evaluate planning quality independently from execution?

Create a test set with expert plans. Measure plan similarity (step overlap, ordering), step granularity, and whether the plan would succeed if executed perfectly (plan correctness).

What happens when the plan is right but execution fails?

That's an execution problem, not a planning problem. Fix tool reliability, executor prompts, or step verification. Don't replan if the plan itself is correct.

References

• ReAct: Synergizing Reasoning and Acting (Yao et al., 2022)
• LangChain Agents Documentation
• OpenAI Agents Guide

Further Reading

• Anthropic Tool Use Documentation
• LangGraph Documentation
• Microsoft AutoGen

Summary

• Agent planning decomposes goals into verifiable, executable steps with explicit success criteria. - Separate planning (capable model) from execution (cheap model or code) for cost and reliability. - Replanning is essential - cap at 2 attempts, provide full failure context to the replanner. - Good steps are specific, observable, and scoped to one capability. Avoid vague instructions.

• Verify each step before proceeding; verify the goal before responding to the user. - 3–7 steps is the sweet spot. Over-decomposition adds latency and error rates. - Log plans and step traces - they're your primary debugging and audit artifact. - Match planning depth to task complexity; not every task needs a planner.