GPT Models Guide

TL;DR

• Current family is GPT-5.6: gpt-5.6-sol (frontier / complex / agentic), gpt-5.6-terra (balanced everyday production), gpt-5.6-luna (fast / cheap volume). Route by workload — never treat one ID as the permanent default.

• Context is large on current GPT-5.6 tiers (on the order of ~1.05M tokens; verify per model and endpoint in OpenAI docs). Long context raises cost and can degrade middle-of-prompt attention.

• Illustrative pricing (approximate — verify): Sol ~$4/$20 (promo through at least Nov 21, 2026), Terra ~$2/$12, Luna ~$0.20/$1.20 per 1M input/output tokens. Confirm against OpenAI pricing before budgeting.

• ChatGPT vs API: Aug 6 2026 ChatGPT update tunes Sol for Plus/Pro with a reasoning slider; Free/Go default to Luna. Codex and ChatGPT Work may stay on earlier July builds — pin API IDs separately from consumer ChatGPT.

• Production surface: streaming, function calling, structured outputs, prompt caching, Batch API, and reasoning-effort / higher-compute modes on Sol-class workloads.

• History is not the lineup: GPT-3.5 → GPT-4 → GPT-4o → o-series → GPT-5.x → GPT-5.6. Keep older IDs only for migration and evolution context.

Quick Decision Guide

Who this guide is for

• Best for: AI engineers · ML engineers · backend engineers · architects
• Difficulty: Intermediate
• Estimated time: 55 min

Learning Path

Large Language Models → Prompt Engineering → GPT Models → Function Calling → Cost Optimization → Evaluation

On this page

• Why This Matters
• The Problem GPT Models Solve
• How We Got Here
• What Is the GPT Model Family?
• How GPT Models Work
• Architecture
• Step-by-Step Flow
• Real Production Example
• Design Decisions
• Comparisons
• Common Mistakes
• Where It Breaks Down
• When NOT to Default to GPT
• Running in Production
• Production Checklist
• Related Guides
• Interview Questions
• Key Takeaways
• FAQs
• References
• Further Reading

Why This Matters

OpenAI's GPT models still power a large share of production LLM applications — direct API, Azure OpenAI, or indirectly through IDE agents and orchestration frameworks. Model selection is not a one-time brand choice. OpenAI ships new tiers, retires snapshots, and changes pricing. Teams that treat "GPT" as a single static product overpay, hit rate limits, or send hard agent loops to a volume tier that was never meant for them.

If you call an LLM API, you need a mental model of what each GPT-5.6 tier is for, what it costs, and how to pin IDs so yesterday's eval still matches tomorrow's traffic. Capability, latency, and cost are the routing axes — not marketing names.

Engineering Insight

Most production failures trace back to weak routing and evaluation, not to picking the "wrong" provider. Choose tiers by workload and eval-gate every change.

The Problem GPT Models Solve

Before chat-scale foundation models, natural-language interfaces usually meant task-specific models: one for summarization, one for classification, one for extraction. Each needed labeled data, training infra, and ongoing maintenance.

GPT collapses that stack into a general conditional-generation API. You describe the task in prompts (and tools), and one family covers generation, classification, extraction, translation, and code — without retraining per task.

For product teams that means:

• Faster iteration — change prompts and routing instead of retraining pipelines
• Lower upfront CapEx — no GPU cluster before product-market fit
• Unified surface — chat, tools, vision (where supported), and structured data through one SDK

The tradeoff: you inherit OpenAI pricing, rate limits, data-handling terms, knowledge cutoff, and residual hallucination. Reliability is an application property — grounding, schemas, eval — not a property of the brand alone. See large language models.

How We Got Here

Diagram: GPT family evolution

timeline
    title From GPT-3.5 chat to GPT-5.6 workload tiers
    2022-2023 : GPT-3.5 / early GPT-4
              : Chat API goes mainstream
    2023-2024 : GPT-4 Turbo / GPT-4o
              : Longer context, multimodal, cheaper mini tiers
    2024-2025 : o-series reasoning
              : Extra internal compute for hard tasks
    2025-2026 : GPT-5.x → GPT-5.6
              : Sol / Terra / Luna workload routing

Capability shipped first as single flagships; production practice converged on tiered routing by cost and difficulty.

Keep GPT-3.5 / GPT-4 / o-series in migration and history only. New systems should start on GPT-5.6 IDs and verify current aliases in OpenAI docs.

What Is the GPT Model Family?

GPT (Generative Pre-trained Transformer) is OpenAI's line of autoregressive models exposed through the Chat Completions / Responses APIs. Current production chat workhorses are the GPT-5.6 tiers:

OpenAI also ships embeddings, image, audio, and fine-tuning products. When engineers say "GPT models" in application design, they usually mean the chat/completions line above.

Note

Model names, context limits, and prices change. Always verify against OpenAI model docs and pricing before locking architecture or finance forecasts.

How GPT Models Work

At inference time a GPT model predicts the next token given prior tokens in the context window. System messages, user turns, tool definitions, and retrieved documents are tokenized, processed through transformer layers with attention, and decoded until a stop condition.

Production-relevant mechanisms:

Context window. Input and output share a budget. Current GPT-5.6 tiers advertise very large windows (~1.05M class — verify). Longer prompts cost more and can weaken attention to middle spans ("lost in the middle").

Function calling. The model emits structured tool calls; your app executes tools and returns results — the core of function calling agents.

Structured outputs. JSON Schema-constrained generation reduces parse failures for machine pipelines — see structured outputs.

Reasoning effort / higher-compute modes. Sol-class workloads can allocate extra internal compute (reasoning tokens / effort controls where exposed). Latency and cost rise; hard math and planning often improve. Treat this as a dial, not a permanent setting for all traffic.

Streaming, prompt caching, Batch. Streaming improves perceived latency. Prompt caching discounts repeated prefixes. Batch (~50% off typical) fits offline jobs with multi-hour SLAs.

Multimodal inputs. Where the tier supports vision/audio, non-text inputs share the same stack — no separate OCR model for many document screenshots.

The original Transformer stacks attention and feed-forward blocks with residuals and layer norm:

Source: Google Research

Architecture

GPT in production is not "call ChatGPT." It is a routing + API + verification stack.

Diagram: GPT-5.6 production architecture

flowchart TB
    subgraph Client [Application]
        U[User / Job]
        R[Router: intent + complexity]
    end
    subgraph Models [OpenAI / Azure OpenAI]
        L[gpt-5.6-luna]
        T[gpt-5.6-terra]
        S[gpt-5.6-sol]
    end
    subgraph Controls [Controls]
        Cache[Prompt / response cache]
        Schema[Structured outputs]
        Tools[Tool execution sandbox]
        Eval[Eval + observability]
    end
    U --> R
    R -->|volume / FAQ| L
    R -->|balanced RAG / chat| T
    R -->|hard / agentic| S
    L --> Schema
    T --> Tools
    S --> Tools
    Schema --> Cache
    Tools --> Eval

Route first; constrain outputs; measure cost and quality per model ID.

Current lineup (July 2026)

Illustrative pricing (verify)

Cost levers: prompt caching, Batch API, route-down with evaluation gates, context compression, shorter system prompts. Details: cost optimization.

Step-by-Step Flow

Diagram: Request path with workload routing

sequenceDiagram
    participant U as User
    participant App as App API
    participant Rt as Router
    participant LLM as GPT-5.6
    participant Tool as Tools
    U->>App: Request
    App->>Rt: Classify complexity / risk
    Rt-->>App: luna | terra | sol + effort
    App->>LLM: Chat + tools + schema
    alt tool call
        LLM-->>App: tool_call
        App->>Tool: Execute
        Tool-->>App: Result
        App->>LLM: tool result
    end
    LLM-->>App: Final content
    App-->>U: Streamed / validated response

Classify before calling; escalate only when eval or confidence requires it.

1. Profile the workload — latency SLO, stakes, expected tokens, tool needs.
2. Pick a starting tier — Luna for volume; Terra for most product chat/RAG; Sol for hard agentic or deep reasoning. Do not set Sol as org-wide default.
3. Pin the model ID — env var / config map; record the exact ID in traces.
4. Add schemas and tools early — structured outputs for parsers; tools for live facts.
5. Enable streaming for interactive UX; Batch for offline.
6. Instrument — model ID, input/output/reasoning tokens, TTFT, cost estimate, refusal rate.
7. Load-test rate limits — RPM/TPM; backoff; Azure OpenAI if you need enterprise networking.
8. Eval-gate upgrades — golden set before changing Sol/Terra/Luna or effort defaults. See evaluation.

Real Production Example

Support classifier on Luna; response generation on Terra or Sol by complexity. Model IDs come from the environment.

from __future__ import annotations

import json
import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

VOLUME = os.environ.get("OPENAI_VOLUME_MODEL", "gpt-5.6-luna")
BALANCED = os.environ.get("OPENAI_MODEL", "gpt-5.6-terra")
COMPLEX = os.environ.get("OPENAI_COMPLEX_MODEL", "gpt-5.6-sol")


def classify_ticket(ticket_text: str) -> dict:
    response = client.chat.completions.create(
        model=VOLUME,
        messages=[
            {
                "role": "system",
                "content": (
                    "Classify the support ticket. Return JSON keys: "
                    "category (billing|technical|account|other), "
                    "complexity (simple|complex), "
                    "sentiment (positive|neutral|negative)."
                ),
            },
            {"role": "user", "content": ticket_text},
        ],
        response_format={"type": "json_object"},
        temperature=0,
    )
    return json.loads(response.choices[0].message.content)


def generate_response(ticket_text: str, classification: dict) -> str:
    model = COMPLEX if classification["complexity"] == "complex" else BALANCED
    stream = client.chat.completions.create(
        model=model,
        messages=[
            {
                "role": "system",
                "content": (
                    "You are a support agent. Be concise. "
                    "If you lack evidence, say so and offer escalation."
                ),
            },
            {"role": "user", "content": ticket_text},
        ],
        stream=True,
        temperature=0.3,
    )
    return "".join(
        chunk.choices[0].delta.content or "" for chunk in stream
    )


ticket = "I've been charged twice for my Pro subscription this month."
meta = classify_ticket(ticket)
print(meta)
print(generate_response(ticket, meta))

Tool-calling sketch (Terra default; escalate to Sol for long agent loops):

tools = [
    {
        "type": "function",
        "function": {
            "name": "lookup_order",
            "description": "Look up a customer order by order ID",
            "parameters": {
                "type": "object",
                "properties": {"order_id": {"type": "string"}},
                "required": ["order_id"],
            },
        },
    }
]

response = client.chat.completions.create(
    model=BALANCED,
    messages=[{"role": "user", "content": "Where is order #ORD-9281?"}],
    tools=tools,
    tool_choice="auto",
)

Design Decisions

Choose OpenAI GPT when:

• You need a mature API for tools, schemas, streaming, and a large SDK ecosystem
• Azure OpenAI private networking / compliance path matters
• Multimodal + tools in one vendor stack simplifies ops
• Your golden set shows Sol/Terra/Luna meeting quality SLOs at acceptable cost

Prefer another family when:

• Long-context coding with Anthropic strengths → Claude
• GCP-native grounding / Vertex → Gemini
• Self-host / open weights → Llama, Mistral
• Aggressive API cost on reasoning → evaluate DeepSeek on your suite

Provider choice is rarely permanent. Keep an abstraction (thin client wrapper or gateway) so swapping Terra for Sonnet 5 on one route does not require rewriting the product. The expensive lock-in is usually prompt format drift, proprietary Assistants state, and missing twin evals — not the HTTP call itself.

Common patterns

Cascade routing pays twice on hard queries and saves on easy ones. Cap escalations per session and alert when escalation rate spikes — that usually means the classifier degraded or Luna quality dropped after a silent alias change.

Diagram: When to spend on Sol

flowchart TD
    Q[Incoming query] --> Risk{High stakes?}
    Risk -->|Yes| Sol[gpt-5.6-sol + verify]
    Risk -->|No| Easy{FAQ / classify / extract?}
    Easy -->|Yes| Luna[gpt-5.6-luna]
    Easy -->|No| Hard{Agentic / deep reason?}
    Hard -->|Yes| Sol
    Hard -->|No| Terra[gpt-5.6-terra]

Stakes and difficulty drive spend; volume paths stay on Luna unless eval fails.

Fine-tuning GPT is a late lever. Prefer prompt engineering, RAG, and tools first. Fine-tune when you need stable format/tone that few-shot cannot deliver economically — and re-run the same golden set you use for tier routing so fine-tunes do not silently regress factuality.

Comparisons

Common Mistakes

1. Sol for everything — highest cost and latency; reserve for work that needs it.
2. Ignoring RAG token burn — fifty chunks into any tier destroys budget; retrieve less, rerank.
3. Hardcoding aliases — pin versioned IDs; test before promoting "latest."
4. Regex-parsing free text — use structured outputs / JSON schema mode.
5. No fallback — rate limits and outages need Azure twin, Claude/Gemini backup, or queue.
6. PII without a data policy — API vs ChatGPT consumer terms differ; use enterprise agreements.
7. Skipping eval on route-down — cheaper Luna that fails faithfulness is not savings. See evaluation.

Where It Breaks Down

• Knowledge cutoff — use RAG or live tools for current facts.
• Arithmetic — offload to calculators; do not trust fluent math.
• Non-determinism — even temperature 0 varies; never use GPT for crypto or safety-critical deterministic logic.
• Long-context degradation — put critical instructions at start and end; measure recall on long packs.
• Policy refusals — moderation false positives on medical/legal; plan escalation UX.
• Vendor lock-in — Assistants-specific or fine-tune formats raise migration cost; keep an abstraction boundary.
• Reasoning-token surprise bills — Sol with high effort can emit large hidden token counts; budget and alert on reasoning tokens separately from visible completions.
• Multi-tool agent loops — each hop is a full billable call; cap steps and disable tools on Luna FAQ routes.

When NOT to Default to GPT

Do not make GPT (or Sol) the permanent org default when:

• Your eval suite shows Claude or Gemini winning on the actual workload (coding pack, multimodal, GCP data plane)
• You must self-host for residency or air-gap → open weights
• Traffic is almost all cheap classification → Luna or another provider's volume tier after bake-off
• You cannot instrument tokens/cost yet — fix observability first, then route
• Finance is quoting blog prices — freeze forecasts until you verify live OpenAI rates

Warning

A single permanent model ID for all traffic is an anti-pattern. Route by capability, latency, and cost; pin versions; re-eval on every change.

Running in Production

Best Practice

Pin model IDs per environment, log tokens and $ per request, and block deploys when golden-set quality regresses after a tier change.

Separate interactive chat from overnight Batch jobs in config and dashboards. Sol with high reasoning effort can be correct and still miss a 2s TTFT SLO — move those paths to async UX (progress states, email follow-up, ticket comments) rather than blocking the request thread. When using Azure OpenAI, treat deployment names as the pinned IDs and keep a matrix of Azure vs direct OpenAI parity for features you rely on (structured outputs, caching, Batch).

Continue Learning

• Next guide: Claude Models
• Then: Function Calling

Production Checklist

• Model ID pinned per environment (no floating latest aliases)
• Fallback model configured and load-tested
• Routing policy defined (capability / latency / cost tiers)
• Prompt cache enabled on static-prefix paths
• Structured outputs enforced on machine-consumed paths
• Tools used for live facts; parametric knowledge not treated as current
• Streaming for interactive; Batch for offline volume
• Token and cost metrics attributed per tenant/route
• Golden-set upgrade evaluation completed before ID changes
• Rollback / circuit-break path documented
• Deprecation calendar monitored

Related Guides

Prerequisites

• Large Language Models — next-token prediction and production boundaries
• Tokens — billing and context units
• Prompt Engineering — contracts before tier upgrades

Core Concepts

• Claude Models — Anthropic flagship / volume tiers
• Gemini Models — Google Flash / Pro-class routing
• Llama Models · Mistral Models · DeepSeek Models

Implementation

• Function Calling · Structured Outputs
• ChatGPT · LangChain · LlamaIndex

Optimization

• Cost Optimization

Advanced Topics

• Evaluation

Diagram: GPT learning path

flowchart LR
    LLM[LLMs] --> GPT[GPT models]
    GPT --> FC[Function calling]
    FC --> SO[Structured outputs]
    SO --> CO[Cost opt]
    CO --> EV[Evaluation]
    GPT --> CL[Claude]
    GPT --> GM[Gemini]

Learn the family, then tools and cost controls; compare peers on your golden set.

Interview Questions

1. How do you choose among Sol, Terra, and Luna?
   By workload: volume/latency → Luna; balanced product → Terra; hard reasoning/agents → Sol. Confirm with evals; never a permanent single default.

2. Why pin versioned model IDs?
   Aliases can change behavior overnight. Pins make traces and golden sets reproducible.

3. What is prompt caching good for?
   Discounting repeated static prefixes (system + large docs). Put variables last or you destroy the cache key.

4. When is the Batch API appropriate?
   Offline jobs with hours of SLA — labeling, nightly summaries — not interactive chat.

5. How do GPT tools relate to hallucinations?
   Tools supply live facts; the model still needs schemas and verification. Tools reduce parametric fabrication; they do not eliminate unfaithful generation.

6. GPT vs Claude for coding?
   Bake off GPT-5.6 Terra/Sol against Anthropic's current flagship Claude tiers on your repo metrics — correctness, edit discipline, latency, cost.

7. What breaks at long context?
   Cost, latency, and middle-context attention. Measure retrieval and instruction adherence on long packs.

8. How do you control spend?
   Route by complexity, cache, compress context, Batch offline, attribute $ per tenant — see cost optimization.

Key Takeaways

• GPT-5.6 is three workload tiers (Sol / Terra / Luna), not one eternal default.
• Pin IDs, route by capability/latency/cost, and verify pricing against OpenAI.
• Use streaming, tools, structured outputs, caching, and Batch as first-class controls.
• Historical GPT-4 / o-series belong in migration history, not new defaults.
• Compare Claude, Gemini, and open models on your eval suite before locking a vendor.

FAQs

How do GPT-5.6 tiers differ?

Sol targets the hardest coding, agentic, and reasoning work; Terra balances capability, latency, and cost; Luna targets volume. Start from workload and evals.

When should I use Sol instead of Terra?

When Terra fails your golden set on hard multi-step, agentic, or deep reasoning tasks — or when stakes demand maximum capability plus verification. Do not use Sol for FAQ traffic.

How much does GPT-5.6 cost?

Illustrative ballparks: Luna ~$0.20/$1.20, Terra ~$2/$12, Sol ~$4/$20 per 1M input/output tokens (Sol promotional from Aug 21 2026 through at least Nov 21). Verify live OpenAI pricing, caching discounts, and reasoning-token billing.

How does ChatGPT routing differ from the API?

As of Aug 6 2026, Plus/Pro ChatGPT conversations use an updated Sol with a reasoning slider; Free/Go default to Luna (unlimited text chats rolling out). Treat ChatGPT product routing as separate from pinned gpt-5.6-sol / terra / luna API IDs used in apps, Codex, and Work.

Can I self-host GPT?

No. Use the API or Azure OpenAI. For self-host, see Llama / Mistral / DeepSeek guides.

What context window does GPT-5.6 support?

Current tiers are in the ~1.05M-token class depending on model/endpoint — confirm in model docs. Longer is not always better for quality or cost.

Does OpenAI train on API data?

API terms generally exclude training by default for API traffic; verify current terms for your contract (API vs Enterprise vs consumer ChatGPT).

Is GPT always better than Claude or Gemini?

No. Run task-specific evals. Claude often shines on long-context careful coding; Gemini on multimodal + GCP grounding; GPT on ecosystem maturity and tool/schema ergonomics.

Should I fine-tune or use RAG?

RAG for changing private facts; fine-tune for style/format that prompting cannot deliver economically. Most teams prompt → RAG → fine-tune last.

What happens on deprecation?

OpenAI announces windows. Pin IDs, watch changelogs, re-run evals before cutting over.

References

• OpenAI API Documentation
• OpenAI Models
• OpenAI Pricing
• Azure OpenAI Service
• Attention Is All You Need (Vaswani et al., 2017)

Further Reading

• Anthropic Documentation
• Google AI for Developers
• LangChain Documentation
• LlamaIndex Documentation