Query Transformation Guide

TL;DR

• Query transformation improves retrieval by changing the query before search — rewriting vague questions, generating alternative phrasings, expanding terminology, or decomposing complex questions into subqueries.

• User questions are optimized for communication, not retrieval — documentation uses different terms than users (TLS handshake vs "gateway disconnecting"), and short questions embed poorly compared to passage-length text.

• Query transformation is a pre-retrieval stage — it runs before hybrid search or vector search; re-ranking runs after retrieval on the merged candidate pool.

• Multi-query retrieval is one technique within query transformation — generate a small bounded set of alternative queries, retrieve independently, merge and deduplicate, then rerank. Not every pipeline needs multiple queries.

• Transformation is not guaranteed to help — it can introduce query drift, increase latency and cost, and change user intent. Gate it with a classifier and evaluate with retrieval evaluation metrics.

Why This Matters

Retrieval quality is the ceiling for RAG answer quality. When the right document never reaches the LLM, no prompt engineering fixes the response. Many retrieval failures happen before search executes: the query sent to the index is a poor match for how knowledge is stored.

Users ask questions in conversational language, with pronouns, missing product names, and domain terms their documentation never uses. A support engineer asks "Why does my gateway keep disconnecting?" while runbooks describe TLS handshake failures, connection timeouts, certificate rotation, and keepalive settings. The retrieval system searches the representation of whatever query it receives — not the user's underlying information need.

Query transformation attempts to bridge that gap. It is one of the highest-leverage pre-retrieval improvements when recall is low on natural-language questions, but it adds cost, latency, and failure modes that naive implementations ignore. Production teams that transform every query without measurement often increase noise without improving recall.

The Problem Query Transformation Solves

A user's question is optimized for communicating intent to a human or a chat interface. It is not necessarily optimized for:

• Embedding similarity — short questions occupy a different region of vector space than passage-length documentation.
• Lexical matching — users say "disconnecting"; docs say "connection reset" or "TLS handshake failure."
• Multi-part information needs — one sentence may require evidence from four separate sources.
• Conversational context — "What did it say about that problem we saw yesterday?" is unsearchable without session history resolution.

Consider a concrete mismatch:

Direct retrieval embeds the user's phrasing. If the embedding model and corpus vocabulary diverge, recall drops even when the answer exists. Query transformation creates retrieval queries that better expose the information need — without changing what the user actually asked (the original question still drives generation and reranking).

This is distinct from problems solved elsewhere in the pipeline:

How We Got Here

Information retrieval research has long separated query formulation from document ranking. Query expansion, relevance feedback, and pseudo-relevance feedback predated neural RAG by decades. Dense retrieval made the vocabulary mismatch problem more visible: bi-encoder models compress queries and documents independently, so phrasing differences hurt more than in keyword systems where exact term overlap still scores.

Diagram: Evolution of query transformation in RAG

timeline
    title Query transformation in IR and RAG
    1990s : Query expansion and relevance feedback
    2017 : Neural query rewriting for web search
    2020 : RAG pipelines with fixed queries
    2022 : Multi-query and HyDE patterns in frameworks
    2023 : Step-back and decomposition prompts
    2024 : Production query classifiers and gated transforms

Framework docs codified rewrite and multi-query patterns; production teams added gating, provenance logging, and retrieval eval before shipping transforms broadly.

LangChain query transformation and LlamaIndex query pipelines documented rewrite, decomposition, and multi-query patterns. The engineering question shifted from whether transformation exists to when to apply which technique and how to measure impact on recall@k.

What Is Query Transformation?

Query transformation is any pre-retrieval step that converts a user's input into one or more search queries better suited to your index. The umbrella covers several distinct techniques:

Query transformation is not synonymous with multi-query retrieval. Multi-query is one important pattern inside the broader concept. It is also not reranking, hybrid search, or agentic planning — those operate at different pipeline stages.

Diagram: Query transformation in the RAG pipeline

flowchart TD
    Q[User Question] --> T[Query Transformation]
    T --> R[Retrieval]
    R --> RR[Reranking]
    RR --> C[Context]
    C --> G[Generation]

Query transformation sits immediately before retrieval; reranking and generation consume its output.

How Query Transformation Works

Query rewriting (one-to-one)

Rewriting produces a single standalone retrieval query from a conversational or underspecified input. Common cases:

• Conversational follow-ups — "What did it say about that problem we saw yesterday?" → "TLS handshake failure troubleshooting for Acme Edge gateway"
• Vague queries — "connection issues" → "Edge gateway connection reset and timeout diagnostics"
• Acronym mismatch — "mTLS setup" → "mutual TLS certificate configuration NebulaAPI"
• Verbose questions — compress to searchable keywords while preserving constraints

The rewrite model (often a small LLM with a structured prompt) receives session context when available. The original user question is retained for reranking and generation — the rewrite is for retrieval only.

Warning

Rewriting can drop exact identifiers, version numbers, or negations. Prompts must instruct the model to preserve constraints like NebulaAPI v2.3, not broaden to NebulaAPI.

Query expansion (enrichment)

Expansion adds terms to the query without necessarily replacing it. Sources include synonym lists, domain ontologies, acronym tables, and LLM-generated related terms.

Example: "gateway disconnect" may expand toward connection reset, TLS handshake, keepalive timeout.

Expansion differs from keyword stuffing. Effective expansion adds discriminative domain terms; ineffective expansion adds generic related words that pull irrelevant documents. Monitor for query drift — when expanded terms dominate retrieval and pull off-topic chunks.

Multi-query retrieval

Multi-query generates several alternative retrieval queries from one user question, retrieves against each independently, then merges results.

Diagram: Multi-query retrieval flow

flowchart TD
    O[Original query] --> G[Generate alternatives]
    G --> Q1[Q1]
    G --> Q2[Q2]
    G --> Q3[Q3]
    Q1 --> R1[Retrieve]
    Q2 --> R2[Retrieve]
    Q3 --> R3[Retrieve]
    R1 --> U[Candidate union]
    R2 --> U
    R3 --> U
    U --> D[Deduplicate]
    D --> F[Fusion or rerank]
    F --> FC[Final context]

Alternative phrasings improve recall; merge and dedup prevent duplicate chunks from flooding context.

Why alternative queries help recall: different phrasings activate different regions of embedding space and different BM25 term matches. A question about "rate limiting" may retrieve docs indexed under "429 throttling" only when an alternative query uses that terminology.

Why naive concatenation fails:

• Duplicate chunks — the same passage ranks highly for multiple queries
• Rank dilution — concatenating lists without fusion loses which query found which doc
• Noise amplification — weak queries add irrelevant hits to the union
• Context overflow — duplicates consume token budget before reranking

Production multi-query pipelines deduplicate by stable chunk ID, track provenance (which query retrieved each candidate), cap per-query top-k, and rerank the merged pool against the original user question. Usually retain the original query alongside transformed queries when it is meaningful for retrieval; for underspecified conversational follow-ups, evaluation may support retrieving only with the resolved standalone query.

Query decomposition

Decomposition splits a complex question into independently retrievable subqueries when the original requires evidence from multiple sources.

Example: "Compare the latency and pricing implications of using vector search versus hybrid search for a 50M-document RAG system."

Each subquery retrieves independently; results merge for generation. Decomposition helps when no single document answers the full question.

Risk: subqueries may lose cross-constraints encoded in the original — the comparison framing, scale assumptions (50M documents), or required pairing of latency with pricing. Mitigate by including shared constraints in every subquery prompt and reranking against the full original question.

Diagram: Query decomposition

flowchart TD
    CQ[Complex question] --> D[Decompose]
    D --> S1[Subquery 1]
    D --> S2[Subquery 2]
    D --> S3[Subquery 3]
    S1 --> M[Retrieve and merge]
    S2 --> M
    S3 --> M
    M --> A[Answer synthesis]

Decomposition retrieves evidence per subproblem; generation must reassemble relationships.

Step-back prompting

Step-back generates a more abstract query alongside the specific one. A question about configuring keepalive on a specific gateway product might step back to "TCP connection lifecycle and keepalive principles."

Useful when answers require background concepts the specific query omits. Risk: abstract queries retrieve overly broad passages that dilute precision. Often combined with multi-query (specific + step-back) rather than used alone.

HyDE (Hypothetical Document Embeddings)

HyDE generates a hypothetical passage that would answer the query, embeds that passage, and uses the hypothetical embedding for vector retrieval:

Query → LLM generates hypothetical relevant passage → Embed passage → Vector search

The hypothetical document may occupy embedding space closer to corpus passages than a short question does — addressing query-document representation mismatch.

HyDE is a distinct strategy, not a replacement for rewriting or multi-query. Risks include unsupported assumptions in the generated passage, extra generation latency, and retrieval driven by hallucinated content. Use when eval shows representation mismatch; skip when queries already match document style or when exact lexical match matters.

Architecture

Three common architectural shapes:

Simple rewrite path:

User Question → Rewrite → Retrieval → (optional rerank) → Generation

Multi-query path:

                    User Question
                          |
                          v
                Query Transformation
                          |
             +------------+------------+
             |            |            |
             v            v            v
            Q1           Q2           Q3
             |            |            |
             +------------+------------+
                          |
                          v
                     Retrieval
                          |
                          v
                 Merge / Deduplicate
                          |
                          v
                      Reranking
                          |
                          v
                    Final Context
                          |
                          v
                         LLM

Production gated pipeline:

Security filters and transformed query text are separate concerns. An LLM rewrite must not remove or weaken tenant filters, ACL constraints, or version scoping — filters attach at the retrieval API layer, not inside generated query strings.

Step-by-Step Flow

Step 1: Classify the query. Is it already specific and searchable (exact doc ID, error code, SKU)? Pass through unchanged. Is it conversational, multi-part, or vocabulary-mismatched? Route to the appropriate transform.

Step 2: Apply transformation. Cap alternative query count with a small bounded configuration and tune it using retrieval metrics, latency, cost, duplicate rate, and candidate noise. Preserve exact identifiers and structured constraints in prompts. Retain the original query for provenance, and retrieve with it when evaluation shows it adds value.

Step 3: Retrieve in parallel. Each query runs against the same index with identical metadata filters. Record per-query latency and hit lists.

Step 4: Merge and deduplicate. Union candidates by stable chunk ID. Track provenance: which queries retrieved each chunk.

Step 5: Fuse or rerank. Apply reciprocal rank fusion across per-query ranked lists, or pass the deduplicated pool to a cross-encoder reranker scored against the original user question.

Step 6: Generate. Pass top chunks and the original question to the LLM. Cite sources; log the full trace for debugging.

Diagram: Production request sequence

sequenceDiagram
    participant U as User
    participant API as RAG API
    participant C as Classifier
    participant T as Transformer
    participant R as Retriever
    participant M as Merger
    participant RR as Reranker
    participant L as LLM
    U->>API: question + filters
    API->>C: classify query
    C->>T: transform if needed
    T-->>API: Q0 Q1 Q2
    par Parallel retrieval
        API->>R: search Q0
        API->>R: search Q1
        API->>R: search Q2
    end
    R-->>M: candidate lists
    M-->>RR: deduped pool
    RR-->>API: top chunks
    API->>L: original Q + context
    L-->>U: answer

Every stage logs provenance; filters apply uniformly across parallel retrievals.

Real Production Example

Scenario: Enterprise documentation RAG for NebulaAPI — an internal API platform. User question:

"Our staging pods keep getting 503s after we rotated certs on the edge gateway last night — what should we check?"

Transformation decision: Classifier flags conversational + multi-symptom query → multi-query retrieval (3 alternatives + original).

Generated retrieval queries:

Metadata filters (unchanged across all queries): tenant_id=acme, product=NebulaAPI, env=staging, doc_acl ⊆ user.groups

Per-query retrieval (top-5 each, hybrid search):

After deduplication: 4 unique chunks (A, B, C, E). Duplicate A appeared 3 times; counted once with provenance {Q1, Q2, Q3}.

Reranking (cross-encoder, original user question): B → A → C → E

Final context: Chunks B + A + C (top-3). Chunk E dropped — too generic.

Generated answer: Points to staging health check config and TLS handshake verification steps after cert rotation, citing runbook sections. Correct because Q2 surfaced terminology the original query lacked.

Without transformation, Q0 alone retrieved E and a generic troubleshooting page — missing chunk A (the specific TLS post-rotation runbook).

Design Decisions

Common Patterns

Original + alternatives. Usually search the original query alongside generated alternatives when it is already meaningful as a retrieval query. For underspecified conversational follow-ups, a resolved standalone rewrite may replace original-query retrieval when evaluation supports that choice.

Classifier-first. Route exact lookups and identifier queries to direct retrieval. Transform only when classification confidence exceeds threshold.

Constraint envelope. Pass version, tenant, product, and date filters as structured API parameters — never rely on the LLM to embed them in query text.

Rerank after merge. Multi-query union increases recall but hurts precision. Reranking against the original question is standard in production.

Cache transforms. Cache alternative queries for identical questions within a session TTL. Invalidate on corpus version change.

Eval before rollout. A/B test recall@k on a golden set before enabling transforms in production traffic.

Comparisons

Technique comparison

Pipeline position comparison

Common Mistakes

1. Transforming every query — adds latency and noise on exact lookups that direct search handles well.

2. Generating too many alternatives — 8+ queries multiply retrieval cost with diminishing recall gains.

3. Losing exact identifiers during rewrite — ERR_NEBULA_8842 becomes "connection error"; retrieval misses the runbook.

4. Dropping version or date constraints — rewrite broadens NebulaAPI v2.3 to NebulaAPI; wrong docs surface.

5. Dropping original-query retrieval without evaluation — rewrites can miss phrasing that matched; test whether original-query retrieval improves recall before removing it.

6. Concatenating results without dedup — duplicate chunks waste context window and confuse rerankers.

7. No candidate provenance — impossible to debug which alternative query caused a bad retrieval.

8. Skipping rerank after multi-query merge — union improves recall but floods the pool with noise.

9. Treating transformation as guaranteed improvement — measure recall@k; some corpora need no transform.

10. Evaluating answer quality without retrieval metrics — fluent wrong answers hide retrieval regression.

Running in Production

Best Practice

Retain the original query for provenance, keep transformed-query count bounded and evaluation-driven, preserve exact identifiers, apply security filters independently of LLM rewrites, retrieve in parallel, deduplicate, retain provenance, rerank merged candidates, and gate deploys on recall@k.

Observability per request

Record at minimum:

• Original query and session context
• Transformation decision and technique used
• All transformed queries (Q0…Qn)
• Transformation model and latency
• Per-query retrieval latency and top-k IDs
• Query → candidate provenance map
• Duplicate rate and pre/post dedup counts
• Fusion or rerank output and final selected context

Debugging from original question + final answer alone is insufficient. When a user reports a wrong answer, you need to know whether retrieval failed on all queries or only the rewrite.

Code: multi-query retrieval core

from __future__ import annotations

import asyncio
from dataclasses import dataclass, field
from typing import Callable


@dataclass
class RetrievedChunk:
    chunk_id: str
    text: str
    score: float
    source_queries: set[str] = field(default_factory=set)


MAX_ALTERNATIVES = 3
TRANSFORM_TIMEOUT_S = 8.0
RETRIEVE_TIMEOUT_S = 5.0


async def generate_alternative_queries(
    user_query: str,
    llm_complete: Callable[..., str],
    *,
    max_alternatives: int = MAX_ALTERNATIVES,
) -> list[str]:
    """Return original + up to max_alternatives LLM-generated search queries."""
    prompt = f"""Generate {max_alternatives} diverse search queries for retrieving
technical documentation. Preserve exact product names, error codes, and version
numbers from the user question. Do not answer — output queries only.

User question: {user_query}

Output one query per line."""
    try:
        raw = await asyncio.wait_for(
            asyncio.to_thread(llm_complete, prompt),
            timeout=TRANSFORM_TIMEOUT_S,
        )
    except (asyncio.TimeoutError, OSError):
        return [user_query]

    alternatives = [line.strip() for line in raw.splitlines() if line.strip()]
    # Retain original by default; dedupe near-identical strings
    queries = [user_query]
    for q in alternatives:
        if q.lower() != user_query.lower() and q not in queries:
            queries.append(q)
        if len(queries) > max_alternatives + 1:
            break
    return queries


async def retrieve_for_queries(
    queries: list[str],
    retrieve: Callable[[str], list[RetrievedChunk]],
    *,
    filters: dict | None = None,
) -> dict[str, list[RetrievedChunk]]:
    """Retrieve in parallel; filters apply at retrieve() — not in query text."""

    async def _one(q: str) -> tuple[str, list[RetrievedChunk]]:
        try:
            hits = await asyncio.wait_for(
                asyncio.to_thread(retrieve, q, filters or {}),
                timeout=RETRIEVE_TIMEOUT_S,
            )
        except (asyncio.TimeoutError, OSError):
            hits = []
        return q, hits

    pairs = await asyncio.gather(*[_one(q) for q in queries])
    return dict(pairs)


def deduplicate_candidates(
    results_by_query: dict[str, list[RetrievedChunk]],
) -> list[RetrievedChunk]:
    """Merge by chunk_id; union provenance and keep best score."""
    by_id: dict[str, RetrievedChunk] = {}
    for query, hits in results_by_query.items():
        for hit in hits:
            if hit.chunk_id in by_id:
                existing = by_id[hit.chunk_id]
                existing.source_queries.add(query)
                existing.score = max(existing.score, hit.score)
            else:
                by_id[hit.chunk_id] = RetrievedChunk(
                    chunk_id=hit.chunk_id,
                    text=hit.text,
                    score=hit.score,
                    source_queries={query},
                )
    return sorted(by_id.values(), key=lambda c: c.score, reverse=True)


async def multi_query_retrieve(
    user_query: str,
    llm_complete: Callable[..., str],
    retrieve: Callable[[str, dict], list[RetrievedChunk]],
    rerank: Callable[[str, list[RetrievedChunk]], list[RetrievedChunk]],
    filters: dict,
) -> list[RetrievedChunk]:
    queries = await generate_alternative_queries(user_query, llm_complete)
    results_by_query = await retrieve_for_queries(queries, retrieve, filters=filters)
    candidates = deduplicate_candidates(results_by_query)
    return rerank(user_query, candidates)[:5]

Filters pass separately to retrieve() — tenant ACL, product version, and environment never depend on LLM-generated query strings.

Cost and latency scaling

One user query with three transformed alternatives typically incurs:

• 1 transformation LLM call
• 4 embedding operations (if original included)
• 4 vector searches (+ 4 BM25 searches if hybrid)
• 1 merge/dedup step
• 1 reranker pass over the deduplicated pool

Cost scales roughly linearly with alternative query count. Start with a small bounded number and tune it using recall@k (or other retrieval metrics), latency, cost, duplicate rate, and candidate noise. More transformed queries can improve recall, but they also increase retrieval work, duplication, noisy candidates, latency, and spend. Parallel retrieval reduces wall-clock latency but not total compute. Use query complexity gating to skip transformation on exact-match patterns.

Where It Breaks Down

Exact identifiers rewritten incorrectly. Error codes, CVE IDs, and ticket numbers must pass through unchanged. LLM rewrites paraphrase them into generic terms.

Version constraints removed. A rewrite from NebulaAPI v2.3 to NebulaAPI retrieves deprecated docs.

Expansion introduces unrelated concepts. Expanding "gateway" with generic networking terms pulls irrelevant infrastructure pages.

Decomposition loses cross-question dependencies. Subqueries about latency and pricing may retrieve incompatible assumptions when the original required a paired comparison.

Generated queries become near-duplicates. Three alternatives differing only in stopwords triple retrieval cost without recall gain.

Multi-query floods the candidate pool. Union without dedup and rerank passes repetitive, noisy context to the LLM.

Transformation latency exceeds benefit. A 2-second LLM rewrite on a corpus where direct search already achieves 95% recall@5 wastes budget.

Small or highly precise corpora. A 200-page internal wiki with consistent terminology may not benefit from transformation.

Domain terminology already matches users. Support portals trained on user language may need hybrid search or filters, not query rewriting.

When NOT to Use Query Transformation

Skip or gate query transformation when:

1. Exact lookup queries — document IDs, error codes, SKUs, CVE numbers. Use hybrid search or direct keyword match.

2. Highly structured search — faceted catalog browse where metadata filters encode the query.

3. Known identifiers present — query already contains discriminative tokens that match the index.

4. Retrieval recall already high — golden set recall@5 above your threshold without transforms.

5. Strict latency budgets — sub-200ms retrieval SLA cannot absorb LLM transform + N searches.

6. Very small corpora — brute-force search with reranking may suffice under ~500 chunks.

7. Structured metadata solves the problem — metadata filtering on product, version, and tenant narrows better than query rewriting.

Alternatives: Improve chunking, switch embedding models, enable hybrid search, add synonym rules at index time, or expand the golden set before adding transform complexity.

Decision tree: should you transform this query?

flowchart TD
    A[Incoming query] --> B{Specific and searchable?}
    B -->|Yes| Z[Direct retrieval]
    B -->|No| C{Conversational or vague?}
    C -->|Yes| R[Rewrite]
    C -->|No| D{Multiple info needs?}
    D -->|Yes| DC[Decompose]
    D -->|No| E{Terminology mismatch?}
    E -->|Yes| MQ[Multi-query or expand]
    E -->|No| F{Embed mismatch?}
    F -->|Yes| H[Consider HyDE]
    F -->|No| Z

Gate transforms by query type; default to direct retrieval when uncertain.

Key Takeaways

• Query transformation bridges the gap between how users ask questions and how documentation is indexed.
• It runs before retrieval; hybrid search and reranking solve different problems at later stages.
• Multi-query retrieval is one technique — not the whole concept. Rewriting, expansion, decomposition, step-back, and HyDE address different failure modes.
• Transformation can improve recall but introduces drift, cost, latency, and debugging complexity.
• Retain the original query for provenance, preserve filters separately, deduplicate, rerank, and measure recall@k — not answer fluency alone.

Related Guides

• Foundations: RAG · Embedding Models · Semantic Search · Chunking Strategies

• Retrieval stack: Hybrid Search · Vector Search · Metadata Filtering · Re-ranking · Late Interaction Retrieval

• Quality & ops: Retrieval Evaluation · Vector Databases

• Vector stores: Qdrant · Weaviate · Pinecone · Milvus — compare in Best Vector Databases.

• Head-to-heads: Qdrant vs Pinecone · Qdrant vs Weaviate

If you understood this topic, read next:

Diagram: Recommended learning path

flowchart LR
    A[RAG] --> B[Query Transform]
    B --> C[Hybrid]
    C --> D[Rerank]
    D --> E[Eval]

Prerequisites: RAG · Embedding Models · Hybrid Search

Next topics: Re-ranking · Retrieval Evaluation · Metadata Filtering

Estimated time: 50 min · Difficulty: Intermediate

Interview Questions

1. What is the difference between query rewriting and multi-query retrieval?

   • Expected: rewriting is one-to-one (single improved query); multi-query generates several alternatives retrieved independently and merged.

2. Why can multi-query retrieval improve recall?

   • Expected: alternative phrasings match different corpus terminology and embedding regions; union increases chance relevant doc appears in candidate pool.

3. What is query drift?

   • Expected: expansion or rewrite adds terms that pull irrelevant documents, changing retrieval focus away from user intent.

4. Why retain the original query alongside transformed queries?

   • Expected: user phrasing may match docs rewrite misses; reranker scores against original question.

5. How should tenant and version filters interact with LLM-generated queries?

   • Expected: filters applied as structured retrieval parameters, independent of query text — rewrite must not broaden scope.

6. Why is deduplication necessary after multi-query retrieval?

   • Expected: same chunk retrieved by multiple queries; duplicates waste context and distort ranking without dedup.

7. Where does HyDE fit in the query transformation taxonomy?

   • Expected: generates hypothetical passage, embeds it, searches — addresses query-document representation mismatch, not synonym diversity.

8. How do you evaluate query transformation?

   • Expected: recall@k, MRR, nDCG on golden set comparing with/without transform; also latency, cost, duplicate rate.

9. Query transformation vs reranking — which fixes what?

   • Expected: transformation improves query formulation before search (recall); reranking improves precision within retrieved candidates.

10. When would decomposition hurt retrieval?

    • Expected: when subqueries lose cross-constraints (comparisons, conditional relationships) encoded in the original complex question.

FAQs

What is query transformation in RAG?

Query transformation is any pre-retrieval step that rewrites, expands, splits, or otherwise reformulates a user question into search queries that better match indexed documentation. It runs before vector or hybrid search.

Is query rewriting the same as multi-query retrieval?

No. Rewriting produces one improved query. Multi-query retrieval generates several alternative queries, retrieves against each, and merges results. Both are techniques under the query transformation umbrella.

When should I use multi-query retrieval?

When eval shows recall gains from alternative phrasings — common with natural-language questions over technical corpora with terminology mismatch. Start with a small bounded number of transformed queries and tune it using retrieval quality metrics, latency, cost, duplicate rate, and candidate noise.

Does query transformation always improve RAG?

No. It can introduce noise, change intent, and increase cost. Measure recall@k on your golden set. Skip for exact lookups, high-recall baselines, and strict latency budgets.

What is query drift?

Query drift occurs when expansion or rewriting adds terms that dominate retrieval and pull documents unrelated to the user's actual information need.

Should the original query also be searched?

Usually, yes when the original phrasing is still meaningful for retrieval. In underspecified conversational follow-ups, a resolved standalone rewrite may replace original-query retrieval if evaluation shows better retrieval quality. In all cases, retain the original query for provenance and debugging.

How many alternative queries should I generate?

Start with a small bounded number of transformed queries and tune based on retrieval quality (for example recall@k), latency, cost, duplicate rate, and candidate noise. Increasing query count can improve recall, but it also increases retrieval work, duplicate candidates, irrelevant candidates, latency, and cost.

What is the difference between query transformation and reranking?

Query transformation runs before retrieval to improve what you search for. Reranking runs after initial retrieval to reorder candidates by relevance. They complement each other in multi-query pipelines.

Where does HyDE fit?

HyDE generates a hypothetical answer passage, embeds it, and uses that vector for search. It addresses embedding-space mismatch between short questions and long documents — a distinct strategy from rewriting or multi-query.

How do I evaluate query transformation?

Build a golden test set and compare recall@k, MRR, and nDCG with and without transformation. The question is whether relevant evidence retrieval improved — not whether the rewrite sounds better. See Retrieval Evaluation.

References

• Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE, Gao et al., 2022)
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)
• Take a Step Back: Evoking Reasoning via Abstraction in LLMs (Zheng et al., 2023)
• LangChain — Query transformation and retrieval
• LlamaIndex — Query transformations
• Elasticsearch — Query expansion concepts

Further Reading

• Dense Passage Retrieval (Karpukhin et al., 2020)
• BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation (Thakur et al., 2021)