PROBE: Prospective Retrieval with Oracle-Based End-Goal Distillation for Multi-Hop Question Answering¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/luxu-zju/PROBE
Area: LLM Reasoning
Keywords: Multi-Hop Question Answering, Iterative Retrieval, Prospective Query, End-Goal Distillation, Dense Retriever Alignment
TL;DR¶
PROBE decouples the multi-hop reasoning state into factual summaries and prospective queries with context denoising, and optimizes dense retrievers via final-hop cross-encoder Margin-MSE distillation, enabling rerank-free, label-free multi-hop reasoning.
Background & Motivation¶
Multi-hop Question Answering (MHQA) requires models to locate and synthesize scattered pieces of evidence across multiple documents to answer complex queries. Because full evidence chains cannot be retrieved reliably in a single pass, modern systems predominantly adopt iterative Retrieval-Augmented Generation (RAG) paradigms, interleaving retrieval steps with intermediate reasoning steps to progressively construct multi-hop reasoning trajectories (e.g., IRCoT, ReSCORE). However, the overall performance of such iterative systems remains fundamentally constrained by the underlying dense retriever (e.g., Contriever). Adapting dense retrievers to iterative multi-hop reasoning is notoriously difficult because gold query-document relevance annotations are unavailable for the dynamic, path-dependent intermediate queries that emerge during iterative reasoning.
To overcome the lack of manual annotations, recent label-free methods such as ReSCORE utilize step-wise generative likelihoods from large language models to supervise intermediate queries based on accumulated evidence. Nevertheless, this step-wise paradigm suffers from two severe failure modes: first, because the reasoning state passively summarizes existing context without anticipating unresolved dependencies, queries frequently remain anchored to known entities, leading to repetitive surface-form entity loops that fail to retrieve crucial bridging documents; second, step-wise generative supervision strongly favors answer-bearing documents over intermediate bridging evidence that does not directly contain the answer tokens, penalizing early exploratory retrieval and shrinking trajectory diversity.
Addressing these twin bottlenecks requires rethinking both how intermediate information needs are formulated and how retrieval supervision is distributed across reasoning hops. Instead of entangling factual verification with speculative exploration, models should explicitly isolate forward-looking search intent. Furthermore, instead of imposing noisy, path-dependent supervision on ambiguous intermediate steps, retriever optimization should focus on the trajectory's fully resolved end goal. Core idea: decouple the multi-hop reasoning state into a factual summary for grounded generation and a prospective query for forward-looking retrieval (with context denoising), while supervising the dense retriever via cross-encoder Margin-MSE distillation exclusively at the final hop of resolved trajectories to achieve reranker-free inference.
Method¶
Overall Architecture¶
The PROBE framework operates in two distinct stages: Stage 1 performs iterative reasoning with prospective queries and context denoising to solve questions and collect silver trajectories; Stage 2 applies Oracle-Based End-Goal Distillation to train a lightweight Bi-Encoder using continuous Cross-Encoder ranking signals exclusively at the final hop.
During iterative inference, given question \(Q\) and corpus \(\mathcal{D}\), the system executes multi-hop retrieval. At hop \(i\), the query combines the original question with prior factual summaries and the latest prospective query. The Reader LLM evaluates the evidence under strict context denoising—receiving only grounded factual summaries and the newly retrieved passages without speculative queries. If the evidence is insufficient, it outputs "Unknown" and generates a structured thought tuple consisting of a factual summary \(s_i\) and a prospective query \(q_i^{\text{next}}\). Once the trajectory resolves at final hop \(I\), the cross-encoder teacher scores the final candidate pool, and pairwise margin differences supervise the student Bi-Encoder via Margin-MSE loss. At deployment, the cross-encoder is completely discarded.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
Q["Multi-Hop Question Q"] --> Ret1["Initial Retrieval (Hop 1)"]
Ret1 --> Denoise["Context-Denoised Reader Check"]
Denoise -->|Unknown| StateDecouple["Decoupled Thought State Generation<br/>Factual Summary + Prospective Query"]
StateDecouple --> RetIter["Multi-Hop Prospective Concatenated Retrieval<br/>Anchor on Verified Facts & Explore Missing Entity"]
RetIter --> Denoise
Denoise -->|Valid Answer| EndGoalDistill["End-Goal Distillation & Margin Filtering<br/>Align Final-Hop Cross-Encoder Signals"]
EndGoalDistill --> Deploy["Deploy Rerank-Free Bi-Encoder Retriever"]
Key Designs¶
1. Decoupled Thought State Generation: Resolving intermediate query anchoring and surface-form loops
Traditional iterative methods accumulate unstructured thoughts, causing successive queries to become trapped around familiar entities mentioned in earlier hops. PROBE explicitly structures the reasoning state at each hop into a tuple \(t_j = \langle s_j, q_j^{\text{next}} \rangle\). The factual summary \(s_j\) strictly captures verified entities and relations extracted from newly retrieved documents, while the prospective query \(q_j^{\text{next}}\) forms an autonomous, forward-looking natural language question aimed specifically at the missing bridging dependency (for instance, anticipating "Who created the NBC sitcom 'Superstore'?" after verifying that Johnny Pemberton plays Bo Thompson in Superstore). Both parts are generated simultaneously in a single LLM call, but routed to separate downstream modules.
2. Context Denoising for Answer Generation: Preventing speculative hypotheses from contaminating the reader
Exposing speculative search directives to the answer-generation reader risks inducing hallucinations or causing premature convergence on ungrounded hypotheses. PROBE enforces context denoising by removing all prospective queries \(\{q_1^{\text{next}}, \dots, q_{i-1}^{\text{next}}\}\) from the reader's prompt context, supplying only the original question, the factual summaries, and the newly retrieved passages: $\(\hat{A}_i = \text{LLM}(Q, \{s_1, \dots, s_{i-1}\}, \mathcal{D}_i)\)$ If the reader determines the verified evidence is sufficient, it generates the concise final answer \(\hat{A}_i\) and terminates the loop; otherwise, it returns "Unknown", preserving rigorous factual grounding throughout multi-hop reasoning.
3. Multi-Hop Prospective Concatenated Retrieval: Balancing historical factual grounding with forward exploration
When retrieving at intermediate hop \(i\) (\(i > 1\)), discarding the original question leads to semantic drift, yet over-relying on initial queries fails to incorporate newly uncovered entities. PROBE constructs a structured search query: $\(q_{\text{search}, i} = Q \parallel s_1 \parallel q_1^{\text{next}} \parallel \dots \parallel s_{i-1} \parallel q_{i-1}^{\text{next}}\)$ The concatenated historical summaries anchor retrieval in verified context, while the latest prospective query \(q_{i-1}^{\text{next}}\) guides the dense retriever toward the missing bridging entity. Across extensive evaluations, this query formulation averages only 67.65 tokens and exceeds the 512-token limit in only 0.32% of instances, ensuring compact and targeted retrieval.
4. Oracle-Based End-Goal Distillation with Margin Filtering: Eliminating intermediate path noise while preserving exploration
In multi-hop QA, intermediate retrieval targets are highly ambiguous and path-dependent, as multiple valid reasoning paths may lead to the correct answer. Forcing the retriever to match specific intermediate step labels overfits to single paths and penalizes exploratory diversity. PROBE distills ranking signals from a teacher Cross-Encoder (BGE-Reranker-v2-m3) exclusively at the final hop \(I\) of successfully resolved trajectories. To discard ambiguous supervisory signals, instances are filtered using a margin threshold (\(\epsilon = 10^{-4}\)): only instances where the score difference between the top-1 document \(d^+\) and top-2 document \(d^-\) within candidate pool \(\mathcal{D}_{\text{cand}}\) satisfies \(r_t(q_{\text{search}, I}, d^+) - r_t(q_{\text{search}, I}, d^-) \ge \epsilon\) are retained for distillation. This provides trajectory-level alignment without constraining intermediate exploratory steps.
Loss & Training¶
During retriever training, only the question encoder \(E_Q\) of Contriever is updated while the document encoder \(E_D\) remains frozen. Student relevance scores are computed as inner products: \(r_s(q, d) = E_Q(q)^\top E_D(d)\).
Given the retained \(M = 32\) candidate documents at the final hop, we construct the set of all document pairs \(\mathcal{P}\). For any document pair \((d_u, d_v) \in \mathcal{P}\), the score margins under teacher model \(t\) and student model \(s\) are defined as: $\(\delta_m(d_u, d_v) = r_m(q_{\text{search}, I}, d_u) - r_m(q_{\text{search}, I}, d_v), \quad m \in \{s, t\}\)$ The student retriever is optimized using the Margin-MSE objective: $\(\mathcal{L} = \frac{1}{|\mathcal{P}|} \sum_{(d_u, d_v) \in \mathcal{P}} \left( \delta_s(d_u, d_v) - \delta_t(d_u, d_v) \right)^2\)$ The model is trained using AdamW with a learning rate of \(1 \times 10^{-5}\), weight decay 0.01, batch size 32 (with 8 gradient accumulation steps), warmup ratio 0.1, and gradient clipping norm 1.0. Distillation on 12,738 filtered final-hop instances requires only 147 minutes on a single RTX 3090 GPU for 20 epochs.
Key Experimental Results¶
Main Results¶
On three challenging MHQA benchmarks (MuSiQue, HotpotQA, and 2WikiMultiHopQA), PROBE achieves superior Exact Match (EM) and F1 performance over both retriever-fixed reasoning baselines and trained retriever systems:
| Method | MuSiQue (EM / F1) | HotpotQA (EM / F1) | 2WikiMHQA (EM / F1) | Avg (EM / F1) | Training Scale / Protocol |
|---|---|---|---|---|---|
| ReAct (GPT-3.5 + BM25) | 10.2 / 19.7 | 36.0 / 46.9 | 28.0 / 37.3 | 24.7 / 34.6 | Inference-only prompt |
| IRCoT (Flan-T5-XL + BM25) | 22.0 / 31.8 | 44.4 / 56.2 | 49.7 / 54.9 | 38.7 / 47.6 | Interleaved CoT & retrieval |
| Search-R1 (Qwen2.5-7B + Contriever) | 24.4 / 34.0 | 52.4 / 64.9 | 54.4 / 63.0 | 43.7 / 54.0 | RL-trained LLM generator |
| STRIDE (Qwen3-8B + Contriever) | 24.6 / 33.6 | 49.4 / 59.4 | 57.0 / 65.6 | 43.7 / 52.9 | Explicit decision planning |
| ReSCORE (Llama-3.1-8B + Contriever) | 23.4 / 32.7 | 47.2 / 59.3 | 50.0 / 59.7 | 40.2 / 50.6 | Full training sets (278K) |
| PROBE Untrained (Llama-3.1-8B + Contriever) | 20.4 / 30.9 | 48.0 / 60.8 | 52.2 / 61.7 | 40.2 / 51.1 | Prospective pipeline only |
| PROBE (Trained Contriever, Ours) | 25.0 / 34.5 | 53.4 / 66.7 | 59.2 / 67.6 | 45.9 / 56.3 | 15K HotpotQA source (12.7K instances) |
Ablation Study¶
Ablations on reasoning components (Factual Summary and Prospective Query) and distillation supervision hops (Inter-hop, Final-hop, and All-hop), alongside final-hop Multi-Hop Recall (\(\text{MHR}_{\text{final}}@8\)):
| Config | MuSiQue (EM / F1 / MHR@8) | HotpotQA (EM / F1 / MHR@8) | 2WikiMHQA (EM / F1 / MHR@8) | Note |
|---|---|---|---|---|
| Vanilla RAG (w/o Sum, w/o PQ, w/o Dist) | 11.2 / 18.0 / 48.7 | 36.0 / 49.0 / 64.9 | 35.4 / 42.0 / 62.3 | Standard iterative baseline |
| Factual Summary only (+Sum, w/o PQ, w/o Dist) | 19.0 / 28.5 / 59.3 | 44.0 / 58.1 / 77.4 | 44.0 / 52.5 / 76.3 | Retains context only |
| Prospective Query only (w/o Sum, +PQ, w/o Dist) | 21.2 / 31.1 / 59.1 | 45.2 / 57.9 / 71.5 | 45.0 / 53.9 / 77.1 | Active forward exploration |
| Full Reasoning Pipeline (+Sum, +PQ, w/o Dist) | 20.4 / 30.9 / 61.5 | 48.0 / 60.8 / 79.0 | 52.2 / 61.7 / 80.4 | Competitive without retriever tuning |
| + Intermediate Distillation (+Sum, +PQ, Inter-hop) | 23.2 / 32.8 / 69.8 | 53.8 / 66.1 / 87.3 | 54.4 / 62.9 / 92.2 | Intermediate noise hurts QA |
| + All-hop Distillation (+Sum, +PQ, All-hop) | 24.6 / 34.3 / 70.0 | 51.8 / 64.7 / 86.9 | 56.4 / 64.5 / 93.0 | Highest MHR but degraded EM/F1 |
| + Final-hop Distillation (+Sum, +PQ, Final-hop, PROBE) | 25.0 / 34.5 / 68.3 | 53.4 / 66.7 / 85.9 | 59.2 / 67.6 / 90.9 | Highest QA accuracy & robustness |
Key Findings¶
- Final-hop distillation outperforms step-wise intermediate supervision: While intermediate-hop (Inter-hop) and all-hop distillation achieve marginally higher raw recall (e.g., 92.2% - 93.0% MHR@8 on 2WikiMHQA), their downstream QA performance degrades (54.4/62.9 and 56.4/64.5 vs. 59.2/67.6 EM/F1 for Final-hop). Supervising ambiguous intermediate steps overfits the retriever to single speculative paths, whereas final-hop supervision provides clean, trajectory-level alignment.
- Extreme sample efficiency: While ReSCORE trains dataset-specific models on 278K examples across all datasets using costly generative likelihood calculations, PROBE trains a single universal retriever from 15K HotpotQA source queries (filtered to 12,738 instances), surpassing ReSCORE by 5.7 average points and demonstrating zero-shot cross-benchmark generalization.
- Rerank-free efficiency and robust error recovery: Deploying the distilled Bi-Encoder without the Cross-Encoder preserves competitive accuracy while reducing inference latency by 33% to 34% (e.g., from 4.27s to 2.82s on MuSiQue). Moreover, on trajectories with incomplete first-hop retrieval, PROBE achieves a 62.7% retrieval recovery rate, a 50.9% full-chain recovery rate, and a 37.1% semantic answer recovery rate, substantially outperforming ReSCORE (25.6%).
Highlights & Insights¶
- Decoupled reasoning state with context denoising: Separating backward-looking verification (factual summaries) from forward-looking exploration (prospective queries), and filtering speculative queries out of reader context, prevents speculative drift from inducing hallucinations while preserving strong exploratory search momentum.
- End-goal supervision generalizes backwards: PROBE proves that supervising only the final hop of multi-hop trajectories allows learned ranking capabilities to transfer naturally to intermediate steps, challenging the common assumption that multi-hop retrieval requires explicit step-wise supervision.
- Reranker-free deployment via offline margin distillation: Distilling fine-grained cross-encoder preferences into bi-encoders allows systems to eliminate expensive cross-attention rerankers at inference time, delivering high efficiency in production RAG systems.
Limitations & Future Work¶
- Reliance on clean Wikipedia-style corpora: Evaluations were conducted on curated Wikipedia benchmarks. Real-world information-seeking involves noisy, heterogeneous, unstructured documents and underspecified queries where entity boundaries are blurred.
- Persistence of entity and relation selection errors: The error taxonomy reveals that 43.1% (261 of 606 genuine failures) stem from entity/relation selection errors within the retrieved context, showing that retrieval enhancements alone cannot solve generator-side fine-grained constraint reasoning.
- Pre-deployment offline trajectory synthesis overhead: Generating silver trajectories and scoring candidate pools with Cross-Encoders requires substantial offline GPU resources (~15 hours on 3 GPUs). Future work could explore lighter bootstrapping mechanisms and dynamic backtracking retrieval controls.
Related Work & Insights¶
- vs ReSCORE (Lee et al., 2025): ReSCORE uses LLM generative likelihoods to supervise intermediate queries, conflating parametric knowledge with relevance and biasing retrieval toward answer-bearing documents. PROBE adopts prospective queries and final-hop Cross-Encoder distillation, achieving higher accuracy with only ~5% of the training data.
- vs Search-R1 (Jin et al., 2025) & STRIDE (Chen et al., 2026): Search-R1 and STRIDE freeze the retriever and train or prompt the LLM to control search actions. PROBE directly optimizes the dense retriever itself, proving that the aligned retriever is model-agnostic and delivers consistent plug-and-play gains across diverse LLM generators and iterative pipelines (e.g., Iter-Retgen, IRCoT).
- vs IRCoT (Trivedi et al., 2023) & FLARE (Jiang et al., 2023): IRCoT and FLARE rely on interleaved chain-of-thought generation or masked prediction heuristics that frequently suffer from entity drift or loss of entity bindings across multiple hops. PROBE preserves explicit entity and relation constraints in self-contained natural language prospective queries.
Rating¶
- Novelty: ⭐⭐⭐⭐☆ Decouples factual summaries from prospective queries and replaces noisy intermediate step-wise supervision with clean final-hop end-goal distillation.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Evaluated across three multi-hop and three single-hop benchmarks, accompanied by rigorous ablations, latency-accuracy trade-off studies, error recovery rates, query length profiling, and comprehensive failure taxonomies.
- Writing Quality: ⭐⭐⭐⭐⭐ Well-structured, mathematically rigorous, with lucid conceptual explanations and insightful qualitative comparisons.
- Value: ⭐⭐⭐⭐⭐ Provides a highly practical, data-efficient, and rerank-free paradigm for aligning dense retrievers in complex iterative multi-hop reasoning architectures.