BEAR: Budgeted Evidence Allocation for Multi-Document Reasoning¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Evidence Allocation, Hierarchical Indexing, Budget-Aware Reasoning, Multi-Document Reasoning, Exploration and Recovery
TL;DR¶
Addressing the core tension between prohibitive compute costs in long-context LLMs and fragmented contextual isolation in flat retrieval, BEAR organizes multi-document collections into hierarchical semantic forests offline and dynamically balances high-level LLM exploration with structural TreeExpansion recovery under a strict token budget, outperforming prior retrieval baselines across nine benchmarks with an approximately 4K-token evidence context.
Background & Motivation¶
Multi-document reasoning in real-world applications is fundamentally constrained not merely by how many tokens an LLM generator can digest in its context window, but by how a strictly budgeted query-time evidence bandwidth should be distributed across heterogeneous source documents and hierarchical semantic granularities. In complex knowledge-intensive tasks—such as multi-hop question answering, long-form scientific analysis, and agentic multi-source research—relevant facts rarely reside in a single cohesive paragraph. Instead, critical premises are often dispersed across distinct documents or buried amidst extensive distracting discourse, making unconstrained context ingestion both computationally unsustainable and logically brittle.
Directly feeding full document corpora into extreme-context models incurs steep per-query token and latency costs while exacerbating the well-documented "Lost-in-the-Middle" phenomenon, where non-selective exposure dilutes parametric attention and triggers catastrophic hallucinations. Conversely, standard retrieval-augmented generation relying on flat chunking over-retrieves locally similar snippets that lack overarching discourse coherence and global narrative structure, leaving the generator with isolated fragments ill-suited for cross-document synthesis. While recent structured systems (e.g., GraphRAG, RAPTOR, and HiRAG) introduce graph or tree topologies, they predominantly treat hierarchy as an offline indexing convenience rather than an active query-time resource allocation substrate, failing to provide principled mechanisms for trading coarse abstraction against fine-grained local facts under strict latency and token budgets.
This paper tackles the challenge by reframing multi-document reasoning as a structured, budget-aware evidence allocation problem. Under a fixed final context budget, an optimal system should expose coarse semantic outlines to locate promising regions globally and only descend into local detailed evidence when fine discrimination is strictly warranted. Core idea: formulate multi-document reasoning as budgeted evidence allocation over offline hierarchical semantic trees, using dual-path LLM-guided exploration and tree-structural recovery to dynamically deliver high-coverage, low-hallucination evidence under a compact 4K-token budget.
Method¶
Overall Architecture¶
BEAR operates across two primary stages: offline hierarchical knowledge organization and online budget-aware dual-path evidence routing. Offline, for each document in the corpus \(\mathcal{D} = \{d_1, \dots, d_N\}\), BEAR applies LLM-guided semantic chunking to preserve discourse boundaries, constructs a hierarchical semantic tree \(T_i\) with summary-bearing internal nodes and raw chunk leaves, and establishes multi-granularity vector indices. Online, given a user query \(q\) and a final evidence budget \(B_{\max}\) (defaulting to approximately 4K tokens), BEAR first performs coarse-grained document candidate retrieval and fusion. If the fused document text satisfies \(B_{\max}\), it is returned directly as BEAR(docs); otherwise, the system invokes BEAR(nodes) for fine-grained node-level refinement, running an LLM-guided exploration path alongside a structure-propagating TreeExpansion recovery path. The retrieved candidates are reconciled in the NodeFusion module via deduplication, structural prioritization, and document-order sorting before being passed to a shared Qwen3-32B generator.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Query & Document Corpus"] --> B["Offline Hierarchical Knowledge Organization<br/>Semantic Chunking & Tree Forest Construction"]
B --> C["Multi-Granularity Dense Indexing<br/>Path-Summary & Leaf Embeddings"]
C --> D["Budget-Aware Coarse Evidence Access<br/>Document Candidate Fusion & Bmax Check"]
D -->|Within Budget| E["BEAR(docs) Document-Level Context"]
D -->|Exceeds Budget| F["Complementary Exploration & Recovery<br/>LLM Summary Navigation + TreeExpansion Scoring"]
F --> G["NodeFusion & Budget Truncation<br/>Ancestor-Descendant Deduplication & Ordering"]
E --> H["Qwen3-32B Final Generation"]
G --> H
Key Designs¶
1. Offline Hierarchical Knowledge Organization: replacing arbitrary fixed-length slicing with discourse-coherent trees and multi-granularity dual embeddings
Traditional retrieval pipelines chop documents into rigid, fixed-length token windows, inevitably fracturing critical relational propositions and paragraph-level discourse coherence. BEAR overcomes this by deploying an LLM semantic segmenter \(C_i = \text{LLMsegment}(d_i) = \{c_1, \dots, c_{m_i}\}\), ensuring that every chunk aligns with a coherent semantic proposition. Next, an LLM structures these chunks into a rooted semantic hierarchy \(T_i = \text{LLMstructure}(C_i \mid d_i)\) consisting of document-level root, section-level thematic groups, sub-sections, and original chunk leaves. To enable scalable construction on long documents, BEAR applies a progressive partition-and-merge strategy: building partial trees over partitions and merging them via \(\text{TREEMERGE}\). For multi-granularity indexing, BEAR creates two distinct embedding representations: non-leaf internal nodes \(v\) are encoded using their hierarchical path and summary: $\(e_v = \text{Embed}(\text{toc\_path}(v) \oplus \text{summary}(v))\)$ while leaf nodes \(c\) encode raw chunk content \(e_c = \text{Embed}(\text{content}(c))\). This allows the retriever to index and query both global thematic summaries and local evidence details within a unified forest representation.
2. Budget-Aware Coarse-to-Fine Routing: token-budget gating between document-level fusion and node-level structural refinement
To prevent premature degradation of document-level discourse, BEAR establishes a strict token budget \(B_{\max}\) on the final evidence context fed into the generator. At query time, BEAR first executes coarse evidence access: an exploration path prompts an LLM with high-level document outlines and level-1 section summaries to select relevant document IDs \(\mathcal{D}_{\text{llm}}\), while a dense retrieval path retrieves top document candidates \(\mathcal{D}_{\text{vector}}\). Their union forms a deduplicated set \(\mathcal{D}_{\text{fusion}}\). If the aggregated content of \(\mathcal{D}_{\text{fusion}}\) fits within \(B_{\max}\), the system directly outputs this document-level context (BEAR(docs)), maintaining complete contextual narrative for the generator. Only when the coarse context exceeds \(B_{\max}\), or when fine-grained discrimination is explicitly necessary, does the system trigger BEAR(nodes) to drill down into node-level allocation, preventing unnecessary evidence shredding when global document context is already affordable.
3. Complementary Exploration and Recovery: combining LLM semantic reasoning with tree-restricted structural score propagation
In the node-level refinement phase, relying purely on LLM selection suffers from "conclusion-node selection bias"—the model gravitates toward high-level summary nodes whose subtrees fail to cover necessary evidence-bearing sibling branches (such as auxiliary clinical findings or timeline steps). Conversely, pure vector search struggles across multi-hop reasoning branches. BEAR resolves this by deploying two orthogonal, complementary retrieval paths: - Exploration Path: The LLM reviews the table of contents and summaries of internal nodes within the candidate documents to reason about which semantic subtrees contain supporting evidence, selecting explicit node IDs \(\mathcal{N}_{\text{llm}}\). - Recovery Path (TreeExpansion): A lightweight, parameter-free tree propagation mechanism scores any node \(v\) by combining direct query similarity, ancestral contextual inheritance, and descendant evidentiary support: $\(S(v) = \alpha S_{\text{sim}}(v) + \beta S_{\text{inh}}(v) + \gamma S_{\text{child}}(v)\)$ Using an untuned equal-weight default (\(\alpha = \beta = \gamma = \frac{1}{3}\)), TreeExpansion rescues deep leaf nodes that direct semantic similarity misses or that fall outside the LLM's narrow summary selection.
4. NodeFusion and Budget Truncation: structure-aware deduplication, priority-based partitioning, and document-position preservation
Merging candidate nodes from disparate retrieval paths inevitably introduces redundancy and structural disorder. NodeFusion applies a deterministic three-step post-processing protocol: First, it eliminates ancestor-descendant overlaps—if both a parent summary and its descendant chunk are recalled, it removes redundant parent abstractions to maximize evidence density. Second, it partitions nodes into those explicitly selected by the LLM (high semantic priority) and those introduced via TreeExpansion (structural support). Third, candidate nodes within each document are restored to their original physical reading order in the source document. Finally, the ordered evidence list is truncated exactly at the token budget \(B_{\max}\), feeding the downstream generator with a coherent, non-redundant, and budget-compliant context.
Benchmark & Evaluation Protocol¶
BEAR is evaluated across nine distinct benchmarks encompassing synthetic knowledge allocation, multi-hop reasoning, long-document QA, conversational memory, open-domain QA, and agentic multi-turn retrieval. To eliminate parametric Wikipedia memory leakage in LLMs, primary controlled comparisons are performed on the synthetic DragonBall benchmark across Recall, Completeness, Hallucination, and Irrelevance metrics. Real-world evaluations span HotpotQA, 2WikiMultiHopQA, MuSiQue, PopQA, NarrativeQA, LoCoMo, and QASPER. Downstream agentic transfer is assessed on BrowseComp-plus with a fixed Tongyi-DeepResearch-30B-A3B agent. In all matched retrieval experiments, offline structuring and online retrieval LLMs are standardized to DeepSeek-V3.2, dense retrieval uses BGE-M3, and final generation is performed by Qwen3-32B restricted to an approximately 4K-token retrieved context budget.
Key Experimental Results¶
Main Results¶
The table below reports the primary experimental results on DragonBall (synthetic knowledge) as well as HotpotQA and 2WikiMultiHopQA (real-world multi-hop knowledge), presenting mean \(\pm\) standard deviation across 3 repeated runs:
| Method Category & Model | DragonBall Recall(%) ↑ | DragonBall Comp.(%) ↑ | DragonBall Hall.(%) ↓ | DragonBall Irr.(%) ↓ | HotpotQA EM(%) ↑ | HotpotQA Ans F1(%) ↑ | 2Wiki EM(%) ↑ | 2Wiki Ans F1(%) ↑ |
|---|---|---|---|---|---|---|---|---|
| Matched Retrieval Baselines | ||||||||
| BM25 | 29.17 | 35.13 | 32.55 | 32.25 | 20.67 | 32.69 | 10.33 | 18.12 |
| BGE-M3 | 58.17 ± 1.06 | 59.56 ± 1.84 | 28.81 ± 2.40 | 11.54 ± 1.95 | 22.83 ± 0.76 | 36.25 ± 0.53 | 15.83 ± 1.44 | 26.94 ± 1.32 |
| BGE-M3 + BGE-Reranker | 66.82 ± 1.94 | 61.33 ± 2.42 | 28.78 ± 2.26 | 9.81 ± 0.94 | 23.83 ± 1.31 | 38.61 ± 1.28 | 20.00 ± 0.40 | 29.21 ± 1.00 |
| RAPTOR | 31.93 ± 1.66 | 59.78 ± 2.70 | 26.50 ± 2.48 | 13.67 ± 1.29 | 29.33 ± 1.26 | 44.72 ± 1.70 | 23.50 ± 1.00 | 36.96 ± 1.30 |
| LongRefiner | 29.24 ± 2.32 | 43.86 ± 1.98 | 34.88 ± 2.25 | 21.26 ± 1.87 | 22.33 ± 0.85 | 33.97 ± 0.71 | 9.00 ± 0.87 | 13.16 ± 0.83 |
| HippoRAG2 | 68.80 ± 1.75 | 73.37 ± 2.24 | 19.55 ± 1.84 | 6.93 ± 0.93 | 33.83 ± 0.58 | 48.99 ± 1.22 | 29.67 ± 0.76 | 44.19 ± 0.73 |
| Full-Context Controls / References | ||||||||
| Qwen3-32B (Full-Context) | – | 66.86 ± 0.30 | 25.36 ± 0.33 | 7.70 ± 0.01 | – | – | – | – |
| Gemini-2.5-Flash (Full-Ctx) | – | 88.62 ± 0.34 | 6.30 ± 0.14 | 5.09 ± 0.09 | 29.83 ± 1.31 | 45.96 ± 1.06 | 38.67 ± 0.24 | 55.21 ± 0.27 |
| Gemini-2.5-Pro (Full-Ctx) | – | 90.86 ± 0.07 | 5.46 ± 0.24 | 3.67 ± 0.04 | 41.74 ± 0.33 | 56.66 ± 0.90 | 53.01 ± 1.41 | 64.69 ± 0.72 |
| Proposed Method | ||||||||
| BEAR(docs) | 85.55 ± 0.47 | 91.86 ± 1.32 | 5.64 ± 1.02 | 2.40 ± 0.49 | 31.00 ± 1.15 | 45.80 ± 1.18 | 34.63 ± 2.17 | 48.02 ± 1.45 |
| BEAR(nodes) | 88.64 ± 0.84 | 88.64 ± 1.52 | 7.60 ± 1.12 | 3.71 ± 0.86 | 33.50 ± 1.15 | 48.44 ± 0.54 | 33.00 ± 1.73 | 46.30 ± 1.48 |
On broader real-world benchmarks, BEAR sets new state-of-the-art results across varied modalities (official benchmark answer-level F1%): - LoCoMo (Conversational Memory): BM25 14.04% / BGE-M3 17.21% / HippoRAG2 14.60% \(\to\) BEAR(docs) 26.42% (+9.21% over the best baseline); - QASPER (Scientific Paper QA): BM25 12.70% / BGE-M3 16.14% / HippoRAG2 15.55% \(\to\) BEAR(nodes) 27.15% (+11.01% over the best baseline); - MuSiQue (Multi-Hop QA): BM25 14.99% / BGE-M3 22.68% / HippoRAG2 25.52% \(\to\) BEAR(nodes) 32.46% (+6.94% over HippoRAG2); - PopQA (Open-Domain QA): BM25 40.25% / BGE-M3 49.16% / HippoRAG2 47.69% \(\to\) BEAR(docs) 58.39%; - NarrativeQA (Long-Document Story QA): BM25 5.67% / BGE-M3 9.25% / HippoRAG2 11.47% \(\to\) BEAR(nodes) 14.58%.
In the BrowseComp-plus agentic evaluation, plugging BEAR(nodes) into Tongyi-DeepResearch-30B boosted task accuracy from 44.46% to 66.60% (+22.14 pp) and recall from 62.32% to 76.60%, while reducing search tool calls from 30.37 to 21.74.
Ablation Study¶
Progressive ablation of node-level evidence access components on DragonBall under a 4K token evidence budget:
| Routing Configuration | Recall(%) ↑ | Completeness(%) ↑ | Hallucination(%) ↓ | Irrelevance(%) ↓ |
|---|---|---|---|---|
| Leaf-only (Flat dense retrieval over semantic chunks) | 50.55 | 54.83 | 26.20 | 18.82 |
| Tree-aware (Dense retrieval over hierarchical tree nodes) | 87.91 | 84.12 | 9.96 | 5.92 |
| Tree + LLM-guided (Exploration path only) | 89.25 | 88.13 | 7.67 | 4.20 |
| Tree + TreeExp. (Recovery path only) | 91.55 | 85.93 | 9.03 | 5.04 |
| BEAR(nodes) (Dual-path bi-path fusion) | 98.19 | 90.36 | 5.28 | 4.23 |
Attribution analysis of gold-supporting evidence reveals non-redundant coverage across query types: - Shared coverage by both paths accounts for 80.7%; - TreeExpansion uniquely recovers 10.3% of gold evidence; - LLM-guided path uniquely recovers 6.1% of gold evidence; - Neither path misses only 2.9% overall (concentrated in deep medical comparison leaves).
Key Findings¶
- Hierarchical indexing serves as the primary performance driver: Moving from Leaf-only to Tree-aware retrieval yields the single largest performance jump, boosting Recall from 50.55% to 87.91% (+37.36 pp) and slashing Hallucination from 26.20% to 9.96%. Exposing internal summary nodes allows the retriever to anchor global semantics before local chunk matching.
- Exploration and recovery exhibit asymmetric complementarity: The recovery path (TreeExpansion) outperforms on factual and chronological queries (FQ: 95.42% vs 87.50%; TSQ: 95.42% vs 89.65%) where localized order matters. Conversely, the LLM-guided exploration path excels in synthesis-heavy reasoning (IIQ: 88.39%; NCQ: 90.35%). Fusing both paths provides an average boost of +6.01 pp across all answerable question types.
- Structured allocation trumps unselective context expansion: Supplying Qwen3-32B with the unpruned full context achieves only 66.86% Completeness and 25.36% Hallucination. BEAR with a strict 4K budget achieves 91.86% Completeness with only 5.64% Hallucination. Organization, not raw context capacity, dictates reasoning reliability.
Highlights & Insights¶
- Repositioning hierarchy from storage to allocation: BEAR reframes tree-structured indices from passive vector storage hierarchies into active, budget-governed routing vehicles that dynamically regulate evidence granularity.
- Countering conclusion-node bias via structural propagation: The paper formally identifies a critical failure mode in LLM-guided search—where the model prematurely latches onto a summary node and misses essential sibling data—and overcomes it through an elegant, parameter-free tree diffusion formula.
- Latency-quality Pareto efficiency: Despite expending 27.95K tokens in intermediate retrieval reasoning, BEAR reduces the generator's search space and hesitation tokens, finishing generation in 16.71s for an end-to-end latency of 23.52s, outperforming Standard RAG (27.48s) and HippoRAG2 (32.57s).
Limitations & Future Work¶
- Structural prerequisite of document discourse: BEAR's performance gains depend heavily on whether source documents possess coherent topical organization from which an LLM can distill meaningful parent summaries; gains attenuate on noisy, fragmented corpora.
- Absence of explicit cross-document relational edges: Semantic trees are constructed per document without cross-document entity knowledge graphs. Consequently, on entity-centric multi-hop benchmarks like HotpotQA, BEAR(nodes) achieves 48.44% Answer F1, remaining slightly behind HippoRAG2 (48.99%).
- Offline indexing and maintenance overhead: Building hierarchical summaries requires nontrivial offline compute (~2.65M output tokens per corpus). Incremental updates currently necessitate rebuilding affected document subtrees.
Related Work & Insights¶
- vs RAPTOR: RAPTOR relies on recursive bottom-up clustering and flattens all tree levels into a single dense vector pool; BEAR respects document-native discourse trees and executes controlled, top-down budgeted routing, avoiding context contamination by mixed-granularity duplicates.
- vs HippoRAG / HippoRAG2: HippoRAG constructs graph structures over extracted entity-relation triples and executes Personalized PageRank; BEAR circumvents schema-dependent triple extraction by operating over semantic document forests, demonstrating superior robustness on non-entity-centric text (e.g., narrative QA and financial reports).
- vs LongRefiner: LongRefiner relies on a separately trained auxiliary model for chunk pruning; BEAR utilizes off-the-shelf LLMs with parameter-free TreeExpansion propagation, offering seamless zero-shot transfer across languages and domains.
Rating¶
- Novelty: ⭐⭐⭐⭐ [Principled formulation of budgeted coarse-to-fine allocation with orthogonal dual-path exploration and recovery]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive 9-benchmark evaluation encompassing synthetic controls, real-world QA, latency, and agentic workflows]
- Writing Quality: ⭐⭐⭐⭐⭐ [Exemplary clarity in problem formulation, intuitive diagrams, and insightful failure mode analysis]
- Value: ⭐⭐⭐⭐⭐ [Provides a definitive blueprint for building cost-effective, high-accuracy RAG architectures in the long-context era]