ToToFilter: Topic- and Topology-Aware Noise Filtering Framework over Medical KGs for Evidence-Based Medical Q&A¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Knowledge Graphs, Evidence-Based Medical QA, Noise Filtering, Topology-Aware, Entity Localization
TL;DR¶
This paper proposes ToToFilter, a train-free topic- and topology-aware noise filtering framework that addresses entity-level noise from generic high-frequency hubs via Topic-Aware Entity Localization (TEL) and path-level noise from redundant or spurious multi-hop chains via Topology-Aware Path Filtering (TPF), significantly outperforming competitive graph-augmented baselines across four medical QA benchmarks and PrimeKGQA.
Background & Motivation¶
Evidence-Based Medical Question Answering (EB-MQA) seeks to deliver accurate, clinically verifiable decisions grounded in solid biomedical evidence. While large language models (LLMs) paired with retrieval-augmented generation (RAG) offer a promising avenue to mitigate hallucinations, unstructured medical literature frequently introduces lengthy textual contexts and disperses critical diagnostic facts. Retrieving structured relational paths from large-scale general medical knowledge graphs (KGs), such as PrimeKG, provides an attractive alternative with broad relational coverage and explicit reasoning structures. However, directly querying dense, general-purpose medical KGs often injects severe retrieval noise that ironically degrades LLM performance below that of vanilla, unaugmented models.
This performance collapse stems from two interconnected failure modes in existing KG-augmented pipelines. First, Entity-Level Noise occurs when query mentions are aligned to the graph: standard semantic similarity matching tends to drift toward ubiquitous, non-diagnostic hub nodes (such as "Hospital" or "Patient") rather than disease-specific core etiology (such as "Insulin Resistance"). Second, Path-Level Noise emerges even after correct anchors are identified: dense medical KGs provide countless alternative multi-hop paths connecting core entities, many of which pass through high-degree generic hubs or constitute circuitous, weakly diagnostic reasoning chains (such as attributing diabetic etiology to indirect inflammation via "Bacterial Infection"). In natural language prompts, these hub-mediated chains appear deceptively coherent while actively diluting option-discriminative evidence and triggering spurious clinical inferences.
Existing knowledge-augmentation methods attempt to prune graph retrieval using agentic multi-hop expansion, learned subgraph scoring, or iterative self-reflection, but they struggle to balance structural graph constraints with inference latency, incurring prohibitive computational overhead on large graphs. The authors' angle of attack is to decouple and sequentially execute topic-driven anchor selection and topology-regularized path scoring. Core idea: Build a lightweight, train-free two-level filtering framework (ToToFilter) that anchors diagnostically prioritized core entities via Topic-Aware Entity Localization (TEL) and prunes redundant, hub-heavy paths using multi-criteria Topology-Aware Path Filtering (TPF) combined with LLM reranking, delivering compact and option-discriminative evidence chains at minimal latency.
Method¶
Overall Architecture¶
ToToFilter operates as a train-free, two-level filtering pipeline structured across domain screening, entity localization, path filtering, and evidence-grounded answer generation. Given a clinical query \(Q\) and candidate options \(O = \{o_1, \dots, o_n\}\):
1. Domain Pre-screening: A lightweight prompt \(P_{dom}\) categorizes the question as either BIOMEDICAL or OTHER. Non-biomedical queries (e.g., healthcare administration or ethics) bypass graph retrieval entirely and are answered directly by the base LLM, preventing irrelevant graph wandering.
2. Topic-Aware Entity Localization (TEL): Biomedical entities extracted from the question stem and answer choices are ranked by their diagnostic relevance roles (core anchors, key discriminators, target candidates, context) and vector-aligned to KG nodes to construct the anchored node set \(V_Q\).
3. Topology-Aware Path Filtering (TPF): Shortest paths within a hop limit \(H\) are recalled between anchor pairs in \(V_Q\). A coarse topological score balances semantic similarity, path length penalty, and intermediate node hub penalty to prune the candidate set to a compact subset \(T_{cr}\), which is subsequently reranked by an LLM (\(P_{rerank}\)) to retain option-discriminative paths \(\hat{T}\).
4. Answer Generation: Retained paths are translated into verified natural language statements via official PrimeKG verbalization rules (\(P_{verb}\)) and fed alongside \(Q\) and \(O\) into the backbone LLM (\(P_{gen}\)) for step-by-step reasoning and answer selection.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}}%%
flowchart TD
A["Clinical Input: Query Q & Options O"] --> B["Domain Screening Pdom<br/>BIOMEDICAL vs. OTHER"]
B -->|OTHER| Z["Vanilla LLM Direct Answer"]
B -->|BIOMEDICAL| C["Topic-Aware Entity Localization (TEL)<br/>Clinical role priority + Soft vector matching"]
C --> D["Candidate Path Recall ASP≤H<br/>Bounded shortest path retrieval to Tcand"]
D --> E["Multi-Criteria Topology Coarse Filtering<br/>Semantic relevance - Length - Hub penalty"]
E --> F["LLM Discriminative Reranking Prerank<br/>Select top causal evidence paths T_hat"]
F --> G["Final Answer Generation Pgen<br/>Verbalized clinical evidence + CoT prediction"]
Key Designs¶
1. Topic-Aware Entity Localization (TEL): Clinical Role-Prioritized Selection and Soft Vector Anchoring
Standard entity linking over long medical vignettes often extracts superficial context words (e.g., "patient", "clinic"), allowing high-degree graph hubs to hijack downstream path search. TEL overcomes this by decomposing entity anchoring into topic extraction with clinical role ranking followed by vector-based soft matching. First, an LLM extracts all biomedical mentions and explicitly reranks them according to four clinical tiers: (i) Core Anchors (highest priority: primary disease history, administered drugs, exposures, or genetic mutations serving as reasoning origins); (ii) Key Discriminators (specific signs, laboratory anomalies, and pathological mechanisms); (iii) Target Candidates (key entities from answer choices representing potential endpoints); and (iv) Context (lowest priority: non-specific symptoms or general anatomy). The top entities are deduplicated and truncated to at most 5 core concepts \(E_{core} = \{e_i\}\). Next, Sentence-BERT (MPNet) maps each mention to its nearest KG node via cosine similarity:
Applying a strict confidence threshold \(\tau = 0.92\), only matches satisfying \(\cos(\text{Enc}(e_i), \text{Enc}(\hat{v}_i)) > \tau\) enter the anchored set \(V_Q\). If \(|V_Q| < 2\), graph retrieval safely aborts and the system falls back to parametric generation.
2. Multi-Criteria Topology-Aware Coarse Filtering: Joint Semantic Alignment, Length Compactness, and Hub Node Penalization
Even between valid anchor nodes, dense medical KGs yield hundreds of alternative multi-hop paths. TPF first queries Neo4j for all shortest paths between unordered anchor pairs \(\{u, v\} \subseteq V_Q\) up to a maximum hop limit \(H=4\), deduplicating and bounding candidates to at most \(N_{max}=500\) paths \(T_{cand}\). For each candidate path \(p = (v_0, v_1, \dots, v_\ell)\), TPF computes a multi-dimensional coarse score:
where each term imposes complementary semantic and structural constraints: - Semantic Relevance (\(S_{sem}\)): Measures cosine alignment between the textualized path and query \(Q\): \(S_{sem} = \cos(\text{Enc}(Txt(p)), \text{Enc}(Q))\), ensuring evidence remains on-topic. - Structural Compactness Penalty (\(S_{len}\)): Uses path length \(S_{len} = \ell\), applying a linear penalty because longer reasoning chains accumulate confounding noise and dilute causal directness. - Information Specificity Penalty (\(S_{deg}\)): Penalizes paths traversing generic, high-connectivity intermediate nodes via:
Query anchor nodes in \(V_Q\) are exempt from this penalty. The max operator reflects the topological insight that a single generic hub anywhere along an intermediate path destroys its diagnostic specificity. With hyperparameters \((\alpha, \beta, \gamma) = (8.0, 0.5, 0.3)\), coarse ranking retains the top \(N_c = 50\) paths as \(T_{cr}\).
3. LLM Discriminative Path Reranking and High-Fidelity Verbalization (Prerank & Pverb)
Topological coarse filtering purges unspecific and circuitous paths, but subtle option discrimination requires fine-grained reasoning. TPF feeds \(T_{cr}\) into an LLM guided by prompt \(P_{rerank}\) to select up to \(N_f = 10\) paths \(\hat{T}\) that explicitly contrast the options (falling back to top-\(N_f\) coarse paths if the call produces no valid index). Finally, to eliminate logical inversion caused by reading undirected KG triples backwards, \(P_{verb}\) converts \(\hat{T}\) into clinical sentences following official PrimeKG relation templates (e.g., standardizing contraindication to "Patients with [condition] should NOT use [drug]"). These natural language evidence statements are concatenated with the query for final chain-of-thought answer prediction.
Benchmark & Evaluation Protocol¶
- Benchmarks: Evaluated on four clinical QA datasets—MedQA (USMLE-style), MMLU (biomedical/clinical subsets), PubMedQA (biomedical literature reasoning), and MedExQA (expert explanations)—along with PrimeKGQA for ground-truth retrieval quality.
- Backbone Models & Infrastructure: Deployed on two NVIDIA A800 GPUs using vLLM for Llama3.1-8B and Qwen2.5-7B. Hyperparameters are fixed across all benchmarks: \(\tau = 0.92\), \(H = 4\), \(N_{max} = 500\), \(N_c = 50\), and \(N_f = 10\).
- Baselines: Compared against Vanilla LLM, BasicKG (unfiltered shortest-path retrieval), graph prompting agents (GoG, MindMap), subgraph/hypothesis retrievers (SubgraphRAG, HyKGE), and dynamic KG co-augmentation (DALK).
Key Experimental Results¶
Main Results¶
ToToFilter achieves the highest accuracy across all eight backbone–dataset configurations, notably outperforming strong retrieval-augmented systems like DALK.
| Method / Benchmark | MedQA | MMLU | PubMedQA | MedExQA | Average Acc. (%) |
|---|---|---|---|---|---|
| Llama3.1-8B | |||||
| Vanilla LLM | 60.21 | 70.81 | 74.00 | 70.85 | 68.97 |
| BasicKG | 49.33 | 57.43 | 74.40 | 70.11 | 62.82 |
| GoG | 55.93 | 64.70 | 68.40 | 55.53 | 61.14 |
| MindMap | 60.80 | 70.71 | 74.60 | 70.11 | 69.06 |
| SubgraphRAG | 59.54 | 73.55 | 73.40 | 72.23 | 69.68 |
| HyKGE | 59.35 | 77.52 | 74.40 | 72.28 | 70.89 |
| DALK (Prev. SOTA) | 65.99 | 77.32 | 76.20 | 74.79 | 73.58 |
| ToToFilter (Ours) | 68.03 | 79.87 | 78.80 | 76.81 | 75.88 (+2.30) |
| Qwen2.5-7B | |||||
| Vanilla LLM | 51.14 | 65.33 | 68.80 | 73.94 | 64.80 |
| BasicKG | 54.99 | 74.92 | 71.20 | 73.40 | 68.63 |
| GoG | 57.74 | 74.29 | 67.60 | 72.66 | 68.07 |
| MindMap | 51.14 | 67.86 | 71.60 | 74.89 | 66.37 |
| SubgraphRAG | 59.39 | 74.39 | 72.23 | 73.40 | 69.85 |
| HyKGE | 56.95 | 78.71 | 73.80 | 76.06 | 72.11 |
| DALK (Prev. SOTA) | 60.30 | 77.66 | 69.20 | 75.64 | 69.97 |
| ToToFilter (Ours) | 63.08 | 79.35 | 74.60 | 76.28 | 73.33 (+3.36) |
On PrimeKGQA, ToToFilter demonstrates superior evidence retrieval: on Llama3.1-8B, it achieves 43.25% Answer Recall@10 (vs. DALK's 32.67%) and 32.19% Triple Recall@10 (vs. DALK's 13.68%, a +18.51 point lead); on Qwen2.5-7B, it attains 40.49% Triple Recall@10, outperforming all baselines by substantial margins.
Ablation Study¶
Ablating the scoring terms in TPF reveals their individual and combined effects on reasoning accuracy and graph topological compactness.
| ID | Sem (\(S_{sem}\)) | Len (\(S_{len}\)) | Deg (\(S_{deg}\)) | MedQA Acc (%) | Avg. Hops \(\downarrow\) | Avg. Degree \(\downarrow\) | MMLU Acc (%) |
|---|---|---|---|---|---|---|---|
| 0 | ✗ | ✗ | ✗ | 61.35 | 2.93 | 2380.39 | 78.40 |
| 1 | ✓ | ✗ | ✗ | 61.84 | 2.86 | 2142.04 | 78.81 |
| 2 | ✗ | ✓ | ✗ | 61.32 | 2.67 | 1138.59 | 78.45 |
| 3 | ✗ | ✗ | ✓ | 61.43 | 2.73 | 1018.21 | 78.75 |
| 4 | ✓ | ✓ | ✗ | 62.97 | 2.53 | 1183.57 | 78.97 |
| 5 | ✓ | ✗ | ✓ | 62.69 | 2.59 | 1105.76 | 79.16 |
| 6 | ✗ | ✓ | ✓ | 62.57 | 2.53 | 1034.76 | 79.24 |
| 7 (Full) | ✓ | ✓ | ✓ | 63.08 | 2.53 | 1058.64 | 79.35 |
In broader module ablations, removing TEL (w/o TEL: naive top-5 entity truncation without threshold) drops MedQA accuracy on Llama3.1-8B from 68.03% to 60.21% (collapsing to vanilla LLM level). Removing TPF (w/o TPF: selecting raw top-10 recalled paths) drops accuracy to 67.01%, corroborating the necessity of two-level filtering.
Key Findings¶
- Unfiltered Graph Evidence Degrades LLMs: BasicKG drops by up to 10.88 percentage points below the vanilla LLM on MedQA (49.33% vs. 60.21%), proving that injecting noisy, hub-mediated graph paths severely misleads clinical inference unless rigorous filtering is enforced.
- Degree Penalization (\(S_{deg}\)) Drastically Tames Hub Drift: While semantic relevance alone leaves the average intermediate node degree at 2142.04, adding the logarithmic hub penalty (\(S_{deg}\)) cuts average degree by more than half to 1058.64, successfully stripping non-specific nodes like "Hospital" and "Patient".
- Pareto Efficiency Frontier: ToToFilter firmly occupies the upper-left Pareto frontier of accuracy versus inference latency. On Llama3.1-8B/MedQA, ToToFilter achieves 68.03% accuracy in 12.32s per iteration, whereas DALK requires 50.08s for 65.99%, and HyKGE requires 47.39s for 59.35%. Under Qwen2.5-7B, ToToFilter consumes 46.9% fewer API tokens than DALK and 38.2% fewer than MindMap.
Highlights & Insights¶
- Zero-Parameter Topological Proxy: By utilizing the simple closed-form term \(\max \log(1+\text{deg}(v))\), ToToFilter suppresses graph hub contamination without training graph neural networks or computing dense subgraph metrics, achieving instant computation on large KGs.
- Train-Free, Zero-Overhead Adaptation: The framework requires no fine-tuning or parameter updates on the backbone LLM or KG embeddings, making it immediately transferable across different model scales (7B to 8B) and evolving clinical knowledge graphs.
- Target-Aware Bidirectional Constraint: Anchoring answer choice mentions alongside query stem concepts establishes a target-oriented search space that avoids open-ended, undirected graph exploration.
Limitations & Future Work¶
- Static Fact Incompatibility with Dynamic Medicine: General KGs primarily store static relational assertions, making it difficult to capture temporal disease progression, staging, dosage dependencies, and individual patient comorbidities.
- Shortest-Path Truncation Horizon: Restricting path recall to shortest paths within \(H=4\) hops and \(N_{max}=500\) candidates may occasionally prune multi-system, non-shortest causal chains in rare complex syndromes.
- Future Directions: Exploring query-adaptive dynamic weight adjustment for \((\alpha, \beta, \gamma)\), joint structural-semantic representation learning, and extending ToToFilter to heterogeneous clinical corpora (such as EHRs and clinical practice guidelines).
Related Work & Insights¶
- vs. DALK (Li et al., 2024): DALK uses dynamic KG augmentation with iterative self-reflection loops, resulting in high latency (>50s/query); ToToFilter uses lightweight geometric and topological pruning with a single reranking pass, attaining superior accuracy at a fraction of the inference time.
- vs. MindMap (Wen et al., 2024): MindMap builds graph-of-thought prompts from retrieved subgraphs but lacks explicit mathematical penalties for high-degree hub nodes; ToToFilter actively suppresses generic nodes via TEL and degree penalization before prompt construction.
- vs. SubgraphRAG (Li et al., 2025): SubgraphRAG scores individual triples using an MLP to assemble subgraphs, which can sever continuous multi-hop reasoning chains; ToToFilter directly evaluates and ranks coherent multi-hop relational paths, preserving intact causal evidence.
Rating¶
- Novelty: ⭐⭐⭐⭐ [Clear and practical two-level formulation addressing entity- and path-level graph noise via lightweight topological scoring]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Evaluated across four MQA benchmarks and PrimeKGQA with paired significance tests, detailed ablations, and Pareto latency analysis]
- Writing Quality: ⭐⭐⭐⭐⭐ [Well-structured narrative, rigorous mathematical formulations, and clear clinical case studies]
- Value: ⭐⭐⭐⭐ [Provides an efficient, deployable blueprint for evidence-grounded medical RAG over complex knowledge graphs]