Skip to content

MedF^2Route: Evidence-Routed Graph Verification for Hallucination Detection in Clinical LVLMs

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/Agentic-CliniAI/MedF2Route
Area: Medical NLP
Keywords: Hallucination Detection, Clinical LVLM, Evidence Routing, Clinical Relational Graph, Faithfulness and Factuality

TL;DR

Addressing the intertwined nature of faithfulness and factuality hallucinations and evidence misalignment in clinical LVLMs, MedF²Route structures clinical contexts into graphs, dynamically routes response triplets to heterogeneous patient and knowledge evidence sources for dual-path verification, and achieves substantial gains in detection precision and overall Macro-F1 across diverse backbones.

Background & Motivation

Clinical large vision-language models (LVLMs) are increasingly deployed for auxiliary diagnostic reasoning, yet their clinical adoption and reliability are fundamentally undermined by pervasive hallucinations. In real-world clinical consultations, diagnostic evaluation demands rigorous cross-verification across heterogeneous, multi-source evidence. On one hand, faithfulness-oriented hallucinations arise when the model diverges from patient-specific evidence, including the clinical history, symptoms, laboratory findings, and medical imaging. On the other hand, factuality-oriented hallucinations occur when diagnostic deductions or pathophysiological assertions contradict established medical knowledge bases and clinical guidelines. In practice, these two dimensions are deeply intertwined: a clinically sound diagnostic statement must faithfully ground itself in the patient's individual findings while strictly conforming to universally accepted medical facts.

Existing black-box hallucination detection pipelines predominantly employ claim-level decomposition protocols that evaluate hallucinations along a single dimension. Faithfulness-focused frameworks break responses into atomic claims and verify them against patient text or images using natural language inference (NLI) classifiers or visual grounding heuristics. Conversely, factuality-oriented pipelines decompose text into standalone claims to perform retrieval-augmented fact-checking against external biomedical literature. Crucially, such fine-grained atomic fragmentation severs contextual dependencies and multi-step causal reasoning chains essential for clinical deduction—such as linking lesion anatomical morphology, invasive margins, and secondary malignancy risk. While recent methods like UniHD attempt joint multi-source verification, they aggregate heterogeneous multimodal evidence into an undifferentiated prompt, risking severe evidence misalignment. Meanwhile, graph-based methods like GraphEval remain confined to unimodal text-grounded faithfulness.

To overcome both contextual fragmentation and evidence misalignment, the core essence of black-box clinical hallucination detection must be recognized as an evidence routing problem. Without an explicit mechanism to guide which specific claim should be verified against which reference source, monolithic prompts and undifferentiated classifiers inevitably suffer from acute trade-offs between precision and recall. Core idea: Model multimodal clinical cases and LVLM responses as structured relational clinical graphs partitioned into query-following and diagnostic inference subgraphs, dynamically dispatch each response triplet to its dedicated heterogeneous evidence source (patient text, medical image, or sufficiency-expanded knowledge graph) for dual-path verification, and aggregate triplet signals via conservative max-pooling for sentence-level detection.

Method

Overall Architecture

MedF²Route operates through a principled four-stage verification architecture: (1) Clinical Graph Construction, which extracts structured, attribute-preserving clinical relational graphs \(G_{x_t}\) and \(G_y\) from free-text patient queries \(X=\{x_t, x_v\}\) and model responses \(Y\); (2) Evidence-Aware Routing, which semantically disentangles \(G_y\) into a query-following subgraph \(G_y^{qry}\) and an inference subgraph \(G_y^{inf}\), further dispatching triplets to patient text, medical images, or medical knowledge; (3) Dual-Path Verification, which validates faithfulness via graph matching and multimodal visual grounding while evaluating factuality against a compact, sufficiency-expanded medical reference knowledge subgraph \(G_{ref}\); and (4) Granularity Alignment, which projects fine-grained triplet-level verification labels back to sentence-level binary predictions using conservative max-pooling.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multimodal Clinical Input<br/>Case Text x_t + Medical Image x_v + LVLM Response Y"] --> B["Clinical Graph Construction<br/>Extract Structured Relational Graphs G_xt and G_y"]
    B --> C["Evidence-Aware Routing<br/>Semantic Partition (G_y^qry vs G_y^inf) + Evidence Dispatch"]
    C -->|Text / Image Branch| D["Dual-Path Verification<br/>Faithfulness Branch (Text Graph Match / Visual Grounding)"]
    C -->|Knowledge Graph Branch| E["Dual-Path Verification<br/>Factuality Branch (Sufficiency-Guided Multi-Hop KG)"]
    D --> F["Granularity Alignment<br/>Conservative Max-Pooling to Sentence Predictions"]
    E --> F
    F --> G["Sentence-Level Detection z_hat<br/>Precise Identification of Unsupported Clinical Relations"]

Key Designs

1. Clinical Graph Construction: Preserving contextual relationships and fine-grained clinical attributes
To resolve the contextual fragmentation inherent in atomic claim-level decomposition, MedF²Route implements an instruction-guided clinical relation extractor \(\mathcal{G}\) that maps unstructured text into directed clinical relational graphs \(G = (C, E)\). The concept set \(C\) encompasses clinically salient entities including symptoms, anatomical structures, imaging observations, diagnoses, and procedures, while directed edges represent clinical relation triplets \(r = \langle c_h, \ell, c_t \rangle\). Crucially, the extractor explicitly preserves essential clinical modifiers—such as negation, certainty, anatomical laterality, temporality, and severity attributes—whenever they alter clinical semantics. The patient case text is transformed into query graph \(G_{x_t} = (C_{x_t}, E_{x_t})\), and the response text is partitioned into numbered sentences and mapped to response graph \(G_y = (C_y, E_y)\), where each extracted triplet retains an explicit sentence index anchor. This relational representation preserves multi-step clinical reasoning context while eliminating assignment ambiguity during downstream sentence-level aggregation.

2. Evidence-Aware Routing: Preventing cross-dimensional evidence misalignment via two-step dispatch
To prevent cross-dimensional signal interference caused by concatenating heterogeneous evidence into undifferentiated prompts, MedF²Route introduces a sequential evidence-aware routing mechanism \(\rho\): $\(\rho(G_y) \to \{G_y^{qry,txt},\, G_y^{qry,img},\, G_y^{inf,kg}\}\)$ The first step executes dimensional partitioning: it classifies each response triplet as either query-following (\(G_y^{qry}\)) or diagnostic inference (\(G_y^{inf}\)). Triplets that summarize, restate, or directly follow patient-specific findings from \(X\) are routed to the faithfulness verification pathway. Conversely, triplets that formulate diagnostic deductions, etiological interpretations, differential diagnoses, or prognostic evaluations are routed to the factuality pathway. The second step performs evidence-source routing: within \(G_y^{qry}\), triplets specifying directly observable imaging attributes (e.g., lesion morphology, density, laterality, signal intensity, enhancement, or spatial borders) are allocated to the medical image \(x_v\), forming \(G_y^{qry,img}\), while the remainder form \(G_y^{qry,txt}\). For \(G_y^{inf}\), the evidence source is designated as the reference medical knowledge subgraph \(G_{ref}\), forming \(G_y^{inf,kg}\). This constrained routing isolates verification scopes and prevents spurious evidence cross-talk.

3. Dual-Path Verification: Decoupled multimodal grounding and sufficiency-guided multi-hop knowledge verification
To accommodate distinct verification demands, MedF²Route deploys evidence-specific verifiers \(\hat{y}_r = \mathcal{H}(r, E_r)\). In the faithfulness branch, for \(r \in G_y^{qry,txt}\), subgraph matching against \(G_{x_t}\) identifies whether the case fact is directly stated or semantically entailed, permitting minor lexical variants while flagging ungrounded patient claims. For \(r \in G_y^{qry,img}\), a vision-language judge compares the core visual finding against image \(x_v\), flagging a hallucination if the visual finding is contradicted or unsupported by observable image evidence. In the factuality branch, recognizing that raw knowledge graphs are either excessively large or relationally incomplete, the system constructs a compact reference subgraph \(G_{ref}\) from LMKG (containing 327,707 processed biomedical entities). Using query and response concepts as seeds, one-hop neighbors are retrieved and pruned by semantic embedding similarity (\(\delta = 0.7\)). Subsequently, a sufficiency-guided multi-hop expansion iteratively incorporates frontier nodes that complete relation paths or bridge subject and object concepts under a depth budget \(T=5\), stopping as soon as the subgraph provides sufficient evidence to support or refute the assertion. The factuality verifier then evaluates \(r \in G_y^{inf,kg}\) against \(G_{ref}\).

4. Granularity Alignment: Conservative max-pooling enforcing clinical zero-tolerance safety
To align fine-grained triplet-level verification with sentence-level diagnostic assessment, MedF²Route executes deterministic aggregation based on the sentence anchors established during graph construction. Let \(R(s_t)\) denote the set of response triplets extracted from sentence \(s_t\). The sentence-level binary prediction \(\hat{z}_t \in \{0, 1\}\) is computed via conservative max-pooling: $\(\hat{z}_t = \max_{r \in R(s_t)} \hat{y}_r\)$ In clinical diagnostic contexts, safety considerations dictate that partial correctness cannot redeem an unsupported assertion; even if an entire sentence accurately describes normal anatomy, introducing a single fabricated lesion or an anatomically impossible tumor metastasis invalidates the clinical reliability of the statement. Therefore, a sentence is classified as hallucinated (\(\hat{z}_t = 1\)) if any constituent clinical relation is unsupported by patient evidence or inconsistent with established medical knowledge.

Key Experimental Results

Main Results

Evaluations were conducted on MedF²Bench, comprising 500 multimodal clinical cases, 585 medical images spanning five modalities (CT, ECG, MRI, ultrasound, and X-ray), and 2,383 annotated response sentences. Performance is reported under the case-averaged sentence-level protocol across three distinct VLM backbones: GPT-5.4-nano, InternVL-3.5, and Gemini-3.1-flash-lite.

Backbone Target Type Method Granularity Hall. P (%) Hall. R (%) Hall. F1 (%) Overall Acc (%) Overall Mac.F1 (%)
GPT-5.4-nano Faithfulness VLM-as-Judge Sentence 29.79 61.33 37.76 64.01 53.67
SelfCheckGPT Sentence 20.24 48.00 26.41 53.87 43.13
FActScore Claim 20.47 70.22 30.97 38.60 34.60
UNIHD Claim 24.83 91.96 37.74 36.60 32.21
FactSelfCheck Graph 21.02 99.11 34.63 21.99 18.57
MedF²Route Graph 36.68 66.67 45.03 67.44 59.64
Factuality VLM-as-Judge Sentence 42.96 84.73 54.11 68.13 62.76
SelfCheckGPT Sentence 42.94 86.18 53.55 63.36 59.19
FActScore Claim 25.46 80.36 37.36 42.87 39.12
UNIHD Claim 29.73 96.70 43.59 41.41 38.24
FactSelfCheck Graph 21.63 99.27 35.42 22.40 18.80
MedF²Route Graph 48.40 93.45 60.84 70.36 67.32
InternVL-3.5 Faithfulness VLM-as-Judge Sentence 27.44 36.00 29.85 78.44 57.74
FactSelfCheck Graph 20.71 98.22 34.15 21.96 18.20
MedF²Route Graph 44.85 56.00 48.33 79.73 67.20
Gemini-3.1-flash-lite Faithfulness VLM-as-Judge Sentence 31.13 76.89 42.30 57.27 51.44
MedF²Route Graph 46.20 73.78 54.61 76.46 68.58

Ablation Study

The ablation study was conducted under the GPT-5.4-nano backbone using a context-preserving design where bypassed modules maintain their output structures to avoid downstream disruption. Both final sentence-level Macro-F1 and intermediate routed triplet-level Macro-F1 are reported.

Ablation Variant Sent. Mac.F1 Joint Triplet Mac.F1 Fact. Triplet Mac.F1 Faith. Triplet Mac.F1 Description
MedF²Route (Full Model) 63.86 67.12 66.93 41.38 Full evidence-routed verification pipeline
w/o Evidence-aware Routing 51.91 60.27 58.07 44.83 Severe drop in Sent. F1 (-11.95) and Fact. F1 (-8.86)
w/o KG Evidence Retrieval 60.60 65.28 64.24 41.38 Factuality Macro-F1 drops by -2.69
w/o Multi-hop KG Expansion 63.18 66.56 65.86 41.38 Factuality Macro-F1 drops by -1.07
w/o Visual Evidence Selection 60.95 63.69 66.93 20.02 Catastrophic collapse in Faithfulness (-21.36)

Key Findings

  • Baselines suffer from inflated recall and excessive false alarms: While graph and claim baselines like FactSelfCheck and UNIHD achieve extreme recall (>90% to 99%), their precision remains dismal (20%–29%), dragging overall accuracy down to 21%–41%. In contrast, MedF²Route elevates hallucination precision to 36.68%–48.40% while preserving high recall, establishing a favorable precision-recall trade-off vital for downstream error correction.
  • Evidence-aware routing is the central architectural pillar: Disabling evidence routing incurs the steepest performance penalty, dropping sentence-level Macro-F1 by 11.95 points and factuality by 8.86 points. This confirms that without explicit boundary separation between patient evidence and biomedical knowledge, multi-source evidence conflation severely degrades detection accuracy.
  • Visual evidence routing is strictly indispensable for faithfulness: Removing visual evidence selection causes a catastrophic 21.36-point plunge in the faithfulness branch (falling from 41.38 to 20.02), demonstrating that clinical imaging findings cannot be reliably validated through textual abstractions alone.
  • Context matters: finer decomposition is not universally superior: Error analysis on InternVL reveals that 69.90% of misclassifications stem from backbone verification limitations rather than KG gaps. Breaking complex clinical narratives into overly granular atomic units isolates claims from broader pathophysiological context, indicating that granularity must be dynamically calibrated against backbone reasoning capabilities.

Highlights & Insights

  • Formulating hallucination detection as an evidence routing problem: Transcending the conventional view of hallucination detection as homogeneous NLI or generic RAG, the paper conceptualizes multi-source clinical validation around dynamic evidence dispatch, eliminating cross-modal signal interference.
  • Sufficiency-guided dynamic KG subgraph expansion: Instead of querying static multi-hop neighborhoods that introduce semantic drift or high latency, the framework employs an LLM-guided sufficiency check to incrementally add bridging entities only when necessary to judge medical veracity.
  • Superior precision critical for clinical deployment: False positives in clinical AI create alert fatigue and mislead clinicians during automated error correction. MedF²Route delivers marked precision improvements across closed-source and open-source backbones, establishing a trustworthy foundation for clinical LVLM auditing.

Limitations & Future Work

  • Domain confinement to diagnostic consultation: The evaluation is centered on diagnostic reasoning. Other specialized clinical workflows—such as radiology report generation (almost exclusively visual faithfulness) or medication recommendation (dominated by drug-drug interaction factuality)—exhibit distinct evidence priorities and require adapted routing topologies.
  • Dependence on underlying backbone reasoning capacity: As shown in the InternVL error audit, a substantial proportion of verification failures stem from intrinsic reasoning errors of the verifier model rather than retrieval inadequacy. Detection performance remains bound by backbone clinical competence.
  • Knowledge graph coverage and timeliness boundaries: Factuality verification relies on LMKG coverage (~70% entity matching). Rare diseases, novel pathogens, or evolving clinical guidelines outside the KG horizon cannot be fully verified through structured paths alone.
  • vs VLM-as-Judge & SelfCheckGPT: Direct black-box sentence evaluators lack structured evidence routing, fail to isolate localized anatomical discrepancies, and require costly repeated sampling (SelfCheckGPT). MedF²Route provides interpretable, single-pass graph verification with substantially higher precision and statistical significance (\(p < 0.01\)).
  • vs FActScore & UNIHD: Claim-level approaches shatter clinical narrative coherence and, in the case of UNIHD, blend heterogeneous evidence into undifferentiated prompts leading to evidence confusion. MedF²Route retains relational context via graph topology and guarantees evidence alignment.
  • vs FactSelfCheck & GraphEval: GraphEval is restricted to unimodal textual faithfulness, while FactSelfCheck relies on stochastic KG consistency between sampled outputs. MedF²Route is the first framework to integrate patient text, raw medical images, and external biomedical knowledge graphs into a unified routing architecture.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ First framework to formalize multimodal clinical hallucination detection as an evidence routing problem with structured graph disentanglement.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous evaluation on 500-case MedF²Bench across five imaging modalities, three VLM backbones, paired bootstrap tests, and granular error audits.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear mathematical formulation, rigorous ablation methodology, and complete prompt disclosures.
  • Value: ⭐⭐⭐⭐⭐ Provides an indispensable, high-precision auditing framework and benchmark for advancing the trustworthiness and clinical safety of diagnostic LVLMs.