Lung-R1: A Knowledge Graph-Guided LLM for Pulmonary Diagnostic Reasoning¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: pulmonary diagnostic reasoning, medical knowledge graph, reinforcement learning, GRPO, electronic medical records
TL;DR¶
Addressing the "Pulmonary Knowledge-to-Diagnosis Gap" where LLMs memorize textbook facts but struggle with patient-specific EMR deduction, this paper introduces LungKG (a 59k-node structured pulmonary graph) and Lung-R1 (trained with graph-constrained CoT and graph-guided RL), achieving state-of-the-art clinical EMR diagnosis performance.
Background & Motivation¶
Diagnosing pulmonary diseases requires synthesizing heterogeneous clinical narratives, chest radiologic imaging, pulmonary function tests, and microbiologic laboratory results under marked phenotypic variability and pervasive cross-disease symptom overlap. While recent general and medical-specific large language models have achieved impressive scores on medical board examination questions, medical question answering, and clinical information extraction, authentic bedside pulmonary diagnosis demands patient-specific, relation-aware deduction across distributed clinical evidence rather than isolated textbook factual recall.
The root cause of this failure mode is what the authors formally define as the Pulmonary Knowledge-to-Diagnosis Gap: the profound mismatch between knowledge-centric medical benchmark training and case-centric bedside diagnostic reasoning. Standard pre-training and supervised fine-tuning corpora typically present respiratory facts as isolated examination multiple-choice questions or fragmented instructions. When confronted with real-world electronic medical records (EMR), models frequently lack the structured clinical logic required to map observed symptoms (such as dyspnea, fever, and hypoxemia) and imaging findings to candidate pathogen hypotheses, differential diagnoses, and disease severity stages, leading models to either enumerate superficial secondary complications or miss life-threatening primary etiologies entirely.
To resolve this limitation, structured medical knowledge must be transformed into training-time relational constraints and reward signals rather than relying solely on brittle inference-time retrieval. Core idea: construct LungKG, the first diagnosis-oriented pulmonary knowledge graph comprising 59,038 nodes and 164,308 edges across 112 relation types, and train Lung-R1 via inverse-degree graph-constrained Chain-of-Thought fine-tuning followed by multi-objective reinforcement learning guided by diagnostic correctness, graph faithfulness, and relation-path consistency rewards.
Method¶
Overall Architecture¶
The Lung-R1 training pipeline operationalizes structured pulmonary diagnostic knowledge into an internal reasoning mechanism across three major phases: LungKG construction, KG-constrained Chain-of-Thought (CoT) supervised fine-tuning (SFT), and KG-guided reinforcement learning (RL). First, LungKG is synthesized from clinical practice guidelines, medical textbooks, and real EMR records into a normalized graph topology spanning 15 entity types and 112 directed relation types. Second, during SFT, inverse-degree subgraph sampling generates diverse KGQA pairs, while clinical records are anchored to LungKG subgraphs to generate natural-language diagnostic reasoning traces without exposing raw graph tokens. Finally, using Group Relative Policy Optimization (GRPO), the model is optimized against a tripartite reward combining diagnostic outcome accuracy, graph claim grounding, and path directionality consistency.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Pulmonary Guidelines / Textbooks / De-identified EMRs"] --> B["Construct Pulmonary Knowledge Graph LungKG<br/>59,038 nodes / 164,308 edges / 15 entity types / 112 relation types"]
B --> C["KG-Constrained CoT Construction & Supervised Fine-Tuning<br/>Inverse-degree sampling KGQA + EMR anchor subgraph expansion EMR-CoT"]
C --> D["KG-Guided Reinforcement Learning Alignment<br/>GRPO group advantages + Correctness/Faithfulness/Path rewards"]
D --> E["Zero-Retrieval Record-Grounded Diagnostic Output<br/>Held-out evaluation on Choice, Pulmonary-QA, and EMR Diagnosis"]
Key Designs¶
1. Structured Pulmonary Knowledge Graph LungKG: Unorganized Facts and Clinical Overlap -> Relation-Centric Topological Organization -> Providing a Rigorous Diagnostic Evidence Substrate General medical ontologies lack specialized relation definitions tailored to pulmonary differential diagnosis, impeding the modeling of disease-symptom-treatment interdependencies. LungKG organizes 59,038 nodes and 164,308 directed edges across 15 entity categories (including 11,544 Western medicines, 10,976 other treatments, 8,448 diseases, 6,481 drugs, 5,120 symptoms, 4,098 examinations, and 2,804 pathogens). It explicitly codifies four core diagnostic axes: pathogen-to-infection relations, symptom-to-differential-diagnosis links, imaging-examination-to-disease mappings, and drug-contraindication safety profiles. Clinician-supervised verification confirms an inter-annotator F-agreement of 88.2% on entity extraction and 83.8% on relation identification, establishing a high-fidelity topological foundation for downstream training.
2. KG-Constrained Reasoning Chain Construction: Frequency Bias and Hallucinatory Drift -> Inverse-Degree Subgraph Sampling and EMR Anchor Expansion -> Internalizing Robust Clinical Deductive Trajectories Standard fine-tuning corpora suffer from severe head-disease bias and often produce ungrounded causal claims. Lung-R1 addresses this via a dual-source graph-constrained reasoning generation pipeline. For knowledge-centric QA, an inverse tail-degree sampling algorithm assigns selection probabilities inversely proportional to node degrees \(P(v) \propto 1/(\text{deg}(v) + \varepsilon)\) for nodes with degree below graph median \(\tau\), ensuring comprehensive coverage of rare and discriminative conditions across 11,578 curated KGQA samples. For clinical record reasoning, 3,569 de-identified EMR cases are mapped to graph anchor nodes \(A_x = \text{Match}(x, \text{LungKG})\), which are expanded via breadth-first search into constrained evidence subgraphs \(g_x = \text{Expand}(A_x, d; \text{LungKG})\) targeting the reference diagnosis \(d\). Natural language reasoning traces \(r_x = \text{GenEMR-CoT}(x, d, \text{Serialize}(g_x))\) are then generated under strict graph constraints, enabling the model to learn relation-grounded clinical deduction without relying on graph traversal at inference time.
3. KG-Guided Multi-Component Reinforcement Learning: Misaligned Reasoning and Pure Outcome Exploitation -> Tripartite Outcome, Faithfulness, and Directed Path Consistency Rewards -> Enforcing Rigorous Bedside Clinical Alignment Optimizing solely for outcome diagnosis accuracy via sparse rewards frequently leads models to generate hallucinated intermediate reasoning or produce over-conservative lists of diffuse secondary findings. Lung-R1 introduces a multi-component reward function under the GRPO framework, generating \(G\) candidate responses per prompt and computing group-normalized advantage scores. The reward function \(R_i = \lambda_1 R_{\text{dx}}(y_i) + \lambda_2 R_{\text{graph}}(y_i) + \lambda_3 R_{\text{path}}(y_i)\) integrates three normalized signals (weighted with \(\lambda_1 = 0.50, \lambda_2 = 0.25, \lambda_3 = 0.25\)): $\(R_{\text{dx}}(y_i) = \frac{s_{\text{dx}}}{5}\)$ $\(R_{\text{graph}}(y_i) = \frac{N_{\text{sup}} + 0.5 N_{\text{partial}}}{\max(N_{\text{claim}}, 1)}\)$ $\(R_{\text{path}}(y_i) = \frac{N_{\text{valid}}}{\max(N_{\text{path}}, 1)}\)$ Here \(s_{\text{dx}} \in \{0, \dots, 5\}\) assesses alignment against clinical rubrics, \(R_{\text{graph}}\) measures the proportion of extracted factual claims verified by LungKG subgraphs, and \(R_{\text{path}}\) verifies whether reasoning steps follow valid directed diagnostic paths (such as symptom \(\to\) examination \(\to\) pathogen \(\to\) disease). This setup drives the policy \(\pi_\theta\) to simultaneously maximize clinical accuracy and maintain rigorous path faithfulness.
Loss & Training¶
Lung-R1 initializes from Qwen2.5-7B-Instruct and Qwen2.5-14B-Instruct backbones. The SFT stage is trained with a learning rate of \(2 \times 10^{-6}\), cosine learning rate scheduling with 0.03 warmup ratio, global batch size of 64, and cutoff context lengths of 2048 (7B) and 3072 (14B) on BF16 precision with DeepSpeed ZeRO-3. The RL phase keeps reward weights fixed at \(\lambda_1=0.50, \lambda_2=0.25, \lambda_3=0.25\) and optimizes policy parameters via the clipped surrogate objective \(J(\theta) = \mathbb{E}[\min(r_i \hat{A}_i, \text{clip}(r_i, 1-\epsilon, 1+\epsilon)\hat{A}_i)]\), strictly excluding all held-out evaluation cases to prevent data contamination.
Key Experimental Results¶
Main Results¶
The evaluation suite comprises 250 multiple-choice questions (Choice), 250 physician-reviewed respiratory QA prompts (Pulmonary-QA), and 300 held-out de-identified clinical records (EMR Diagnosis). Scoring is conducted across a 5-model LLM-as-Judge ensemble alongside Quadratic Weighted Kappa (QWK) calculated against senior pulmonary physician adjudication.
| Model Category | Model Name | Choice Overall Acc (%) | Pulmonary-QA Score (0-5) | Pulmonary-QA QWK | EMR Diagnosis Score (0-5) | EMR Diag QWK | EMR Micro R |
|---|---|---|---|---|---|---|---|
| Small Language Models | Qwen2.5-7B | 58.40 | 3.660 | 0.8863 | 3.7007 | 0.8116 | 0.7281 |
| Small Language Models | Qwen2.5-14B | 63.20 | 3.996 | 0.8727 | 3.7709 | 0.7322 | 0.7931 |
| Small Language Models | Qwen3-30B-A3B | 64.80 | 4.292 | 0.8630 | 3.3617 | 0.6904 | 0.8249 |
| General Frontier LLMs | Gemini-3-Pro | 67.20 | 4.396 | 0.8460 | 4.0688 | 0.5431 | 0.7056 |
| General Frontier LLMs | DeepSeek-V3.2 | 66.80 | 4.256 | 0.8394 | 4.0573 | 0.6371 | 0.8489 |
| General Frontier LLMs | DeepSeek-R1 | 63.60 | 4.264 | 0.8847 | 3.9891 | 0.7129 | 0.8103 |
| General Frontier LLMs | Claude-Sonnet-4.5 | 60.40 | 4.396 | 0.9046 | 4.2107 | 0.5887 | 0.8753 |
| General Frontier LLMs | GPT-5.1 | 54.00 | 4.1488 | 0.9182 | 4.2075 | 0.6120 | 0.7868 |
| Medical-Specific LLMs | ClinicalGPT-R1 | 59.20 | 3.588 | 0.8598 | 3.7449 | 0.7452 | 0.7958 |
| Medical-Specific LLMs | HuatuoGPT-o1-7B | 62.80 | 3.636 | 0.8686 | 3.3687 | 0.8278 | 0.6578 |
| Medical-Specific LLMs | Baichuan-M2 | 62.40 | 4.016 | 0.8822 | 3.9339 | 0.7271 | 0.8196 |
| Proposed Models | Lung-R1-7B | 67.20 | 4.120 | 0.9123 | 4.3550 | 0.8295 | 0.8939 |
| Proposed Models | Lung-R1-14B | 67.60 | 4.416 | 0.9239 | 4.3583 | 0.8275 | 0.8939 |
Ablation Study¶
The ablation experiments isolate the individual impacts of SFT supervision data sources, explicit Chain-of-Thought (CoT) format, and KG-guided reinforcement learning.
| Ablation Category | Variant Specification | Choice Acc (%) | Pulmonary-QA Score | EMR Diagnosis Score | Observations and Key Findings |
|---|---|---|---|---|---|
| SFT Source Ablation (7B) | Base-7B (Qwen2.5-7B) | 58.4 | 3.660 | 3.7007 | Unadapted open-weights backbone |
| 7B w/o EMR (KGQA-only) | 61.2 | 4.084 | 3.4949 | High QA capability but fails to ground in authentic EMR records | |
| 7B w/o KGQA (EMR-only) | 62.0 | 3.664 | 4.2626 | High diagnostic performance but degraded medical QA recall | |
| 7B Full SFT (Dual-source) | 64.1 | 4.112 | 4.3471 | Optimal balanced performance across QA and case diagnosis | |
| CoT Format Ablation (SFT) | 7B w/o CoT (direct answer) | 63.8 | 4.172 | 4.1918 | Lack of reasoning trace impairs diagnostic accuracy (-0.155) |
| 7B w/ CoT (explicit reasoning) | 64.1 | 4.112 | 4.3471 | Explicit chains anchor multi-stage differential decisions | |
| 14B w/o CoT | 65.5 | 4.244 | 4.3041 | Removing CoT degrades 14B EMR diagnostic score (-0.053) | |
| 14B w/ CoT | 67.2 | 4.298 | 4.3575 | Explicit deduction yields robust clinical record synthesis | |
| KG-RL Ablation (14B CoT) | 14B w/o KG-RL (SFT checkpoint) | 67.2 | 4.298 | 4.3575 | SFT provides strong preliminary diagnostic accuracy |
| 14B w/ KG-RL (Full Pipeline) | 67.6 | 4.416 | 4.3583 | RL strengthens QA scores and enforces graph path fidelity |
Key Findings¶
- Complementarity of Dual SFT Sources: Models trained without EMR data suffer a severe drop on EMR Diagnosis from 4.3471 to 3.4949, showing that pure textbook knowledge graphs cannot generalize directly to noisy, real-world clinical records. Conversely, training without KGQA drops Pulmonary-QA to 3.664, confirming that both sources are essential.
- Critical Role of Explicit Reasoning Chains: Omitting explicit CoT reduces EMR Diagnosis scores by 0.1553 on 7B and 0.0534 on 14B. Error analysis reveals that models without explicit reasoning frequently produce over-extended lists of secondary findings (Overprediction) without isolating the primary acute pathology.
- Superior Agreement with Human Clinicians: While top commercial frontier models (Claude-Sonnet-4.5, Gemini-3-Pro) achieve high absolute scores, their QWK score agreement with clinician consensus on EMR Diagnosis remains modest (0.54-0.64). In contrast, Lung-R1 achieves a QWK of 0.8275–0.8295, demonstrating that its diagnostic criteria match the severity-aware prioritization of practicing pulmonologists.
Highlights & Insights¶
- Internalizing Knowledge into Model Parameters vs. External RAG: Rather than querying graph databases during inference—which incurs substantial latency and entity-matching errors on multi-page clinical notes—Lung-R1 embeds graph-grounded paths directly into the model's parameters and reasoning policies during SFT and RL.
- Inverse-Degree Sampling for Clinical Long-Tail Diseases: High-degree nodes (e.g., broad conditions like pneumonia or common antibiotics) easily dominate medical datasets. Inverse-degree sampling ensures equitable representation of low-frequency clinical entities, such as rare fungal infections and interstitial lung diseases.
- Multi-Objective Process Supervision via Graph Consistency: By validating intermediate textual reasoning claims and relation trajectories against LungKG edges in the RL reward loop, the framework establishes effective guardrails against reward hacking and spurious reasoning.
Limitations & Future Work¶
- Text-Only Modality Constraints: Real-world pulmonary diagnoses heavily depend on native high-resolution computed tomography (HRCT) DICOM scans, digital stethoscope lung sounds, and dynamic blood-gas trends. Future work should expand Lung-R1 into a multi-modal foundation architecture.
- Diagnosis-Only Task Scope: The current system evaluates disease identification and differential diagnosis, without extending to prescriptive management such as personalized antibiotic dosing, mechanical ventilation weaning strategies, or longitudinal adverse drug monitoring.
- Single-Center Disease Distribution: The EMR dataset exhibits an ICD-10 concentration in pneumonia (64.7%) and chronic lower respiratory diseases (19.1%). Evaluating across diverse geographic and multi-center cohorts is necessary to ensure generalizability.
Related Work & Insights¶
- vs ClinicalGPT-R1 / HuatuoGPT-o1-7B: General medical reasoning LLMs rely primarily on broad biomedical examination benchmarks and generic medical corpora. In contrast, Lung-R1 demonstrates that complex specialty domains benefit substantially from dedicated domain-specific knowledge graphs with fine-grained directed relations.
- vs GraphRAG (Edge et al., 2024): While GraphRAG constructs dynamic retrieval subgraphs during inference, it remains vulnerable to retrieval errors on ambiguous clinical acronyms. Lung-R1 proves that utilizing the graph offline to synthesize constrained CoT data and design process-level RL rewards achieves superior clinical reasoning fidelity with zero inference latency overhead.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Introduces the first dedicated pulmonary diagnostic knowledge graph and pairs it with graph-constrained CoT and multi-objective RL.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous benchmark across 20 state-of-the-art systems on three distinct tasks, featuring multi-judge ensembles and physician-adjudicated QWK metrics.
- Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear narrative, thorough documentation of hyper-parameters, and comprehensive appendix detailing prompt templates and error taxonomies.
- Value: ⭐⭐⭐⭐⭐ Provides a concrete, reproducible engineering paradigm for adapting open-weights LLMs into trustworthy medical reasoning systems.