Skip to content

MedSPA: Self-planning Medical Agents

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Medical Vision-Language Models, Self-Planning, Evidence Acquisition, Reinforcement Learning Alignment, Radiology Report Generation

TL;DR

MedSPA addresses the tension between uncontrolled chain-of-thought hallucination and rigid clinical template traversal in medical vision-language reasoning by decomposing diagnosis into an iterative planning-and-reasoning agent (PR Agent) and an evidence-grounded summary agent (SUM Agent), achieving state-of-the-art clinical efficacy (+0.086 CE-F1 on MIMIC-CXR) through soft template distillation and role-specific reinforcement alignment.

Background & Motivation

Recent medical vision-language models (Medical VLMs) have demonstrated impressive predictive capabilities across diagnostic benchmarks, including automated radiology report generation (MRG) and medical visual question answering (MedVQA). However, in high-stakes clinical practice, generating a fluent final report or an accurate multiple-choice answer is insufficient; diagnostic predictions must be strictly grounded in observable radiographic evidence. To enhance transparency and mimic clinical decision-making, recent studies have embraced multi-step reasoning paradigms, primarily bifurcated into step-by-step chain-of-thought (CoT) prompting and template-based checklist traversal.

Nevertheless, both existing paradigms exhibit fundamental clinical shortcomings. Unconstrained CoT reasoning offers flexible, case-by-case exploration, but it frequently lacks disciplined clinical guidance—models easily get sidetracked by irrelevant background artifacts, jump to premature diagnostic conclusions, or introduce hallucinated pathological entities. Conversely, template-based approaches enforce standardized domain knowledge by compelling the model to traverse a rigid, pre-compiled anatomical checklist (such as a fixed 10-step inspection protocol covering lungs, pleura, heart, and mediastinum). In real clinical practice, board-certified radiologists do not blindly follow a static checklist regardless of patient condition; instead, they treat standardized protocols as clinical priors, dynamically adapting their diagnostic focus based on presenting abnormalities and terminating the search once sufficient findings are collected.

The fundamental tension lies in how to enable a multimodal model to balance explicit procedural knowledge from clinical templates with flexible visual-semantic reasoning without falling into the extremes of uncontrolled hallucination or rigid, inefficient over-inspection. Furthermore, when an end-to-end model produces an erroneous diagnosis, standard outcome supervision provides only delayed rewards, making it impossible to determine whether the failure stemmed from incomplete evidence acquisition, flawed intermediate inference, or an ungrounded final summary. Core idea: Decouple clinical diagnostic reasoning into a two-agent cooperative architecture—an iterative Planning-and-Reasoning Agent (PR Agent) that adaptively decides what anatomical evidence to inspect and when to stop, and a Summary Agent (SUM Agent) that synthesizes the final report strictly conditioned on the accumulated memory—optimized via soft template distillation and role-specific reinforcement alignment (R-Align).

Method

Overall Architecture

MedSPA formulates diagnostic evidence exploration and report generation as a cooperative interaction between two specialized agents, both instantiated with Qwen3-VL-8B-Instruct. Given an input medical image \(I\), the system initiates an iterative loop driven by the Planning-and-Reasoning (PR) Agent. At each step \(t\), the PR Agent evaluates the image alongside its dynamic working memory \(M_t\), generating an action-reasoning-decision triple: selecting an anatomical focus \(a_t\), conducting image-grounded diagnostic reasoning \(r_t\), and deciding whether to proceed (\(d_t \in \{\text{continue}, \text{summarize}\}\)). Once the PR Agent concludes evidence collection at step \(T\) (\(d_T = \text{summarize}\)), the accumulated reasoning chain \(Z = M_{T+1}\) is passed to the Summary (SUM) Agent, which generates the final clinical report or answer \(\hat{o}\) grounded in \(I\) and \(Z\). The framework is trained via a two-stage paradigm: medical knowledge distillation with clinical quality filtering, followed by role-specific reinforcement alignment (R-Align).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Medical Image and Prompt<br/>Image I + Prompt"] --> B["Soft Template Knowledge Distillation and Quality Control<br/>Teacher LVLM curation + Clinical entity alignment filter"]
    B --> C["Iterative Evidence Acquisition and Memory Maintenance<br/>PR Agent generates (Action, Reason, Decision) triples"]
    C -->|Decision: continue| C
    C -->|Decision: summarize| D["Role-Specific Reinforcement Alignment<br/>PR state-space rewards + SUM fidelity rewards"]
    D --> E["Evidence-Grounded Diagnostic Summary Generation<br/>SUM Agent synthesizes final report / VQA answer"]

Key Designs

1. Soft Template Knowledge Distillation and Quality Control: Structuring Unsupervised Evidence Chains Addressing the severe scarcity of intermediate clinical reasoning annotations, this design develops an automated curation pipeline to synthesize structured reasoning trajectories from paired images and reference reports. Using a strong teacher model (Qwen3-VL-32B), the pipeline incorporates a "soft medical reasoning template" formulated from radiology textbooks and clinician consultations. Rather than enforcing rigid step-by-step traversal, the soft template guides the model to first establish the global radiographic context (projection view, image quality, support devices), followed by a flexible, clinician-like exploration of major anatomical zones prioritizing normal baseline observations before anomalous findings. Each step produces an action \(a_t\) (formatted as a short V-ing phrase), an image-grounded rationale \(r_t\), and an explicit termination decision \(d_t\). To eliminate hallucinations and flawed reasoning before supervised fine-tuning, the curated chains undergo a two-tier quality control protocol: first, clinical entity labels extracted via RadGraph and a predefined label set \(\mathcal{L}\) are compared between the chain \(\phi(Z)\) and reference report \(\psi(y)\); second, an LLM-based clinical evaluator scores the chain on factual faithfulness and diagnostic completeness. Chains failing label consistency or falling below the quality threshold are discarded, yielding the high-purity dataset \(\mathcal{D}_{\text{MedPR}}\).

2. Iterative Evidence Acquisition and Memory Maintenance: Decoupling Dynamic Planning from Grounded Vision Reasoning To prevent the model from missing subtle findings or getting locked into single-pass inductive biases, the PR Agent operates as a stateful sequential decision-maker. At iteration \(t\), conditioned on image \(I\) and historical memory \(M_t = \{(a_k, r_k, d_k)\}_{k=1}^{t-1}\) (\(M_1 = \emptyset\)), the PR Agent autoregressively outputs \((a_t, r_t, d_t)\). The action \(a_t\) specifies the diagnostic objective (e.g., "Evaluating lower lung zones" or "Assessing pleural space"); \(r_t\) records the localized visual evidence; and \(d_t \in \{\text{continue}, \text{summarize}\}\) explicitly determines whether evidence accumulation should continue. The dynamic memory \(M_t\) acts as a clinical scratchpad that tracks previously inspected structures and confirmed pathologies, effectively preventing redundant re-examination and ensuring that subsequent findings do not contradict earlier observations. Once the agent determines that all salient evidence is gathered, it outputs \(d_T = \text{summarize}\), passing the consolidated chain \(Z = M_{T+1}\) to the SUM Agent.

3. Role-Specific Reinforcement Alignment: Eliminating Delayed Reward Interference and Premature Stopping Supervised fine-tuning alone teaches agents to mimic expert trajectories but does not optimize for exploration efficiency or final diagnostic fidelity. Optimizing both agents end-to-end on final report accuracy causes credit assignment confusion—penalizing the PR Agent for summary generation errors or vice-versa. MedSPA introduces Role-Specific Reinforcement Alignment (R-Align) to optimize each agent against its exact functional boundary. In Stage 1, the PR Agent is optimized while freezing the SUM Agent, using rewards calculated strictly over the gathered evidence labels \(\phi(Z)\) versus reference labels \(\psi(y)\): $\(R_R(Z) = R_{\text{con}}(Z) + R_{\text{fmt}}(Z) + \beta R_{\text{cov}}(Z)\)$ Here, diagnostic consistency \(R_{\text{con}}(Z, y) = \frac{\phi(Z)^\top \psi(y)}{|\mathcal{L}|}\) rewards correct evidence identification; structural validity \(R_{\text{fmt}}(Z)\) enforces strict formatting compliance; and coverage penalty \(R_{\text{cov}}(Z, y) = -\frac{C(\phi(Z), \psi(y))}{|\psi(y)|}\) penalizes premature stopping based on the count of missed positive reference findings \(C(\phi(Z), \psi(y))\). In Stage 2, with the aligned PR Agent frozen, the SUM Agent is optimized to preserve acquired evidence faithfully: its reward \(R_S(Z, y, \hat{o})\) computes fine-grained label payoffs penalizing the omission of PR-discovered findings (assigning \(-1\) if a finding was captured in \(Z\) but omitted from \(\hat{o}\)). Both stages are trained using Group Relative Policy Optimization (GRPO) with KL regularization against their SFT reference policies.

4. Evidence-Grounded Diagnostic Summary Generation: Hallucination-Resistant Synthesis After comprehensive evidence exploration terminates, the SUM Agent is tasked with condensing the unstructured multi-step trajectory into a coherent, professional medical report or concise VQA answer. Unlike standard generative models that produce descriptions directly from raw visual features—often falling prey to frequency-biased statistical hallucinations—the SUM Agent is conditioned jointly on the original image \(I\) and the verified reasoning chain \(Z\). Supervised on clinical report targets and aligned via the role-specific summary reward, the SUM Agent learns to synthesize disparate anatomical observations (such as clear apical fields, basilar atelectasis, and normal cardiac contours) into a concise Impressions and Findings section without hallucinating unobserved pathologies or discarding verified abnormalities.

Loss & Training

Both optimization stages under R-Align employ Group Relative Policy Optimization (GRPO). For the PR Agent, \(N\) candidate reasoning trajectories \(\{Z_i\}_{i=1}^N\) are sampled per input to compute group-relative advantages \(A_i^R = R_R(Z_i) - \frac{1}{N}\sum_{j=1}^N R_R(Z_j)\), optimizing the trajectory loss: $\(\mathcal{L}_R = -\mathbb{E}_{Z_i \sim \pi_\theta^R} \left[ \sum_{t=1}^{T_i} A_i^R \log \pi_\theta^R(z_{i,t} \mid I, q, z_{i,<t}) \right] + \lambda_{\text{KL}} \mathbb{D}_{\text{KL}}(\pi_\theta^R \parallel \pi_{\text{ref}}^R)\)$ Similarly, the SUM Agent samples \(M\) candidate report outputs \(\{Y_i\}_{i=1}^M\) conditioned on the frozen PR chain \(Z\), computing advantage \(A_i^S = R_S(Y_i) - \frac{1}{M}\sum_{j=1}^M R_S(Y_j)\) with an analogous KL-regularized objective \(\mathcal{L}_S\). Training is performed on 8 \(\times\) 80GB A100 GPUs using LoRA (\(r=64, \alpha=128\), dropout 0.05). SFT is conducted for 10 epochs on MRG and 3 epochs on MedVQA. R-Align is trained on 50,000 samples for 2 epochs per agent with learning rate \(2 \times 10^{-5}\) and coverage coefficient \(\beta=0.5\).

Key Experimental Results

Main Results

The framework is evaluated on medical report generation (official MIMIC-CXR test set and out-of-distribution IU X-ray) and medical visual question answering (SLAKE and VQA-RAD). In report generation, performance is measured using clinical efficacy (CE) metrics (Precision, Recall, F1 based on 14 CheXpert clinical labels) alongside standard lexical metrics (BLEU-1, BLEU-4, ROUGE-L, METEOR).

Dataset Model CE-Prec. ↑ CE-Rec. ↑ CE-F1 ↑ BLEU-1 ↑ BLEU-4 ↑ ROUGE-L ↑ METEOR ↑
MIMIC-CXR R2Gen (Chen et al., 2020) 0.333 0.273 0.276 0.353 0.103 0.277 0.142
METransformer (Wang et al., 2023b) 0.364 0.309 0.311 0.386 0.124 0.291 0.152
PromptMRG (Jin et al., 2024) 0.501 0.509 0.476 0.398 0.112 0.268 0.157
DART (Park et al., 2025) 0.520 0.546 0.533 0.437 0.137 0.310 0.175
Direct SFT (Baseline) 0.483 0.466 0.474 0.298 0.090 0.231 0.132
SFT MedSPA (Ours, Intermediate) 0.549 0.543 0.546 0.334 0.125 0.247 0.152
R-Aligned MedSPA (Ours, Final) 0.569 0.552 0.560 0.341 0.129 0.260 0.169
IU-Xray (OOD) R2Gen (Chen et al., 2020) 0.141 0.136 0.138 0.325 0.059 0.253 0.131
PromptMRG (Jin et al., 2024) 0.213 0.229 0.211 0.401 0.098 0.281 0.160
Direct SFT (Baseline) 0.182 0.221 0.200 0.215 0.060 0.176 0.132
SFT MedSPA (Ours, Intermediate) 0.322 0.252 0.283 0.311 0.084 0.253 0.147
R-Aligned MedSPA (Ours, Final) 0.324 0.301 0.312 0.337 0.086 0.267 0.152

On MedVQA across modalities (Chest X-ray, Brain MRI, Abdominal CT), MedSPA demonstrates substantial advantages on open-ended questions: on SLAKE, MedSPA-Qwen3VL achieves an open-ended average F1 of 74.2% (a +35.9% gain over native Qwen3VL at 38.3%) and closed-ended accuracy of 81.0%; on VQA-RAD, it reaches an open-ended average of 39.5% and closed-ended accuracy of 76.8%, significantly outperforming general-purpose MLLMs including Gemini 2.5 Pro (50.6% / 40.3% open-ended) and LLaVA.

Ablation Study

A component-wise ablation on the MIMIC-CXR test set isolating the PR Agent, SUM Agent, and R-Align reinforcement stages is detailed below (PR-only configurations are evaluated on labels extracted directly from the accumulated reasoning memory):

Model Configuration PR Agent SUM Agent R-Align CE-Precision CE-Recall CE-F1 Key Finding & Description
Direct SFT – – – 0.483 0.466 0.474 Single-pass baseline without intermediate planning
PR only ✓ – – 0.540 0.550 0.545 Planning memory alone yields massive +0.084 recall boost
SFT MedSPA ✓ ✓ – 0.549 0.543 0.546 SUM agent filters noisy observations (+0.009 precision)
R-Aligned PR ✓ – ✓ 0.549 0.563 0.556 RL with coverage penalty expands reliable evidence retrieval
R-Aligned MedSPA ✓ ✓ ✓ 0.569 0.552 0.560 Synergistic full model achieves highest precision and F1

Key Findings

  • PR Agent Drives Recall while SUM Agent Guards Precision: Introducing the PR Agent alone raises recall from 0.466 to 0.550, confirming that dynamic planning ensures comprehensive visual inspection across lung zones and mediastinal structures. Integrating the SUM Agent improves precision by filtering speculative or redundant intermediate steps.
  • Superiority Over Fixed 10-Step Templates with Fewer Turns: Comparative analysis across DeepSeek-VL2, Llama-4 Scout, and Gemini 2.5 Pro reveals that while a rigid 10-step medical checklist improves recall over CoT, it incurs heavy computational waste on simple cases. MedSPA achieves higher CE-F1 scores with an average of only 5.4 reasoning steps, proving that adaptive termination focuses compute on genuine pathology rather than mechanical traversal.
  • Generalization Beyond the CheXpert Label Space: Evaluated on GREEN (clinical error notation) and RadCliQ (report clinical quality score), R-Aligned MedSPA achieves state-of-the-art performance (GREEN: 0.295 vs. GPT-4V 0.160; RadCliQ: 0.661 vs. PromptMRG 1.169 and GPT-4V 1.786). In independent double-blind evaluation by LLM judges (Gemini 2.5 Pro and GPT-4o), both reasoning chains and final reports scored above 4.0/5.0, confirming robust clinical reliability.

Highlights & Insights

  • Distilling Rules Then Learning to Selectively Ignore Them: The core insight is that rigid domain standards (checklists, guidelines) should not be hardcoded into generation constraints. Instead, distilling them as soft priors and subsequently training the agent via reinforcement learning to decide when sufficient evidence is gathered enables the model to achieve both clinical rigor and case-adaptive flexibility.
  • Role-Specific Credit Assignment in Multi-Agent RL: By isolating intermediate evidence rewards from downstream generation rewards, MedSPA eliminates the delayed reward dilemma that plagues end-to-end medical reasoning, providing an effective blueprint for training multi-agent systems in complex knowledge domains.
  • Transferable "Probe-and-Summarize" Architecture: The decoupled loop of stateful evidence probing followed by grounded summarization directly extends to other high-stakes verification domains, such as legal contract audit, multi-step math theorem proving, and safety-critical software debugging.

Limitations & Future Work

  • Trade-off on Standard Lexical Overlap Metrics: MedSPA scores lower on lexical n-gram metrics (e.g., BLEU-4 of 0.129 vs. DART's 0.137) because it optimizes for factual clinical findings rather than exact reference phrasing, occasionally expressing identical clinical diagnoses with divergent syntax. Future work could introduce fluency-preserving reward objectives.
  • Absence of Explicit Region-of-Interest (ROI) Visual Grounding: The PR Agent performs image-level textual planning without bounding box supervision or explicit spatial cross-attention masks, relying on implicit visual grounding. Incorporating explicit visual grounding could enhance interpretability for focal micro-lesions.
  • Label-Space Confinement in R-Align: The coverage penalty and consistency rewards currently depend on the 14 CheXpert label categories, leaving rare findings (e.g., specific hardware positions, atypical bone fractures) dependent entirely on the SFT stage.
  • vs PromptMRG / RGRG (Static Prompts and Detection-Driven Generation): PromptMRG relies on pre-classified diagnosis prompts, and RGRG requires pre-trained bounding box detectors for fixed anatomical regions. MedSPA requires no external object detectors, using an LLM-driven autonomous state machine to flexibly navigate the image based on emerging clinical context.
  • vs MedCoT / Med-PRM (Process Rewards and Hierarchical Experts): MedCoT implements multi-expert routing without adaptive termination, while Med-PRM requires step-by-step guideline verification models with high computational overhead. MedSPA achieves efficient, lightweight alignment via role-specific group-relative policy optimization without external reward verifier networks.

Rating

  • Novelty: ⭐⭐⭐⭐☆ Decouples diagnostic exploration and grounded summarization into specialized agents, creatively balancing clinical protocol priors with RL-driven adaptive planning.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive benchmarks spanning MRG and VQA, zero-shot OOD testing, step-count efficiency analyses, bias-free clinical metrics (GREEN/RadCliQ), and LLM-as-a-judge validation.
  • Writing Quality: ⭐⭐⭐⭐⭐ Rigorous conceptual framing, transparent mathematical formalization, clean experimental contrasts, and insightful qualitative analyses.
  • Value: ⭐⭐⭐⭐☆ Highly impactful for clinical AI deployment, offering a practical paradigm to mitigate multimodal hallucinations through verifiable evidence gathering.