Skip to content

Fathom-Vaidya: Advancing Medical Reasoning with Rubric-Based Rewards

Conference: EMNLP 2026
Paper: ACL Anthology
Area: Medical NLP
Keywords: Medical LLM, Clinical Reasoning, Rubric-Based Rewards, Reinforcement Learning, Multi-Turn Doctor-Patient Interaction

TL;DR

Fathom-Vaidya introduces a sequential reinforcement learning and rubric-generation framework that pairs rule/error-rubric RL on diagnostic questions with actor-critic synthesis of 5.3k multi-turn clinical dialogues and multi-dimensional continuous rubrics under GSPO, boosting 30B performance on HealthBench-Hard to 50.1% and outperforming GPT-5 Thinking.

Background & Motivation

Deploying Large Language Models into clinical healthcare workflows demands competence across two fundamentally distinct yet tightly coupled reasoning paradigms. The first is diagnostic reasoning—a convergent, evidence-based process where the model integrates presenting complaints, laboratory biomarkers, imaging findings, and past medical history to arrive at a single accurate diagnosis. The second is clinical healthcare reasoning—an open-ended, navigational cognitive process required to gather incomplete history, communicate with empathy, calibrate uncertainty, provide urgent triage, and formulate multi-step management plans across multi-turn interactions where no single unique ground-truth answer exists. Prior works have largely targeted exam-oriented multiple-choice benchmarks using deterministic rule-based verification or sparse binary accuracy rewards, leaving models vulnerable to severe factual hallucinations, logical shortcuts, and rigid conversational mannerisms when confronted with authentic, unstructured patient encounters.

Existing preference alignment and reinforcement learning paradigms exhibit critical structural limitations across both fronts. In diagnostic reasoning, binary accuracy feedback provides no dense supervisory signal to penalize flawed intermediate causal deductions or subtle clinical misconceptions, while gathering human expert Chain-of-Thought demonstrations remains prohibitively expensive to scale. In real-world clinical interactions, recent stress-testing benchmarks such as HealthBench and HealthBench-Professional expose stark failure modes in commercial foundation models. While several concurrent efforts have explored rubric-based reinforcement learning, they typically restrict their focus to narrow diagnostic scenarios, focus solely on superficial fluency or accuracy while neglecting context-seeking, completeness, and clinical triage, and lack robust automated generation pipelines capable of synthesizing realistic multi-turn clinical interactions accompanied by mathematically grounded, fine-grained scoring standards without massive manual clinician intervention.

To resolve the dual bottlenecks of non-verifiable clinical outcomes and sparse reward signals in patient-centered dialogue, the critical angle of attack lies in formalizing expert clinical criteria into dense, polar, multi-dimensional evaluation rubrics and bootstrap-synthesizing high-fidelity clinical trajectories via an adversarial actor-critic dynamic. Core idea: build Fathom-Vaidya, a sequential reinforcement learning framework that first strengthens internal diagnostic reasoning on challenging MedBullets problems using rule-based and online error-rubric RL, and subsequently bootstraps a 5.3k multi-turn clinical dialogue dataset with continuous, multi-dimensional rubrics across varied patient personas via an actor-critic feedback loop, steering policy alignment under Group Sequence Policy Optimization (GSPO) without any supervised Chain-of-Thought data.

Method

Fathom-Vaidya decomposes the alignment of medical intelligence into a sequential curriculum targeting convergent diagnostic precision and divergent clinical navigational reasoning. In the initial phase, the framework curates challenging clinical vignettes from MedBullets, deploying two-stage reinforcement learning: binary outcome rewards for moderately difficult cases, followed by online rubric-guided RL targeting identified error categories. In the second phase, to govern open-ended patient conversations with non-verifiable completions, an actor-critic loop synthesizes thematic multi-turn dialogues alongside continuous multi-dimensional scoring rubrics, which then provide dense sequence-level rewards under Group Sequence Policy Optimization (GSPO).

Overall Architecture

The end-to-end framework comprises diagnostic curriculum construction, two-stage diagnostic RL, actor-critic conversational and rubric bootstrapping, and rubric-guided sequence policy optimization, structured as follows:

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: MedBullets / MedQA vignettes and real patient complaints"] --> B["Stage 1: Two-Stage Diagnostic RL via MedBullets Difficulty Filtering<br/>Binary outcome RL on moderate cases + GPT-5 Thinking error-rubric penalty on hard cases"]
    B --> C["Stage 2: Actor-Critic Synthesis of Multi-Turn Clinical Dialogues and Rubrics<br/>5.3k diverse patient-persona dialogues + dual quality/sanity validation and polarity-inverted rubrics"]
    C --> D["Stage 3: Rubric-Based Group Sequence Policy Optimization (GSPO)<br/>Continuous rubric scoring aggregation + system prompt guidance and dynamic epoch filtering"]
    D --> E["Output: Fathom-Vaidya Medical Reasoning Family (4B / 8B / 30B-A3B)"]

Key Designs

1. Two-Stage Diagnostic RL via MedBullets Difficulty Filtering: Resolving the inability of binary rewards to rectify deep causal fallacies and the absence of dense feedback Standard reinforcement learning on medical examinations relies exclusively on 0/1 correctness rewards, allowing models to exploit statistical shortcuts rather than mastering robust pathophysiology. To remedy this, the authors crawled 7,801 high-yield, complex multiple-choice questions from MedBullets spanning USMLE Step 1/2 and Orthopedics boards—spanning basic medical sciences (35%), organ-system pathology (30%), and disease mechanisms (20%)—combined with MedQA and MedMCQA. Before RL, the base model evaluates each question 8 times at temperature 1.0 to compute its empirical pass rate \(\bar{s}_i\). Retained questions are bifurcated by difficulty: moderately difficult items with \(\bar{s}_i \in (0.4, 0.7]\) undergo first-stage rule-based GSPO using binary answer verification; exceedingly difficult items where the model consistently stumbles (\(\bar{s}_i \in [0.1, 0.4]\)) are reserved for second-stage online rubric RL. In this second stage (FV-8B-Diag), rollouts are evaluated by GPT-5 (thinking mode), which parses trajectories into three discrete failure modes: factual knowledge errors/hallucinations, faulty causal deductions, and clinical context misinterpretations. Minor errors receive a severity score of 1, while major errors receive 2. The cumulative severity is normalized into a dense rubric reward: $\(R_{\text{rubric}} = \frac{15 - \sum \text{severity}}{15}\)$ The final training signal linearly combines rubric and correctness rewards: \(R_{\text{final}} = \alpha \cdot R_{\text{rubric}} + (1 - \alpha) \cdot R_{\text{accuracy}}\). Ablations demonstrate that setting \(\alpha = 1.0\) yields optimal performance, allowing rich clinical reasoning to self-emerge purely via reinforcement learning without any human-authored Chain-of-Thought demonstrations.

2. Actor-Critic Synthesis of Multi-Turn Clinical Dialogues and Rubrics: Overcoming the scarcity of realistic interactive medical dialogues and subjective scoring variance Authentic clinical consultations entail dynamic multi-turn history-taking, emotional ambiguity, and implicit needs that cannot be reduced to simple answer matching. To synthesize realistic encounters, the framework builds an actor-critic generation pipeline. For core interactive themes like "responding under uncertainty" and "context-seeking", seed queries are sourced from MedRedQA (verified clinical questions on Reddit's r/askdocs). The critic model \(M_{\text{critic}}\) (Grok-3) inspects the clinical vignette to prescribe the optimal turn count and structural trajectory, while the actor model \(M_{\text{actor}}\) (GPT-5 Thinking) instantiates the conversation across four distinct patient personality archetypes governed by introversion-extroversion and logical-emotional axes. Other themes (emergency triage, global health, health data interpretation) are synthesized using in-context calibration examples from HealthBench-Easy. Raw dialogues undergo iterative verification by \(M_{\text{critic}}\) across completeness and medical consistency (up to 3 iterations). Simultaneously, \(M_{\text{actor}}\) generates up to 5 rubric criteria per theme-axis pair (covering Accuracy, Completeness, Context Awareness, Communication Quality, and Instruction Following) with continuous scores \(p_{ij} \in [-10, +10]\). \(M_{\text{critic}}\) inspects these rubrics via quality and score sanity checks. Crucially, the pipeline injects a 10% random polarity inversion into positive rubrics (e.g., flipping "Advises immediate ambulance" into "Fails to advise ambulance" with a negative score penalty). The authors provide a formal mathematical proof showing that replacing a positive rubric with its negative-polarity contrapositive strictly reduces the normalized score unless the model attains a perfect score, thereby preventing reward hacking and making the synthesized Vaidya-Rubrics dataset (5,351 validated items) exceptionally challenging.

3. Rubric-Based Group Sequence Policy Optimization (GSPO): Eliminating coarse binary reward signals in open-ended text and aligning long-horizon clinical interactions In the clinical healthcare reasoning phase, final model responses across multi-turn dialogues cannot be evaluated deterministically. An LLM-as-a-judge (GPT-4.1) evaluates candidate responses against each generated rubric criterion \(j \in \{1, \dots, M_i\}\). Letting \(\mathbf{1}\{r_{ij}\}\) indicate whether criterion \(j\) is satisfied, and \(p_{ij} \in [-10, 10]\) (\(p_{ij} \neq 0\)) denote its calibrated weight, the normalized scalar reward for dialogue instance \(i\) is formulated as: $\(s_i = \frac{\sum_{j=1}^{M_i} \mathbf{1}\{r_{ij}\} p_{ij}}{\sum_{j=1}^{M_i} \max(0, p_{ij})}\)$ This scalar score serves as the sequence reward within Group Sequence Policy Optimization (GSPO). For each dialogue context, \(K\) candidate trajectories are sampled, and normalized advantages \(\hat{A}(k)\) are computed relative to group mean and standard deviation to update policy parameters \(\pi_\theta\) using clipped importance sampling. To maximize training efficacy, a clinical system prompt clarifying the operational definitions of the five evaluation axes is prepended to the policy during training (yielding an immediate 3–5% absolute gain). Furthermore, dynamic difficulty filtering is executed at the onset of each epoch, strictly pruning samples whose rolling average score over 8 generations falls outside the active learning window of \([0.15, 0.7]\).

Loss & Training

All reinforcement learning runs are implemented within the open-source verl framework using bfloat16 precision and the Adam optimizer (\(\beta_1 = 0.9, \beta_2 = 0.95\)) with a peak learning rate of \(1\times 10^{-6}\) and a 5-step cosine warmup scheduler. 1. Diagnostic RL Phase: Qwen3-8B is trained at batch size 32 with a maximum generation limit of 10,000 completion tokens. It undergoes rule-based GSPO followed by one epoch of online error-rubric RL to produce FV-8B-Diag. 2. Clinical Healthcare RL Phase: Starting from FV-8B-Diag, training continues for 5 epochs. To test architectural scaling, Qwen3-4B and Qwen3-30B-A3B base models are trained for 3 epochs under identical hyperparameters. This phase scales context length to 20,000 tokens and maximum completion tokens to 16,000, accommodating extensive conversational multi-turn histories.

Key Experimental Results

Main Results

The Fathom-Vaidya models were benchmarked across two primary clinical evaluation regimes: the expert-level multiple-choice diagnostic examination benchmark MedXpertQA-Text (2,450 questions across 11 organ systems) and the challenging multi-turn conversational benchmark HealthBench-Hard (1,000 multi-turn patient-doctor interactions).

Table 1: Main performance comparison on MedXpertQA and HealthBench-Hard

Model Category Model Name Parameters MedXpertQA Score (%) HealthBench-Hard (w/o Prompt) HealthBench-Hard (w/ Prompt)
Open-Source Base / Med UltraMedical-3.1 8B 17.20 - -
Open-Source Base / Med Qwen3-8B (Base) 8B 17.40 10.20 13.10
Open-Source Base / Med Huatuo-o1 8B 18.90 - -
Open-Source Base / Med II-Medical-8B 8B 25.00 13.20 17.95
Open-Source Base / Med Baichuan-M2 32B 27.80 34.70 37.92
Open-Source Base / Med Qwen3-30B-A3B 30B (A3B) 23.30 18.70 23.42
Open-Source Base / Med Qwen3-Next-80B-A3B 80B (A3B) 32.50 26.20 30.49
Open-Source Base / Med GPT-oss-120B 120B 36.80 27.50 35.52
Proprietary Thinking Models o4-mini - - 17.60 23.20
Proprietary Thinking Models o3 - - 31.60 35.10
Proprietary Thinking Models gpt-5-thinking - - 39.95 43.79
Proprietary Thinking Models gpt-5.2-thinking - - 38.58 43.60
Proprietary Thinking Models gpt-5-mini-thinking - - 41.15 44.60
Fathom-Vaidya (Ours) FV-4B 4B - 31.44 35.30
Fathom-Vaidya (Ours) FV-8B-Diag 8B 27.90 - -
Fathom-Vaidya (Ours) FV-8B 8B 28.30 30.30 41.15
Fathom-Vaidya (Ours) FV-30B-A3B 30B (A3B) - 37.75 50.10

Table 2: Diagnostic generalization and clinical practice evaluation (MedQA, MedCase, DiagArena, GPQA-Bio, HealthBench-Pro)

Model MedQA Acc (%) MedCase Acc (%) DiagArena Acc (%) GPQA-Bio Acc (%) HealthBench-Pro Acc (%)
Qwen3-4B - - - 56.14 15.90
FV-4B (Ours) - - - 62.82 (+6.68) 28.60 (+12.70)
Qwen3-8B 79.80 32.00 33.10 61.53 17.40
FV-8B-Diag (Ours) 85.40 (+5.60) 32.20 (+0.20) 36.30 (+3.20) - -
FV-8B (Ours) 86.20 (+6.40) 35.00 (+3.00) 37.50 (+4.40) 64.10 (+2.57) 29.40 (+12.00)
Qwen3-30B-A3B - - - 69.20 23.20
FV-30B-A3B (Ours) - - - 70.52 (+1.32) 39.80 (+16.60)

Ablation Study

The paper reports targeted ablations isolating the impact of rubric reward weighting \(\alpha\) during diagnostic RL and evaluating the synthetic Vaidya-Rubrics dataset against baseline dataset distributions:

Table 3: Ablation on diagnostic rubric weight \(\alpha\) (MedXpertQA evaluation)

Model Variant Rubric Weight (\(\alpha\)) Accuracy Weight (\(1-\alpha\)) MedXpertQA Score (%) Note
FV-8B-Ru1 \(\alpha = 1.0\) \(0.0\) 27.76 Pure rubric error penalty; prevents shortcut learning
FV-8B-Ru0.7 \(\alpha = 0.7\) \(0.3\) 27.31 Blended rubric and correctness signal
FV-8B-Ru0 \(\alpha = 0.0\) \(1.0\) 27.24 Pure binary outcome reward without error attribution

Table 4: Dataset ablation comparing training on HealthBench-Easy vs. Vaidya-Rubrics (HealthBench-Hard, with prompt)

Model Scale Base Model Score (%) Trained on HealthBench-Easy (%) Trained on Vaidya-Rubrics (Ours) (%) Gain over Easy Baseline
4B 8.56 ~28.00 35.30 +7.30% absolute
8B 13.10 ~30.50 41.15 >10.60% absolute
30B-A3B 23.42 ~39.20 50.10 >10.90% absolute

Key Findings

  1. Rubric penalties outperform outcome-only signals: Setting \(\alpha=1.0\) yields the highest diagnostic score (27.76 on MedXpertQA). Categorizing trajectory errors into knowledge, causal, and contextual defects provides the dense, structured gradient signal necessary to overcome diagnostic plateaus without supervised CoT demonstrations.
  2. Breakthrough on interactive conversational axes: Radar chart analysis on HealthBench-Hard reveals that Fathom-Vaidya achieves its most pronounced gains on Completeness and Context Awareness (25–30% absolute increases), with critical themes such as Emergency Referrals, Responding under Uncertainty, and Context-Seeking improving by over 30%.
  3. Synthetic curation trumps closed benchmark tuning: Fine-tuning directly on HealthBench-Easy stagnates, whereas synthesizing multi-turn trajectories grounded in MedRedQA patient queries unlocks out-of-distribution transfer, allowing FV-30B-A3B to reach 50.1% on HealthBench-Hard and surpass leading frontier proprietary models.

Highlights & Insights

  • SFT-Free Emergence of Medical Reasoning: While contemporary medical models (e.g., Huatuo-o1, MedReason) rely on massive human-curated CoT traces, Fathom-Vaidya demonstrates that pure reinforcement learning coupled with difficulty filtering and structured error attribution can activate and organize intrinsic clinical reasoning.
  • Polarity Inversion as a Defense Against Reward Hacking: Randomly inverting 10% of positive rubrics and rigorously penalizing errors prevents policy models from generating bland, evasive medical boilerplate that superficially scores well, enforcing strict accountability in clinical triage.
  • Sub-30B Open Model Surpasses Frontier Thinking LLMs: Achieving 50.1% on HealthBench-Hard sets a state-of-the-art benchmark, eclipsing GPT-5 (thinking) (43.79%) and o3 (35.10%), while scoring 39.80% on HealthBench-Professional, within close reach of practicing human physicians (43.70%).

Limitations & Future Work

  • Restricted Model Lineage: Empirical evaluations are conducted exclusively across the Qwen-3 architecture family (4B, 8B, 30B-A3B), leaving the cross-architecture transferability of the rubric RL recipe to models like LLaMA untested.
  • Exclusion of Multimodal Clinical Modalities: The current framework operates entirely over text transcripts. Incorporating diagnostic imaging (CT, MRI, X-rays), pathology slides, and continuous ICU telemetry remains an open frontier.
  • Reliance on Proprietary Models for Synthesis and Judging: Dialogue generation and online trajectory auditing rely on GPT-5 Thinking and Grok-3. Developing self-rewarding, fully open-source clinical verifiers represents a vital next step.
  • vs Baichuan-M2: Baichuan-M2 trains a 32B model with clinical evaluator rubrics on de-identified hospital data; however, its rubrics center primarily on diagnostic accuracy and language fluency, neglecting context-seeking, completeness, and instruction following, and under-representing emergency referrals. Fathom-Vaidya establishes comprehensive coverage across seven conversational themes and introduces polarity-inverted rubrics with tighter mathematical guarantees.
  • vs II-Medical-8B / Beyond Distillation: While these works establish minimalist rule-based RL for medical QA, they are strictly constrained to single-turn multiple-choice problems. Fathom-Vaidya pioneers a sequential curriculum bridging structured diagnostic RL and non-verifiable, multi-turn clinical dialogue optimization under GSPO.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Elegant sequential formulation marrying rule/rubric diagnostic RL with actor-critic clinical dialogue synthesis and polarity-inverted rubrics.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive cross-benchmark evaluation covering MedXpertQA, MedQA, MedCase, DiagArena, HealthBench-Hard, and HealthBench-Professional with clinician agreement audits.
  • Writing Quality: ⭐⭐⭐⭐⭐ Rigorous conceptual structure, lucid prose, detailed appendices with mathematical proofs, and physician validation data.
  • Value: ⭐⭐⭐⭐⭐ Demonstrates that targeted synthetic data and dense rubric RL can bridge the performance gap between open-weight models and frontier commercial reasoning systems in high-stakes healthcare.