Fathom-Vaidya: Advancing Medical Reasoning with Rubric-Based Rewards¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Medical Reasoning, Reinforcement Learning, Multi-dimensional Rubrics, Diagnostic Reasoning, Clinical Interactions
TL;DR¶
Addressing the decoupling between convergent diagnostic inference and multi-turn open-ended clinical dialogue, Fathom-Vaidya introduces a sequential reinforcement learning framework without supervised Chain-of-Thought data, combining two-stage rule/error-rubric RL with an Actor-Critic synthetic clinical rubric reward system to achieve a state-of-the-art 50.1% on HealthBench-Hard with a 30B model.
Background & Motivation¶
Deploying Large Language Models (LLMs) in real-world healthcare demands rigorous clinical reasoning capabilities that extend well beyond standard multiple-choice medical examination benchmarks. Authentic clinical practice inherently couples two distinct yet interdependent cognitive dimensions: diagnostic reasoning, which requires evidence-driven, convergent inference over symptoms, medical history, laboratory tests, and imaging findings to establish an accurate differential diagnosis; and clinical healthcare reasoning, which entails broader navigational judgment to actively elicit missing context, navigate diagnostic uncertainty, triage critical emergencies, and deliver empathetic, patient-centered communication across multi-turn interactions where a single deterministic answer does not exist. Existing general-purpose and specialized medical models frequently stumble in complex clinical scenarios, exhibiting persistent vulnerabilities on challenging evaluations such as HealthBench and MedXpertQA.
The fundamental tension stems from the severe mismatch between conventional reinforcement learning (RL) training paradigms and the multi-faceted nature of real clinical decision-making. Prior reinforcement learning efforts in medicine, such as II-Medical and Huatuo-o1, almost exclusively rely on rule-based verifiable rewards derived from structured multiple-choice exams. While binary correctness signals provide reliable feedback for convergent answers, they entirely fail to penalize flawed intermediate causal steps, medical hallucinations, or critical diagnostic omissions within the generated reasoning trajectories. Furthermore, in open-ended doctor-patient interactions where objective ground-truth answers are absent, scalar binary rewards become unusable. Although recent works like Baichuan-M2 explore subjective rubric-based RL on synthetic dialogues, their criteria remain skewed toward basic correctness and surface communication quality, largely overlooking crucial clinical axes such as proactive context-seeking, completeness of differential considerations, and safety-critical emergency referral guidelines.
To resolve the lack of dense, informative training signals across both diagnostic and conversational axes, this work introduces a sequential reinforcement learning framework that eliminates reliance on costly human-annotated supervised Chain-of-Thought (SFT CoT) data. The authors first bootstrap diagnostic reasoning using difficulty-filtered exam questions from MedBullets, progressing from rule-based verification to online error-rubric severity penalization. Subsequently, they deploy an iterative Actor-Critic feedback loop to synthesize realistic multi-turn clinical scenarios and fine-grained, continuous-scoring multi-dimensional rubrics (the Vaidya-Rubrics dataset). Core idea: sequentially decouple medical reasoning into two-stage rule/error-rubric diagnostic RL and multi-dimensional Actor-Critic rubric-guided clinical interaction RL, supplying dense, multi-axial scalar feedback to optimize non-verifiable healthcare decision-making under GSPO.
Method¶
Overall Architecture¶
The end-to-end training pipeline of Fathom-Vaidya is structured into two sequential stages: diagnostic reasoning bootstrapping and multi-turn clinical healthcare reasoning refinement. The input comprises curated medical question pools (MedBullets, MedQA, MedMCQA) alongside patient query seeds from MedRedQA, and the output is the Fathom-Vaidya model family across various scales (FV-4B, FV-8B, and FV-30B-A3B). In the first stage, models undergo initial rule-based RL on structured exam questions, followed by online rubric RL where an LLM judge evaluates rollout trajectories to penalize categorized clinical reasoning errors. In the second stage, an Actor-Critic generation loop synthesizes 5,351 realistic multi-turn clinical conversations and aligned multi-dimensional evaluation rubrics across 7 healthcare themes, driving sequence-level policy optimization via Group Sequence Policy Optimization (GSPO).
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input: Medical Exam Questions & Clinical Query Seeds"] --> B["Rule-Based Diagnostic RL<br/>Difficulty filtering + GSPO binary reward"]
B --> C["Online Error-Rubric RL<br/>Tri-class clinical error severity penalty"]
C --> D["Actor-Critic Conversation & Rubric Synthesis<br/>Persona-grounded multi-turn dialogues + Polarity inversion"]
D --> E["Multi-Dimensional Rubric-Guided Clinical RL<br/>Multi-axis continuous scoring + System prompt alignment"]
E --> F["Output: Fathom-Vaidya Medical Reasoning Models"]
Key Designs¶
1. Rule-Based Diagnostic RL: Difficulty-Calibrated Reasoning Elicitation without SFT CoT To circumvent the prohibitive cost and potential cognitive biases associated with human-annotated Chain-of-Thought demonstrations, the pipeline initiates diagnostic reasoning through purely exploration-driven reinforcement learning. The authors harvest 7,801 high-caliber, five-option multiple-choice questions from MedBullets, spanning USMLE Step 1, Step 2, and specialized Orthopedics curricula with comprehensive rationale explanations, complemented by MedQA and MedMCQA. To ensure maximal gradient informativeness, questions undergo difficulty-based partitioning using the base model (e.g., Qwen3-8B) evaluated over 8 runs at temperature 1.0. Questions with mean accuracy \(\bar{s}_i \in (0.4, 0.7]\) are allocated to the rule-based RL phase, while harder questions with \(\bar{s}_i \in [0.1, 0.4]\) are reserved for rubric refinement. Models are trained on binary outcome rewards (1 for correct, 0 otherwise) via GSPO, successfully coaxing the base model to discover and reinforce rigorous reasoning chains purely through self-exploration.
2. Online Error-Rubric RL: Trajectory Deconstruction and Severity-Weighted Penalization Recognizing that binary final-answer rewards cannot detect hallucinated facts or logically invalid causal hops along an otherwise correctly guessing trajectory, the authors introduce a second diagnostic stage termed online error-rubric RL. Building upon the rule-trained checkpoint (FV-8B-Diag), rollouts are evaluated online by an advanced LLM judge (GPT-5 in thinking mode) that systematically detects and classifies reasoning flaws into three clinical categories: knowledge errors (e.g., physiological or pharmacological hallucinations), faulty causal reasoning (e.g., confusing correlation with causation in disease progression), and context errors (e.g., misinterpreting patient vitals). Each defect is assigned a severity score (2 for major, 1 for minor), aggregated into a normalized trajectory rubric reward: $\(R_{\text{rubric}} = \frac{15 - \sum \text{severity}}{15}\)$ where the threshold 15 is calibrated against empirical baselines (Qwen3-8B achieves a mean severity score of 11.4 on MedXpertQA samples). The composite training reward is defined as: $\(R_{\text{final}} = \alpha \cdot R_{\text{rubric}} + (1 - \alpha) \cdot R_{\text{accuracy}}\)$ Ablation studies reveal that setting \(\alpha = 1.0\) yields optimal performance, effectively steering policy updates toward eliminating structural reasoning vulnerabilities without compromising diagnostic precision.
3. Actor-Critic Conversation & Rubric Synthesis: Persona Diversity and Polarity Inversion To address the scarcity of interactive healthcare data with non-verifiable ground truth, the framework implements an iterative Actor-Critic synthesis loop to construct the Vaidya-Rubrics dataset. In dialogue generation, queries for themes such as "responding-under-uncertainty" and "context-seeking" are seeded from MedRedQA (real-world patient queries from r/askdocs vetted by clinicians), while themes like emergency referrals, global health, and health-data interpretation are guided by HealthBench-Easy in-context anchors. The Actor model is prompted with four distinct patient persona profiles defined along introversion–extroversion and logical–emotional axes. A Critic model performs up to 3 iterative passes evaluating completeness (detecting missing notes or data placeholders) and medical consistency. In rubric creation, the Critic verifies criterion uniqueness and score reasonableness within \([-10, +10]\). Crucially, to prevent policy exploitation via superficial sycophancy or generic pleasantries, positive rubrics are inverted with a 10% probability into negative criteria (e.g., "Fails to advise calling an ambulance, -8 points"). Proposition 1 formally proves that negative rubric inversion strictly bounds and reduces normalized conversation scores unless performance is flawless, establishing a challenging, un-gameable supervision set of 5,351 dialogues.
4. Multi-Dimensional Rubric-Guided Clinical RL: Continuous Scalar Aggregation for Non-Verifiable Tasks Transitioning to open-ended clinical consultation, the final model response is evaluated by an LLM-as-a-judge against all \(M_i\) generated rubrics for the given case. Denoting \(\mathbf{1}\{r_{ij}\}\) as the binary indicator of whether criterion \(j\) is met and \(p_{ij} \in [-10, 10]\) as its assigned point value, the continuous conversation score is normalized against the maximum attainable positive points: $\(s_i = \frac{\sum_{j=1}^{M_i} \mathbf{1}\{r_{ij}\} p_{ij}}{\sum_{j=1}^{M_i} \max(0, p_{ij})}\)$ This scalar score serves as the sequence reward within the GSPO framework. Furthermore, a structured clinical system prompt is incorporated during training and evaluation, explicitly orienting the model across five foundational clinical axes: Communication (structured, concise, and empathetic), Instruction Following, Medical Accuracy, Context Awareness (eliciting clarifications before definitive advice), and Completeness (highlighting red-flag emergency symptoms). To maintain informative exploration throughout training, dynamic dataset filtering retains datapoints with average 8-run scores within \([0.15, 0.7]\) at the start of each epoch.
Loss & Training¶
All policy training is executed within the verl distributed RL framework utilizing Group Sequence Policy Optimization (GSPO). For each dialogue context, \(K\) full-sequence rollouts \(\tau^{(k)}\) are sampled from the reference policy \(\pi_{\theta_{\text{old}}}\), and sequence-level normalized advantages \(\hat{A}(k)\) are computed by subtracting group mean rewards and dividing by standard deviations. The policy is optimized using the clipped surrogate objective:
$\(\mathcal{L}(\theta) = \mathbb{E}_{G} \left[ \frac{1}{K} \sum_{k=1}^K \min \left( r_\theta(\tau^{(k)}) \hat{A}(k), \, \text{clip}(r_\theta(\tau^{(k)}), 1-\epsilon, 1+\epsilon) \hat{A}(k) \right) \right]\)$
Training uses bfloat16 mixed precision, a global batch size of 32, and Adam optimizer (\(\beta_1=0.9, \beta_2=0.95\)) with a cosine learning rate scheduler peaking at \(1\times 10^{-6}\) after 5 warmup steps. The context length is set to 20,000 tokens with maximum completion tokens capped at 16,000 for clinical healthcare reasoning (10,000 for diagnostic reasoning). FV-4B and FV-30B-A3B are trained for 3 epochs, whereas FV-8B is trained for 5 epochs to reach empirical convergence.
Key Experimental Results¶
Main Results¶
The models were evaluated on the challenging, exam-grade MedXpertQA benchmark (2,450 expert questions across 11 body systems) and the primary clinical conversation benchmark HealthBench-Hard (1,000 most difficult cases from HealthBench), comparing against state-of-the-art open-source and proprietary reasoning systems.
| Dataset | Metric | Ours (FV-30B-A3B / FV-8B) | Prev. SOTA / Baseline | Gain |
|---|---|---|---|---|
| MedXpertQA (Text) | Accuracy (%) | 28.3 (FV-8B) | 17.4 (Qwen3-8B) / 25.0 (II-Medical-8B) | +10.9% absolute over Qwen3-8B |
| MedXpertQA (Text) | Accuracy (%) | 27.9 (FV-8B-Diag) | 27.8 (Baichuan-M2-32B) | 8B surpasses 32B proprietary baseline |
| HealthBench-Hard (w/ Sys) | Accuracy (%) | 50.1 (FV-30B-A3B) | 23.42 (Qwen3-30B-A3B) / 43.79 (GPT-5-thinking) | +26.68% over base; new global SOTA |
| HealthBench-Hard (w/ Sys) | Accuracy (%) | 41.15 (FV-8B) | 13.1 (Qwen3-8B) / 37.92 (Baichuan-M2) | 8B outperforms Baichuan-M2-32B |
| HealthBench-Hard (w/o Sys) | Accuracy (%) | 37.75 (FV-30B-A3B) | 18.7 (Qwen3-30B-A3B) / 31.6 (OpenAI o3) | Exceeds OpenAI o3 and gpt-oss-120B |
| HealthBench-Professional | Accuracy (%) | 39.8 (FV-30B-A3B) | 23.2 (Qwen3-30B) / 36.1 (Grok-4.2) | +16.6% absolute; near physician (43.7) |
| GPQA-Biology | Accuracy (%) | 70.52 (FV-30B-A3B) | 69.20 (Qwen3-30B) | +1.32% general domain transfer |
Ablation Study¶
The authors conducted granular ablation studies evaluating the error-rubric weighting parameter \(\alpha\) in diagnostic training, as well as the training efficacy of Vaidya-Rubrics versus fine-tuning exclusively on HealthBench-Easy.
| Config | Metric (MedXpertQA / HealthBench-Hard) | Note |
|---|---|---|
| FV-8B-Ru1 (\(\alpha = 1.0\)) | 27.76% (MedXpertQA) | Pure error-rubric severity penalization during diagnostic RL |
| FV-8B-Ru0.7 (\(\alpha = 0.7\)) | 27.31% (MedXpertQA) | Combined error-rubric penalty and binary accuracy reward |
| FV-8B-Ru0 (\(\alpha = 0.0\)) | 27.24% (MedXpertQA) | Pure binary accuracy reward without intermediate trajectory auditing |
| Qwen3-8B + HealthBench-Easy | 30.2% (HealthBench-Hard, w/ Sys) | Trained solely on the 4k easy subset of HealthBench |
| Qwen3-8B + Vaidya-Rubrics (FV-8B) | 41.15% (HealthBench-Hard, w/ Sys) | Synthetic actor-critic rubrics yield +10.95% absolute improvement |
| Qwen3-30B + HealthBench-Easy | 38.6% (HealthBench-Hard, w/ Sys) | Baseline scaling on existing benchmark data |
| Qwen3-30B + Vaidya-Rubrics (FV-30B) | 50.10% (HealthBench-Hard, w/ Sys) | Full synthetic curriculum yields +11.50% absolute improvement |
Key Findings¶
- Intermediate error penalization outperforms outcome-only rewards: Placing full weight on the rubric penalty (\(\alpha=1.0\)) yields the highest diagnostic accuracy (27.76%), demonstrating that suppressing reasoning hallucinations directly enhances final answer correctness.
- Completeness and context awareness show the largest leaps: Axial breakdowns on HealthBench-Hard reveal that Fathom-Vaidya achieves 25%–30% absolute improvements in completeness and context awareness, proving that the multi-dimensional rubrics successfully incentivize proactive questioning and red-flag identification.
- Robust out-of-distribution clinical generalization: On HealthBench-Professional (525 doctor-assistant daily workflows), FV-30B-A3B improves by 16.6% over baseline, reaching 39.8% and outperforming Grok-4.2 (36.1%) while approaching the practicing physician baseline (43.7%).
Highlights & Insights¶
- SFT-free clinical reasoning bootstrapping: The framework proves that medical LLMs can break free from costly, brittle supervised Chain-of-Thought fine-tuning. Pure exploration guided by difficulty filtering, rule-based RL, and error severity rubrics extracts latent clinical competence directly from the base model.
- Mathematical rigor in negative rubric inversion: Inverting 10% of positive rubrics into negative criteria mathematically guarantees that shortcuts and generic agreeable prose are heavily penalized when safety-critical clinical standards are omitted.
- High clinical consensus on synthetic data: A randomized verification study with 8 licensed physicians across 500 cases and 5,570 rubrics showed 99.4% conversation validity, 97.7% rubric relevance, and 92.2% agreement between clinicians and the GPT-5 error judge.
Limitations & Future Work¶
- Evaluation restricted to the Qwen3 architecture: All experimental validation is conducted within the Qwen3 model family (up to 30B parameters), leaving open questions regarding cross-architecture generalizability to LLaMA or DeepSeek foundations.
- Absence of multimodal clinical grounding: The study evaluates strictly text-based reasoning, excluding diagnostic modalities such as radiology scans, ECG signals, histopathology slides, and electronic health record time-series.
- Computational overhead of online judge rollouts: Online error-rubric extraction relies on GPT-5 thinking rollouts during Stage 1 RL, incurring substantial inference latency and API cost during training.
Related Work & Insights¶
- vs Baichuan-M2: Baichuan-M2 trains a 32B model using clinical evaluator rubrics on de-identified hospital data, but focuses narrowly on diagnostic correctness and communication. Fathom-Vaidya expands coverage to 7 themes including emergency triage and context-seeking with negative rubric inversion, surpassing Baichuan-M2 on HealthBench-Hard (50.1% vs 37.92%).
- vs II-Medical-8B / Huatuo-o1: Prior models heavily rely on SFT distillation and binary verifiers. Fathom-Vaidya proves that two-stage rule and error-rubric RL without SFT CoT achieves superior diagnostic reasoning (28.3% vs 25.0% on MedXpertQA).
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ [Decoupled two-stage diagnostic and multi-axis clinical interaction RL with negative rubric inversion]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Evaluated on MedXpertQA, HealthBench-Hard, HealthBench-Professional, and physician blind reviews]
- Writing Quality: ⭐⭐⭐⭐⭐ [Clear mathematical formulations, thorough prompt disclosures, and structured presentation]
- Value: ⭐⭐⭐⭐⭐ [Establishes open-weights SOTA surpassing proprietary models on premier healthcare benchmarks]