GPAgentBench-2K: Benchmarking Large Language Model Agents in Complex Clinical Action Space¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/jianing-lab/GPAgentBench
Area: medical_nlp
Keywords: clinical agents, constrained Markov decision process, action space scaling, primary-care decision making, safety-informed abstention
TL;DR¶
Addressing the oversimplified action spaces and unconstrained objectives in current clinical agent benchmarks, this paper introduces GPAgentBench-2K, the first Constrained Markov Decision Process (CMDP) benchmark for primary care featuring a 6-action clinical space, action masking priors, and safety-abstention referral outcomes, exposing a critical quality-safety gap across frontier LLMs and establishing a reference baseline via C-GRPO.
Background & Motivation¶
Large language models show great promise as autonomous clinical agents capable of taking patient histories, calling diagnostic tools, and executing complex medical reasoning. However, existing benchmarks predominantly formulate diagnosis as static, single-step prediction over structured electronic health records, bypassing the sequential, active evidence-gathering process central to clinical practice. Even recent interactive environments that incorporate Markov Decision Processes (MDPs) heavily truncate the agent's action space to coarse two-action subsets (e.g., restricted to ask and diagnose), or divide clinical duties across multi-agent pipelines where individual agents operate within microscopic textual sub-tasks rather than navigating the holistic clinical encounter of a general practitioner.
The deeper tension lies in the optimization formulation: existing interactive medical environments treat clinical decision-making as an unconstrained MDP with a single terminal goal—identifying the correct disease. Whenever secondary considerations such as diagnostic monetary cost are incorporated, they are treated merely as soft, additive penalties in a scalar reward. In real-world primary care, however, consultation is inherently a Constrained MDP (CMDP). Prescribing treatments prior to gathering sufficient evidence, ordering excessive diagnostic workups, or failing to refer a high-risk patient with red-flag symptoms constitute severe clinical safety violations. The asymmetric real-world harm of missing an urgent condition cannot be compensated for by downstream diagnostic accuracy bonuses.
Bridging this gap requires an interactive benchmark that models the complete clinical spectrum—including physical exams, diagnostic tests, and specialist referrals—under strict safety boundaries. Core idea: formalize primary-care clinical consultations as a Constrained MDP spanning six foundational actions (ask, body_exam, test, diagnose, treat, refer), governed by expert topological workflow action masks and persona-driven patient simulation, while operationalizing safety-informed abstention (referral) as a first-class terminal outcome alongside local management to evaluate and constrain agent behaviors.
Method¶
GPAgentBench-2K models primary-care clinical encounters through a verified CMDP formulation. It comprises an expert-validated dataset of 2,428 real-world GP records, a persona-driven patient simulator, an environment with dynamic topological action masking, and an optimization framework using Constrained Group Relative Policy Optimization (C-GRPO).
Overall Architecture¶
At each episode, the environment initialises a consultation from an anonymized clinical record coupled with a sampled patient persona spanning personality, language proficiency, history-recall fidelity, and cognitive clarity. The agent observes the initial chief complaint (with true diagnosis and disposition hidden) and iteratively selects actions from an admissible action set \(M(s_t) \subseteq \mathcal{A}\) enforced via expert topological workflow priors. The agent recursively collects evidence through conversation (ask), physical examinations (body_exam), and laboratory or imaging investigations (test). Once sufficient information is synthesized, the agent faces a terminal branching choice: safely escalate high-risk cases via refer to a specialist, or issue a diagnose action followed by a treat action for manageable conditions. The trajectory is scored against clinical quality rewards (\(R_{\text{Dx}}\), \(R_{\text{Tx}}\), and management accuracy) alongside safety cost (\(C_{\text{safe}}\)) and diagnostic expenditure cost (\(C_{\text{diag}}\)).
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}%%
flowchart TD
A["Initial Patient Presentation<br/>Chief complaint & persona setup"] --> B["Persona-Driven Patient Simulator<br/>Communication hurdles & fidelity shifts"]
B --> C["Full-Spectrum Clinical Action Space with Topological Prior<br/>Action masking across 6 actions"]
C --> D["Branching Clinical Disposition Outcomes<br/>Safety abstention vs local management"]
D --> E["Constrained Policy Optimization C-GRPO<br/>Dual variables decouple safety from reward"]
E --> F["Evaluation Metrics: Quality, Safety, Efficiency"]
Key Designs¶
1. Persona-Driven Patient Simulator: overcoming static structured-record assumptions
Standard benchmarks assume cooperative patients who volunteer complete medical histories upfront, failing to test an agent's ability to navigate communication friction. To address this, GPAgentBench-2K builds on 2,428 clinician-validated GP visit records by conditioning the virtual patient (powered by GPT-4o-mini) on four orthogonal persona dimensions: personality traits (impatient, overanxious, verbose, distrustful, symptom-minimizing, or neutral), language proficiency (basic, intermediate, advanced), anamnestic fidelity (high or low), and cognitive disorientation (normal, moderate, high). When interacting with an anamnestic patient who omits critical prior surgeries or a disoriented patient providing vague timelines, the agent cannot passively parse text; it must actively cross-examine and clarify ambiguous statements. Independent physician audits confirm high fidelity across factual consistency (96.9%), clinical realism (93.8%), and persona adherence (87.5%, Cohen's \(\kappa \ge 0.81\)).
2. Full-Spectrum Clinical Action Space with Topological Prior: preventing unconstrained action explosion
Expanding the action space from a binary ask/diagnose set to realistic clinical practice risks degenerate sequence transitions and parser failures. GPAgentBench-2K establishes a 6-action space \(\mathcal{A} = \{\text{ask}, \text{body\_exam}, \text{test}, \text{diagnose}, \text{treat}, \text{refer}\}\), incorporating 6 physical examination domains and 584 standardized laboratory, imaging, and functional diagnostic tests. To enforce clinical order without eliminating exploratory agency, the environment introduces a state-dependent admissible action mask \(M(s_t) \subseteq \mathcal{A}\). An episode begins in the evidence-collection phase, restricting actions to recursive info-gathering (ask, body_exam, test); only after gathering evidence can the agent issue a refer action (terminating the episode) or a diagnose action, which advances the state to unlock treat as the mandatory terminal step. This topological prior prevents premature treatments and redundant backtracking, allowing evaluation to reflect diagnostic reasoning rather than grammatical parser compliance.
3. Branching Clinical Disposition Outcomes: establishing safety-abstention as a first-class result
Conventional diagnostic benchmarks force agents to guess an exact diagnosis and prescribe therapy, inadvertently penalizing humility and incentivizing dangerous overconfidence on conditions requiring urgent specialist escalation. GPAgentBench-2K explicitly bifurcates terminal outcomes into "diagnose-and-manage" (45.1% of cases manageable in primary care) and "abstain-and-refer" (54.9% of cases across 29 medical specialties). The safety cost is formulated as a binary violation indicator of improper disposition:
$\(C_{\text{safe}} = \mathbf{1}_{\text{mgmt\_wrong}} \in \{0, 1\}\)$
This penalizes both high-risk missed referrals (Miss-Refer, treating a surgical or malignant pathology locally) and unnecessary over-referrals (Miss-GP, overloading tertiary centers with common self-limiting conditions), directly measuring safety-informed triage calibration.
4. Constrained Policy Optimization C-GRPO: preventing compensatory safety tradeoffs in scalar RL
Applying standard unconstrained policy optimization (such as GRPO) with scalarized penalty terms leads policies to absorb severe safety penalties if the diagnostic accuracy reward is sufficiently attractive, effectively trading safety for task performance. To provide a robust alignment baseline, the authors introduce Constrained GRPO (C-GRPO). For each trajectory \(\tau_{i,g}\) within a sampled group, quality rewards and cost violations are standardized independently into \(Z_R(\tau_{i,g})\) and \(Z_{C_k}(\tau_{i,g})\). The constrained advantage combines these components using adaptive Lagrangian dual variables:
$\(\hat{A}^c_{i,g} = Z_R(\tau_{i,g}) - \sum_{k \in \mathcal{K}} \lambda_k Z_{C_k}(\tau_{i,g})\)$
Dual variables are updated via projected subgradient ascent based on empirical constraint violations: \(\lambda_k \leftarrow [\lambda_k + \eta_\lambda (\hat{C}_k - \epsilon_k)]_+\), with the safety multiplier capped at \(p_{\text{mgmt}} = 6\) to stabilize training under severe misses. This decoupled gradient direction forces the policy to respect safety boundaries before optimizing diagnostic accuracy.
Loss & Training¶
C-GRPO updates the masked policy \(\pi^m_\theta\) using a clipped surrogate objective: $\(\mathcal{J}_{\text{C-GRPO}}(\theta) = \mathbb{E}\left[ \min\left( \rho^m_{i,g,t}(\theta) \hat{A}^c_{i,g}, \, \text{clip}\left(\rho^m_{i,g,t}(\theta), 1-\epsilon_c, 1+\epsilon_c\right)\hat{A}^c_{i,g} \right) - \beta D_{\text{KL}}\left(\pi^m_\theta(\cdot \mid s_t) \parallel \pi^m_{\text{ref}}(\cdot \mid s_t)\right) \right]\)$ Hyperparameters include reward weights \(w_{\text{diag}} = +5\), \(w_{\text{treat}} = +3\), safety threshold \(\tau_{\text{safety}} = 0\) (\(p_{\text{mgmt}} = 6\)), test cost budget \(\tau_{\text{cost}} = 0.2\) (equivalent to $100 USD), workflow violation penalty \(c_{\text{workflow}} = 5.0\), turn cap of 10 turns, and timeout penalty \(c_{\text{timeout}} = 5.0\).
Key Experimental Results¶
Main Results¶
Zero-shot evaluation was conducted on a stratified test split of 284 diagnose-and-manage cases and 201 abstain-and-refer cases across 16 frontier proprietary and open-source models, evaluated using GPT-5.4 as judge (validated with human expert agreement \(\kappa \ge 0.89\)).
| Model Type | Model | Dx Acc. (%) | Tx Score (%) | Mgmt Acc. (%) | Miss Refer (%) ↓ | Miss GP (%) ↓ | Avg. Turns ↓ | Test Cost (USD) ↓ |
|---|---|---|---|---|---|---|---|---|
| Proprietary | CLAUDE OPUS 4.7 | 70.0 (2.5) | 46.0 (2.0) | 55.9 (2.3) | 67.2 (3.3) | 27.8 (2.7) | 5.57 (0.06) | $253 (13) |
| Proprietary | GEMINI 3.1 PRO | 66.8 (3.1) | 41.6 (2.0) | 55.9 (2.3) | 47.8 (3.5) | 41.5 (2.9) | 6.07 (0.07) | $231 (12) |
| Proprietary | GPT-5.4 | 64.1 (2.8) | 53.2 (2.4) | 56.3 (2.3) | 51.2 (3.5) | 38.4 (2.9) | 6.91 (0.07) | $147 (10) |
| Proprietary | CLAUDE SONNET 4.6 | 61.7 (1.6) | 36.0 (1.8) | 60.2 (2.2) | 77.1 (3.0) | 13.4 (2.0) | 6.57 (0.05) | $218 (11) |
| Open-Source General | DEEPSEEK-V4-PRO | 59.5 (2.8) | 41.2 (2.0) | 56.7 (2.3) | 57.2 (3.5) | 33.5 (2.8) | 6.98 (0.07) | $146 (10) |
| Open-Source General | KIMI K2.6 | 56.7 (4.1) | 49.2 (3.5) | 41.2 (2.2) | 40.3 (3.5) | 71.8 (2.7) | 8.74 (0.08) | $34 (5) |
| Open-Source General | MINIMAX M2.7 | 60.0 (2.5) | 31.3 (1.9) | 53.0 (2.3) | 78.1 (2.9) | 25.0 (2.6) | 6.95 (0.07) | $278 (14) |
| Open-Source General | QWEN2.5 (72B) | 54.1 (3.2) | 25.4 (1.8) | 46.4 (2.3) | 54.7 (3.5) | 52.8 (3.0) | 6.86 (0.09) | $473 (20) |
| Open-Source General | QWEN2.5 (7B) | 40.4 (2.5) | 18.4 (1.3) | 47.4 (2.3) | 89.6 (2.2) | 21.1 (2.4) | 6.27 (0.09) | $445 (20) |
| Medical Domain | HUATUO-O1 (72B) | 62.3 (3.5) | 33.2 (2.4) | 48.5 (2.3) | 39.3 (3.5) | 60.2 (2.9) | 6.79 (0.08) | $325 (13) |
| Medical Domain | MEDGEMMA (27B) | 49.5 (3.4) | 29.7 (2.2) | 52.0 (2.7) | 65.7 (4.0) | 35.2 (3.4) | 8.51 (0.08) | $145 (12) |
| Medical Domain | DOCTORAGENT-RL (7B) | 41.0 (3.5) | 22.4 (2.0) | 15.7 (1.7) | 89.1 (2.2) | 81.0 (2.3) | 9.44 (0.06) | $89 (9) |
Ablation & RL Alignment Study¶
Table 2 & 10 report the optimization gains of Qwen2.5 (7B) under standard scalarized GRPO versus Lagrangian C-GRPO, along with performance drops across action space scaling.
| Strategy / Action Space Configuration | Dx Acc. (%) | Tx Score (%) | Mgmt Acc. (%) | Miss Refer (%) ↓ | Miss GP (%) ↓ | Test Cost (USD) ↓ |
|---|---|---|---|---|---|---|
| Qwen2.5 (7B) Base | 40.4 | 18.4 | 47.4 | 89.6 | 21.1 | $445 |
| GPAgent (GRPO) | 48.5 (+8.1) | 20.0 (+1.6) | 48.7 (+1.3) | 83.6 (-6.0) | 28.5 (+7.4) | $438 (-7) |
| GPAgent (C-GRPO) | 52.2 (+11.8) | 20.5 (+2.1) | 51.0 (+3.6) | 82.1 (-7.5) | 26.1 (+5.0) | $435 (-10) |
| Action space: 2 actions (Dx, Tx) | ~58.2 | ~24.2 | - | - | - | - |
| Action space: 3 actions (+ ask) | 54.1 (-4.1) | ~23.5 | - | - | - | - |
| Action space: 4 actions (+ test) | 47.1 (-7.0) | ~22.0 | - | - | - | - |
| Action space: 6 actions (Full: + body_exam, refer) | 40.4 (-6.7) | 18.4 (-3.6) | 47.4 | 89.6 | 21.1 | $445 |
Key Findings¶
- Severe Degradation under Action Space Scaling: Scaling the action space from a minimal 2-action set to the full 6-action CMDP causes a universal, monotonic performance drop in diagnostic accuracy across all models (dropping up to 21 percentage points). The largest drops occur when adding
ask(-4.1 pp on average, turning static classification into interactive dialog) andtest(-7.0 pp on average, introducing combinatorial diagnostic selection). - The Clinical Quality-Safety Gap: Current frontier models split into two uncalibrated extremes. Frontier proprietary models (Claude Opus 4.7, Claude Sonnet 4.6) suffer from severe over-confidence, missing 67.2% to 77.1% of required specialist referrals by insisting on treating complex surgical/oncological cases locally. Conversely, models like Kimi K2.6 and Huatuo-o1 (72B) exhibit over-cautious behavior, defaulting to referrals and failing local GP management on 60.2% to 71.8% of routine cases.
- Shotgun Medicine and Parameter Scaling Failure: Parameter scaling alone fails to resolve this safety gap. While scaling Qwen2.5 from 7B to 72B improves Dx accuracy (40.4% \(\to\) 54.1%), the 72B model still misses 54.7% of high-risk referrals and incurs an exorbitant diagnostic cost of $473 by excessively ordering tests without clinical discrimination.
- Rambling Effect of Restricted-Action RL: DoctorAgent-RL (7B), pre-trained exclusively in a 2-action (
ask,diagnose) dialogue sandbox, fails when deployed in the 6-action environment. Lacking the ability to leverage physical exams and tests, it enters repetitive questioning loops (36.7% duplicated queries, 9.44 average turns) and collapses to 15.7% management accuracy. - C-GRPO Superiority and the Residual Safety Deficit: C-GRPO outperforms unconstrained GRPO across all dimensions on Qwen2.5-7B, improving Dx accuracy by 11.8 pp and management accuracy by 3.6 pp while cutting redundant questions to 1.0% (on par with GPT-5.4). However, its absolute Miss-Refer rate remains at 82.1%, demonstrating that achieving clinically acceptable safety in complex clinical action spaces remains an unsolved challenge.
Highlights & Insights¶
- Safety-Informed Abstention as a First-Class Citizen: Elevates the clinician's decision to "know what not to treat" into a formal evaluation target, realigning clinical agent benchmarks with genuine medical ethics and referral practices.
- Lagrangian Constraint Decoupling: Solves the failure mode where models compensate for life-threatening clinical errors with high diagnostic test scores under scalarized RL rewards.
- Standardized Diagnostic Cost Grounding: Maps lab tests and imaging orders to real-world Medicare and urgent care pricing schedules, providing a realistic cost-efficiency penalty rather than arbitrary synthetic loss terms.
Limitations & Future Work¶
- English-Only Primary Care Focus: Constructed exclusively from English-language electronic records, leaving performance across multilingual settings, regional disease prevalences, and different national healthcare triage hierarchies unaddressed.
- Simulated Patient Fidelity Boundaries: While validated by senior physicians, the virtual patient agent (GPT-4o-mini) cannot fully replicate non-verbal physical signs or nuanced psychological dynamics of human clinical encounters.
- Absence of Formal Safety Proofs: Lagrangian relaxation with clipped dual multipliers provides empirical constraint steering rather than certified, provable safety guarantees in deployment.
Related Work & Insights¶
- vs DoctorAgent-RL (Feng et al., 2025): DoctorAgent-RL restricts actions to dialogue and diagnosis; GPAgentBench-2K expands to physical exams, diagnostic testing, and branching dispositions, demonstrating that policies fine-tuned in coarse action spaces degrade into repetitive rambling when confronted with realistic workflows.
- vs LA-CDM (Bani-Harouni et al., 2025): LA-CDM models test selection but applies diagnostic cost as a soft additive penalty without modeling the refer action; GPAgentBench-2K formulates the problem as a formal CMDP with dual optimization and safety-abstention.
- vs AgentClinic (Schmidgall et al., 2024): AgentClinic pioneered interactive clinical dialogue, but does not formulate primary-care practice as a CMDP balancing diagnostic quality, cost constraints, and safety-critical triage thresholds.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Introduces the first comprehensive CMDP clinical agent benchmark with 6 foundational actions, topological masks, and safety abstention.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Evaluates 16 frontier LLMs, provides detailed action scaling ablations, step-level behavioral metrics, and constrained RL fine-tuning.
- Writing Quality: ⭐⭐⭐⭐⭐ Clear problem formulation, rigorous mathematical grounding of CMDP and C-GRPO, and sharp clinical insights.
- Value: ⭐⭐⭐⭐⭐ Exposes the critical quality-safety tradeoff in medical AI, setting an essential evaluation standard for trustworthy clinical agents.