Skip to content

RASPER: Reward-Aligned Summarization of Clinical Notes for EHR Outcome Prediction

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/aryahm1375/RASPER
Area: Medical NLP
Keywords: Electronic Health Records, Clinical Note Summarization, Reinforcement Learning, Prediction Feedback, Multimodal EHR

TL;DR

This paper proposes RASPER, a reward-aligned clinical note summarization framework for EHR outcome prediction that integrates longitudinal structured codes into soft prompts and trains a tunable LLM summarizer via reinforcement learning from downstream predictor feedback (RLPF), preserving patient-specific textual evidence that effectively complements structured codes.

Background & Motivation

Electronic Health Records (EHRs) capture a patient's medical trajectory through complementary multimodal views: structured codes (such as diagnoses, procedures, and medications) reflect standardized cohort-level clinical patterns across populations, whereas unstructured discharge notes record granular, patient-specific narratives including symptom severity, physiological responses to therapies, and physician reasoning that standardized coding vocabularies cannot capture. However, clinical discharge notes are primarily authored for documentation, care coordination, and billing purposes rather than predictive modeling. Consequently, raw notes are notoriously lengthy, redundant, and susceptible to institutional documentation style variations, which dilutes predictive signals and biases models toward reporting artifacts when fed directly into deep architectures.

To mitigate this narrative noise, practitioners commonly employ large language models (LLMs) to condense clinical text into concise summaries prior to downstream modeling. Yet, conventional summarization pipelines treat note compression as a disconnected preprocessing step optimized purely for generic coverage, readability, and syntactic fluency. This design introduces a fundamental objective mismatch: generic summarizers routinely discard subtle, idiosyncratic clinical cues that a specific prognostic outcome relies upon, while preserving plausible but uninformative background statements. Moreover, distinct clinical endpoints (such as 30-day hospital readmission versus multi-label next-visit medication recommendation) inherently depend on divergent aspects of a patient's history, rendering any outcome-agnostic summarizer inherently sub-optimal.

This discrepancy highlights the necessity of co-optimizing note summarization directly with the target clinical objective. To bridge this divide, this paper pursues the core idea: co-train a tunable LLM note summarizer directly against the downstream EHR outcome prediction objective via reinforcement learning from prediction feedback (RLPF), utilizing soft-prompt embeddings of structured codes as patient-level anchors and rewarding summaries based on their relative improvement over historical predictive baselines.

Method

Overall Architecture

The RASPER framework comprises two interacting modules: a tunable LLM summarizer and an LLM-aware outcome predictor. Given a patient's longitudinal history of visits containing structured code sequences and unstructured discharge notes, the tunable summarizer transforms each raw note into a standardized visit-level summary, which are then concatenated in reverse chronological order (placing the most recent encounter first) to form a unified patient-level textual narrative. Simultaneously, a longitudinal EHR encoder compresses the structured code trajectory into a compact patient representation and projects it into continuous soft prompt tokens. The outcome predictor LLM receives the concatenated soft prompts and summary text embeddings to generate task-specific predictions. During training, after supervised fine-tuning of the summarizer and warm-up of the predictor, the summarizer policy is optimized via Proximal Policy Optimization (PPO), driven by scalar rewards derived directly from the predictor's loss.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}}%%
flowchart TD
    A["Patient Longitudinal EHR Visits<br/>{V_i,t, S_i,t}"] --> B["Structured Medical Codes V_i,t"]
    A --> C["Unstructured Discharge Notes S_i,t"]
    subgraph DES1["1. Structured Code Soft Prompting and Multimodal Joint Prediction"]
        direction TB
        B --> D["RETAIN Longitudinal Attention Encoder<br/>Visit-level & variable-level dual pooling"]
        D --> E["Linear Projection Layer Proj_ehr<br/>Generates P continuous soft prompt tokens E_i"]
        C --> F["Tunable LLM Summarizer generates S_hat_i<br/>Reverse-chronological concatenation"]
        E --> G["Combined Predictor Input I_i = [E_i; Embed(S_hat_i)]"]
        F --> G
        G --> H["Gemma-2-2B Outcome Predictor Forward<br/>Avg-Pooling + Linear Head โ†’ y_i"]
        H --> I["Downstream Prediction Loss L_Pred (BCE)"]
    end
    subgraph DES2["2. Teacher-Distilled Initialization and Staged Warm-up Strategy"]
        direction TB
        J["GPT-5.4-mini Target Summaries G_i,t"] --> K["Summarizer SFT Phase: Minimize NLL Loss<br/>Master clinical note structuring (10 epochs)"]
        K --> L["Predictor Independent Warm-up (20 epochs)<br/>Freeze summarizer, train predictor LoRA to stabilize"]
    end
    subgraph DES3["3. Prediction Feedback Reinforcement Learning with Adaptive Patient Baselines"]
        direction TB
        I --> M["Compute Individual Patient Reward<br/>Margin over best-5 historical loss: r_i = l_bar_5 - l_i"]
        M --> N["Mini-batch Z-score Normalization & Clipping [-2, +2]"]
        N --> O["PPO Clipped Objective + Value Head Fitting<br/>Frozen SFT reference policy KL penalty + Entropy bonus"]
        O -.Update Summarizer LoRA.-> F
    end
    I --> P["Final Clinical Outcome Predictions<br/>Readmission Risk / Next-Visit Medication Set"]

Key Designs

1. Structured Code Soft Prompting and Multimodal Joint Prediction: Alleviating modality isolation and narrative dilution

Prior multimodal EHR approaches either linearize codes into long text sequences that inflate context windows or build disjoint models where the modalities cannot mutually contextualize each other. RASPER resolves this by introducing a lightweight prompt-tuning integration: a RETAIN encoder (utilizing dual reverse-time GRU attention over visit importance and variable-level weights) first processes the patient's multi-visit diagnostic and procedure history into a dense representation \(h^{\text{ehr}}_i\). A linear projection module \(\text{Proj}_{\text{ehr}}\) then maps this vector into \(P=8\) continuous soft prompt embeddings \(E_i \in \mathbb{R}^{P \times D}\). These soft prompts are concatenated with the token embeddings of the summarized narrative \(\text{Embed}(\hat{S}_i)\) in the input representation space of the Gemma-2-2B predictor model. By grounding the LLM predictor with cohort-level inductive biases at the very first layer, the predictor can effectively identify complementary clinical signals in the accompanying note summary.

2. Teacher-Distilled Initialization and Staged Warm-up Strategy: Preventing cold-start collapse and noisy policy gradients

Initiating reinforcement learning directly from an unadapted base model on free-form clinical text generation leads to extreme training instability, as the open-ended token space is vast while binary outcome labels provide sparse feedback. RASPER addresses this via a rigorous two-stage curriculum. First, target summaries \(G_{i,t}\) are generated across visit notes using GPT-5.4-mini with standardized clinical extraction prompts, and the local summarizer policy \(\pi_\theta\) is trained for \(Q_{\text{SFT}}=10\) epochs via supervised fine-tuning (SFT) to minimize negative log-likelihood. Second, before policy alignment begins, the LLM-aware predictor is trained independently for \(Q_{\text{warm}}=20\) epochs while holding the summarizer fixed. This ensures that the predictor establishes a reliable decision boundary and calibrated loss landscape, preventing an under-trained predictor from emitting erratic reward feedback that could permanently degrade the summarizer policy.

3. Prediction Feedback Reinforcement Learning with Adaptive Patient Baselines: Driving summaries to preserve task-critical complementary evidence

To guide the summarizer beyond generic syntactic compression toward preserving patient-specific predictive evidence, RASPER employs Reinforcement Learning from Predictor Feedback (RLPF). Because raw prediction cross-entropy exhibits substantial variance across patients with differing baseline illness severities, directly utilizing raw loss would result in noisy, uncalibrated updates. RASPER constructs a robust individualized reference loss \(\bar{\ell}^{(u)}_{i,5}\) defined as the average of the five lowest historical losses recorded for patient \(i\) during the predictor warm-up stage. The instantaneous unnormalized reward is computed as the margin of improvement over this strong personal benchmark:

\[r^{(u)}_i = \bar{\ell}^{(u)}_{i,5} - \ell^{(u)}_i\]

This formulation explicitly rewards the policy only when the generated summary actively resolves diagnostic ambiguity and enhances predictor confidence relative to the patient's own best baseline. The reward is subsequently normalized via mini-batch Z-scoring, clipped to \([-2.0, 2.0]\), and utilized within a PPO objective comprising a clipped surrogate policy loss (\(\epsilon_{\text{PPO}}=0.2\)), a value function regression head, an entropy regularization term (\(\lambda_H=0.01\)), and a KL divergence constraint (\(\lambda_{\text{KL}}=0.02\)) against the frozen SFT reference policy to prevent stylistic language drift.

Loss & Training

The overall model training alternates across distinct phases: 1. Predictor Objective: At all training stages, the outcome predictor minimizes binary cross-entropy: $\(\mathcal{L}_{\text{Pred}} = \text{BCE}(\hat{Y}_i, Y_i)\)$ where only the RETAIN encoder, the soft prompt projection layer, the linear pooling head, and the predictor LoRA adapters are updated, keeping the backbone LLM weights frozen. 2. PPO Summarizer Objective: For training epochs \(e \ge Q_{\text{warm}}\), the summarizer updates its LoRA adapters using an independent AdamW optimizer on the composite PPO objective: $\(\mathcal{L}_{\text{PPO}} = \mathcal{L}_{\text{policy}} + \lambda_v \mathcal{L}_{\text{value}} - \lambda_H \mathbb{E}[H^{\ell}_{i,t}] + \lambda_{\text{KL}} \mathcal{L}_{\text{KL}}\)$ with value loss weight \(\lambda_v=0.5\), reward clipping threshold \(r_{\text{max}}=2.0\), and maximum generated summary length \(M_{i,t}=128\) tokens per visit.

Key Experimental Results

Main Results

The authors evaluate RASPER on MIMIC-III and MIMIC-IV across 30-day hospital readmission prediction (evaluated by AUROC and PRAUC) and medication recommendation (evaluated by multi-label F1 and Jaccard similarity score), comparing against competitive deep EHR and recent LLM-based healthcare architectures:

Category Model MIMIC-III Readmission AUROC MIMIC-III Readmission PRAUC MIMIC-III Med F1 MIMIC-III Med Jaccard MIMIC-IV Readmission AUROC MIMIC-IV Readmission PRAUC MIMIC-IV Med F1 MIMIC-IV Med Jaccard
Deep Learning Deepr 0.671 0.702 0.255 0.160 0.621 0.651 0.228 0.189
Deep Learning RETAIN 0.657 0.678 0.253 0.156 0.673 0.698 0.349 0.255
Deep Learning GRASP 0.618 0.635 0.153 0.096 0.535 0.578 0.267 0.195
Deep Learning AdaCare 0.645 0.687 0.235 0.133 0.615 0.644 0.251 0.184
Deep Learning StageNet 0.678 0.705 0.239 0.139 0.659 0.690 0.278 0.202
Deep Learning Adore 0.665 0.698 0.152 0.125 0.615 0.643 0.206 0.165
Deep Learning ARCI 0.648 0.669 0.283 0.184 0.668 0.698 0.303 0.246
LLM-based GraphCare 0.665 0.705 0.294 0.195 0.688 0.711 0.355 0.259
LLM-based LINKO 0.674 0.711 0.304 0.202 0.695 0.715 0.367 0.264
LLM-based RePrompT 0.682 0.718 0.312 0.206 0.702 0.723 0.375 0.273
Ours RASPER 0.704 0.738 0.326 0.218 0.725 0.742 0.388 0.282

Ablation Study

Systematic ablations were performed to isolate the empirical contribution of the structured EHR encoder, the summarization mechanism, and the RL alignment stage:

Model Variant MIMIC-III Readmission AUROC MIMIC-III Readmission PRAUC MIMIC-III Med F1 MIMIC-III Med Jaccard MIMIC-IV Readmission AUROC MIMIC-IV Readmission PRAUC MIMIC-IV Med F1 MIMIC-IV Med Jaccard
RASPER w/o EHR encoder 0.661 0.698 0.288 0.187 0.685 0.704 0.353 0.258
RASPER w/o Summarization (Raw Notes) 0.674 0.712 0.298 0.191 0.691 0.713 0.363 0.262
RASPER w/o RL (GPT-only summary) 0.689 0.725 0.309 0.201 0.708 0.723 0.371 0.272
RASPER w/o RL (SFT-only summary) 0.684 0.721 0.305 0.198 0.701 0.718 0.367 0.269
RASPER Full Model 0.704 0.738 0.326 0.218 0.725 0.742 0.388 0.282

Backbone Architecture Sensitivity

Performance across varying open-weight language model backbones:

LLM Backbone MIMIC-III Readmission AUROC MIMIC-III Readmission PRAUC MIMIC-III Med F1 MIMIC-III Med Jaccard MIMIC-IV Readmission AUROC MIMIC-IV Readmission PRAUC MIMIC-IV Med F1 MIMIC-IV Med Jaccard
Qwen2.5-0.5B 0.697 0.730 0.315 0.209 0.716 0.731 0.379 0.276
Llama-3.2-1B 0.701 0.734 0.321 0.214 0.722 0.737 0.384 0.278
Gemma-2-2B 0.704 0.738 0.326 0.218 0.725 0.742 0.388 0.282

Key Findings

  • Structured codes serve as an indispensable clinical anchor: Removing the structured EHR encoder (w/o EHR encoder) induces the steepest performance collapse across all configurations (e.g., MIMIC-III readmission AUROC drops by 0.043 from 0.704 to 0.661), demonstrating that cohort-level medical codes supply foundational physiological context that free text alone cannot substitute.
  • Raw discharge narratives severely compromise predictive attention: Feeding truncated raw notes directly into the model (w/o Summarization) achieves only 0.674 AUROC on MIMIC-III readmissions, significantly trailing even the generic GPT-summarized baseline (0.689). This highlights that clinical "note bloat" and non-standardized narrative boilerplate dilute model attention.
  • Task-guided alignment outperforms generic summarization: Moving from generic GPT-5.4-mini summaries (0.689) and local SFT summaries (0.684) to RLPF-aligned summaries (0.704) yields statistically significant gains at the 95% confidence level across both tasks and datasets. The prediction-loss reward effectively forces the summarizer to discard irrelevant administrative text and isolate decisive clinical markers.
  • Moderate inference overhead for substantial predictive gains: Profiling a batch of 12 patients reveals that pure code models run in 0.06s - 0.11s, while RePrompT requires 1.38s. RASPER incurs 1.87s due to the two-stage summarization and prediction inference. This modest computational increase represents an acceptable trade-off for the substantial +2.3% AUROC gain in offline clinical decision-support workflows.

Highlights & Insights

  • Inverting downstream loss into an upstream reinforcement learning reward: Instead of viewing summarization as an isolated NLP preprocessing step, RASPER establishes a closed-loop learning paradigm where the outcome predictor functions as the training environment for the text generation policy.
  • Personalized best-of-5 historical baseline formulation: Computing reward improvements relative to each patient's top-5 lowest warm-up losses circumvents inter-patient variance, ensuring that reward signals reflect genuine evidence enhancement rather than arbitrary variations in patient baseline disease severity.
  • Continuous soft prompt grounding: Projecting structured visit codes into continuous soft prompts via RETAIN provides a computationally efficient bridge that conditions the language model on temporal medical trajectories without squandering token budget on hard-coded linearizations.

Limitations & Future Work

  • Single-institution data validation: Evaluations are restricted to MIMIC-III and MIMIC-IV from Beth Israel Deaconess Medical Center. Validating robustness against differing EHR vendor schemas (e.g., Epic, Cerner) and distinct clinical writing conventions across external healthcare systems remains necessary.
  • Fixed summary budgeting and prompt rigidity: The framework enforces a rigid limit of 128 tokens per visit summary and 512 tokens per patient summary. Developing dynamic, uncertainty-aware length-allocation mechanisms could allow the model to allocate more tokens to clinically turbulent visits.
  • Clinical hallucination and factuality auditing: Optimizing purely for downstream predictive loss could incentivize the policy to generate spurious, over-confident medical entities that act as predictive shortcuts. Future work should integrate biomedical knowledge-graph entity verification rewards to guarantee clinical factuality.
  • vs Structured EHR Deep Models (RETAIN, StageNet): While earlier deep learning models efficiently model temporal code trajectories and disease staging, they discard unstructured clinical notes entirely. RASPER demonstrates that integrating reward-aligned note summaries unlocks latent patient-specific information missed by structured codes.
  • vs Knowledge-Graph Augmentation (GraphCare, LINKO): Knowledge-graph frameworks rely on external ontology propagation, which adds substantial graph construction overhead and can introduce out-of-context entity noise. RASPER stays faithful to intrinsic patient EHR records by directly mining and compressing raw clinical notes.
  • vs Prompt-tuned Multimodal LLMs (RePrompT): RePrompT (Moghaddam et al., ACL 2026) combines RETAIN with LLM prompt tuning but relies on static, task-agnostic clinical summaries. RASPER proves that closing the loop with prediction feedback (RLPF) produces markedly superior, task-informative summaries.

Rating

  • Novelty: โญโญโญโญโญ [Pioneering use of prediction feedback reinforcement learning (RLPF) to align clinical note summarization with EHR outcome prediction]
  • Experimental Thoroughness: โญโญโญโญโญ [Extensive evaluations across MIMIC-III and MIMIC-IV, detailed ablations, encoder benchmarks, backbone comparisons, and inference time profiling]
  • Writing Quality: โญโญโญโญโญ [Clear mathematical formulations, well-motivated problem framing, and coherent methodological progression]
  • Value: โญโญโญโญโญ [Provides a powerful, practical blueprint for multimodal clinical modeling by bridging unstructured clinical narratives and structured medical codes]