Skip to content

Learning Personalized Prompts for Healthcare Guidance

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/CGCL-codes/PPL
Area: Medical LLM
Keywords: personalized prompt learning, hard prompt optimization, reinforcement learning, clinical healthcare guidance, electronic health records

TL;DR

This paper introduces Personalized Prompt Learning (PPL), a framework that integrates longitudinal self-informed medical history with peer-informed collaborative clinical cases into tailored prompts, subsequently refining them via reinforcement learning against real physician recommendations to steer proprietary black-box LLMs toward high-quality personalized healthcare guidance without model fine-tuning.

Background & Motivation

The rapid advancement of large language models (LLMs) has sparked widespread interest across clinical medicine, where hospitals and patients increasingly seek automated systems capable of interpreting longitudinal personal health records and delivering individualized health guidance. However, existing medical LLM paradigms exhibit substantial limitations: most prior efforts focus on fine-tuning lightweight open-source models for a single isolated disease (such as diabetes or prostate cancer). In these systems, personalization is restricted to trivial numerical fluctuations in patient inputs, learning generic, disease-level treatment protocols rather than true patient-level variability. Consequently, they fail to address the heterogeneous health profiles and multi-morbidity conditions commonly encountered in real-world clinical practice.

The core tension stems from two competing constraints: on the one hand, comprehensive healthcare guidance across diverse clinical conditions demands the vast medical knowledge and sophisticated multi-step reasoning capabilities concentrated in cutting-edge proprietary LLMs (e.g., GPT-4, Gemini 1.5 Pro), whose parameters are strictly closed-source and cannot be directly fine-tuned. On the other hand, conventional prompt engineering relies on rigid, hand-crafted templates where sparse clinical values are inserted into fixed prompts. Such static templates fail to capture longitudinal multi-visit laboratory trajectories, lack awareness of similar peer clinical experiences, and cannot adapt to the shifting generation behaviors of distinct downstream LLMs. Furthermore, passing raw, extensive health indicators directly to external commercial APIs raises severe patient privacy concerns and often elicits generalized, non-actionable boilerplate responses.

This paper tackles this bottleneck by freezing the downstream foundation model and training an external, patient-adaptive discrete prompt optimization engine. Core idea: propose personalized prompt learning (PPL), an adaptive framework that constructs initial coarse prompts by fusing longitudinal self-informed predictions with collaborative peer-informed clinical diagnoses, and refines them through reinforcement learning via word-level discrete editing aligned with real physician recommendations, enabling proprietary black-box LLMs to generate high-fidelity personalized healthcare guidance.

Method

Overall Architecture

The PPL framework operates across three tightly coordinated stages: personalized information extraction, coarse-grained prompt construction, and reinforcement learning-driven fine-grained prompt refinement. Given a target patient \(x_i\) represented by longitudinal records of 35 physical examination indicators over variable numbers of hospital visits, PPL first uses a fine-tuned time-series predictor to extract self-informed disease condition predictions. Concurrently, a multi-layer perceptron (MLP) patient encoder projects the zero-padded longitudinal metrics into a unified latent space, using collaborative filtering cosine similarity to retrieve the top-\(k\) clinically similar historical patients and their verified discharge diagnoses (peer-informed personalization). These dual signals are assembled into a structured background and input template to create a coarse-grained personalized prompt \(s_0\). Next, an MDP-based policy network evaluates the prompt at the discrete token level using BERT embeddings augmented with patient representations, iteratively performing \(n\) word-deletion steps to produce a refined prompt \(P\). The resulting discrete hard prompt is passed directly to a black-box proprietary LLM \(\mathrm{LLM}_r\) to produce personalized guidance \(\hat{r}\), with policy parameters updated via REINFORCE using the BERTScore improvement against the patient's actual physician recommendations as the reward signal.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Longitudinal Patient Examination Records<br/>(35 continuous clinical indicators)"] --> B["Self-Informed Longitudinal Predictor<br/>(Fine-tuned LLaVA-1.5-7B predicts condition)"]
    A --> C["Peer-Informed Case Retrieval<br/>(MLP encoder + cosine similarity Top-k)"]
    B --> D["Coarse-Grained Prompt Construction<br/>(Combines context + self-prediction + peer diagnoses)"]
    C --> D
    D --> E["RL-Driven Word-Level Prompt Refinement<br/>(BERT encoder + MLP policy network n-step deletion)"]
    E --> F["Proprietary LLM Black-Box Inference<br/>(GPT-4 / Gemini generates customized guidance)"]
    F --> G["Physician Reference Reward & Policy Update<br/>(Relative BERTScore gain + REINFORCE optimization)"]

Key Designs

1. Dual-Source Self- and Peer-Informed Information Extraction: Resolving Longitudinal Heterogeneity and Clinical Experience Sharing To handle irregular visit counts \(N_{x_i}\) across different patients, PPL introduces complementary extraction mechanisms. For self-informed personalization, multi-visit laboratory examination sequences are cast as text instructions to fine-tune LLaVA-1.5-7B via LoRA as the dedicated predictor \(\mathrm{LLM}_p\), yielding the target patient's projected condition \(\hat{Y}_i = \mathrm{LLM}_p(x_i)\). Empirical evaluations verify that this instruction-tuned predictor significantly outperforms traditional time-series baselines (LSTM, CNN, ROCKET, Transformer) on padded sequence classification. For peer-informed personalization, a 2-layer MLP encoder maps the concatenated, zero-padded records into a \(d=128\) dimensional latent representation \(\hat{x}_i \in \mathbb{R}^{1 \times d}\). The model computes the cosine similarity against all historical patients \(x_j\) in the training archive: $$ w_{ij} = \frac{\hat{x}_i \cdot \hat{x}_j}{|\hat{x}_i|_2 |\hat{x}_j|_2} $$ The ground-truth discharge diagnoses and corresponding similarity scores of the top-\(k\) matching patients are extracted and integrated alongside the target patient's predicted diagnosis into a coarse-grained personalized prompt \(s_0\).

2. Patient-Conditioned Word-Level MDP Prompt Refinement: Overcoming Rigid Templates and Semantic Redundancy Standard hard prompt optimization typically learns a static prompt prefix shared across an entire task, which cannot adapt to individual patients with unique clinical trajectories. PPL formulates prompt optimization as an \(n\)-step Markov Decision Process \(S = \{s_0, s_1, \dots, s_{n-1}\}\). Because the length of initial prompts \(N_s\) varies per patient, PPL executes word-level deletions rather than sentence-level swaps. At optimization step \(i\) with state \(s_i\), BERT extracts contextual embeddings \(E = \mathrm{BERT}(s_i) = (e_1, e_2, \dots, e_{N_s})\) and calculates the mean prompt semantics \(\bar{E} = \frac{1}{N_s}\sum_{j=1}^{N_s} e_j\). To ensure modification decisions reflect patient-specific clinical characteristics, each word vector \(e_j\), the global prompt representation \(\bar{E}\), and the patient embedding \(\hat{x}\) are concatenated and fed into a 3-layer MLP policy network \(f(\cdot)\) to generate the word deletion probability distribution: $$ \pi(e_j \mid s_i) = \frac{\exp(f(e_j \parallel \bar{E} \parallel \hat{x}))}{\sum_{k=1}^{N_s} \exp(f(e_k \parallel \bar{E} \parallel \hat{x}))} $$ Sampling and removing one word per step across \(n\) iterations yields the final personalized hard prompt \(P = s_{n-1}\), dynamically pruning misleading or redundant tokens according to the patient's physiological embedding.

3. Physician-Aligned Relative BERTScore Reward and Black-Box Policy Gradient: Non-Invasive Adaptation for Proprietary LLMs To ensure generated recommendations conform to professional medical communication standards, the refined text prompt \(P\) is fed directly to the downstream proprietary model \(\mathrm{LLM}_r\) to obtain guidance \(\hat{r} = \mathrm{LLM}_r(P)\). To neutralize baseline quality differences across heterogeneous patient prompts, PPL defines a relative reward based on the BERTScore improvement over the initial unrefined response \(\hat{r}_0 = \mathrm{LLM}_r(s_0)\) against physician reference guidance \(r\): $$ \mathrm{Reward} = \mathrm{BS}(\hat{r}, r) - \mathrm{BS}(\hat{r}0, r) $$ The policy network is trained using the REINFORCE objective: $$ \mathcal{L} = -\sum $$ Because the reward computation operates purely through textual prompt inputs and output completions, the optimization requires no gradients, activation caching, or architectural access to }^{n-1} \log(\pi(e_{j_i} \mid s_i)) \cdot \mathrm{Reward\(\mathrm{LLM}_r\), enabling frictionless plug-and-play integration with proprietary commercial models.

Loss & Training

PPL decoupling training into two phases: first, the predictor \(\mathrm{LLM}_p\) is supervised fine-tuned on the training set using LoRA. Second, the MLP patient encoder (2 layers, output dimension 128) and the MLP policy network (3 layers, hidden dimension 256, dropout 0.4, output dimension 1) are optimized jointly using the Adam optimizer with a learning rate of 0.005. Default hyperparameters are set to \(k=10\) retrieved similar patients and \(n=10\) prompt refinement steps. During test-time inference, prompts are constructed using the fixed training set index and processed through \(n\) deterministic policy steps before querying the downstream LLM.

Key Experimental Results

Main Results

Experiments were conducted on a real-world obstetrics and gynecology dataset collected across multiple hospitals in Wuhan between 2020 and 2022, encompassing 38,817 longitudinal records across 35 clinical indicators for 2,373 pregnant patients, accompanied by authentic physician discharge recommendations across 12 diagnostic categories. PPL is evaluated across five proprietary LLMs (Gemini 1.5 Pro, GLM-4, GLM-4-plus, GPT-3.5-turbo, GPT-4) and compared against four lightweight open-source models supervised fine-tuned on the full training split (Llama-3-8B, GLM-4-9B, Qwen-2-7B, LLaVA-1.5-7B).

Model / Framework BLEU-4 ROUGE-1 ROUGE-2 ROUGE-L BS-Precision BS-Recall BS-F1
Gemini 1.5 Pro (Direct) 6.49 27.57 6.19 15.46 62.79 65.15 63.94
Gemini 1.5 Pro + PPL 10.65 37.01 13.67 22.39 69.54 70.75 70.13
GLM-4 (Direct) 8.39 30.93 9.95 19.46 63.06 64.95 63.99
GLM-4 + PPL 9.42 34.67 11.57 21.31 66.80 69.51 68.12
GLM-4-plus (Direct) 7.79 33.15 10.48 15.45 59.43 63.26 61.26
GLM-4-plus + PPL 9.04 36.22 11.96 18.47 65.20 69.58 67.31
GPT-3.5-turbo (Direct) 8.73 29.48 9.69 20.08 65.56 64.90 65.19
GPT-3.5-turbo + PPL 9.39 32.41 10.49 22.84 69.25 68.59 68.89
GPT-4 (Direct) 7.76 28.62 9.38 15.49 61.03 64.59 62.75
GPT-4 + PPL (Ours) 10.88 39.03 14.30 22.69 69.80 71.21 70.50
Fine-tuned Llama-3-8B 9.56 38.44 14.12 23.52 65.61 65.93 65.77
Fine-tuned GLM-4-9B 8.79 36.40 12.02 19.72 64.37 64.29 64.33
Fine-tuned Qwen-2-7B 8.03 36.64 12.18 19.78 64.38 64.22 64.30
Fine-tuned LLaVA-1.5-7B 7.45 33.26 13.59 16.61 61.30 63.29 62.28

Ablation Study

Ablation experiments conducted on the GPT-4 backbone evaluate the individual contributions of Self-informed Personalization (SP), Peer-informed Personalization (PP), and Prompt Refinement (PR).

Config BLEU-4 ROUGE-1 ROUGE-2 ROUGE-L BS-Precision BS-Recall BS-F1 Note
PPL Full Model 10.88 39.03 14.30 22.69 69.80 71.21 70.50 Full model achieves peak performance across metrics
w/o Self-informed (SP) 8.42 34.01 10.85 18.39 61.10 65.01 62.99 Removing condition prediction causes substantial degradation
w/o Peer-informed (PP) 9.29 34.54 11.51 21.38 65.92 68.61 67.24 Removing peer diagnoses yields moderate but consistent drop
w/o Prompt Refinement (PR) 7.91 33.07 10.39 17.68 63.62 66.09 64.83 Relying on coarse prompts drops BLEU-4 by 2.97 points

Key Findings

  • Proprietary LLMs with PPL Outperform Supervised Lightweight Models: On clinical semantic alignment (BERTScore F1), GPT-4 + PPL achieves 70.50%, substantially exceeding fine-tuned lightweight models such as Llama-3-8B (65.77%). Fine-tuned lightweight models only retain a slight edge on ROUGE-L (23.52 vs 22.69) due to memorizing fixed phrasing in the training set, while lacking the diagnostic breadth of foundation LLMs.
  • Prompt Refinement (PR) Serves as the Primary Performance Driver: Eliminating RL-based prompt refinement drops BLEU-4 from 10.88 to 7.91 and ROUGE-1 from 39.03 to 33.07, demonstrating that raw concatenated clinical prompts introduce substantial noise that must be filtered through adaptive discrete editing.
  • Peer-Informed Knowledge Enables Fine-Grained Risk Differentiation: In qualitative case analyses involving three patients who shared an identical premature rupture of membranes (PROM) diagnosis, differing peer-retrieved histories (uterine scar, gestational hypertension, or pure PROM) guided PPL to tailor distinct preventive advice for each individual. In contrast, unguided zero-shot baselines misclassified benign laboratory fluctuations as serious pathologies.
  • Decisive Preference in Expert Human Evaluation: In a double-blind clinical study with 25 medical evaluators (including 7 licensed obstetricians) evaluating 100 randomized cases, PPL won 97 out of 100 head-to-head comparisons (86.12% vote share), decisively outperforming GPT-3.5-turbo (2 wins, 8.32%) and GLM-4 (1 win, 5.56%).

Highlights & Insights

  • Bridging Continuous Health Data and Black-Box Prompts via Discrete MDP Optimization: Rather than attempting inaccessible weight updates, PPL establishes a discrete RL editing mechanism that dynamically adapts hard text prompts according to a patient's continuous longitudinal embedding.
  • Relative Semantic Reward Decouples Baseline Prompt Discrepancies: Utilizing the differential BERTScore gain removes baseline prompt variance across diverse clinical conditions, ensuring the policy network optimizes strictly for the marginal alignment gain relative to authentic physician advice.
  • Inherent Privacy Protection with Negligible Inference Overhead: Raw laboratory numbers and longitudinal tables are converted into abstracted diagnostic states and tailored instructions, preventing the transmission of sensitive raw medical records to external APIs while keeping inference latency low.

Limitations & Future Work

  • Single-Domain Clinical Cohort: The current empirical validation is restricted to obstetrics and gynecology records from a single metropolitan region (Wuhan), leaving cross-specialty generalizability (e.g., oncology, cardiology) unexplored.
  • Strictly Deletion-Based Editing Space: Restricting prompt refinement to word deletion simplifies the action space but precludes synonym replacement, medical term paraphrasing, or conditional token insertion, occasionally creating grammatically disjointed prompts.
  • Heuristic Zero-Padding for Missing Clinical Indicators: Longitudinal visits are aligned using zero-padding without explicitly modeling irregularly sampled continuous physiological dynamics.
  • vs Single-Disease Fine-Tuned Medical LLMs (e.g., Benary et al., 2023; Yang et al., 2023): Prior systems fine-tune small open-source models for single conditions (e.g., diabetes or prostate cancer), which limits generalization and clinical reasoning; PPL leverages general-purpose proprietary models across 12 diagnostic categories using a unified prompt optimization pipeline.
  • vs Task-Level Hard Prompt Optimization (e.g., RLPrompt, Deng et al., 2022; GrIPS, Prasad et al., 2023): Existing methods search for a single global prompt template per downstream task; PPL personalizes the discrete prompt per patient by conditioning the policy network directly on patient embeddings.
  • vs Standard Medical Retrieval-Augmented Generation (RAG): Standard RAG naively concatenates retrieved EHR snippets into the prompt, risking context pollution; PPL treats retrieved similar cases as an initial state and refines it with an RL policy to match the downstream model's specific prompt sensitivities.

Rating

  • Novelty: โญโญโญโญโ˜† [Pioneers patient-conditioned discrete hard prompt optimization via reinforcement learning for personalized clinical guidance]
  • Experimental Thoroughness: โญโญโญโญโญ [Evaluates longitudinal real-world clinical records, extensive proprietary and open-source models, comprehensive ablations, and a 25-expert human blind study]
  • Writing Quality: โญโญโญโญโญ [Clear mathematical formulation, cohesive narrative flow, and informative case comparisons]
  • Value: โญโญโญโญโญ [Provides a practical, privacy-conscious engineering paradigm for deploying proprietary LLMs in personalized healthcare delivery]