title: >- [Paper Note] Learning Personalized Prompts for Healthcare Guidance description: >- [EMNLP 2026][Medical LLM][Personalized Prompt Learning] Proposes PPL, a framework that constructs coarse-grained prompts from longitudinal health records and clinically similar cases, and refines them via reinforcement learning with physician recommendations as reward signals, enabling proprietary LLMs to generate highly tailored healthcare guidance without modifying model weights. tags: - EMNLP 2026 - Medical LLM - Medical NLP - Personalized Prompt Learning - Reinforcement Learning - Healthcare Guidance date: 2026-11-03 content_hash: 58850a5f9797bb52
Learning Personalized Prompts for Healthcare Guidance¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/CGCL-codes/PPL
Area: Medical LLM
Keywords: medical LLM, personalized prompt learning, reinforcement learning, collaborative filtering, hard prompt optimization
TL;DR¶
Proposes PPL (Personalized Prompt Learning), an end-to-end framework that constructs coarse prompts from a patient's historical medical checkups and peer-informed similar cases, and refines them into discrete patient-tailored hard prompts via reinforcement learning with physician recommendations as reward signals, delivering superior personalized healthcare advice with proprietary LLMs.
Background & Motivation¶
With the rapid integration of large language models across clinical and everyday healthcare, hospitals and patients increasingly seek LLM-based systems capable of interpreting longitudinal personal health records and providing individualized health guidance. However, prevailing healthcare LLM approaches primarily rely on generic medical domain knowledge or narrow parameter fine-tuning tailored to a single specific disease (such as diabetes or prostate cancer). In these existing systems, personalization arises almost entirely from surface-level variations in patient query inputs, overlooking personalization at the prompt level and failing to capture the longitudinal complexity and physiological variability of real-world patients across diverse clinical conditions.
Achieving fine-grained clinical personalization introduces two fundamental technical challenges. First, medical guidance systems must support multiple diverse disease conditions while adapting to individual patient profiles. Real-world patients present with multifaceted conditions, variable-length medical examination trajectories, and co-morbidities; developing standalone lightweight models for each single disease fails to offer holistic care. Second, prompt adaptation must be achieved automatically without manual crafting or direct parameter access. Conventional pipelines rely on static templates populated with sparse metrics, which cannot adapt to evolving patient records, contextual nuances, or variations across underlying model architectures—especially for top-tier proprietary models (e.g., GPT-4, Gemini-1.5-Pro) whose model weights remain strictly inaccessible.
To overcome these barriers, the authors shift the personalization mechanism from internal weight adaptation to discrete prompt optimization. The core idea is to propose Personalized Prompt Learning (PPL), which constructs coarse-grained prompts from self-informed historical condition predictions and peer-informed clinically similar cases, and refines them into instance-specific hard prompts via word-level reinforcement learning guided by real-world physician recommendations.
Method¶
Overall Architecture¶
The PPL framework is structured around two primary pillars: Personalized Information Extraction and Personalized Prompt Generation and Refinement. The system takes as input a target patient's variable-length longitudinal physical examination indicators and a historical clinical cohort, and outputs tailored healthcare guidance generated by an off-the-shelf proprietary LLM.
First, to handle variable-length medical visits, a self-informed predictor parameterized by a LoRA-tuned LLaVA-1.5-7B predicts the patient's current disease condition; simultaneously, a two-layer MLP encoder maps standardized physical checkup records into a compact latent space, where collaborative filtering retrieves the top-\(k\) most clinically similar past patients to extract peer-informed diagnostic context. Second, self-informed predictions and peer-informed references are integrated into an initial coarse-grained prompt. Finally, a word-level policy network—conditioned jointly on BERT token representations, global prompt context, and the patient's latent feature—executes \(n\) discrete word-deletion steps under a Markov Decision Process (MDP). The refined prompt is queried against the frozen proprietary LLM, and the policy network is optimized with Adam using policy gradients driven by a relative BERTScore gain against actual physician advice written for the patient.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Patient Longitudinal Health Records<br/>Variable-length multi-visit metrics"] --> B["Self-informed Personalization<br/>LoRA-tuned LLaVA-1.5 condition prediction"]
A --> C["Peer-informed Personalization<br/>MLP encoding & Top-k collaborative filtering"]
B --> D["Coarse-grained Prompt Construction<br/>Integrate self prediction + peer diagnoses"]
C --> D
D --> E["Word-level Prompt Refinement Policy<br/>Conditioned on token, global semantic & patient vector"]
E -->|MDP n-step word deletion editing| F["Fine-grained Personalized Hard Prompt"]
F --> G["Proprietary LLM Backbone<br/>GPT-4 / Gemini-1.5-Pro / GLM-4"]
G --> H["Personalized Healthcare Guidance Output"]
H -->|Relative BERTScore gain vs physician reference| I["RL Reward-driven Policy Gradient<br/>Backpropagation updating policy MLP"]
I -.->|Parameter update| E
Key Designs¶
1. Dual-Source Personalization via Self-Informed and Peer-Informed Extraction: Unifying Longitudinal Trajectories with Cohort Priors
Clinical health records naturally form non-uniform time series with missing dates and varying visit frequencies. Directly feeding raw longitudinal tabular data into general language models induces substantial noise and dimension mismatch. PPL decouples this extraction into two complementary pathways. In the self-informed pathway, longitudinal checkup sequences are formatted as instruction-tuning inputs and modeled with a LoRA-fine-tuned LLaVA-1.5-7B predictor \(\text{LLM}_p\): $\(\hat{Y} = \text{LLM}_p(X)\)$ Empirical analysis demonstrates that the multimodal language model architecture is remarkably resilient to zero-padding artifacts compared to conventional sequential baselines (e.g., LSTM, ROCKET). In the peer-informed pathway, an MLP encoder transforms zero-padded examination indicators into a standardized embedding \(\hat{x}_i \in \mathbb{R}^{1 \times d}\). Collaborative filtering computes cosine similarities between the target patient and historical cases in the training corpus: $\(w_{ij} = \frac{\hat{x}_i \cdot \hat{x}_j}{\|\hat{x}_i\|_2 \cdot \|\hat{x}_j\|_2}\)$ The top-\(k\) nearest peer records and their verified discharge diagnoses are gathered. By combining individualized predictive outcomes with proven historical treatments from peer patients, PPL injects clinical crowd intelligence while honoring the target patient's distinct physiological trends.
2. Patient-Conditioned Discrete Prompt Refinement Policy: Context-Aware Word Editing under MDP
To maintain seamless compatibility with closed-source proprietary LLMs without requiring gradient access, PPL represents prompts as discrete textual hard prompts and formulates optimization as a finite-horizon Markov Decision Process (MDP). The state sequence is denoted as \(S = \{s_0, s_1, \dots, s_{n-1}\}\), initialized with the coarse-grained prompt \(s_0\). While previous discrete prompt optimization methods (such as RLPrompt) determine transition probabilities strictly from pre-trained token embeddings, they produce prompt edits that lack patient-level differentiation.
PPL introduces a tripartite conditioned policy network. At state \(s_i\), prompt tokens are embedded via BERT to obtain \(E = (e_1, e_2, \dots, e_{N_s})\) along with the global semantic mean embedding \(\bar{E} = \frac{1}{N_s}\sum_{j=1}^{N_s} e_j\). The policy network computes token modification (deletion) probabilities by concatenating the token embedding \(e_j\), the global contextual embedding \(\bar{E}\), and the patient representation \(\hat{x}\): $\(\pi(e_j|s) = \frac{\exp(f(e_j \,\|\, \bar{E} \,\|\, \hat{x}))}{\sum_{m=1}^{N_s} \exp(f(e_m \,\|\, \bar{E} \,\|\, \hat{x}))}\)$ where \(f(\cdot)\) is a 3-layer MLP. Over \(n\) sequential editing steps, exactly one token is deleted per step. This patient-conditioned formulation forces the policy network to discard uninformative phrases or misleading distractors while safeguarding clinically salient terminology specific to that individual patient.
3. Physician-Aligned Relative Gain Reward Function: Black-Box Adaptation without Model Fine-Tuning
Supervised token-level losses cannot easily steer discrete prompts toward clinically valid prose. PPL directly utilizes authentic personalized discharge guidance written by physicians (\(r\)) as the reference target.
Because proprietary base models (\(\text{LLM}_r\)) already produce fluent prose from raw prompts, using raw similarity as a reward would reward superficial fluency rather than genuine personalization gains. PPL implements a differential BERTScore reward: $\(\text{Reward} = \text{BS}(\hat{r}, r) - \text{BS}(\hat{r}_0, r)\)$ where \(\text{BS}(\cdot)\) denotes BERTScore, \(\hat{r} = \text{LLM}_r(P)\) is the output generated from the final refined prompt \(P = s_{n-1}\), and \(\hat{r}_0 = \text{LLM}_r(s_0)\) is the baseline response from the unrefined initial prompt. This formulation isolates the marginal value added by prompt editing, explicitly penalizing semantic degradation and rewarding precise alignment with genuine clinical recommendations. The policy loss follows REINFORCE: $\(L = -\sum_{i=0}^{n-1} \log(\pi(e_{j_i}|s_i)) \cdot \text{Reward}\)$ and optimizes the lightweight policy network and encoder via Adam without propagating gradients into the backbone LLM.
Loss & Training¶
Training proceeds sequentially: first, the disease condition predictor \(\text{LLM}_p\) (LLaVA-1.5-7B) is trained via LoRA on multi-visit records across 12 target clinical labels; subsequently, \(\text{LLM}_p\) and the target generator \(\text{LLM}_r\) are kept frozen while the 2-layer MLP encoder (output dim 128, dropout 0.4) and 3-layer MLP policy network (hidden dim 256, dropout 0.4) are trained jointly. Hyper-parameters are configured with learning rate 0.005, neighborhood size \(k=10\), and editing horizon \(n=10\). The entire pipeline trains on 38,817 longitudinal records from 2,373 patients.
Key Experimental Results¶
Main Results¶
On the real-world obstetrics and gynecology benchmark, PPL was evaluated across multiple proprietary models (Gemini-1.5-pro, GLM-4, GLM-4-plus, GPT-3.5-turbo, GPT-4) against their unoptimized baselines, and compared with lightweight open-source models fine-tuned directly on physician guidance. Metrics include lexical overlap (BLEU-4, ROUGE-L, ROUGE-1, ROUGE-2) and semantic embedding alignment (BERTScore Precision, Recall, F1).
| Model Category | Backbone Model | Optimization Setting | BLEU-4 | ROUGE-L | ROUGE-1 | ROUGE-2 | BERTScore F1 |
|---|---|---|---|---|---|---|---|
| Proprietary LLM | Gemini-1.5-pro | Before PPL | 6.49 | 15.46 | 27.57 | 6.19 | 63.94 |
| Proprietary LLM | Gemini-1.5-pro | After PPL | 10.65 | 22.39 | 37.01 | 13.67 | 70.13 |
| Proprietary LLM | GLM-4 | Before PPL | 8.39 | 19.46 | 30.93 | 9.95 | 63.99 |
| Proprietary LLM | GLM-4 | After PPL | 9.42 | 21.31 | 34.67 | 11.57 | 68.12 |
| Proprietary LLM | GLM-4-plus | Before PPL | 7.79 | 15.45 | 33.15 | 10.48 | 61.26 |
| Proprietary LLM | GLM-4-plus | After PPL | 9.04 | 18.47 | 36.22 | 11.96 | 67.31 |
| Proprietary LLM | GPT-3.5-turbo | Before PPL | 8.73 | 20.08 | 29.48 | 9.69 | 65.19 |
| Proprietary LLM | GPT-3.5-turbo | After PPL | 9.39 | 22.84 | 32.41 | 10.49 | 68.89 |
| Proprietary LLM | GPT-4 | Before PPL | 7.76 | 15.49 | 28.62 | 9.38 | 62.75 |
| Proprietary LLM | GPT-4 | After PPL | 10.88 | 22.69 | 39.03 | 14.30 | 70.50 |
| Fine-tuned Open LLM | LLaMA-3-8B | Supervised Fine-Tuning | 9.56 | 23.52 | 38.44 | 14.12 | 65.77 |
| Fine-tuned Open LLM | GLM-4-9B | Supervised Fine-Tuning | 8.79 | 19.72 | 36.40 | 12.02 | 64.33 |
| Fine-tuned Open LLM | Qwen-2-7B | Supervised Fine-Tuning | 8.03 | 19.78 | 36.64 | 12.18 | 64.30 |
| Fine-tuned Open LLM | LLaVA-1.5-7B | Supervised Fine-Tuning | 7.45 | 16.61 | 33.26 | 13.59 | 62.28 |
Ablation Study¶
Ablations on GPT-4 systematically decouple Self-informed Personalization (SP), Peer-informed Personalization (PP), and Prompt Refinement (PR).
| Variant ID | SP | PP | PR | BLEU-4 | ROUGE-L | ROUGE-1 | ROUGE-2 | BERTScore F1 |
|---|---|---|---|---|---|---|---|---|
| 1 | ✗ | ✓ | ✓ | 8.42 | 18.39 | 34.01 | 10.85 | 62.99 |
| 2 | ✓ | ✗ | ✓ | 9.29 | 21.38 | 34.54 | 11.51 | 67.24 |
| 3 | ✓ | ✓ | ✗ | 7.91 | 17.68 | 33.07 | 10.39 | 64.83 |
| Full PPL | ✓ | ✓ | ✓ | 10.88 | 22.69 | 39.03 | 14.30 | 70.50 |
In expert clinical evaluations conducted across 100 randomly sampled test cases by 25 medical evaluators (including 7 licensed obstetricians, 3 PhD candidates, and 15 medical master's students), PPL secured an 86.12% vote share and won 97 comparisons, compared to 1 win for GLM-4 (5.56% vote share) and 2 wins for GPT-3.5-turbo (8.32% vote share).
Key Findings¶
- Prompt learning outperforms small-model fine-tuning: Equipping GPT-4 with PPL achieves a BERTScore F1 of 70.50, substantially surpassing the fine-tuned LLaMA-3-8B (65.77). While fine-tuned lightweight models memorize stock clinical phrases and slightly lead on ROUGE-L (23.52 vs 22.69), PPL produces markedly better clinical semantics and relevance by unlocking the broad reasoning capabilities of proprietary models.
- Prompt refinement is the most critical component: Removing the RL refinement policy (Variant 3) causes BLEU-4 to drop sharply from 10.88 to 7.91 and BERTScore F1 from 70.50 to 64.83, demonstrating that naive coarse concatenation leaves excessive prompt bloat that obscures patient-specific directives.
- Sensitivity peaks at balanced hyper-parameters: Varying \(k\) and \(n\) reveals an inverted U-shape. Values of \(k > 10\) incorporate lower-similarity patients that inject conflicting clinical priors, while \(n > 10\) causes excessive token deletion that fragments prompt syntax and harms generator comprehension.
Highlights & Insights¶
- Non-invasive black-box personalization: Successfully decouples personalized clinical steering from internal weight updates, solving a major bottleneck in deploying closed-source commercial foundation models within healthcare ecosystems.
- Surprising efficacy of multimodal LLMs on tabular time series: Repurposing a visual instruction-tuned model (LLaVA-1.5-7B) for clinical time-series diagnosis achieved a Macro-F1 of 74.13, outperforming specialized sequential models (Transformer 24.58, LSTM 5.51) under irregular zero-padded multi-visit data.
- High transferability across domains: The tripartite design—self-condition predictor, peer-informed retrieval, and RL-driven discrete prompt pruning—transfers readily to other high-stakes personalization domains such as customized financial planning, adaptive education, and legal case advisory.
Limitations & Future Work¶
- Single-specialty cohort: Because publicly accessible healthcare datasets rarely provide paired longitudinal physiological test panels alongside free-form physician guidance, validation is currently restricted to obstetrics and gynecology data from Wuhan hospitals. Evaluation across multi-center oncology or cardiovascular cohorts remains necessary.
- Deletion-only editing space: Constraining edits strictly to token deletions ensures search stability and tractability under RL, but precludes insertion, token substitution, or synonym paraphrasing, occasionally yielding prompts that appear disjointed to human readers despite being effective for LLMs.
Related Work & Insights¶
- vs Single-Disease Fine-Tuned Models (e.g., Yang et al., 2023; Benary et al., 2023): Prior medical LLMs fine-tune small models for specific pathologies, achieving personalization solely through input variable shifts. PPL preserves full multi-disease coverage and harnesses stronger proprietary foundation models.
- vs Task-Level Hard Prompt Optimization (e.g., RLPrompt, Deng et al., 2022; GrIPS, Prasad et al., 2023): Standard discrete prompt optimization seeks a universal static prompt prefix for an entire downstream task. PPL formulates instance-level prompt learning, tailoring dynamic word-level policies to each patient's individual physiological state.
Rating¶
- Novelty: ⭐⭐⭐⭐ [Innovative fusion of discrete hard prompt reinforcement learning with peer collaborative filtering in healthcare]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive validation on 38k clinical records, multiple proprietary LLMs, and rigorous double-blind expert physician evaluation]
- Writing Quality: ⭐⭐⭐⭐⭐ [Clear mathematical formulation, cohesive narrative flow, and well-structured figures]
- Value: ⭐⭐⭐⭐⭐ [Presents a practical, privacy-conscious blueprint for deploying proprietary foundation models in clinical advisory roles]