Skip to content

Do Implicit Personalization and Explicit Styles Conflict? PsPLUG: A Lightweight Plug-in for Balancing Personalization and Style in Customized LLMs

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/RainieLLM/PsPLUG
Area: LLM (Other)
Keywords: Large Language Models, Personalized Generation, Controllable Generation, Preference Optimization, Parameter-Efficient Plug-in

TL;DR

PsPLUG frames personalization as a conditional distributional residual against style-guided reference distributions, injecting a compact 3-token continuous prefix tuned via Bradley-Terry preference optimization to resolve "personalization collapse" and enable continuous inference-time strength control.

Background & Motivation

Large language models deployed in personalized interactive settings are increasingly expected to balance two distinct dimensions of customization: they must mirror individual linguistic habits, personal preferences, and nuanced communicative styles, while simultaneously respecting explicit task directives such as tone or format constraints (e.g., "write in a concise and formal tone" or "reply in a warm and humorous manner"). Existing personalization approaches typically pursue one of three paradigms: retrieval-augmented generation (RAG) that prepends historical user behaviors into the prompt context, per-user parameter-efficient fine-tuning (such as individual LoRA modules), or lightweight continuous soft-prompt plug-ins. However, empirical investigations reveal a critical vulnerability across all conventional personalization pipelines: when strong explicit style instructions are introduced, the generation distribution becomes overwhelmingly dominated by the stylistic directive, severely suppressing user-specific traits and causing what the authors identify as "personalization collapse."

This fundamental failure arises because current methods fail to theoretically distinguish what constitutes an individual's unique personalized signal from the neutral or instruction-following behavior of the base model under identical inputs. Conventional pipelines either optimize standard causal language modeling objectives on user dataโ€”which encourages the model to memorize surface lexical patterns that easily get overwritten by strong prompt directivesโ€”or incur prohibitive computational footprints. In multi-tenant environments, RAG suffers from prompt bloat, high latency, and context-window competition, whereas per-user LoRA adapters incur unsustainable parameter storage and memory swap overhead while tightly overfitting to past expressions, rendering them inflexible when novel conflicting style constraints appear.

To reconcile the tension between explicit task directives and implicit user priors, this work proposes an insightful theoretical shift: modeling personalization as a distributional residual. Rather than attempting to fit an absolute output distribution from user logs, personalization is formalized as the conditional divergence between the user's ground-truth language distribution and the base LLM's neutral style-following distribution under the exact same input. The core idea is to model personalization as a distributional residual against style-conditioned reference policies, injecting a modular tri-part continuous prefix trained via Bradley-Terry preference optimization with style negatives, and employing a scalar multiplier \(\alpha\) at inference time for seamless, zero-overhead adjustment between style compliance and personalization fidelity.

Method

Overall Architecture

The PsPLUG architecture operates over a fully frozen backbone language model \(\pi_{\text{ref}}\) by prepending a compact continuous prefix \(z_{u, x_s}\) consisting of exactly 3 virtual token embeddings into the input representation space. This prefix disentangles the contextual conditioning into three distinct representations: a shared, trainable system instruction vector \(z_{\text{sys}}\), an offline-encoded and projected user vector \(z_u\), and an input-aware query vector \(z_x\). During training, PsPLUG isolates the personalization residual by constructing preference pairs comparing user-authored target outputs against unpersonalized style-only generations produced by the frozen base model under the Bradley-Terry preference objective. At inference time, the backbone parameters remain unchanged, and users can dynamically modulate the influence of their persona relative to system instructions using a continuous scaling coefficient \(\alpha\).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Task Input x + Style Instruction s<br/>x_s = (x, s)"] --> B["Tri-part Continuous Prefix<br/>Concatenate z_sys, z_u, z_x"]
    U["User History H_u"] -->|PAG Profile + Frozen BGE| C["Cached User Vector e_u"]
    C -->|Trainable MLP Projector| B
    B --> D["Prefix-Augmented Frozen LLM<br/>ฯ€_ฯ•(y | x_s, u)"]
    A -->|Frozen Base Forward| E["Style-only Negative Generation<br/>y_s ~ ฯ€_ref(ยท | x_s)"]
    D --> F["Style-Conditioned Preference Residual Optimization<br/>Align (y_u, y_s) via Bradley-Terry Loss"]
    E --> F
    F --> G["Inference-time Strength Control<br/>Scale z_u via Scalar Parameter ฮฑ"]

Key Designs

1. Tri-part Continuous Prefix: Modular Representation for Concurrent Serving

To bypass the substantial latency overhead of RAG and avoid the GPU memory thrashing of switching individual LoRA weights in multi-user deployments, PsPLUG injects a fixed 3-token continuous prefix \(z_{u, x_s} \triangleq [z_{\text{sys}}; z_u; z_x]\) directly at the embedding layer of the frozen LLM backbone. Each component is decoupled: - Shared System Vector \(z_{\text{sys}} \triangleq \gamma \theta_{\text{sys}}\): Employs a shared, trainable vector \(\theta_{\text{sys}} \in \mathbb{R}^d\) scaled by \(\gamma\) (matching the backbone input embedding norm) to establish an invariant instruction anchor across all users; - User Vector \(z_u \triangleq \gamma f_{\text{pag}}^\phi(e_u)\): User history \(H_u\) is compressed offline into a concise natural language Profile-Augmented Generation (PAG) descriptor \(p_u\) using a frozen summarizer, then encoded into an \(\ell_2\)-normalized embedding \(e_u = E(p_u)\) via a frozen sentence encoder (BGE-base-en-v1.5). During online serving, this static embedding is mapped into the LLM hidden space through a lightweight, trainable two-layer MLP projector \(f_{\text{pag}}^\phi\). Thus, each user requires storing only a single low-dimensional vector \(O(d_e)\); - Query Vector \(z_x \triangleq \gamma f_{\text{qry}}^\phi(h(x_s))\): Captures how user expressions vary across specific requests by pooling the token embeddings of the current style-augmented prompt \(h(x_s) = \text{Pool}(\text{Emb}_{\text{ref}}(x_s))\) and mapping them through a trainable query MLP \(f_{\text{qry}}^\phi\).

2. Style-Conditioned Preference Residual Objective: Mathematical Disentanglement of Persona Signals

Standard fine-tuning with causal language modeling fails when explicit styles are applied because generic style tokens drown out subtle user-specific wording. PsPLUG mathematically isolates the user persona as a residual over a style-conditioned reference distribution. Given a task prompt \(x\) and style directive \(s\) forming \(x_s = (x, s)\), the system generates a style-only negative output using the unadapted frozen reference model: $\(y_s \sim \pi_{\text{ref}}(\cdot \mid x_s)\)$ Since \(y_s\) adheres to style \(s\) but contains zero user nuance, pairing it with the user-authored target \(y_u\) yields a style-conditioned preference pair \((y^+, y^-) \triangleq (y_u, y_s)\). The implicit reward score \(r_\phi(y)\) is formulated as the log-likelihood ratio between the personalized policy and the reference policy: $\(r_\phi(y) \triangleq \log \pi_\phi(y \mid x_s, u) - \log \pi_{\text{ref}}(y \mid x_s)\)$ The plug-in parameters \(\phi\) are then optimized using the Bradley-Terry pairwise objective: $\(\mathcal{L}_{\text{style}}(\phi; x_s, u, y^+, y^-) = -\log \sigma \left( \beta \left( r_\phi(y_u) - r_\phi(y_s) \right) \right)\)$ where \(\sigma(\cdot)\) is the sigmoid function and \(\beta\) is a temperature hyperparameter. Because the reference model already captures the style constraint \(s\), the objective provides positive gradient updates only when the personalized policy deviates to capture idiosyncratic user traits beyond generic style compliance. In the special case without style instructions (\(s = \emptyset\)), \(y_s\) reduces to a neutral baseline \(y_0 \sim \pi_{\text{ref}}(\cdot \mid x)\), seamlessly accommodating standard personalization tasks.

3. Inference-Time Personalization Strength Control: Zero-Cost Continuous Modulation

Real-world applications require varying degrees of trade-off between strict adherence to system-level rules and subjective user expressiveness. Rather than re-training separate models or maintaining multiple adapter sets, PsPLUG introduces an explicit scalar scaling coefficient \(\alpha \ge 0\) applied exclusively to the user vector: $\(z_{u, x_s}^{(\alpha)} = [z_{\text{sys}}; \alpha \cdot z_u; z_x]\)$ The conditioned policy evaluates tokens with the scaled prefix: \(\pi_\phi^{(\alpha)}(y \mid x_s, u) = \pi_{\text{ref}}(y \mid [z_{u, x_s}^{(\alpha)}; \text{Emb}_{\text{ref}}(x_s)])\). When \(\alpha \to 0\), the personalized signal is fully suppressed, allowing the model to default strictly to the neutral style-following behavior. When \(\alpha = 1\), the model operates at its trained equilibrium, and when \(\alpha > 1\), it amplifies personalized habits. This continuous modulation operates purely on a pre-computed vector without modifying LLM weights, executing with \(O(1)\) computational complexity.

Key Experimental Results

Main Results

All experiments are evaluated on the standardized LaMP benchmark using a frozen Qwen3-8B backbone across 8 \(\times\) NVIDIA H100 GPUs. Baselines include: non-personalized zero-shot (Non-pers.), Okapi BM25 retrieval-augmented generation (RAG), profile-augmented generation (PAG), standard continuous plug-in (PPlug), and the theoretical per-user parameter-efficient ceiling (OPPU). Below are the primary results in the standard setup without explicit style constraints:

Task / Benchmark Metric Non-pers. RAG PAG PPlug OPPU PsPLUG (Ours)
LaMP-1: Citation Identification ACC โ†‘ 0.518 0.441 0.562 0.563 0.556 0.584
LaMP-1: Citation Identification Macro F1 โ†‘ 0.448 0.397 0.481 0.493 0.556 0.589
LaMP-2M: Movie Tagging Macro F1 โ†‘ 0.254 0.283 0.311 0.307 0.314 0.334
LaMP-2M: Movie Tagging ACC โ†‘ 0.375 0.381 0.393 0.382 0.425 0.392
LaMP-3: Product Rating Prediction MAE โ†“ 0.516 0.615 0.435 0.339 0.347 0.332
LaMP-3: Product Rating Prediction RMSE โ†“ 0.805 0.981 0.714 0.583 0.613 0.464
LaMP-4: News Headline Generation ROUGE-1 โ†‘ 0.146 0.165 0.164 0.158 0.152 0.167
LaMP-4: News Headline Generation ROUGE-L โ†‘ 0.128 0.144 0.146 0.138 0.128 0.148
LaMP-4: News Headline Generation METEOR โ†‘ 0.107 0.108 0.098 0.092 0.079 0.109
LaMP-5: Scholarly Title Generation ROUGE-1 โ†‘ 0.426 0.459 0.415 0.462 0.426 0.463
LaMP-5: Scholarly Title Generation ROUGE-L โ†‘ 0.342 0.387 0.352 0.386 0.342 0.391
LaMP-7: Tweet Paraphrasing ROUGE-1 โ†‘ 0.497 0.500 0.507 0.502 0.498 0.523
LaMP-7: Tweet Paraphrasing ROUGE-L โ†‘ 0.440 0.441 0.435 0.443 0.422 0.457

Style-Constrained Personalization Results

To assess resilience against personalization collapse under explicit style directives, models are evaluated across four distinct predefined styles: Warm, Critical, Concise, and Elaborative. Below are ROUGE-1 (R-1) and ROUGE-L (R-L) performance metrics on generation tasks:

Task / Style Instruction Metric Non-pers. RAG PAG PPlug PsPLUG (Ours) Relative Margin vs. Best Baseline
LaMP-4: News Headline - Warm R-1 / R-L 0.120 / 0.102 0.114 / 0.099 0.131 / 0.114 0.138 / 0.119 0.143 / 0.124 +0.005 / +0.005
LaMP-4: News Headline - Critical R-1 / R-L 0.129 / 0.112 0.138 / 0.120 0.144 / 0.126 0.150 / 0.132 0.159 / 0.137 +0.009 / +0.005
LaMP-4: News Headline - Concise R-1 / R-L 0.140 / 0.123 0.167 / 0.150 0.139 / 0.121 0.143 / 0.126 0.149 / 0.129 -0.018 / -0.021
LaMP-4: News Headline - Elaborative R-1 / R-L 0.134 / 0.113 0.147 / 0.137 0.135 / 0.121 0.147 / 0.126 0.139 / 0.128 -0.008 / +0.002
LaMP-5: Scholarly Title - Warm R-1 / R-L 0.248 / 0.196 0.259 / 0.202 0.365 / 0.295 0.470 / 0.402 0.476 / 0.403 +0.006 / +0.001
LaMP-5: Scholarly Title - Critical R-1 / R-L 0.304 / 0.249 0.312 / 0.255 0.302 / 0.223 0.422 / 0.352 0.441 / 0.368 +0.019 / +0.016
LaMP-5: Scholarly Title - Concise R-1 / R-L 0.441 / 0.360 0.464 / 0.391 0.458 / 0.356 0.471 / 0.395 0.465 / 0.376 -0.006 / -0.019
LaMP-5: Scholarly Title - Elaborative R-1 / R-L 0.206 / 0.167 0.217 / 0.178 0.285 / 0.245 0.322 / 0.312 0.346 / 0.385 +0.024 / +0.073
LaMP-7: Tweet Paraphrase - Warm R-1 / R-L 0.426 / 0.368 0.455 / 0.400 0.448 / 0.392 0.454 / 0.396 0.457 / 0.402 +0.003 / +0.002
LaMP-7: Tweet Paraphrase - Critical R-1 / R-L 0.472 / 0.418 0.476 / 0.431 0.466 / 0.417 0.473 / 0.410 0.478 / 0.414 +0.005 / +0.004
LaMP-7: Tweet Paraphrase - Concise R-1 / R-L 0.475 / 0.421 0.472 / 0.434 0.451 / 0.401 0.487 / 0.423 0.487 / 0.425 0.000 / +0.002
LaMP-7: Tweet Paraphrase - Elaborative R-1 / R-L 0.461 / 0.403 0.467 / 0.405 0.455 / 0.401 0.460 / 0.402 0.470 / 0.415 +0.010 / +0.010

In system efficiency audits, RAG scales storage linearly with history length (\(O(|P_u| \cdot d_e)\)) and increases inference latency, while per-user PEFT/LoRA demands \(O(rHL)\) storage per user alongside substantial adapter-swapping overhead. In contrast, PsPLUG requires only static user vector storage (\(O(d_e)\)) and maintains constant inference latency (\(O(H(d_e + H))\)), completely decoupled from user history size \(|P_u|\).

Key Findings

  • Asymmetric Interference Across Style Categories: Affect- and tone-oriented styles (Warm, Critical) drastically disrupt baseline personalization, causing steep drops in lexical overlap metrics. In contrast, structural constraints (Concise) harmonize more naturally with underlying task targets, even lifting unpersonalized baseline scores.
  • Robust Resistance to Personalization Collapse: PsPLUG achieves top or second-best performance across more than 80% of style-task-metric configurations. Under the difficult Elaborative style on LaMP-5, PsPLUG outperforms the strongest baseline by 7.3 points in ROUGE-L.
  • Continuous Controllability via \(\alpha\): Varying \(\alpha\) between 0.2 and 5.0 demonstrates a smooth, predictable trade-off. Personalization scores increase monotonically with \(\alpha\) while style adherence remains stable across moderate ranges, although excessive scaling (e.g., \(\alpha = 5.0\) in Elaborative title generation) can cause verbose drift.
  • Cross-Architecture Scalability: Experiments spanning Qwen3 4B, 8B, and 32B demonstrate that the residual prefix design reliably generalizes across backbone model scales without degradation.

Highlights & Insights

  • Distributional Residual Formulation: Reconceptualizes personalization away from crude absolute probability fitting. By optimizing the log-likelihood ratio against a style-conditioned reference output, preference learning isolates the pure user-specific delta.
  • Single-Scalar Dynamic Balancing Interface: Provides an intuitive, runtime-tunable dial (\(\alpha\)) that scales only the pre-computed user vector, giving downstream applications fine-grained control over persona intensity with zero parameter reloading.
  • High Multi-Tenant Efficiency: Compresses rich user histories into a static embedding and prepends a fixed 3-token prefix, eliminating adapter-swapping memory bottlenecks and enabling scalable multi-user deployment.

Limitations & Future Work

  • Coarse Predefined Style Categories: The empirical setup focuses on four predefined styles (Warm, Critical, Concise, Elaborative). Real-world interactive dialogues feature composite, subtle, and open-ended stylistic instructions.
  • Lexical Overlap Metric Distortion: Standard LaMP metrics (ROUGE-1, ROUGE-L) inherently penalize legitimate stylistic reformulations that preserve user preferences while modifying sentence syntax, necessitating more robust disentangled evaluation benchmarks.
  • Scope of Model Backbones and Contexts: Evaluations primarily centered on English benchmarks with Qwen3 backbones; extending validation to multilingual and extreme long-context settings remains open.
  • vs. Retrieval-Augmented Generation (e.g., LaMP, LongLaMP): RAG concatenates historical texts into the prompt, resulting in linear latency scaling and vulnerability to style-instruction override. PsPLUG compresses history into a fixed 3-token vector that maintains constant inference latency and high persona fidelity under style constraints.
  • vs. Per-User Fine-Tuning (e.g., OPPU, Per-User LoRA): Per-user adapters rigidly encode historical style directly into parameter weights, struggling to honor conflicting real-time style directives and incurring high multi-tenant storage costs. PsPLUG keeps the backbone frozen, updating only shared modular projectors.
  • vs. Conventional Soft-Prompt Plug-ins (e.g., PPlug, PAG): Earlier plug-in methods train using standard causal language modeling objectives, failing to disentangle user traits from dominant style signals. PsPLUG introduces style-conditioned preference optimization to isolate the personalization residual.

Rating

  • Novelty: โญโญโญโญโญ Formulates personalization as a distributional residual over style constraints and introduces an elegant 3-token prefix architecture.
  • Experimental Thoroughness: โญโญโญโญ Evaluates multiple LaMP tasks, four style configurations, model scalability across parameter sizes, and includes human-LLM correlation audits.
  • Writing Quality: โญโญโญโญโญ Exceptionally clear formulation, well-grounded motivation, and rigorous empirical analysis.
  • Value: โญโญโญโญโญ Delivers an efficient, scalable, and controllable blueprint for deploying personalized LLM agents in production environments with strict instruction boundaries.