Skip to content

Do Implicit Personalization and Explicit Styles Conflict? PsPLUG: A Lightweight Plug-in for Balancing Personalization and Style in Customized LLMs

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/RainieLLM/PsPLUG
Area: LLM Alignment
Keywords: Personalized LLM, Explicit Style Control, Personalization Collapse, Distributional Residual, Pluggable Prefix

TL;DR

To tackle "personalization collapse"โ€”where explicit style instructions override and suppress implicit user habitsโ€”PsPLUG models personalization as a distributional residual via a lightweight plug-in prefix and style-conditioned preference optimization, achieving a controllable trade-off between style adherence and personalization strength under a frozen backbone.

Background & Motivation

In interactive and conversational applications, large language models are expected not only to provide factually accurate outputs but also to reflect individual user preferences, idiosyncratic phrasing, and communicative styles. Existing personalization paradigms broadly fall into three categories: retrieval-augmented generation (RAG) that prepends historical user behaviors to the context window, parameter-efficient fine-tuning (PEFT) that trains per-user adapters (e.g., per-user LoRA), and continuous soft prompts or plug-in modules. Simultaneously, modern NLP systems are routinely steered by explicit task-level or system-level instructions demanding specific stylistic or tonal profiles, such as responding in a warm humorous tone, adopting a critical perspective, or adhering to strict formal conciseness.

However, when explicit style directives meet implicit user personalization, existing systems experience severe persona degradationโ€”a failure mode identified as "personalization collapse." System-level style prompts tend to dominate the generation space, overriding subtle personal traits and reducing customized outputs to generic style-following templates. Conversely, per-user fine-tuning approaches overfit historical writing styles, rendering models unresponsive to conflicting style prompts at inference time, while retrieval-based context padding incurs substantial latency and memory overhead without resolving the underlying entanglement between instructions and persona priors.

To address this fundamental tension, the authors propose reframing personalization from directly modeling absolute generation likelihoods to learning a "distributional residual"โ€”the distinct divergence between a user's authentic linguistic distribution and a neutral, population-level base model distribution under the exact same task and style conditions. Core idea: model personalization as a style-conditioned distributional residual injected via a compact three-part prefix into a frozen backbone, using style-conditioned preference optimization and a continuous inference-time scaling coefficient to achieve a modular, fine-grained balance between explicit style adherence and implicit user personalization without per-user fine-tuning.

Method

Overall Architecture

PsPLUG constructs a personalized policy \(\pi_\phi(y \mid x_s, u)\) that reproduces user-specific behavioral nuances while strictly respecting explicit style instructions \(s\). The base LLM remains completely frozen throughout training and inference. All trainable parameters are isolated within a lightweight plug-in module. The framework prepends a compact continuous prefix \(z_{u, x_s}\) of three tokens to the input embedding sequence: a global system instruction vector \(z_{\text{sys}}\), a user profile projection vector \(z_u\), and a task-and-style query representation \(z_x\). To enable fine-grained, dynamic control at inference time, a continuous scaling factor \(\alpha\) modulates the user vector, allowing seamless adjustment of personalization strength without altering prompt understanding or requiring model parameter reloads.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    In["Input prompt x and explicit style instruction s"] --> InAware["Input-Aware Query Encoder<br/>Token pooling and trainable query MLP"]
    Hist["User historical interactions Hu"] --> Prof["Profile-Augmented Generation & Offline Encoding<br/>PAG textual descriptor and sentence encoder"]
    Prof --> Proj["Personalization Embedding Projector<br/>Project cached profile eu to user vector zu"]
    SysEmb["Trainable System Instruction Embedding<br/>Shared instruction anchor across users zsys"] --> PrefixAssemble["Compact Prefix Assembly<br/>Concatenate zsys, zu, and zx with scaling factor ฮฑ"]
    InAware --> PrefixAssemble
    Proj --> PrefixAssemble
    PrefixAssemble --> PolicyOpt["Style-Conditioned Preference Optimization<br/>Construct style baseline and residual Bradley-Terry loss"]
    PolicyOpt --> Out["Balanced personalized and styled output"]

Key Designs

1. Modular Three-Part Prefix Representation: Decoupled Feature Injection without Per-User Retraining

To eliminate the massive computational and storage overhead of per-user fine-tuning while decoupling distinct conditioning signals, PsPLUG decomposes the soft prefix into three specialized vector components: $\(z_{u, x_s} \triangleq [z_{\text{sys}}; z_u; z_x]\)$ Here, \(z_{\text{sys}} \triangleq \gamma \theta_{\text{sys}}\) is a shared continuous embedding scaled by \(\gamma\) to match the backbone embedding norm, acting as a global instruction anchor that primes the frozen LLM to process system-level directives. The user-specific vector \(z_u \triangleq \gamma f_{\text{pag}}^\phi(e_u)\) captures idiosyncratic persona traits: Profile-Augmented Generation (PAG) first summarizes raw user history \(H_u\) into a fixed-length text descriptor \(p_u\), which is encoded offline by a frozen sentence encoder \(E(\cdot)\) (BGE-base) into an \(\ell_2\)-normalized vector \(e_u\), subsequently mapped into the LLM hidden space via a trainable MLP \(f_{\text{pag}}^\phi\). Finally, the input-aware vector \(z_x \triangleq \gamma f_{\text{qry}}^\phi(h(x_s))\) pools token embeddings of the task and style prompt \(x_s\) through a trainable MLP, grounding the prefix in current context. By keeping the prefix length strictly fixed to 3 tokens, inference overhead remains constant \(O(d_e)\) regardless of history size \(|P_u|\), enabling instantaneous user switching during multi-tenant serving.

2. Style-Conditioned Preference Alignment: Isolating Personalization Residual via Pairwise Bradley-Terry Loss

Standard preference optimization anchors directly to unconstrained base distributions, leaving style control entangled with individual persona priors. PsPLUG mathematically isolates the user residual under the style-conditioned context \(x_s = (x, s)\). For each query, a style-only negative response \(y_s \sim \pi_{\text{ref}}(\cdot \mid x_s)\) is sampled from the unpersonalized base model. Because \(y_s\) satisfies style instruction \(s\) but contains zero user nuance, contrasting the ground-truth user-authored response \(y_u\) against \(y_s\) creates a style-conditioned pair \((y^+, y^-) \triangleq (y_u, y_s)\). Defining the implicit reward difference as the log-likelihood ratio: $\(r_\phi(y) \triangleq \log \pi_\phi(y \mid x_s, u) - \log \pi_{\text{ref}}(y \mid x_s)\)$ the plug-in parameters \(\phi\) are optimized via a Bradley-Terry objective: $\(\mathcal{L}_{\text{style}}(\phi; x_s, u, y_u, y_s) = -\log \sigma \left( \beta \left( r_\phi(y_u) - r_\phi(y_s) \right) \right)\)$ where \(\sigma(\cdot)\) is the sigmoid function and \(\beta > 0\) is a temperature hyperparameter. Because the reference policy \(\pi_{\text{ref}}\) absorbs the generic stylistic shift induced by \(s\), the loss penalizes generic style mimicry and forces the plug-in to encode only the pure personalization residual. When explicit style instructions are absent (\(s = \emptyset\)), this objective naturally collapses to \(\mathcal{L}_{\text{neutral}}\), isolating personal style relative to an unconditioned baseline \(y_0 \sim \pi_{\text{ref}}(\cdot \mid x)\).

3. Inference-Time Scalar Strength Control: Continuous Dynamic Steering of Persona Intensity

In practical deployment, different tasks and users demand flexible trade-offs between strict style adherence and strong personal expression. Leveraging the structural disentanglement of the three-part prefix, PsPLUG introduces a continuous scaling coefficient \(\alpha \ge 0\) applied exclusively to the user representation vector \(z_u\): $\(z_{u, x_s}^{(\alpha)} = [z_{\text{sys}}; \alpha \cdot z_u; z_x]\)$ Prepending this modulated prefix sequence to the frozen LLM input embeddings enables dynamic control over personalization intensity. As \(\alpha \to 0\), user-specific signals are smoothly dampened, yielding an output dominated purely by the task and style instruction; setting \(\alpha = 1\) recovers the standard balanced equilibrium; scaling \(\alpha > 1\) amplifies idiosyncratic traits. This mechanism requires no online decoding parameter search or multi-adapter re-training, providing instantaneous, real-time steerability.

A Worked Example

Consider the LaMP-7 tweet paraphrasing task. The input query is a generic statement: "I'm exhausted and frustrated because I have a test tomorrow and I haven't studied," paired with an explicit "Warm and Humorous" style directive. Under standard baselines, style dominance triggers personalization collapse, resulting in an overly generic, impersonal reply: "Wendy's restaurant is a good option...".

In PsPLUG, the user's historical profile vector \(e_u\) reflects distinct habits: lowercase sentence starts, informal contractions ("gonna", "havent"), and emoticons (";("). With the default balance parameter \(\alpha = 1\), the prefix-injected model produces: "i'm so tired and fed up, I've got a test tomorrow and i still havent studied a single page... ;(". The output seamlessly satisfies the warm, gentle comedic style while rigorously replicating the user's idiosyncratic punctuation and informal phrasing, completely avoiding personalization collapse.

Loss & Training

The framework follows a two-stage lightweight pipeline: 1. Offline Profile Construction: Using Qwen3-8B-Instruct, user history items are summarized into compact textual profiles \(p_u\) via task-specific prompts. A frozen BGE-base-en-v1.5 model extracts 768-dimensional [CLS] representations with \(\ell_2\) normalization, which are cached offline. 2. Plug-in Optimization: The MLP projectors and system embedding \(\theta_{\text{sys}}\) are trained for 5 epochs using AdamW with early stopping, a learning rate of \(1 \times 10^{-4}\), and bfloat16 precision across 8 NVIDIA H100 GPUs. The base LLM Qwen3-8B and sentence encoder remain frozen throughout training.

Key Experimental Results

Main Results

On the standard LaMP benchmark without explicit style constraints across 6 tasks (classification, rating prediction, and text generation) using Qwen3-8B, PsPLUG consistently matches or outperforms prior personalization baselines:

Task / Metric Non-pers. RAG (BM25) PAG PPlug OPPU (Per-User LoRA) PsPLUG (Ours)
LaMP-1: Citation Identification (ACC โ†‘) 0.518 0.441 0.562 0.563 0.556 0.584
LaMP-1: Citation Identification (F1 โ†‘) 0.448 0.397 0.481 0.493 0.556 0.589
LaMP-2M: Movie Tagging (F1 โ†‘) 0.254 0.283 0.311 0.307 0.314 0.334
LaMP-2M: Movie Tagging (ACC โ†‘) 0.375 0.381 0.393 0.382 0.425 0.392
LaMP-3: Product Rating (MAE โ†“) 0.516 0.615 0.435 0.339 0.347 0.332
LaMP-3: Product Rating (RMSE โ†“) 0.805 0.981 0.714 0.583 0.613 0.464
LaMP-4: News Headline Gen. (ROUGE-1 โ†‘) 0.146 0.165 0.164 0.158 0.152 0.167
LaMP-4: News Headline Gen. (ROUGE-L โ†‘) 0.128 0.144 0.146 0.138 0.128 0.148
LaMP-5: Scholarly Title Gen. (ROUGE-1 โ†‘) 0.426 0.459 0.415 0.462 0.426 0.463
LaMP-5: Scholarly Title Gen. (ROUGE-L โ†‘) 0.342 0.387 0.352 0.386 0.342 0.391
LaMP-7: Tweet Paraphrasing (ROUGE-1 โ†‘) 0.497 0.500 0.507 0.502 0.498 0.523
LaMP-7: Tweet Paraphrasing (ROUGE-L โ†‘) 0.440 0.441 0.435 0.443 0.422 0.457

Ablation Study

To evaluate personalization preservation under explicit stylistic interference, the table below reports performance across 4 target styles (Warm, Critical, Concise, Elaborative) on text generation benchmarks:

Target Style Metric Task Non-pers. RAG PAG PPlug PsPLUG (Ours)
Warm ROUGE-1 / ROUGE-L LaMP-4 0.120 / 0.102 0.114 / 0.099 0.131 / 0.114 0.138 / 0.119 0.143 / 0.124
Critical ROUGE-1 / ROUGE-L LaMP-4 0.129 / 0.112 0.138 / 0.120 0.144 / 0.126 0.150 / 0.132 0.159 / 0.137
Concise ROUGE-1 / ROUGE-L LaMP-4 0.140 / 0.123 0.167 / 0.150 0.139 / 0.121 0.143 / 0.126 0.149 / 0.129
Elaborative ROUGE-1 / ROUGE-L LaMP-4 0.134 / 0.113 0.147 / 0.137 0.135 / 0.121 0.147 / 0.126 0.139 / 0.128
Warm ROUGE-1 / ROUGE-L LaMP-5 0.248 / 0.196 0.259 / 0.202 0.365 / 0.295 0.470 / 0.402 0.476 / 0.403
Critical ROUGE-1 / ROUGE-L LaMP-5 0.304 / 0.249 0.312 / 0.255 0.302 / 0.223 0.422 / 0.352 0.441 / 0.368
Concise ROUGE-1 / ROUGE-L LaMP-5 0.441 / 0.360 0.464 / 0.391 0.458 / 0.356 0.471 / 0.395 0.465 / 0.376
Elaborative ROUGE-1 / ROUGE-L LaMP-5 0.206 / 0.167 0.217 / 0.178 0.285 / 0.245 0.322 / 0.312 0.346 / 0.385
Warm ROUGE-1 / ROUGE-L LaMP-7 0.426 / 0.368 0.455 / 0.400 0.448 / 0.392 0.454 / 0.396 0.457 / 0.402
Critical ROUGE-1 / ROUGE-L LaMP-7 0.472 / 0.418 0.476 / 0.431 0.466 / 0.417 0.473 / 0.410 0.478 / 0.414
Concise ROUGE-1 / ROUGE-L LaMP-7 0.475 / 0.421 0.472 / 0.434 0.451 / 0.401 0.487 / 0.423 0.487 / 0.425
Elaborative ROUGE-1 / ROUGE-L LaMP-7 0.461 / 0.403 0.467 / 0.405 0.455 / 0.401 0.460 / 0.402 0.470 / 0.415

Key Findings

  • Disparate Style Interference: Tone-oriented styles (Warm and Critical) severely disrupt existing personalization methods, leading to noticeable performance drops. In contrast, the Concise directive functions primarily as a length constraint rather than an affective shift, maintaining or even improving lexical overlap metrics across baselines.
  • Pareto-Dominant Customization: PsPLUG attains the highest or second-highest scores across over 80% of style-task-metric combinations, outperforming prior baselines by up to 2โ€“4 ROUGE points while recording the highest style-compliance scores in LLM and human evaluations.
  • Predictable Strength Steering: Adjusting \(\alpha \in [0.2, 5.0]\) demonstrates consistent monotonic trade-offs. Concise generation remains robust across scaling ranges, whereas excessive scaling under Elaborative directives conflicts with brevity constraints on title generation tasks, highlighting the utility of dynamic inference-time control.
  • Minimal Per-User Overhead: Unlike RAG's \(O(|P_u| \cdot d_e)\) storage and variable latency or per-user LoRA's \(O(rHL)\) weight footprint and hot-swapping overhead, PsPLUG requires only a single static embedding \(O(d_e)\) per user and introduces a fixed 3-token prefix overhead.

Highlights & Insights

  • Distributional Residual Perspective: Reframing user persona as a conditional residual deviation against an instruction-conditioned base model avoids the entanglement of generic instruction compliance and subtle personal priors.
  • Decoupled Prefix with Zero-Overhead Inference Scaling: Decomposing conditioning into distinct continuous tokens enables instant scalar modulation \(\alpha\) of persona strength at inference time, completely bypassing multi-checkpoint tuning or decoding-time search.
  • Edge-Friendly Privacy Architecture: Decoupling profile summarization from generation permits offline, client-side extraction of static embeddings, eliminating the need to transmit sensitive raw interaction histories to centralized servers.

Limitations & Future Work

  • Closed Set of Style Directives: Experiments rely on four predefined categories (Warm, Critical, Concise, Elaborative). Open-ended, compositional, and fine-grained real-world stylistic requirements require further validation.
  • Lexical Overlap Metric Constraints: Standard LaMP metrics (ROUGE) struggle to separate stylistic word choice from core semantic personalization; deeper reliance on nuanced human and multi-agent judging protocols is needed.
  • Model and Domain Generalization: Empirical verification is centered on Qwen3-8B and English benchmarks; scaling to diverse architectures, multilingual settings, and long-form document domains remains open.
  • vs Retrieval-Based Personalization (LaMP / Pearl / Step-back): RAG prepends raw historical text to the context window, incurring substantial token costs and suffering from persona collapse when strong system instructions dominate attention. PsPLUG compresses history into a fixed single vector and isolates persona signals via preference contrast.
  • vs Parameter-Efficient Fine-Tuning (OPPU / Lifelong LoRA): Per-user PEFT requires saving and dynamically loading individual adapter weights during multi-tenant serving and severely overfits historical style, failing to adapt to explicit style instructions. PsPLUG keeps the backbone frozen and relies on lightweight prefix concatenation.
  • vs Soft-Prompt Plug-ins (PPlug / Persona-Plug): While PPlug injects continuous prompts, it trains via standard causal language modeling without disentangling background model priors or style directives. PsPLUG utilizes style-conditioned preference optimization to explicitly separate personalization residuals.

Rating

  • Novelty: โญโญโญโญโญ Formulates the style-constrained personalization challenge, defines personalization as a distributional residual, and implements modular inference scaling.
  • Experimental Thoroughness: โญโญโญโญโ˜† Covers 6 LaMP tasks across 4 distinct styles, supported by automated LLM evaluation and human correlation analysis.
  • Writing Quality: โญโญโญโญโญ Rigorous task formulation, clear mathematical progression, and coherent structure.
  • Value: โญโญโญโญโญ Offers a practical, privacy-preserving, and computationally efficient paradigm for deploying personalized interactive agents.