Skip to content

Response-Aware User Memory Selection for LLM Personalization

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/jfisher52/Response_Utility_Optimized_Memory_Selection
Area: LLM (Other)
Keywords: LLM personalization, user memory selection, mutual information, predictive entropy reduction, selective abstention

TL;DR

Addressing the flaw that conventional inference-time personalization relies solely on surface similarity between queries and user memories, this paper proposes RUMS, an information-theoretic framework that selects memory subsets by maximizing mutual information with model output distributions, and distills this utility signal into a lightweight DeBERTa classifier that matches or exceeds GPT-4 prompting while slashing token costs by up to 95%.

Background & Motivation

As large language models become ubiquitous in conversational assistants and everyday tools, personalization has emerged as a cornerstone of user satisfaction. Compared to parameter-efficient fine-tuning or low-rank preference adaptation that require expensive model updates per user, inference-time personalizationโ€”prepending relevant user profile attributes or historical facts to the prompt contextโ€”offers a flexible and practical alternative. However, real-world user memory stores grow substantially over time. Naively inserting an entire memory profile into the prompt introduces irrelevant or distracting context that confuses the model's attention, degrades generation quality and naturalness, introduces privacy vulnerabilities, and drives up token latency and cost.

Existing memory selection techniques tackle this by retrieving relevant items via lexical matching (e.g., BM25), dense embedding similarity (e.g., MiniLM, ReContriever), or direct prompting of proprietary LLMs. The foundational limitation shared across these paradigms is their sole reliance on measuring the relationship between the incoming user query and candidate memory items, completely blind to how those memory features actually alter the model's downstream response distribution. Query-based retrieval frequently over-selects redundant or superficial memories while missing non-obvious attributes that crucially narrow the response space. Furthermore, standard retrieval lacks a calibrated mechanism to abstain from retrieval when queries are purely factual or non-personalized, routinely introducing harmful noise.

This paper approaches the problem through an information-theoretic lens: an informative memory item should be characterized not by how much it resembles the input query, but by how effectively it eliminates uncertainty in the model's predictive distribution. Core idea: formalize memory selection as maximizing conditional mutual information between memory subsets and generation outputs (RUMS-Utility), prove that minimizing predictive entropy under latent user preferences approximately maximizes expected user utility, and distill this selection signal into a lightweight DeBERTa classifier (RUMS-Model) enabling adaptive abstention and highly cost-efficient inference.

Method

Overall Architecture

The RUMS framework comprises two principal components: an offline information-theoretic utility evaluator (RUMS-Utility) and an online lightweight predictor (RUMS-Model). During the offline training stage, candidate memory subsets are assessed for a given input query by measuring the reduction in predictive sequence entropy between an unconditioned model and a memory-conditioned model. If the maximum entropy reduction exceeds an optimal calibration threshold \(\tau\), the subset maximizing information gain is retained; otherwise, the query is marked for complete abstention (selecting an empty set \(\emptyset\)). The resulting pairs are then used to supervise a lightweight DeBERTa-v3-large classifier, allowing the system at inference time to perform single-pass or item-wise selection in milliseconds without running expensive Monte Carlo sampling or entropy computations.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    subgraph S1["Offline Utility Quantification & Selective Abstention"]
        direction TB
        A["User Query x & Candidate Memory Pool M"] --> B["Response-Aware Utility Evaluation<br/>Estimate predictive conditional entropy drop"]
        B --> C{"Max Utility > Threshold ฯ„ ?"}
        C -->|No| D["Selective Abstention Mechanism<br/>Return empty set โˆ… for non-personal inputs"]
        C -->|Yes| E["Optimal Memory Subset Mining<br/>Select subset S* maximizing mutual info"]
    end
    subgraph S2["Online Lightweight Inference & Downstream Generation"]
        direction TB
        D --> F["Lightweight Utility Approximation Classifier<br/>Distill DeBERTa-v3-large on mined pairs"]
        E --> F
        F --> G["Inference-Time Memory Subset Prediction<br/>Single-pass multi-label or binary scoring"]
        G --> H["Personalized Downstream Response Generation<br/>Inject selected items into generation prompt"]
    end

Key Designs

1. Response-Aware Utility Evaluation: Quantifying Memory Value via Predictive Entropy Reduction Rather than computing geometric similarity in embedding space, RUMS-Utility defines the utility of a memory subset \(S\) as its conditional mutual information with the generated response variable \(Y\) given query \(x\): $\(U_{\text{RUMS}}(S, u) = H(Y \mid x) - H(Y \mid x, m_{u,S})\)$ Under the theoretical formulation, the authors assume the existence of a latent target variable \(Z_u\) that governs optimal response quality under strictly proper scoring rules, with memory acting as an approximate information bottleneck. Theorem 3.1 formally establishes that minimizing predictive entropy \(H(Y \mid x, m_{u,S})\) is approximately equivalent to minimizing uncertainty about the latent preference \(Z_u\), thereby maximizing expected user utility. To make sequence-level entropy tractable over variable-length outputs, the sequence entropy is decomposed via the chain rule into token-level conditional entropies and estimated through Monte Carlo sampling (\(N=5\) rollouts) truncated at the first 20 tokens to avoid length-dependent entropy dilution.

2. Selective Abstention Mechanism: Calibrated Disentanglement of Personalized and Non-Personal Queries A critical failure mode of existing personalized systems is shoehorning user attributes into generic questions (such as factual trivia), which degrades response quality. RUMS-Utility inherently provides an abstention metric: for non-personalized inputs, the model's factual output distribution remains largely invariant whether user attributes are appended or not, resulting in near-zero utility scores. Conversely, personalization-dependent queries cause broad prior distributions to sharply concentrate around user preferences, yielding high utility scores. By tuning a global decision threshold \(\tau = 0.29\) on held-out validation data via Youden's J statistic, RUMS automatically defaults to selecting no memory items whenever all candidate subsets fall below \(\tau\), providing principled abstention without ad-hoc heuristic filters.

3. Lightweight Utility Approximation Classifier: Distilling Information Gain for Real-Time Serving While RUMS-Utility provides rigorous theoretical selection, calculating entropy online requires multiple autoregressive forward passes and combinatorial subset searches, taking over 7 seconds per query. To achieve production viability, the authors distill offline (query, optimal subset) targets into a 418M parameter DeBERTa-v3-large model. They formulate two practical variants: RUMS-Binary, which scores individual (query, item) pairs independently and tends toward conservative, noise-resistant selection suitable for smaller response LLMs; and RUMS-Multi, a multiclass multi-label classifier trained on merged high-utility subsets from multiple user profiles per query, completing the entire memory selection in a single forward pass (0.04s) and providing broader context suitable for top-tier LLMs.

Loss & Training

The offline scoring uses Llama-3.1-8B as the reference entropy calculator. To scale candidate evaluations, GPT-4 is prompted to filter initial candidate sets from 50 items down to at most 10 candidates before exact entropy evaluation. Offline data generation pairs 2.5K queries from PersonaFeedback and 2K from FreebaseQA with \(u=5\) diverse user profiles, filtering out FreebaseQA queries that exceed \(\tau = 0.29\). For model training, RUMS-Binary converts pairs into triplets and optimizes binary cross-entropy with a learning rate of \(5 \times 10^{-6}\) over 12 epochs. RUMS-Multi trains a multi-label logistic head using a learning rate of \(2 \times 10^{-5}\) over 12 epochs, with decision thresholds tuned on validation splits to \(b = 0.81\) and \(b = 0.93\) respectively.

Key Experimental Results

Main Results

The evaluation spans synthetic benchmarks (PersonaFeedback and FreebaseQA) and real-world ChatGPT dialogue logs (WildChat human-annotated splits). Memory item selection accuracy is measured against majority-vote human gold labels (macro-averaged Precision, Recall, and F1), alongside binary classification accuracy for distinguishing personalized from non-personalized queries (Recall on personalized, Specificity on non-personalized).

Dataset Metric RUMS-Subset RUMS-Binary RUMS-Multi Semantic Sim. GPT4-Few Performance Analysis
Synthetic Memory Selection F1 0.70 [0.65, 0.75] 0.78 [0.74, 0.82] 0.31 [0.26, 0.36] 0.14 [0.11, 0.17] 0.64 [0.60, 0.68] RUMS-Binary beats GPT-4 by +14% F1
Synthetic Memory Selection Precision 0.61 [0.55, 0.66] 0.60 [0.55, 0.64] 0.23 [0.19, 0.26] 0.09 [0.07, 0.11] 0.53 [0.50, 0.56] Surface similarity yields near-zero precision (0.09)
Synthetic Memory Selection Recall 0.70 [0.64, 0.75] 0.89 [0.86, 0.92] 0.39 [0.34, 0.44] 0.23 [0.18, 0.28] 0.65 [0.61, 0.69] Substantially recovers non-obvious relevant attributes
Real World Memory Selection F1 0.53 [0.47, 0.59] 0.45 [0.41, 0.49] 0.35 [0.31, 0.40] 0.20 [0.18, 0.23] 0.39 [0.34, 0.44] RUMS-Subset exceeds GPT-4 by +14% F1
Real World Memory Selection Precision 0.48 [0.42, 0.55] 0.32 [0.29, 0.36] 0.30 [0.26, 0.34] 0.17 [0.15, 0.20] 0.42 [0.38, 0.46] Retains strong precision under domain shift
Real World Memory Selection Recall 0.36 [0.30, 0.41] 0.60 [0.55, 0.64] 0.38 [0.34, 0.43] 0.27 [0.24, 0.30] 0.39 [0.34, 0.44] Strongest recall across out-of-distribution inputs
Synthetic Abstention Recall / Spec. 94.4 / 94.4 โ€” โ€” 100.0 / 0.0 100.0 / 50.0 Near-perfect balance; similarity has 0% specificity
Real World Abstention Recall / Spec. 92.3 / 96.0 โ€” โ€” 7.7 / 100.0 76.9 / 100.0 Robust cross-category balance without extreme skew

Ablation Study & Downstream Response Quality

The impact on downstream response quality was evaluated across two response generators: LLaMA-3.1-70B-Instruct and GPT-4. The table details the win rate margin (Win Rate of Ours minus Win Rate of Baseline, evaluated via CoT prompting with human validation):

Generation Setup Baseline Method RUMS-Utility Margin RUMS-Binary Margin RUMS-Multi Margin Key Observation
Synthetic / LLaMA-70B vs None (No memory) +0.11 +0.25 -0.13 Memory significantly enhances personalization
Synthetic / LLaMA-70B vs All (Full 50 items) +0.18 +0.16 -0.19 Over-contextualization severely distracts LLaMA-70B
Synthetic / LLaMA-70B vs Semantic Similarity +0.43 +0.27 +0.09 Semantic similarity injects irrelevant noise
Synthetic / LLaMA-70B vs GPT4-Few +0.07 +0.18 -0.16 Lightweight 418M model beats 400x larger GPT-4
Real World / GPT-4 vs Semantic Similarity -0.06 -0.02 0.00 Surface similarity underperforms on open-ended queries
Real World / GPT-4 vs BM25 +0.10 +0.18 +0.13 Lexical retrieval suffers severe under-selection
Real World / GPT-4 vs ReContriever +0.07 -0.05 +0.13 Dense retrieval lags behind utility-aware selection
Real World / GPT-4 vs GPT4-Few -0.08 +0.04 +0.11 RUMS-Multi provides richest context for strong LLM

Key Findings

  • Generator capacity dictates the optimal memory selection regime: For less noise-tolerant models like LLaMA-70B, the conservative RUMS-Binary model delivers the highest gains (outperforming all baselines by at least 15% on synthetic data) by strictly suppressing irrelevant attributes. In contrast, for highly capable models like GPT-4 that naturally ignore prompt noise, RUMS-Multi yields superior win rates (+11% margin over GPT4-Few) by offering a richer set of candidate traits.
  • Dramatically reduced deployment and token costs: Assuming a production throughput of 1 million queries per day at standard GPT-4 pricing, prepending all 50 items costs $165.3K monthly, Semantic Similarity costs $16.5K, and ReContriever costs $18.6K. In contrast, RUMS-Binary costs only $6.9K and RUMS-Multi costs $8.4Kโ€”representing up to a 95% reduction in additional prompt token costs. Furthermore, RUMS-Multi requires only 0.04 seconds inference time compared to 1.22 seconds for GPT-4 prompting.
  • Predictive entropy properties: Base models exhibit much clearer distribution divergence between personalized and non-personalized queries compared to instruction-tuned models (JS divergence 0.56 vs 0.32), as RLHF/instruction-tuning tends to pre-sharpen output probabilities. Restricting entropy computation to the first 20 tokens prevents length-induced entropy decay from drowning out the true personalization signal.

Highlights & Insights

  • Reconceptualizing retrieval via output distribution: Grounding memory selection in output uncertainty rather than query-memory lexical overlap provides an actionable, principled alternative to standard retrieval-augmented generation paradigms.
  • Decoupled execution via offline distillation: The study neatly sidesteps the prohibitively high cost of online mutual information estimation (7.28s) by confining it to offline data annotation, successfully distilling the capability into a DeBERTa encoder operating at 0.04s per query.
  • Native abstention from entropy dynamics: By viewing unpersonalized queries as invariant output distributions, the framework solves retrieval-decision and feature-selection simultaneously using a single unified utility threshold \(\tau\).

Limitations & Future Work

  • Static single-turn formulation: The experiments focus on static 50-item user profiles and single-turn queries to isolate selection accuracy; extending RUMS to multi-turn conversational histories where preferences shift dynamically remains an open challenge.
  • Bias toward restrictive constraints: Because mutual information rewards entropy reduction, the argmax selection objective inherently favors memories that narrow the answer space (e.g., location, dietary limits) over memories that might encourage creative, exploratory, or expansive generation.
  • Cross-model transfer assumptions: Generating utility targets using Llama-3.1-8B and evaluating downstream responses on GPT-4 assumes that output uncertainty transfers cleanly across distinct model architectures and calibration profiles.
  • vs Retrieval Baselines (BM25, MiniLM, ReContriever): Surface and dense retrievers select items solely based on prompt similarity, frequently retrieving superficial matches and failing completely to abstain on non-personalized inputs. RUMS directly models response effect, drastically improving precision and response quality.
  • vs LLM In-Context Selection (GPT-4 Prompting): While prompting frontier models provides strong contextual reasoning, it suffers from pronounced over-selection, high token costs, and substantial inference latency. Distilling the RUMS signal into a 418M DeBERTa matches or exceeds GPT-4 selection quality at a fraction of the cost.

Rating

  • Novelty: โญโญโญโญโญ Formulates an information-theoretic utility objective for LLM personalization with rigorous proofs and practical distillation.
  • Experimental Thoroughness: โญโญโญโญโญ Comprehensive evaluation covering synthetic benchmarks, real-world WildChat dialogues, human agreement studies, and wall-clock/cost analyses.
  • Writing Quality: โญโญโญโญโญ Clean narrative flow, intuitive motivations, and well-structured mathematical and empirical arguments.
  • Value: โญโญโญโญโญ Provides an immediately deployable and economically viable blueprint for production-scale personalized LLM applications.