Skip to content

UniMem: Complementary Episodic-to-Parametric Memory for Boundary-Agnostic Task Streams

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Complementary Learning Systems, Boundary-Agnostic Task Streams, Procedural KV Memory, Dynamic Self-Routing, Continual Learning

TL;DR

Inspired by brain Complementary Learning Systems, UniMem decouples lightweight routing tokens from high-capacity Procedural KV Memory, establishing a self-routing lifecycle where sparse unknown queries are buffered for RAG fallback and recurring patterns are clustered to expand dedicated parametric memory units, achieving strong execution stability on boundary-agnostic streams.

Background & Motivation

Large language model (LLM) agents are increasingly tasked with multi-step reasoning, tool invocation, and continuous autonomous interactions, where the ability to accumulate past experience and reuse task-specific execution strategies is fundamental to long-horizon agency. However, practical deployment across real-world workflows exposes agents to streaming inputs that are inherently heterogeneous, boundary-agnostic, and constantly evolving. Current memory paradigms generally bifurcate into two extremes: external retrieval-augmented memory (such as RAG) stores exemplars externally and absorbs new observations without gradient updates, yet it fails to internalize recurring execution patterns into model behavior and incurs substantial inference-time retrieval latency and context bloat; conversely, parametric memory approaches (such as LoRA or prompt tuning) internalize procedural workflows into continuous weights, but they strictly rely on human-annotated task boundaries and static parameter capacities, suffering from severe parameter collision and catastrophic forgetting under non-stationary streams.

This fundamental tension between plasticity and stability cannot be resolved by either memory mechanism in isolation. In cognitive neuroscience, Complementary Learning Systems (CLS) theory shows how mammalian brains resolve this dilemma through coordinated dual structures: the hippocampus facilitates rapid episodic encoding and transient storage of novel experiences, while the neocortex gradually extracts and consolidates recurring structural patterns into stable, interference-free long-term parametric knowledge. Existing modular continual learning techniques (such as orthogonal subspace constraints in O-LoRA, dynamic routing in L2R, or single-token procedural compression in TOKMEM) assume predefined task inventories or static allocations, rendering them incapable of autonomously demarcating emerging task boundaries or timing parametric transitions in unstructured deployment streams.

Addressing this open challenge requires decoupling the identification of task identity from the execution of task procedures, granting the model autonomous meta-control over its own memory expansion. By maintaining learnable routing tokens alongside a calibrated novelty sentinel, the system can gracefully reject unfamiliar queries into an episodic buffer for retrieval-augmented handling, deferring parameter allocation until unsupervised density clustering discovers verified, recurring procedural patterns that justify new dedicated memory blocks. Core idea: Inspired by Complementary Learning Systems, UniMem establishes a self-routing episodic-to-parametric memory lifecycle that buffers sparse unfamiliar queries for retrieval-augmented execution while autonomously clustering recurring patterns into expandable Procedural KV Memory units, resolving the stability-plasticity dilemma on boundary-agnostic streams without task labels.

Method

UniMem introduces an autonomous, open-world memory expansion framework designed for boundary-agnostic task streams. For any incoming query \(q\), the frozen backbone produces the last-token hidden state \(h_q \in \mathbb{R}^d\), which is fed into a lightweight routing module: if the query matches an existing task with high confidence, the system activates the paired Procedural KV Memory block \(P_k\) to inject layer-wise residual transformations into the attention layers; if the query is deemed novel or uncertain, it is routed to a calibrated novelty sentinel \(e_{\text{NOVEL}}\) and stored in an Episodic Buffer while being serviced via \(k\)-NN retrieval-augmented execution. As buffered interactions accumulate, an unsupervised clustering pipeline powered by Normalized Compression Distance (NCD) and HDBSCAN identifies coherent task clusters, applying rigorous outlier filtering and cohesion checks before promoting verified clusters into fresh parametric units \((e_{\text{new}}, P_{\text{new}})\).

Overall Architecture

The execution and evolution pipeline of UniMem operates through a dual-loop lifecycle: real-time self-routing memory selection for incoming queries and asynchronous unsupervised parametric consolidation for buffered episodic data. The backbone language model remains completely frozen throughout all phases, preserving foundational general reasoning capabilities while expanding task-specific modules on demand.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Streaming Query q<br/>Boundary-agnostic & unlabelled"] --> B["Self-Routing Memory Selection<br/>Joint posterior over known tokens & e_NOVEL"]
    B -->|Top known probability >= 0.7| C["Procedural KV Memory Inference<br/>Activate P_k & inject gated residuals into attention"]
    B -->|Low confidence / Routed to e_NOVEL| D["Streaming Episodic Buffering & RAG Fallback<br/>Store in episodic buffer & retrieve k-NN demos"]
    C --> E["Generate Task Response"]
    D --> E
    D --> F["Unsupervised Task Discovery & Quality Gating<br/>Gzip NCD matrix + HDBSCAN + Centroid & Cohesion filter"]
    F -->|Qualified dense recurring cluster| G["Incremental Memory Expansion<br/>Allocate fresh e_new & P_new via supervised fine-tuning"]
    F -->|Noise & unverified sparse queries| H["Retain in Episodic Buffer for future evidence"]
    G --> B

Key Designs

1. Self-Routing Memory Selection: Decoupling task identification from execution via a calibrated novelty sentinel
Standard dynamic routing architectures or Mixture-of-Experts (MoE) routers are notorious for over-committing unknown inputs to the closest known expert, causing catastrophic execution distortion. UniMem resolves this by expanding the routing token matrix to \(E = [e_1, \dots, e_K, e_{\text{NOVEL}}]\). Following the initialization of known task tokens, the novelty sentinel \(e_{\text{NOVEL}}\) is anchored near the centroid of known routing embeddings with a minor Gaussian perturbation and normalized to match the average geometric scale \(\bar{r} = \frac{1}{K}\sum_{k=1}^K \|e_k\|\). The routing layer is then calibrated using a \((K+1)\)-way cross-entropy objective against held-out pseudo-unseen tasks. During inference, UniMem evaluates the full joint Softmax distribution and executes a max-known-probability rejection rule:

\[p_i = \frac{\exp(h_q^\top e_i)}{\sum_{j=0}^K \exp(h_q^\top e_j)}, \quad i^\star = \arg\max_{i \in \{1,\dots, K\}} p_i, \quad \hat{y} = \begin{cases} i^\star, & \text{if } p_{i^\star} \ge \tau_{\text{route}} \\ 0, & \text{otherwise} \end{cases}\]

When the top candidate confidence falls below \(\tau_{\text{route}} = 0.7\), the query is classified as novel (\(\hat{y}=0\)). Because \(e_{\text{NOVEL}}\) participates directly in the Softmax denominator, it acts as an active probability sink for out-of-distribution patterns, suppressing known token probabilities and preventing premature parametric activation.

2. Procedural KV Memory Inference: Layer-wise residual cross-attention with clamped scalar gating
Compressing complex task procedures into a single prefix token (e.g., TOKMEM) creates severe procedural capacity bottlenecks, whereas full adapter fine-tuning induces cross-task weight interference. UniMem pairs each recognized task \(k\) with a dedicated Procedural KV Memory block \(P_k = \{(K_k^{(l)}, V_k^{(l)})\}_{l=1}^L\). For an activated memory unit, layer-wise query vectors \(Q^{(l)}\) undergo bidirectional cross-attention without causal masking or rotary position embeddings (RoPE), treating the procedural slots as global, position-invariant context. The memory output is blended into the frozen self-attention state \(H_{\text{orig}}^{(l)}\) via a learnable clamped scalar gate:

\[H_{\text{out}}^{(l)} = H_{\text{orig}}^{(l)} + g^{(l)} \cdot \text{CrossAttn}\left(Q^{(l)}, K_k^{(l)}, V_k^{(l)}\right), \quad g^{(l)} = \text{clamp}(\tilde{g}^{(l)}, 0, g_{\max})\]

Initializing \(\tilde{g}^{(l)} = 0.01\) ensures smooth training dynamics and avoids sigmoid gradient vanishing near saturation. For architectures utilizing Grouped-Query Attention (such as LLaMA-3.2-3B with 24 query heads and 8 KV heads), key-value slots are parameterized strictly over \(H_{\text{kv}}\) and broadcast across query heads. This parameterization introduces merely 60,444 parameters per task on LLaMA-3.2-3B—several orders of magnitude smaller than the 3B backbone—while guaranteeing physical modular isolation.

3. Unsupervised Task Discovery & Quality Gating: Dual-filtered compression-based clustering for safe consolidation
In continuous task streams lacking explicit domain demarcations, blindly triggering gradient updates on streaming batches invites severe representation collapse. When the Episodic Buffer reaches its trigger threshold \(C = 1600\), UniMem initiates unsupervised task discovery. Sample instruction-query pairs are truncated to 128 characters, and procedural proximity is evaluated using Normalized Compression Distance (NCD) via gzip:

\[\text{NCD}(x_i, x_j) = \frac{C(x_i x_j) - \min\{C(x_i), C(x_j)\}}{\max\{C(x_i), C(x_j)\}}\]

Unlike dense embeddings that can be biased by superficial phrasing, NCD directly estimates Kolmogorov complexity discrepancies, capturing lexical, grammatical, and structural execution patterns. The distance matrix is processed by HDBSCAN (\(S_{\min}=50, \text{min\_samples}=80\)), which autonomously identifies density-based clusters while marking ambiguous instances as noise. UniMem then enforces two quality gates: (1) embedding-based outlier removal, filtering samples whose Qwen3-Embedding centroid cosine similarity falls below \(\bar{s} - 0.5\sigma_s\); and (2) intra-cluster cohesion assessment, requiring average pairwise NCD similarity \(\text{sim}_{\text{NCD}}(C) \ge 0.85\) over sampled pairs. Only qualifying clusters trigger allocation of a new \((e_{\text{new}}, P_{\text{new}})\) pair trained via SFT, whereas rejected clusters and sparse long-tail tasks (e.g., tasks with only 10 samples) remain in the buffer for reliable RAG fallback.

Loss & Training

UniMem employs decoupled optimization objectives to mirror its retrieve-then-execute lifecycle: 1. Routing Objective: Minimizes the negative log-likelihood of selecting the ground-truth task index or the novelty sentinel solely from input query states: $\(\mathcal{L}_{\text{route}} = -\log p(z \mid q_{1:n}; E)\)$ During routing training, all parametric memory blocks remain inactive, ensuring gradients update only routing embeddings \(E\). 2. Execution Objective: Computes standard cross-entropy loss over target response tokens conditioned on the assigned task identity: $\(\mathcal{L}_{\text{exec}}^{(k)} = -\frac{1}{m} \sum_{t=1}^m \log p(y_t \mid q_{1:n}, k, y_{<t}; \Theta_k)\)$ where \(\Theta_k = \{P_k, \{g^{(l)}\}\}\) denotes task-specific parameters while the backbone remains frozen. The overall loss is \(\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{route}} + \mathcal{L}_{\text{exec}}\). The routing tokens are optimized with a learning rate of \(1 \times 10^{-3}\), while Procedural KV Memory modules use \(5 \times 10^{-3}\), balancing routing selectivity with rapid procedural convergence.

Key Experimental Results

Main Results

On the Super-Natural Instructions (SNI) streaming benchmark containing up to 100 novel streaming tasks (with 10% designated as sparse long-tail tasks having only 10 training instances), UniMem was benchmarked against Zero-Shot Base, Standard RAG (\(k=3\)), Replay LoRA (10% cumulative rehearsal), and Adapted TOKMEM. The results below report Exact Match (EM, %) and ROUGE-L (%):

Backbone Method Test-10 (EM / R-L) Test-50 (EM / R-L) Test-100 (EM / R-L) Macro Avg. (EM / R-L)
LLaMA-3.2-3B Base (Zero-Shot) 29.60 / 47.71 32.64 / 45.97 31.80 / 45.72 31.35 / 46.47
Standard RAG 37.00 / 58.43 37.22 / 51.79 35.21 / 49.45 36.48 / 53.22
Replay LoRA (10% replay) 38.40 / 60.36 45.76 / 57.91 43.62 / 56.67 42.59 / 58.31
Adapted TOKMEM 38.40 / 61.80 41.06 / 54.67 40.54 / 55.07 40.00 / 57.18
UniMem (Ours) 51.60 / 67.01 47.22 / 59.53 47.56 / 60.58 48.79 / 62.37
LLaMA-3.1-8B Base (Zero-Shot) 39.80 / 58.09 39.40 / 53.33 39.28 / 53.20 39.49 / 54.87
Standard RAG 45.20 / 64.83 42.61 / 59.12 41.56 / 56.65 43.12 / 60.20
Replay LoRA (10% replay) 55.20 / 73.80 49.84 / 62.69 49.39 / 62.58 51.48 / 66.36
Adapted TOKMEM 56.20 / 74.08 49.27 / 62.87 44.74 / 59.74 50.07 / 65.56
UniMem (Ours) 58.00 / 73.75 51.84 / 64.49 51.93 / 64.46 53.92 / 67.57
Qwen3-8B Base (Zero-Shot) 40.80 / 53.10 45.84 / 57.28 44.36 / 57.72 43.67 / 56.03
Standard RAG 41.80 / 60.35 42.86 / 58.53 42.26 / 58.68 42.31 / 59.19
Replay LoRA (10% replay) 44.80 / 63.38 47.51 / 60.20 50.44 / 63.48 47.58 / 62.35
Adapted TOKMEM 39.80 / 59.67 44.45 / 57.46 44.35 / 57.92 42.87 / 58.35
UniMem (Ours) 53.20 / 67.83 49.71 / 61.38 49.94 / 62.22 50.95 / 63.81

On the boundary-agnostic SuperGLUE Mixed Stream across six diverse reasoning tasks (BoolQ, CB, RTE, COPA, MultiRC, WiC), UniMem attained an average accuracy of 73.89% on LLaMA-1B-Instruct (vs. Replay LoRA 64.91% and TOKMEM 44.27%) and 85.95% on LLaMA-3B-Instruct (vs. Replay LoRA 77.49% and TOKMEM 78.98%), demonstrating superior robustness when implicit task transitions occur at every single sample.

Ablation Study

Systematic ablations were performed on core architectural components across model scales, alongside a dedicated routing investigation comparing explicit novelty sentinel calibration with fixed-threshold rejection:

Backbone & Variant Test-10 (EM / R-L) Test-50 (EM / R-L) Component Contribution Analysis
LLaMA-3.2-3B: UniMem (Full) 51.60 / 67.01 47.22 / 59.53 Complete proposed framework
└─ w/o RAG cache — / — 46.45 / 58.97 Omission of buffer RAG leaves sparse tasks unhandled
└─ w/o KV gate 35.80 / 56.43 31.22 / 45.33 Severe collapse (-16.00 EM): ungated cross-attention corrupts representations
LLaMA-3.1-8B: UniMem (Full) 58.00 / 73.75 51.84 / 64.49 Complete proposed framework
└─ w/o RAG cache — / — 50.90 / 63.10 Disabling episodic retrieval reduces long-tail performance
└─ w/o KV gate 37.40 / 57.10 30.16 / 42.05 Severe collapse (-21.68 EM): uncalibrated procedural signals disrupt the backbone

In the unknown-task routing ablation under a constant rejection cutoff \(\tau = 0.5\): - Incorporating the calibrated novelty sentinel \(e_{\text{NOVEL}}\) achieved routing accuracies of 83.0% (LLaMA-3.2-3B), 93.2% (LLaMA-3.1-8B), and 74.0% (Qwen3-8B); - Removing the sentinel and relying solely on a fixed threshold over known task tokens resulted in drastic drops to 21.6%, 4.0%, and 0.8%, indicating that a dedicated geometric sink is indispensable for discriminating novel tasks.

Key Findings

  • Scalar gating is the vital stabilizer for procedural injection: Eliminating layer-wise gates \(g^{(l)}\) led to precipitous drops of 16 to 21 EM points, demonstrating that learned modulation is essential to prevent uncalibrated cross-attention outputs from corrupting frozen attention dynamics.
  • Decoupled routing and execution outperforms monolithic token memory: While TOKMEM compresses both routing identity and execution procedures into a single token vector, its performance deteriorates on long streams (trailing UniMem by 7.19 EM points on LLaMA-3.1-8B Test-100). In contrast, UniMem utilizes tokens solely as pointers and delegates execution to 60K-parameter KV blocks, lifting procedural capacity.
  • Remarkable resistance to catastrophic forgetting: Tracking the retention of 7 initial memory units across five successive consolidation rounds (R1 to R5) revealed that routing accuracy drifted negligibly from 1.000 to 0.991, while normalized EM response accuracy was almost completely preserved (0.474 to 0.471).
  • Graceful sub-linear parameter scaling: Over 16,000 streaming samples spanning 100 novel tasks, UniMem consolidated only 76 distinct parametric units; 10 sparse tasks were safely preserved in the episodic buffer, and semantically overlapping tasks converged into unified units, preventing uncontrolled parameter proliferation.

Highlights & Insights

  • Neuroscience-grounded CLS operationalization: Systematically translates mammalian dual-memory principles into an actionable architecture combining an external episodic buffer, a calibrated novelty sentinel, and layer-wise Procedural KV Memory.
  • Representation-agnostic NCD task discovery: Leverages gzip-based Normalized Compression Distance to discover recurring procedural structures, bypassing dense semantic embedding biases and capturing algorithmic syntax and format similarities.
  • Extreme parameter and computational efficiency: Each consolidated unit requires only ~60K parameters; processing a 100-task stream takes only 3–4 GPU-hours on a single NVIDIA H20 GPU, compared to 20–24 GPU-hours for Replay LoRA (a 5–8× compute reduction).

Limitations & Future Work

  • Conservative consolidation threshold: The static capacity buffer limit (\(C=1600\)) and high cohesion threshold (0.85) prevent false consolidation but can leave moderately sparse or slow-evolving tasks in the episodic buffer indefinitely.
  • Single parametric instantiation: Memory modules are instantiated exclusively via Procedural KV Memory with a frozen backbone; exploring modular LoRA adapters, soft prompts, or sparse MoE blocks under this routing framework remains open.
  • Episodic buffer memory management: The episodic buffer currently uses simple capacity-based triggering without active sample curation, forgetting heuristics, or prioritized replay across consolidation cycles.
  • vs Replay LoRA: Replay LoRA attempts to fold all streaming task knowledge into a shared low-rank parameter space, requiring heavy rehearsal to counteract forgetting and suffering severe parameter collision as stream length increases; UniMem maintains frozen backbone weights, isolating task knowledge into modular KV blocks with ~5–8× lower compute overhead.
  • vs Adapted TOKMEM: TOKMEM forces a single token embedding to act as both task identifier and execution controller, creating an information bottleneck; UniMem decouples these responsibilities, using tokens as lightweight switches and Procedural KV Memory for high-capacity procedural execution.
  • vs Pure RAG / Agent Memory (A-MEM, MemAgent): Pure external memory approaches rely on in-context demonstration retrieval, leaving internal model behaviors unchanged while accumulating high context overhead and inference latency; UniMem treats RAG as a temporary staging buffer, consolidating frequent patterns into internal weights once sufficient evidence is gathered.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Elegant operationalization of brain CLS theory, seamlessly integrating novelty sentinels, compression-based clustering, and modular parameter expansion.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation across 10 to 100 streaming tasks on multiple LLM backbones (3B to 8B), featuring rigorous retention, compute, and parameter scaling analyses.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear narrative, disciplined mathematical formulation, and exhaustive documentation of implementation hyperparameters.
  • Value: ⭐⭐⭐⭐⭐ Offers a practical, principled foundation for self-evolving, boundary-agnostic LLM agent memory systems.