LifeMem: Enabling Lifelong Experience Reuse for LLM Agents via Memory Distillation¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/BITHLP/LifeMem
Area: LLM Agent
Keywords: LLM Agent, Lifelong Learning, Experience Reuse, Memory Distillation, Workflow Clustering
TL;DR¶
LifeMem addresses action space mismatch and catastrophic forgetting in inter-environment lifelong learning by structuring heterogeneous interaction trajectories into workflow clusters constrained by directional statistics, distilling invariant high-level skills, and activating them alongside environment-aligned exemplars to achieve significant positive backward transfer across 10 benchmarks.
Background & Motivation¶
The pursuit of artificial general intelligence necessitates autonomous agents capable of continually adapting to evolving tasks and heterogeneous environments over their operational lifetime. Under this inter-environment lifelong learning paradigm, large language model (LLM) agents are expected to accumulate experience across diverse domainsโincluding embodied control, tool utilization, web navigation, and data analysisโand seamlessly transfer acquired procedural knowledge to unseen tasks without relying on expensive, environment-specific retraining. However, interactions originating from heterogeneous environments naturally feature disparate state representations, distinct observation spaces, and fundamentally mismatched action grammars, posing steep obstacles to effective cross-environment experience reuse.
Existing paradigms for experience accumulation fall broadly into internal and external memory methods, yet both hit performance bottlenecks in lifelong deployment. Internal-memory approaches that continually fine-tune LLM parameters incur prohibitive computational costs and frequently suffer from parameter overwriting, leading to catastrophic forgetting of earlier environments. Conversely, external-memory systems maintain trajectory databases to support in-context retrieval, but exhibit two critical structural failures as operational history accumulates: first, trajectory-only retrieval frameworks (such as Synapse) index raw interaction logs directly, which fails to bridge mismatched action spaces across domains and causes retrieval clutter that misleads agent actions; second, generic trajectory abstraction frameworks (such as ExpeL and AutoSkill) attempt to distill high-level insights directly over an unpartitioned memory pool, inadvertently pulling irrelevant trajectories into the distillation context and producing coarse, noisy skills that degrade downstream task execution.
The underlying tension resides in the dual requirement of lifelong agent planning: reliable execution demands concrete, environment-aligned demonstrations to anchor syntactically valid actions, while cross-task generalization requires environment-invariant strategic abstractions decoupled from low-level environmental mechanics; yet without structural partitioning, accumulating raw trajectories inevitably compromises retrieval precision. This paper addresses this challenge by structuring memory evolution around latent functional execution workflows, adaptively partitioning trajectory clusters under directional statistical constraints, and distilling purified high-level skills locally within each cluster. Core idea: organize heterogeneous agent trajectories into adaptive workflow clusters via directional diversity constraints and distill local environment-invariant skills, dynamically combining environment-aligned trajectories with abstract skills at inference time to decouple low-level grounding from high-level transfer.
Method¶
Overall Architecture¶
LifeMem models the agent's long-term memory as a dynamic, hierarchically organized system that undergoes single-pass initialization, streaming adaptive evolution, and dual-tier inference activation. When streaming interaction trajectories arrive from diverse environments, LifeMem partitions them into structured clusters based on functional execution workflows rather than surface semantic similarity. To maintain cluster specificity as memory expands, it incorporates an adaptive splitting mechanism governed by directional statistics on the unit hypersphere.
For each purified workflow cluster, an LLM-driven distillation process abstracts environment-invariant procedural rules and behavioral guidelines, forming a high-level skill repository decoupled from domain-specific APIs. During inference on a novel task, LifeMem activates a complementary memory context: it retrieves Top-\(k\) historical trajectories grounded in the target environment to provide precise syntax exemplars, while simultaneously activating the high-level skills linked to those workflows to provide macro-level strategic guidance.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Streaming Trajectory Stream<br/>Heterogeneous Multi-Env Tasks"] --> B["Workflow Clustering<br/>Functional Execution Logic Assignment"]
B --> C["Directional Diversity Splitting<br/>Capacity L & MRL Concentration Constraint"]
C --> D["Local Skill Distillation<br/>Extract Environment-Invariant Rules"]
D --> E["Dual-Tier Memory Inference<br/>Env-Aligned Trajectories + Abstract Skills"]
E --> F["Action Generation & Lifelong Transfer<br/>Positive BwT & Zero-Shot Generalization"]
Key Designs¶
1. Workflow Clustering: Mitigating cross-environment surface interference by grouping trajectories according to functional execution logic
Trajectories originating from distinct environments often differ dramatically in surface textual descriptions and action vocabularies while sharing identical procedural workflows (e.g., verifying user credentials, handling condition branches, executing state changes). Standard dense retrievers relying solely on semantic text embeddings frequently group tasks by topic rather than operational structure, causing cross-environment retrieval collisions. LifeMem initializes memory with an initial set of workflow clusters \(\mathcal{C} = \{c_1, \dots, c_M\}\). When a new trajectory \(\xi\) arrives, the system retrieves its Top-\(k\) closest neighbors and identifies their associated clusters \(\mathcal{C}_\xi\). A lightweight LLM then evaluates functional alignment: if \(\xi\) aligns with the underlying execution logic of an existing cluster \(c \in \mathcal{C}_\xi\), it is incorporated into \(c\) and the cluster summary \(d_c\) is updated; otherwise, a new workflow cluster is created. This ensures trajectories within each cluster remain strictly homogeneous in their behavioral patterns.
2. Directional Diversity Splitting: Preventing cluster bloating and semantic drift via Mean Resultant Length regularization
As trajectories continuously accumulate during lifelong interaction, an existing cluster may gradually encompass divergent behavioral sub-patterns, degrading the specificity of subsequent skill distillation. Arbitrarily splitting clusters based on raw trajectory counts risks fragmenting cohesive workflows into redundant subsets, whereas failing to split introduces noise. LifeMem addresses this by introducing the Mean Resultant Length (MRL) from directional statistics to evaluate structural homogeneity geometrically. Given a cluster \(c\) containing \(n\) trajectories, each trajectory's normalized embedding representation is mapped to a unit vector \(x_i\) (\(\|x_i\| = 1\)). The directional concentration \(R\) is defined as:
The scalar \(R \in [0, 1]\) directly quantifies structural homogeneity on the hypersphere: \(R \to 1\) reflects high redundancy (e.g., repetitive executions of identical routine templates), whereas lower \(R\) indicates divergent operational behaviors. LifeMem enforces a dual-condition splitting criterion: a cluster \(c\) is partitioned into 3 to 6 finer-grained sub-clusters if and only if both the volume limit and diversity threshold are met: \(n > L\) (with \(L=50\)) and \(R < \tau_{\text{mrl}}\) (with \(\tau_{\text{mrl}}=0.95\)). This ensures that cluster partitioning occurs only under genuine behavioral divergence, maintaining adaptive cluster granularity.
3. Local Skill Distillation: Eliminating cross-task noise by extracting environment-invariant principles within homogeneous clusters
Extracting global skills directly across an unpartitioned memory pool causes conflicting action definitions to corrupt the abstracted rules, producing noisy or overly generic heuristics. LifeMem circumvents this by confining distillation locally within each individual workflow cluster. For each cluster \(c_i \in \mathcal{C}\), an advanced language model \(f(\cdot)\) synthesizes the constituent trajectories into a concise skill profile:
The distillation prompt explicitly instructs the model to extract 3 to 5 universally applicable behavioral guidelines and strategic Ways of Thought, forbidding references to specific trial instances, environmental object IDs, or low-level API signatures. Each distilled skill \(s_i\) captures domain-agnostic execution principles, temporal dependencies, and error-recovery heuristics. The resulting hierarchical memory structure is represented as \(\mathcal{H} = \{(c_i, s_i) \mid c_i \in \mathcal{C}\}\), decoupling transferable reasoning rules from environment-specific dynamics.
4. Dual-Tier Memory Inference: Bridging abstract strategy with syntactic grounding via joint trajectory-skill activation
During inference on a target task \(T = \langle E, o_0, g \rangle\), relying solely on high-level skills leaves the agent vulnerable to invalid syntax or incorrect parameter formatting; conversely, relying solely on raw exemplars reintroduces catastrophic forgetting and negative transfer. LifeMem deploys a dual-tier activation scheme: it first retrieves the Top-\(k\) historical trajectories \(\{\xi_0, \dots, \xi_{k-1}\}\) strictly conditioned on the same environment \(E\), providing explicit, syntactically grounded exemplars of valid tool calls and state transitions. Concurrently, it identifies the workflow clusters \(\mathcal{C}_\xi\) linked to these exemplars and activates their associated environment-invariant skills \(\mathcal{S}_\xi = \{s_i \mid c_i \in \mathcal{C}_\xi\}\). Conditioned on the unified active memory context \(M_{\text{act}} = (\{\xi_j\}_{j=0}^{k-1}, \mathcal{S}_\xi)\), the policy generates actions via \(a = P_\theta(g, M_{\text{act}}, o)\), harmonizing structural guidance with precise low-level execution.
Loss & Training¶
LifeMem operates as an external memory system that enhances frozen LLMs without requiring gradient updates or parameter fine-tuning. Trajectory embeddings are indexed using all-MiniLM-L6-v2 via FAISS, while structured cluster metadata and skill texts are managed in SQLite3.
For lightweight routing during streaming evolution, GPT-5-mini determines incremental cluster assignments; for cluster dissolution and high-level skill distillation, GPT-4.1 is employed. During inference, all models are evaluated at a decoding temperature of 0.3 without reasoning mode flags. In the lifelong benchmark setup, tasks are streamed in sequence across 10 environments grouped by operational domain (AlfWorld \(\to\) ScienceWorld \(\to\) APIBank \(\to\) \(\tau\)-bench \(\to\) HotpotQA \(\to\) Webshop \(\to\) LifelongAgentBench \(\to\) ToolQA \(\to\) MiniwoB++ \(\to\) Mind2web), with additional evaluations conducted on fully randomized and buffer-stabilized streams.
Key Experimental Results¶
The experimental benchmark spans 5 scenarios across 10 environments with over 13,000 tasks: embodied action (ALFWorld, ScienceWorld), tool utilization (APIBank, \(\tau\)-bench), web search (HotpotQA, WebShop), data analysis (LifelongAgentBench, ToolQA), and web browsing (MiniWoB++, Mind2Web). For 4 environments lacking training interaction data, 2,310 trajectories were generated via GPT-4.1 under the ReAct framework, with human verification confirming \(>90\%\) step reasonableness. Evaluation metrics include Overall Performance (OP \(\uparrow\), Eq. 8) and Backward Transfer (BwT \(\uparrow\), Eq. 9).
Main Results: Inter-Environment Lifelong Learning Performance (Table 2)¶
| Backbone Model | Evaluation Metric / Category | ReAct (No Mem) | Synapse (Traj Only) | ExpeL (Trial & Error) | AutoSkill (Global Skill) | LifeMem (Ours) |
|---|---|---|---|---|---|---|
| GPT-4o-mini | Overall Performance (OP \(\uparrow\)) | 28.84 | 39.18 | 41.85 | 33.22 | 44.13 (โ5.45% rel.) |
| Backward Transfer (BwT \(\uparrow\)) | โ | -7.04 | +3.10 | -8.54 | +5.66 | |
| Embodied Action OP / BwT | 10.62 / โ | 40.02 / -3.85 | 44.09 / +8.81 | 12.36 / -28.97 | 45.65 / +14.84 | |
| Tool Utilization OP / BwT | 35.18 / โ | 30.93 / -11.33 | 36.87 / -1.63 | 34.40 / +2.84 | 37.93 / +0.42 | |
| Web Search OP / BwT | 32.50 / โ | 31.50 / -9.89 | 33.50 / +4.80 | 29.50 / -6.65 | 39.50 / +6.91 | |
| Deepseek-v3.2-exp | Overall Performance (OP \(\uparrow\)) | 41.14 | 46.38 | 43.85 | 40.62 | 48.02 (โ3.54% rel.) |
| Backward Transfer (BwT \(\uparrow\)) | โ | -0.60 | -0.28 | -9.09 | +3.11 | |
| Qwen3-32b | Overall Performance (OP \(\uparrow\)) | 33.98 | 41.96 | 43.29 | 35.79 | 46.52 (โ7.46% rel.) |
| Backward Transfer (BwT \(\uparrow\)) | โ | -2.24 | -1.17 | -6.17 | +6.68 |
Ablation Study & Zero-Shot Cross-Task Transfer (Table 3 & Table 4)¶
| Evaluation Setting | Configuration | Metric / Accuracy | Comparative Implication |
|---|---|---|---|
| Component Ablation | ReAct Baseline (GPT-4o-mini / GPT-4.1) | 28.84 / 40.42 | Base zero-shot capabilities |
| (GPT-4o-mini Backbone) | + Trajectories Only (+trajs) | 42.84 | Establishes low-level action grounding |
| + Skills Only (+skills, via GPT-4o-mini / 4.1) | 36.16 / 37.61 | Macro-guidance without explicit syntax | |
| + Trajs + Skills (distilled via GPT-4o-mini) | 43.18 | Outperforms zero-shot GPT-4.1 (40.42) | |
| + Trajs + Skills (distilled via GPT-4.1) | 44.13 | Optimal synergistic configuration | |
| Zero-Shot Transfer: FEVER | ReAct / Synapse / ExpeL / AutoSkill | 0.40 / 0.38 / 0.41 / 0.49 | Unseen task in HotpotQA environment |
| (GPT-4o-mini Backbone) | LifeMem (Full) / LifeMem (w/o trajs) | 0.49 (+22.5%) / 0.47 (+17.5%) | Resists irrelevant API retrieval noise |
| Zero-Shot Transfer: \(\tau\)-bench Airline | ReAct / Synapse / ExpeL / AutoSkill | 0.32 / 0.26 / 0.26 / 0.34 | Unseen domain from retail training |
| (GPT-4o-mini Backbone) | LifeMem (Full) / LifeMem (w/o trajs) | 0.38 (+18.8%) / 0.38 (+18.8%) | High-level invariant skills drive transfer |
Key Findings¶
- Mitigation of Catastrophic Forgetting: In baseline methods, unorganized memory accumulation leads to severe negative backward transfer (Synapse scores BwT of -7.04 and AutoSkill drops to -8.54 on GPT-4o-mini). In contrast, LifeMem achieves consistently positive BwT across all backbones (+5.66 for GPT-4o-mini, +3.11 for Deepseek-v3.2-exp, and +6.68 for Qwen3-32b), demonstrating that structural workflow isolation allows subsequent learning to reinforce earlier knowledge rather than overwriting it.
- Greater Relative Gains on Smaller Models: LifeMem provides a +15.29 absolute gain on GPT-4o-mini (from 28.84 to 44.13), enabling the smaller model equipped with LifeMem to surpass the unaugmented performance of GPT-4.1 (40.42). While stronger models like Deepseek-v3.2-exp also improve (from 41.14 to 48.02), the larger relative boost on lightweight models indicates that structured experience reuse partially narrows capability gaps.
- Retrieval Engineering Fails as a Substitute for Abstraction: Upgrading Synapse with dense embeddings (
text-embedding-ada-002) only marginally improves OP from 39.18 to 40.55, while incorporating a cross-encoder re-ranker (BGE-reranker-v2-m3) degrades performance to 37.29 due to surface semantic bias disrupting multi-step execution coherence. This confirms that advanced retrieval alone cannot resolve action-space mismatch without explicit skill abstraction. - Inference Efficiency and High Workflow Purity: LifeMem consumes only 2.18k tokens on average during inference prompt construction (an increase of ~300 tokens over Synapse), whereas AutoSkill's unorganized skill concatenation inflates prompts to 4.68k tokens. Human evaluation demonstrates a 91% workflow purity across learned clusters, with cluster sizes naturally scaling according to task diversity (averaging 29 trajectories per cluster in repetitive WebShop vs. 12 in heterogeneous \(\tau\)-bench).
Highlights & Insights¶
- MRL Directional Regularization for Adaptive Splitting: Incorporating the Mean Resultant Length (MRL) from directional statistics enables the system to measure geometric trajectory dispersion on the unit hypersphere, triggering cluster division only when genuine behavioral divergence occurs rather than reacting to arbitrary sample volume.
- Decoupled Dual-Tier Prompting Architecture: The clean separation between environment-aligned concrete exemplars (for low-level action grammar compliance) and domain-invariant abstracted skills (for high-level reasoning and recovery) provides a robust blueprint for handling heterogeneous tool-use environments.
- Buffer-Based Stream Stabilization: For unpredictable real-world task streams, introducing a minimal task-specific FIFO buffer (size=5) recovers up to 64.9% of the performance drop observed under fully randomized streams, proving that light batching stabilizes streaming workflow clustering.
Limitations & Future Work¶
- Dependence on Advanced LLMs for Offline Distillation: While inference overhead is minimal, cluster splitting and skill abstraction rely on calls to GPT-4.1; deploying this framework across millions of trajectories in production will require migrating distillation to specialized on-device SLMs.
- Absence of Memory Pruning and Eviction Policies: LifeMem focuses on incremental expansion and cluster splitting, lacking mechanisms to prune obsolete workflows or compress redundant sub-clusters when memory storage reaches strict capacity limits.
- Topological Knowledge Graph Extension: Future work could explore interconnecting workflow clusters via a directed graph topology to represent causal transitions between distinct execution phases.
Related Work & Insights¶
- vs. Synapse (Zheng et al., 2024): Synapse relies exclusively on raw trajectory retrieval, causing severe action space mismatch when faced with new tasks (e.g., retrieving APIBank
ToolSearchercalls during FEVER verification). LifeMem enforces environment-aligned exemplar filtering and supplements it with abstract skills, avoiding cross-environment syntax crashes. - vs. ExpeL (Zhao et al., 2024a) & AutoSkill (Yang et al., 2026): ExpeL and AutoSkill perform global experience extraction over flat memory stores, allowing irrelevant domain trajectories to inject severe noise into the skill base. LifeMem performs localized distillation within homogeneous workflow clusters, preserving skill representativeness and keeping prompt tokens under 2.2k.
Rating¶
- Novelty: โญโญโญโญโญ Proposes a principled framework for inter-environment lifelong experience reuse combining workflow clustering, MRL directional diversity constraints, and local skill distillation.
- Experimental Thoroughness: โญโญโญโญโญ Comprehensive evaluation across 10 environments, 13k tasks, and 3 major LLM backbones, supported by rigorous ablation, zero-shot transfer, and human-annotated purity studies.
- Writing Quality: โญโญโญโญโญ Exceptionally clear narrative, elegant mathematical formulations, and rigorous error analysis with transparent case studies.
- Value: โญโญโญโญโญ Establishes a highly practical architectural foundation for mitigating catastrophic forgetting in long-horizon autonomous LLM agent deployments.