Skip to content

From Retrieval to Reconstruction: Constructing Evolvable Cognitive Memory for Long-Term Dialogue

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/Silent-Rain02/CogMem
Area: LLM Agent
Keywords: cognitive memory architecture, long-term dialogue, agentic active recall, memory consolidation, theory of mind

TL;DR

Addressing the critical flaw of semantic collapse between subjective claims and factual records in flat dialogue RAG, CogMem introduces a training-free cognitive architecture governed by the PEC2F graph schema, offline CLS-inspired memory consolidation, and ReAct-driven deterministic graph operators that reconstruct evidence across long conversational sessions.

Background & Motivation

The evolution of Large Language Models (LLMs) from stateless text generators toward autonomous personal agents has made persistent, consistent long-term memory an indispensable requirement. However, simply extending the context window of foundation models is computationally prohibitive and fails to emulate the structured, evolvable nature of human biographical memory. Over extended, multi-session interactions spanning weeks or months, an intelligent agent must not merely store past utterances as inert records, but actively reconstruct narrative timelines, trace shifting user preferences, and navigate the intricate social dynamics of multi-speaker environments where individual perspectives constantly diverge.

Current approaches to long-term dialogue memory predominantly adhere to the passive "store-and-retrieve" paradigm of Retrieval-Augmented Generation (RAG), vectorizing text chunks into dense, flat indices. This paradigm suffers from two fatal limitations. First, vector-based representations inherently induce what the authors term Semantic Collapse: in dense embedding spaces, an objective event record (e.g., "Jon lost his job in January") and a subjective opinion regarding that event (e.g., "Gina thinks Jon losing his job was a huge mistake") exhibit high lexical and semantic overlap, pulling their embeddings tightly together. Consequently, flat retrieval frequently returns an attributed opinion when queried for an objective record, contaminating the agent's knowledge store and fundamentally crippling its Theory-of-Mind (ToM) reasoning. A controlled probe on 150 matched factโ€“claim pairs demonstrates that BGE-M3 yields a high mean cosine similarity of 0.8231, causing standard flat RAG to misattribute speaker opinions as factual records in 42.7% of queries. Second, static retrieval pipelines lack agentic intentionality; rigid top-\(K\) passage matching or fixed-hop graph expansion cannot dynamically adjust search strategies based on intermediate findings, failing on complex multi-hop and temporal reasoning tasks.

To overcome these structural bottlenecks, CogMem draws inspiration from the Complementary Learning Systems (CLS) theory in cognitive science, which characterizes the continuous interplay between fast episodic learning in the hippocampus and gradual semantic consolidation in the neocortex. An effective conversational agent requires an architecture that can simultaneously preserve granular, source-attributed episodic traces and synthesize compact, generalized semantic knowledge without losing provenance. Core idea: construct an evolvable cognitive memory architecture based on the PEC2F (Person-Event-Concept-Claim-Fact) schema, structurally isolating subjective claims from objective records, employing offline dual-resolution memory consolidation with evidence mounting, and empowering a ReAct-based Cognitive Search Agent to reconstruct query-adaptive evidence chains via deterministic graph operators.

Method

Overall Architecture

The CogMem framework is organized into two primary layers: Memory Generation & Evolution (offline/incremental memory lifecycle) and Cognitive Reasoning & Application (online query answering). During ongoing dialogue, an Atomic Extraction Engine continuously parses raw conversational turns into structured entities and relationships within the PEC2F graph, preserving source turn IDs and timestamps. Periodically, an offline consolidation mechanism synthesizes recurring episodic event clusters into generalized Fact nodes while anchoring them back to raw traces via SUPPORTED_BY provenance edges, simultaneously reconciling temporally conflicting claims from identical speakers. When a user poses a question, a Cognitive Search Agent driven by the ReAct loop classifies the high-level query intent and dynamically sequences four deterministic graph operatorsโ€”Anchoring, Traversal, Intersection, and Evidence Groundingโ€”navigating the cognitive graph to reconstruct an evidence-grounded answer.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}}%%
flowchart TD
    A["Raw Multi-Turn Dialogue Stream"] --> B["Stage 1: PEC2F Cognitive Graph Schema<br/>Atomic extraction separating Person/Event/Concept/Claim/Fact"]
    B --> C["Stage 2: Dynamic Stability via Consolidation<br/>Event abstraction to Fact + Evidence mounting provenance"]
    C --> D["User Query Input"]
    subgraph S["Stage 3: Active Plasticity Agentic Recall"]
        direction TB
        E["LLM Intent Parsing & Operator Routing"] --> F1["Oanchor Hybrid Scoring"]
        E --> F2["Otraverse Spatiotemporal Traversal"]
        E --> F3["Ointersect Neighborhood Intersection"]
        E --> F4["Oground Raw Evidence Grounding"]
    end
    D --> S
    S --> G["Dynamic Reconstructed Evidence Subgraph"]
    G --> H["Grounded, Coherent Final Response"]

Key Designs

1. PEC2F Cognitive Graph Schema: Epistemic separation of subjective claims from objective records to eliminate semantic collapse

To resolve semantic collapse where dense vector embeddings fail to distinguish facts from opinions, CogMem defines a typed person-centric graph schema \(G = (V, E)\) comprising five distinct node types: Person (\(V_P\)), Event (\(V_E\)), Concept (\(V_C\)), Claim (\(V_{\text{Claim}}\)), and Fact (\(V_F\)). While Person and Concept nodes serve as stable semantic anchors, and Event/Fact nodes capture episodic experiences and stable personal attributes respectively, subjective opinions are explicitly mediated through intermediate Claim nodes. Rather than directly assigning an attribute edge between persons, an attributed belief (e.g., "Jon thinks Gina is lazy") is encoded as a two-hop path: $\((p_{\text{jon}}, \text{CLAIMS}, n_{\text{claim}}) \in E \quad \wedge \quad (n_{\text{claim}}, \text{ABOUT}, p_{\text{gina}}) \in E\)$ where \(n_{\text{claim}} \in V_{\text{Claim}}\) encapsulates the subjective statement along with its sentiment polarity. This structural separation ensures that even if a dense retriever finds high lexical similarity between a claim and an event, the retrieval controller strictly segregates them by node type, querying Claim nodes only when the user query explicitly demands ToM or perspective analysis.

2. Dynamic Stability via Consolidation: Dual-resolution evidence mounting and speaker-isolated claim reconciliation

Unchecked accumulation of fine-grained dialogue events causes exponential graph sprawl and slows down online traversal. Grounded in CLS theory, CogMem introduces an offline consolidation engine that clusters episodic nodes \(S_{p,c} = \{e_1, \dots, e_k\} \subseteq V_E\) sharing person \(p\) and concept \(c\) within a rolling time window \(\Delta t_{\max} = 30\) days. Whenever a cluster exceeds threshold \(|S_{p,c}| \ge \tau\) (default \(\tau = 3\)), an LLM abstraction function \(\Psi\) transforms the episodic cluster into a high-level Fact node: $\(f_{\text{new}}, T_{\text{env}} \leftarrow \Psi(S_{p,c})\)$ where \(T_{\text{env}} = [t_{\text{start}}, t_{\text{end}}]\) denotes the computed temporal envelope. To prevent irreversible loss of granular context during abstraction, CogMem establishes backward provenance edges via an Evidence Mounting strategy: $\(E_{\text{prov}} = \{(f_{\text{new}}, \text{SUPPORTED\_BY}, e_i) \mid \forall e_i \in S_{p,c}\}\)$ This creates a dual-resolution memory hierarchy: broad inquiries access \(f_{\text{new}}\) directly for fast semantic shortcuts, while detail-oriented questions trace SUPPORTED_BY links down to raw episodic instances. For belief updates, claims from different speakers are never merged to respect subjective disagreement; only conflicting claims originating from the same canonical source, target, and predicate are temporally sequenced into CURRENT, SUPERSEDED, or CONFLICTING views.

3. Active Plasticity via Agentic Recall: Intent-guided composition of deterministic graph operators

Instead of static, heuristic k-hop expansions, CogMem implements an active Cognitive Search Agent governed by a ReAct loop. The agent maps natural-language queries to deterministic graph operators. The workflow always initiates with the Anchoring operator \(O_{\text{anchor}}\), which retrieves the top \(K_{\text{anchor}} = 15\) seed nodes via hybrid lexical-dense scoring: $\(S(v, q) = \alpha \cdot I_{\text{lex}}(v, q) + (1-\alpha) \cdot \cos(\Phi(v), \Phi(q))\)$ with \(\alpha = 0.3\) and BGE-M3 embeddings \(\Phi\). Following anchoring, the agent executes up to \(D_{\text{step}} = 5\) post-anchoring steps driven by LLM intent parsing: comparison queries (COMPARE, COMMON) invoke the Intersection operator \(O_{\text{intersect}}\) over semantic relations \(R_{\text{sem}}\) to extract shared concept subgraphs \(G_{\text{common}} = \bigcap_{i=1}^n \bigcup_{r \in R_{\text{sem}}} N_r(p_i)\); perspective queries (BELIEF, OPINION) trigger the Spatiotemporal Traversal operator \(O_{\text{traverse}}\) filtered by \(R_{\text{claim}}\); temporal queries apply dynamic temporal gating \(T\) during traversal; and verification queries (WHY, VERIFY) trigger the Evidence Grounding operator \(O_{\text{ground}}\) to retrieve raw conversational utterances \(d \in D\). If early evidence satisfies the query (Direct Hit), the agent halts immediately, optimizing token consumption and execution latency.

Loss & Training

CogMem is an entirely training-free, non-parametric cognitive architecture. It requires no model parameter fine-tuning, operating via structured prompting across ingestion, consolidation, and agentic retrieval (detailed prompt specifications are provided in Appendix H of the paper). Experiments validate CogMem using closed-source GPT-4o-mini and locally deployed Qwen2.5-14B-Instruct via vLLM, coupled with 1024-dimensional BGE-M3 embeddings. Generation temperature is strictly set to 0.0 to eliminate stochastic variance and ensure deterministic operator routing.

Key Experimental Results

Main Results

Evaluated on LoCoMo (~1.5K question-answer pairs over long, multi-session dialogues) and LongMemEval (500 cross-session consistency queries), CogMem outperforms standard RAG, dedicated agentic memory baselines (Mem0, A-MEM, LightMem, GAM), heuristic graph retrievers (LightRAG, HippoRAG), and learned graph traversal (RoG).

Benchmark / Category Metric CogMem (GPT-4o-mini) CogMem (Qwen2.5-14B) Best Competing Baseline Performance Margin
LoCoMo Single Hop F1 / BLEU-1 59.20 / 56.40 60.40 / 55.80 58.93 / 53.74 (GAM, Qwen) +1.47 F1 / +2.06 BLEU-1
LoCoMo Multi Hop F1 / BLEU-1 50.70 / 46.10 48.42 / 44.65 42.96 / 34.48 (GAM, Qwen) +5.46 F1 / +10.17 BLEU-1
LoCoMo Temporal F1 / BLEU-1 63.80 / 57.90 56.20 / 49.10 51.52 / 44.43 (GAM, Qwen) +4.68 F1 / +4.67 BLEU-1
LoCoMo Open Domain F1 / BLEU-1 56.10 / 51.20 55.80 / 50.10 54.03 / 48.65 (LightRAG, Qwen) +1.77 F1 / +1.45 BLEU-1
LongMemEval Overall Accuracy (%) 68.40 66.50 65.20 (A-MEM, Qwen) +1.30% to +4.11% Acc

In a blind human preference study on 50 randomly sampled LoCoMo Multi-Hop and Temporal queries against the strongest baseline GAM, CogMem achieved a 68% win rate, 20% tie rate, and only a 12% loss rate.

Ablation Study

Ablations on Qwen2.5-14B isolate the relative contributions of the agentic ReAct loop, the epistemic Claim nodes, and the offline memory consolidation module.

System Configuration LoCoMo Multi-Hop F1 (%) LongMemEval Knowledge Update Acc (%) Architectural Contribution
CogMem (Full Model) 48.42 73.08 Complete cognitive architecture with synergistic components
w/o Consolidation 43.56 (-4.86) 66.67 (-6.41) Traverses raw episodic events without semantic shortcuts
w/o Claim (Flat Graph) 41.20 (-7.22) 56.67 (-16.41) Flattens claims into facts, suffering from severe semantic collapse
w/o Agent (Fixed Flow) 22.81 (-25.61) 48.10 (-24.98) Rigid 3-step pipeline (anchor \(\rightarrow\) 2-hop \(\rightarrow\) ground) fails on multi-hop

An isolated analysis of memory consolidation shows that synthesizing 497 Fact nodes and adding 4,708 SUPPORTED_BY edges improves Multi-Hop F1 from 43.56% to 48.42% while reducing average reasoning steps from 3.42 to 2.67 (a 21.9% reduction in inference calls).

Key Findings

  • Agentic operator routing is critical: Disabling the ReAct search agent in favor of a static three-step heuristic pipeline collapses Multi-Hop F1 by 25.61 percentage points (48.42% \(\rightarrow\) 22.81%), and overall weighted F1 drops by 65.4%, demonstrating that static traversal cannot navigate complex multi-session graphs.
  • Claim nodes eliminate semantic collapse: In the dedicated 150-pair probe, flat RAG misattributed speaker opinions as factual records in 42.7% of cases due to dense embedding overlap (\(0.8231\) mean cosine similarity), whereas CogMem reduced this failure rate to 6.0%. Ablating Claim nodes degrades Knowledge Update accuracy on LongMemEval by 16.41 percentage points.
  • Consolidation trades offline indexing for online agility: Offline consolidation adds 14.8% more nodes and 47.4% more edges via additive evidence mounting, but this investment cuts online multi-hop retrieval steps by 21.9% while boosting F1 by 4.86 points, validating the role of synthesized Fact nodes as cognitive shortcuts.

Highlights & Insights

  • Topological decoupling of Theory of Mind: Introducing intermediate Claim nodes solves the long-standing semantic collapse dilemma in dense vector retrieval, enabling language agents to maintain precise, source-attributed belief representations.
  • CLS-inspired dual-resolution evidence mounting: Emulating hippocampal-neocortical consolidation, the architecture abstracts volatile events into stable facts while anchoring them back to raw utterances via SUPPORTED_BY edges, providing high-level efficiency without forfeiting auditability.
  • Algebraic operator navigation: By casting retrieval as an intent-routed sequence of deterministic set-theoretic and graph operations (Anchoring, Traversal, Intersection, Grounding), CogMem delivers fully interpretable and inspectable reasoning trajectories.

Limitations & Future Work

  • Authors' stated limitations: Consolidation currently runs offline in batch mode (~537.85s for LoCoMo), lacking real-time stream consolidation; the cognitive schema focuses primarily on social interactions, omitting dynamic user goals or affective emotion vectors.
  • Empirical error bottlenecks: A granular root-cause analysis of 100 LoCoMo failure cases reveals that temporal normalization errors (e.g., failing to resolve relative terms like "last year" to absolute calendar dates) account for 44% of temporal query failures, while embedding semantic drift causes 28% of single-hop errors.
  • Future directions: Developing neural tool learning to compose novel domain-specific retrieval primitives dynamically, implementing stream-based incremental consolidation, and integrating hybrid lexical-dense fallbacks to mitigate vector drift.
  • vs Standard RAG / Dense Retrieval: Conventional chunk-based RAG conflates objective facts with subjective viewpoints in dense vector space; CogMem enforces structural epistemic boundaries via the PEC2F schema.
  • vs Heuristic Graph RAG (LightRAG, HippoRAG): While LightRAG and HippoRAG construct entity-relation graphs or run Personalized PageRank, they lack source-attributed claim distinctions and cannot dynamically reconcile evolving temporal beliefs; CogMem provides intentional, operator-guided reasoning.
  • vs Agentic Memory Systems (Mem0, A-MEM, GAM): Unlike flat memory banks or unanchored text summaries, CogMem combines biologically inspired memory consolidation with dual-resolution evidence mounting, achieving superior scalability and zero-shot multi-hop reasoning.

Rating

  • Novelty: โญโญโญโญโญ Formulates the semantic collapse dilemma in conversational RAG and proposes an elegant cognitive graph schema with ToM separation and CLS consolidation.
  • Experimental Thoroughness: โญโญโญโญโญ Rigorous validation across LoCoMo and LongMemEval, extensive cross-backbone comparisons, blind human evaluations, sensitivity curves, and a quantitative 100-case error taxonomy.
  • Writing Quality: โญโญโญโญโญ Exceptionally clear framing, rigorous conceptual distinctions, and transparent documentation of architecture, pseudocode, and hyper-parameters.
  • Value: โญโญโญโญโญ Establishes a foundational, training-free architectural blueprint for building auditable, ToM-aware, and temporally consistent memory systems for autonomous LLM agents.