Skip to content

A retrieval conditioned rebinding circuit for dynamic entity tracking in large language models

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/sori424/dynamicET
Area: LLM Reasoning
Keywords: Dynamic entity tracking, Rebinding circuit, Causal intervention, Mechanistic interpretability, Binding identifier

TL;DR

Using causal mediation analysis, path patching, and subspace interventions, this work proves that LLMs do not eagerly maintain a globally updated situation model during dynamic state transitions; instead, they rely on a compact "retrieval-conditioned rebinding circuit" (comprising 3.9%–10.8% of attention heads) that defers binding resolution to query time via dereferencing pointers, with Gemma models exhibiting symmetric Q/K matching and Llama models being predominantly key-dominated.

Background & Motivation

Accurately maintaining entity-specific properties and dynamically updating their relational states over extended contexts is an indispensable prerequisite for long-context language understanding and complex multi-step reasoning. Classical cognitive theories of language comprehension have long postulated that cognitive agents incrementally construct and update an internal mental simulation or "situation model" of the described world state as discourse unfolds. Recent breakthroughs in mechanistic interpretability demonstrate that Transformer-based large language models represent symbolic relations by binding entities to abstract binding identifiers within intermediate activation spaces. However, the vast majority of prior investigations remain confined to static in-context associations or synthetic setups lacking explicit state transitions, leaving a substantial theoretical void regarding the precise neural computation executed by models during naturalistic state updates.

This theoretical gap centers on the fundamental tension between two mutually exclusive computational accounts: the Global State Update hypothesis versus the Retrieval-Conditioned Rebinding hypothesis. Under the Global State Update account, the model immediately reconstructs a comprehensive, post-transition world representation immediately upon encountering an action instruction (e.g., swapping items between two containers), allowing subsequent queries to simply read off the referenced entity from this eagerly updated internal state. Conversely, the Retrieval-Conditioned Rebinding hypothesis posits that maintaining full situation updates eagerly is computationally inefficient; instead, the model preserves compact transition information and only resolves the necessary bindings lazily upon encountering the final query, dynamically remapping abstract binding identifiers through designated pointer circuits to retrieve target payloads directly from their original context tokens.

To formally adjudicate between these conflicting paradigms, this paper devises an end-to-end mechanistic diagnostic framework combining causal mediation analysis, greedy path patching, and query/key (Q/K) subspace interventions. Evaluated systematically across four instruction-tuned open-source models (Gemma2-9B, Gemma3-12B, Llama3.2-3B, and Llama3.1-8B), the study investigates whether dynamic state tracking is universal across architectures and how its internal representational signatures vary across distinct model families. Core idea: Large language models handle dynamic state tracking not through eager global state simulation, but via a highly sparse "retrieval-conditioned rebinding circuit" that dynamically redirects pointer dereferencing heads at query time to fetch original contextual payloads, revealing distinct structural implementations between symmetric Q/K matching in Gemma and key-dominated routing in Llama.

Method

Overall Architecture

The analytical framework advances across multiple complementary levels of mechanistic resolution: macro-level token information flow routing, meso-level attention head circuit pruning, and micro-level geometric comparison in linear Q/K subspaces. Given a standard entity tracking sequence Box R contains rabbit. Box S contains sock. Box T contains toy. Swap items of Box S and Box R. Question: Which item does Box R contain? Answer: [:], the framework first applies interchange interventions across critical token roles to identify where causal information mediates final token generation. Next, a five-stage greedy path patching protocol disentangles query-mediated (Q) and value-mediated (V) composition, isolating a compact subnetwork of functional attention heads. The isolated heads are subsequently subjected to pointer versus content control interventions to confirm their specific role in memory addressing, followed by linear direction projections in the query and key subspaces to dissect the underlying algebraic matching mechanism.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Sequence: Initial Context + Swap Operation + Question"] --> B["Causal Mediation Analysis & Rebinding Hypotheses<br/>Token-level residual stream interventions dissociate eager update vs lazy rebinding"]
    B --> C["Path Patching & Five-Stage Rebinding Attention Circuit<br/>Isolating Q/V composition and pruning attention heads to a minimal functional subnetwork"]
    C --> D["Role-Specific Head Interventions & Dereferencer Pointer Routing<br/>Content control vs pointer interventions confirm address routing in Group B"]
    D --> E["Q/K Subspace Binding Identifier Interventions & Cross-Family Signatures<br/>Mean binding direction probes reveal Gemma Q/K symmetry vs Llama key dominance"]
    E --> F["Output Prediction: Correctly attend to original context payload token"]

Key Designs

1. Causal Mediation Analysis & Rebinding Hypotheses: Dissociating Eager Global Updates from Lazy Local Rebinding

To ascertain whether state transitions are instantiated eagerly or evaluated lazily, this design deploys interchange interventions across three functionally distinct token categories: the reference container (Box R), the swap target container (Box S), and the target payload object (sock). By transplanting hidden state residual activations from a clean input \(x_{\text{orig}}\) into an aligned counterfactual input \(x_{\text{counter}}\), the Interchange Intervention Accuracy (IIA) is measured across layers. The empirical evidence demonstrates that causal effects for the reference and swap target boxes do not propagate forward from their initial contextual occurrences; instead, they abruptly emerge only after the Swap sentence and travel forward toward the answer prediction position. Crucially, the target object's causal activation flows directly from its original contextual mention in deeper layers to the final output, completely bypassing intermediate updated container tokens. This dissociation decisively refutes the eager global update account, validating the retrieval-conditioned rebinding model wherein the Swap instruction acts as a lightweight deferred operator resolved at the final : token.

2. Path Patching & Five-Stage Rebinding Attention Circuit: Causal Pruning from Coarse Routes to Sparse Heads

To localize the precise computational graph at attention-head granularity, a five-stage greedy path patching protocol is established, strictly distinguishing two types of sender-receiver head interaction: V-patching (replacing value inputs to verify payload transmission from attended source positions) and Q-patching (replacing query inputs to verify whether upstream heads shape downstream attention distributions). The functional candidate roles are structured into five stages: - Group E (Binding Anchor): Writes initial box identity and binding identifier information into the residual stream at contextual container positions; - Group D (Swap Position Transmitter): Reads Group E representations via V-composition and propagates them across the swap transition span; - Group C (Position Updater): Receives swap signals via Q-composition to modulate attention toward the updated antecedent container tokens; - Group B (Dereferencer): Reads the resolved binding identifier via V-composition, operating as an address pointer; - Group A (Answer Retriever): Located at the terminal query token, guided by Group B via Q-composition to attend directly to the correct object payload in the early context, driving correct logit generation.

Following five-stage scoring over 100 prompts, heads are pruned using a rigorous contribution-based criterion: \(\frac{F(\text{Cir}\setminus K) - F(\text{Cir}\setminus(K\cup\{v\}))}{F(\text{Cir}\setminus(K\cup\{v\}))}\). Any attention head exhibiting a marginal contribution below 1% is eliminated.

3. Role-Specific Head Interventions & Dereferencer Pointer Routing: Decoupling Address Selection from Content Transfer

To rigorously demonstrate that Group B attention heads act strictly as address pointers rather than data buses transferring lexical contents, this design introduces two orthogonal intervention regimes: - Content Control Intervention: Holds container identifiers and swap structure fixed while altering only the underlying object tokens (e.g., swapping sock with egg). Because the retrieval address remains identical, patching the attention pattern alone should yield no change in counterfactual object logits (\(\Delta\text{logit}(\text{egg}) \approx 0\)); - Pointer Intervention: Modifies the binding topology in the counterfactual run (e.g., swapping container roles such that counterfactual retrieval requires fetching from a different source position). Patching the counterfactual attention pattern into the original run tests whether the head redirects attention toward the object residing at the counterfactual source position in the original text (i.e., rabbit), expecting \(\Delta\text{logit}(\text{rabbit}) > 0\).

In Gemma-9B, Group B exhibits an unequivocal double dissociation: pointer intervention drives a dramatic surge of \(\Delta\text{logit}(\text{rabbit}) = 6.79\), while content control produces a negligible shift of \(\Delta\text{logit}(\text{egg}) = 0.26\). This confirms that Group B exclusively governs memory dereferencing without copying lexical content.

4. Q/K Subspace Binding Identifier Interventions & Cross-Family Signatures: Probing Algebraic Alignment and Key Dominance

To probe how Group B heads execute address pointer matching at the dot-product level, mean binding identifier directions are estimated across \(N=100\) randomized prompts to average out lexical semantics: $\(\Delta Q_{\ell,h}(i \to j) = \mu_{Q,\ell,h}(j) - \mu_{Q,\ell,h}(i), \quad \Delta K_{\ell,h}(i \to j) = \mu_{K,\ell,h}(j) - \mu_{K,\ell,h}(i)\)$ Three controlled intervention protocols are enacted on Group B: 1. Query-side intervention: Shifts the query vector at the : position from binding \(i\) to \(j\), checking if requesting a counterfactual identifier redirects attention; 2. Key-side intervention: Leaves queries untouched while shifting key vectors at context positions; 3. Diagnostic Q+K cancellation intervention: Simultaneously translates the query at : and the keys at original positions along identical directional vectors \(i \to j\). If the attention head relies on geometric dot-product compatibility \(\text{softmax}(q^\top k / \sqrt{d})\), translating both subspaces in unison will preserve relative alignment and completely cancel out redirection.

This design uncovers striking architectural divergence: Gemma architectures demonstrate crisp Q/K geometric matching where Q+K joint shifts cancel redirection back to baseline (\(\Delta R \approx 0\)), whereas Llama architectures are heavily key-dominated, wherein key-side manipulations dictate behavior, query shifts remain ineffective, and Q+K shifts fail to cancel.

Loss & Training

This work represents a training-free mechanistic investigation conducted on frozen pre-trained and instruction-tuned language models. All experiments were executed on an 80GB NVIDIA H100 PCIe GPU. The formal evaluation metrics include: - Candidate Accuracy \(F\): The fraction of instances where the target gold entity \(O_k\) attains the top logit among candidate choices; - Change in Log Attention Ratio \(\Delta R\): Quantifies the net shift in attention mass from original binding \(i\) to counterfactual binding \(j\): $\(\Delta R = \left[\log(M_j + \epsilon) - \log(M_i + \epsilon)\right]_{\text{patched}} - \left[\log(M_j + \epsilon) - \log(M_i + \epsilon)\right]_{\text{original}}, \quad M_k = \sum_{s \in S_k} A_{\ell,h}(t_{:}, s)\)$ - Output Logit Difference Shift \(\Delta\text{logit}\): The change in final logit margin between counterfactual and original tokens: $\(\Delta\text{logit} = [z(O_j) - z(O_i)]_{\text{patched}} - [z(O_j) - z(O_i)]_{\text{original}}\)$ - Switch Fraction: The empirical frequency with which an intervention causes the argmax attended context binding to switch from \(i\) to \(j\).

Key Experimental Results

Main Results

The table below contrasts the performance of the full model \(M\), the pruned rebinding circuit \(\text{Cir}\), and a matched-size random head baseline \(\text{Rand}\) across \(N=300\) held-out evaluation instances:

Model Full Model Acc \(F(M)\) Circuit Acc \(F(\text{Cir})\) Random Baseline \(F(\text{Rand})\) Heads Retained / Total Circuit Ratio (%) Circuit Margin (Cir) Random Margin (Rand)
Gemma2-9B 1.00 0.89 0.34 38 / 672 5.65% 2.76 (Full: 8.87) -0.57
Gemma3-12B 1.00 0.91 0.38 83 / 768 10.81% 14.10 (Full: 17.83) -0.78
Llama3.2-3B 0.71 0.71 0.34 42 / 672 6.25% 2.32 (Full: 0.84) -0.80
Llama3.1-8B 0.99 0.84 0.38 40 / 1024 3.91% 3.70 (Full: 4.72) -0.52

Across all four architectures, isolating merely 3.9%–10.8% of the attention heads recovers 85%–100% of full-model task accuracy, whereas random head subsets collapse to 0.34–0.38, demonstrating extreme architectural localization.

Ablation & Subspace Interventions

The table below summarizes the causal impact of Q/K binding identifier interventions applied to Group B dereferencer heads across all models:

Model Intervention Condition \(\Delta R\) (Log Attention Ratio) \(\Delta\text{logit}\) (Output Margin) Switch Fraction Mechanistic Interpretation
Gemma2-9B Random Direction Control -0.28 0.48 0.06 Ineffective baseline vector
Query Intervention (Q int.) 2.66 4.25 0.47 Strongly redirects pointer attention
Key Intervention (box+obj) 2.08 3.51 0.31 Keys encode binding identifier
Key Intervention (box only) 1.92 0.04 0.42 Shifts attention without shifting logit
Key Intervention (obj only) 2.25 4.35 0.35 Object keys directly drive output readout
Q+K Diagnostic Intervention -0.12 1.57 0.12 Full cancellation confirms Q/K symmetry
Gemma3-12B Random Direction Control -0.12 -0.69 0.08 Ineffective baseline vector
Query Intervention (Q int.) 0.26 0.41 0.04 Query-driven redirection
Key Intervention (box+obj) 0.29 0.10 0.14 Key-assisted addressing
Key Intervention (box only) -0.33 -0.05 0.12 Modulates attention slots
Key Intervention (obj only) 0.87 0.23 0.18 Drives payload readout
Q+K Diagnostic Intervention -0.04 -0.49 0.08 Full cancellation confirms matching
Llama3.2-3B Random Direction Control -0.10 0.21 0.05 Ineffective baseline vector
Query Intervention (Q int.) -0.46 0.15 0.01 Query side is non-functional
Key Intervention (box+obj) 1.32 1.03 0.30 Keys dominantly guide routing
Key Intervention (box only) 2.23 0.06 0.37 Modulates addressing without logit shift
Key Intervention (obj only) 0.69 0.15 0.29 Directs payload readout
Q+K Diagnostic Intervention 1.69 0.95 0.34 No cancellation; key-dominated routing
Llama3.1-8B Random Direction Control 0.17 1.02 0.12 Ineffective baseline vector
Query Intervention (Q int.) 0.18 0.83 0.06 Weak query-side effect
Key Intervention (box+obj) 1.20 2.92 0.36 Keys dominantly control behavior
Key Intervention (box only) 1.59 -0.02 0.40 Shifts attention without shifting logit
Key Intervention (obj only) 0.78 2.70 0.26 Object keys govern answer selection
Q+K Diagnostic Intervention 0.98 1.05 0.27 No cancellation; key-dominated routing

Key Findings

  • High Circuit Sparsity & Group E Redundancy: Rebinding is mediated by a micro-circuit of fewer than 85 heads. Following pruning, Group E (binding anchor heads) is entirely discarded in Gemma-9B, indicating that initial binding identifiers are encoded during context formation and preserved within residual stream states without requiring active maintenance heads.
  • Addressing versus Readout Dissociation: Granular K-side interventions reveal an elegant division of labor: shifting keys solely at container (box) tokens drastically shifts attention ratios (\(\Delta R = 1.92 \sim 2.23\)) but leaves output logits unaffected (\(\Delta\text{logit} \le 0.06\)). Conversely, shifting keys at object tokens immediately triggers large behavioral logit shifts (up to 4.35). Thus, container tokens serve as structural addressing slots, while object tokens carry the actual readout payload.
  • Profound Cross-Family Mechanistic Divergence: While both architectures implement retrieval-conditioned rebinding functionally, their underlying geometric realizations diverge sharply. Gemma models rely on symmetric, linear Q/K dot-product matching that cancels under joint displacement. Llama models store binding identities almost entirely within key subspaces, rendering query-side perturbations ineffective and preventing Q+K cancellation.
  • Robustness Across Complex Scaling: Experiments scaling up to 5 containers with distractor swap operations (Appendix Figures 8 & 9) confirm identical behavior: prefix activations at the swap point encode only local modifications, whereas full rebinding consistently resolves at the terminal retrieval colon.

Highlights & Insights

  • Computational Parsimony via Lazy Evaluation: Rather than continuously updating a full situation model in memory at every state change—a costly procedure for long sequences—Transformers implement a computational analogue of lazy evaluation, propagating compact relational operators and only executing dereferencing upon query.
  • Clean Decoupling of Pointers from Payloads: Dereferencing heads (Group B) function purely as address pointers that steer downstream attention without carrying lexical semantics, mirroring indirect memory addressing in computer architectures.
  • Implications for Dynamic In-Context Knowledge Editing: Unlike parameter-modifying editing techniques (e.g., ROME, MEMIT) that alter underlying weight matrices, LLMs inherently handle dynamic state updates by altering pointer routes rather than overwriting original contextual bindings. This indicates that non-destructive in-context editing methods can be developed by manipulating attention routing heads.

Limitations & Future Work

  • Constrained Benchmark Synthetic Setup: The current setup uses controlled synthetic templates with alphabetic container names and single-token objects under swap operations. Real-world narrative discourse involves multi-token entities, probabilistic state transitions, and varied verbs (e.g., put, take, hide).
  • Linear Representation Assumption: Interventions rely on difference-in-means directional probes in Q/K subspaces. If binding identities are stored nonlinearly or distributed across complex manifold curvatures, linear interventions may underestimate the circuit's full causal power.
  • Omission of Backup Redundancies: The greedy pruning protocol discards heads with \(<1\%\) contribution, potentially overlooking dormant backup circuits that activate when primary pathways are perturbed.
  • vs. Feng & Steinhardt (2024): While Feng & Steinhardt established that LLMs use linear binding IDs to associate entities in static contexts, this paper extends the framework to explicit dynamic transitions, proving that binding IDs can be dynamically remapped via pointer routing circuits.
  • vs. Prakash et al. (2024, 2025): Complements prior work on lookback mechanisms for belief tracking by establishing a comprehensive five-stage circuit (Swap Transmitter \(\to\) Position Updater \(\to\) Dereferencer \(\to\) Retriever) explicitly responsible for state mutation.
  • vs. Tang et al. (2026): Provides direct circuit-level and causal intervention evidence supporting Tang et al.'s hypothesis that language models aggregate information in parallel upon query arrival rather than tracking state transitions step-by-step.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ [Pioneering identification and formalization of retrieval-conditioned rebinding circuits under dynamic state updates]
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive causal mediation, path patching, pointer decoupling, and subspace interventions evaluated across four models across two architectures]
  • Writing Quality: ⭐⭐⭐⭐⭐ [Rigorous experimental logic, precise formalization of hypotheses, and outstanding clarity in causal diagramming]
  • Value: ⭐⭐⭐⭐⭐ [Provides fundamental circuit-level insights into LLM state tracking, symbolic binding, and lightweight in-context knowledge editing]