LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/jins7/LatentEvolve
Area: Large Language Model Reasoning
Keywords: Test-Time Scaling, Latent Space Reasoning, Complementary Learning Systems, Self-Evolving LLMs, Experience Replay
TL;DR¶
Inspired by Complementary Learning Systems (CLS) theory, LatentEvolve establishes a dual-phase latent test-time scaling framework that alternates between fast daytime episodic retrieval and slow nighttime parametric memory consolidation, empowering frozen large language models to continuously learn how to scale test-time compute in a fully unsupervised manner.
Background & Motivation¶
Large Language Models (LLMs) have achieved remarkable breakthroughs across complex problem-solving domains including mathematical reasoning, scientific discovery, natural language understanding, and embodied decision-making. Historically, these advances have predominantly been driven by training-time scaling laws, where increasing parameter counts and pre-training corpus volumes consistently yield predictable performance gains. However, this classical scaling paradigm is encountering diminishing returns due to prohibitive training costs and the looming exhaustion of high-quality human-annotated data. Consequently, significant research attention has pivoted toward test-time scaling (TTS), which seeks to unlock the latent potential of frozen foundation models by allocating additional computational expenditure dynamically during the inference phase.
Current test-time scaling paradigms span multiple computational dimensions, ranging from explicit natural-language techniques—such as parallel repeated sampling with majority voting, multi-agent debates, and sequential self-refine feedback loops—to continuous latent-space interventions that steer token representations (e.g., SoftCoT and LatentSeek). Despite their diverse formulations, nearly all established TTS methodologies suffer from a fundamental cognitive limitation: inter-task independence. Under prevailing setups, the test-time computations executed for distinct queries are treated as entirely isolated, memoryless episodes. Once a problem is resolved, the search paths explored, the reflective insights gathered, and the directional latent optimization steps discovered are permanently discarded, never informing future reasoning. This inability to accumulate experiential knowledge fundamentally prevents language models from progressively improving their test-time scaling capabilities through ongoing environmental interaction.
To resolve this critical barrier, the authors take inspiration from the Complementary Learning Systems (CLS) theory in cognitive neuroscience. Human intelligence relies on the coordinated interplay between two complementary memory systems: the fast-learning hippocampus, which rapidly encodes specific episodic memory traces during active wakefulness, and the slow-learning neocortex, which gradually consolidates and distills accumulated experiences into structured procedural knowledge during sleep. Mapping this biological mechanism to continuous representation spaces provides an elegant solution for lifelong inference adaptation. Core idea: propose LatentEvolve, a biologically inspired dual-phase latent test-time scaling framework that couples daytime episodic momentum transfer for fast instance-level adaptation with nighttime parametric replay for slow procedural knowledge consolidation, enabling LLMs to self-evolve test-time compute without external supervision.
Method¶
Overall Architecture¶
LatentEvolve reframes the inference pipeline of frozen LLMs into an interleaved, recurrent cycle of daytime exploration and nighttime consolidation. Operating directly on continuous latent prefixes prepended to token sequences, the system modulates model activations without altering the frozen backbone parameters. When a new query arrives, the model derives an initial base latent prefix via greedy decoding; if prior nighttime cycles have occurred, a compact auxiliary model termed the "latent weaver" provides an informed prior transformation. During the daytime phase, an episodic buffer retrieves top-\(k\) semantically relevant historical experiences, and their directional optimization shifts are transferred to the query via weighted momentum transfer. The initialized latent sequence then undergoes iterative self-supervised policy gradient refinement driven by multi-criteria self-evaluation. Periodically (e.g., every \(T=200\) test queries), the system transitions into the nighttime phase, training the latent weaver on accumulated high-confidence latent trajectories through parametric regression, effectively distilling episodic wisdom into a permanent procedural prior for subsequent tasks.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Streaming Query ci<br/>Input Context & Prompt"] --> B["Latent Weaver Prior Refinement<br/>Parametric Neural Prior Transformation"]
B --> C["Associative Retrieval & Momentum Transfer<br/>Episodic Buffer Search & Vector Shift"]
C --> D["Self-Supervised Policy Gradient Refinement<br/>Multi-Criteria Self-Reward Ascent"]
D --> E["High-Confidence Episodic Archiving<br/>Quality-Gated Memory Buffer Retention"]
E -->|Periodic Trigger Every T Queries| F["Nighttime Experience Replay Consolidation<br/>Latent Weaver Replay Optimization"]
F -.->|Distilled Weaver Weights W_psi| B
Key Designs¶
1. Latent Weaver Prior Refinement: overcoming the blind cold-start exploration of unguided base representations
In conventional latent reasoning frameworks, the initial intervention vector is directly extracted from the first \(L'\) hidden representations generated during an initial greedy Chain-of-Thought rollout: \(z_{\text{base},i} = \mathcal{H}_\theta(c_i)_{1:L'}\). This naive initialization merely reflects default autoregressive momentum and lacks any awareness of higher-order reasoning strategies. LatentEvolve deploys a lightweight, parameter-efficient language model (instantiated with Qwen-2.5-1.5B) designated as the latent weaver \(\mathcal{W}_\psi\). Trained through prior nighttime cycles, the weaver ingests the semantic context embedding \(e_{c_i}\) alongside the raw base state \(z_{\text{base},i}\) to project an informed, globally consolidated latent prior: $\(z'_{\text{base},i} = \mathcal{W}_\psi(e_{c_i}, z_{\text{base},i})\)$ This prior transformation embeds abstract procedural memory directly into the starting state, substantially reducing the Euclidean distance between initial candidate activations and the manifold of successful reasoning trajectories before any online search occurs.
2. Associative Retrieval & Momentum Transfer: mitigating negative transfer and semantic conflicts from raw state reuse
The daytime phase functions analogously to hippocampal episodic recall. The framework maintains an episodic buffer \(\mathcal{M}\) dynamically populated with historical triplets \((e_{c_j}, z_{\text{base},j}, z^*_j)\), comprising the query embedding, base state, and refined optimal latent representation. For a newly arriving problem \(c_i\), the system computes its context embedding \(e_{c_i}\) and performs cosine similarity search to retrieve a localized neighborhood \(\mathcal{N}_k(c_i)\) of top-\(k\) relevant experiences. Instead of naively averaging the final latent states \(z^*_j\) of retrieved neighbors—which risks catastrophic semantic distortion due to prompt-specific divergences—LatentEvolve explicitly isolates the optimization journey. It computes the directional momentum vector \(\Delta z_j = z^*_j - z_{\text{base},j}\) and transfers this displacement to the current query weighted by semantic similarity \(\alpha_j \propto \exp(S(e_{c_i}, e_{c_j}))\): $\(z_{0,i} = z'_{\text{base},i} + \sum_{j \in \mathcal{N}_k(c_i)} \alpha_j \Delta z_j\)$ This formulation ensures that proven optimization trajectories steer the starting state into fertile latent regions while preserving context-specific semantic integrity.
3. Self-Supervised Policy Gradient Refinement: driving continuous optimization without external ground-truth supervision
While momentum transfer provides a promising foundation, the latent vector must still adapt to the unique nuances of query \(c_i\). To eliminate dependence on ground-truth answer labels or external verifiers, LatentEvolve implements an intrinsic self-rewarding mechanism. The frozen LLM \(\pi_\theta\) evaluates its own rollouts \(y\) across five normalized diagnostic dimensions: final answer correctness \(s_{\text{ans}}\), problem comprehension \(s_{\text{comp}}\), mathematical validity \(s_{\text{calc}}\), formatting fidelity \(s_{\text{form}}\), and reasoning clarity \(s_{\text{clar}}\), combined under a \(1:1:1:2:2\) weighting scheme: $\(Q(y) = \frac{1}{7} \big[ s_{\text{ans}}(y) + s_{\text{comp}}(y) + s_{\text{calc}}(y) + 2 s_{\text{form}}(y) + 2 s_{\text{clar}}(y) \big]\)$ Using \(M=8\) stochastic rollouts generated under the conditioned distribution \(p(\cdot \mid c_i, z_k; \theta)\), the gradient of the expected self-reward \(\mathcal{J}(z_k)\) is approximated via policy gradient: $\(\nabla_{z_k} \mathcal{J}(z_k) \approx \frac{1}{M} \sum_{m=1}^M Q(y^{(m)}) \nabla_{z_k} \log p(y^{(m)} \mid c_i, z_k; \theta)\)$ The continuous control vector updates iteratively via gradient ascent \(z_{k+1} \leftarrow z_k + \eta \nabla_{z_k} \mathcal{J}(z_k)\) with learning rate \(\eta=0.3\). Refinement halts upon reaching maximum iterations \(K=10\) or when the expected reward fails to increase for three consecutive steps, producing the optimal latent token sequence \(z^*_i\).
4. High-Confidence Episodic Archiving: gating memory quality against hallucinatory noise accumulation
Unsupervised self-evolution carries the inherent danger of error compounding, where flawed self-assessments pollute the episodic archive. To safeguard memory integrity, LatentEvolve enforces a rigorous quality-gated admission protocol. The system evaluates the mean expected reward of the terminal candidate rollouts \(\mathbb{E}[Q(y_k^{(M)})] = \frac{1}{M} \sum_{k=1}^M Q(y_k^{(M)})\). The triplet \((e_{c_i}, z_{\text{base},i}, z^*_i)\) is formally admitted to the episodic buffer \(\mathcal{M}\) if and only if: $\(\mathbb{E}[Q(y_k^{(M)})] \ge \tau\)$ where \(\tau\) is set to a calibrated threshold of \(0.5\). This admission filter prunes unpromising or noisy reasoning paths, ensuring that only robust and high-performing latent transformations are preserved for downstream retrieval and distillation.
Loss & Training¶
Nighttime consolidation is triggered periodically every \(T=200\) test-time queries. Once the threshold is reached, inference pauses temporarily, and the accumulated episodic memory buffer \(\mathcal{M}\) serves as a replay training set for the latent weaver \(\mathcal{W}_\psi\). The training objective minimizes the mean squared reconstruction error between the weaver's contextual projection and the archived empirical optima: $\(\mathcal{L}(\psi) = \mathbb{E}_{(e_{c_j}, z_{\text{base},j}, z^*_j) \sim \mathcal{M}} \left\| \mathcal{W}_\psi(e_{c_j}, z_{\text{base},j}) - z^*_j \right\|_2^2\)$ Following parameter optimization, the episodic buffer \(\mathcal{M}\) is flushed and reset (\(\mathcal{M} \leftarrow \emptyset\)), freeing memory for the next diurnal cycle. Through this recurring cycle, transient episodic experiences are consolidated into stable, permanent neural network parameters.
Key Experimental Results¶
Main Results¶
The authors conduct comprehensive evaluations across eight demanding benchmarks spanning General QA (MMLU), Mathematical Reasoning (GSM8K, MATH-500, AIME 2024, AIME 2025), Scientific Reasoning (SciBench, GPQA-Diamond), and Medical QA (JAMA Clinical). Table 1 presents the performance comparison using Qwen2.5-7B-Instruct against thirteen representative baselines (exact figures directly cited from the paper).
| Paradigm | Method | MMLU | GSM8K | MATH-500 | AIME24 | AIME25 | SciBench | GPQA | JAMA |
|---|---|---|---|---|---|---|---|---|---|
| Prompting (training-free) | Vanilla Model | 72.10 | 89.32 | 75.73 | 11.84 | 8.31 | 11.27 | 35.07 | 47.72 |
| Prompting (training-free) | CoT | 73.50 | 91.38 | 76.21 | 11.95 | 8.92 | 11.99 | 35.86 | 50.96 |
| Reinforcement Learning | Self-Rewarding | 63.10 | 88.30 | 73.14 | 6.25 | 9.89 | 9.36 | 30.30 | 47.07 |
| Reinforcement Learning | Genius | 58.30 | 87.93 | 76.89 | 6.82 | 9.11 | 13.60 | 29.67 | 41.78 |
| Reinforcement Learning | GRPO | 68.90 | 92.30 | 78.20 | 18.49 | 15.10 | — | 36.61 | 51.62 |
| Reinforcement Learning | Reinforce | 63.77 | 92.30 | 77.83 | 17.70 | 14.27 | — | 36.99 | 49.16 |
| Reinforcement Learning | Reinforce++ | 65.90 | 92.60 | 78.01 | 18.75 | 14.94 | — | 35.98 | 50.40 |
| Latent Reasoning | Coprocessor | 68.10 | 83.60 | 77.19 | 14.16 | 10.26 | — | 35.47 | 43.70 |
| Latent Reasoning | SoftCoT | 62.30 | 80.13 | 74.13 | 10.78 | 8.17 | — | 35.10 | 49.70 |
| Test-time Scaling | Self-Consistency | 69.80 | 88.62 | 69.40 | 14.94 | 10.20 | 12.13 | 36.74 | 51.62 |
| Test-time Scaling | Self-Refine | 61.40 | 86.33 | 59.32 | 13.38 | 8.64 | 9.36 | 35.85 | 45.64 |
| Test-time Scaling | LatentSeek | 68.50 | 91.58 | 75.60 | 14.06 | 15.41 | 14.45 | 34.21 | 50.40 |
| Test-time Scaling | TTRL | 70.90 | 92.80 | 77.39 | 23.33 | 16.71 | 13.92 | 36.36 | 49.16 |
| Proposed Framework | LatentEvolve | 77.30 | 94.53 | 80.55 | 20.56 | 18.37 | 19.79 | 43.67 | 52.94 |
Cross-Backbone Generalization and Absolute Gains¶
To demonstrate architecture invariance, LatentEvolve was systematically applied across five prominent open-source model backbones varying from 3B to 12B parameters (exact figures from Table 2).
| Backbone | Variant | MMLU | GSM8K | MATH | SciBench | GPQA | JAMA | AIME24 | AIME25 |
|---|---|---|---|---|---|---|---|---|---|
| Llama3.2-3B | Vanilla +LE \(\Delta\) |
57.60 64.30 +6.70 |
71.31 75.51 +4.20 |
42.30 52.40 +10.10 |
6.79 9.39 +2.60 |
23.31 29.29 +5.98 |
45.14 48.44 +3.30 |
0.00 7.59 +7.59 |
0.00 5.92 +5.92 |
| Qwen2.5-7B | Vanilla +LE \(\Delta\) |
72.10 77.30 +5.20 |
89.32 94.53 +5.21 |
75.73 80.55 +4.82 |
11.27 19.79 +8.52 |
35.07 43.67 +8.60 |
47.72 52.94 +5.22 |
11.84 20.56 +8.72 |
8.31 18.37 +10.06 |
| Qwen3-4B | Vanilla +LE \(\Delta\) |
74.10 78.30 +4.20 |
89.23 97.83 +8.60 |
84.71 95.32 +10.61 |
12.28 31.93 +19.65 |
41.28 58.59 +17.31 |
51.49 53.67 +2.18 |
24.17 33.41 +9.24 |
18.75 25.56 +6.81 |
| Qwen3-8B | Vanilla +LE \(\Delta\) |
75.20 79.80 +4.60 |
90.52 98.21 +7.69 |
86.93 97.11 +10.18 |
6.36 10.83 +4.47 |
39.47 46.18 +6.71 |
53.08 54.60 +1.52 |
29.04 40.13 +11.09 |
19.86 27.78 +7.92 |
| Gemma-3-12B | Vanilla +LE \(\Delta\) |
75.50 77.60 +2.10 |
89.23 92.79 +3.56 |
84.80 89.32 +4.52 |
10.84 18.93 +8.09 |
40.48 47.20 +6.72 |
49.50 55.06 +5.56 |
23.52 32.75 +9.23 |
18.27 26.23 +7.96 |
Ablation Study and Sensitivity Analysis¶
The authors validate each architectural component, testing ablations on daytime retrieval, nighttime consolidation, latent sequence length \(L'\), retrieval metrics, and confidence threshold \(\tau\) (from Figure 4 and Tables 9/10).
| Configuration | MATH-500 (Qwen2.5-7B) | SciBench (Qwen2.5-7B) | Core Takeaway |
|---|---|---|---|
| Full LatentEvolve (\(L'=30\)) | 80.3% | 33.4% | Full dual-phase biological framework |
| w/o Daytime Scaling | 76.5% | 28.5% | Disables episodic retrieval; drops SciBench by 4.9% |
| w/o Nighttime Scaling | 75.2% | 26.6% | Disables weaver consolidation; drops SciBench by 6.8% |
| Latent Length \(L'=10\) | 75.8% | 24.1% | Insufficient steering capacity in latent space |
| Latent Length \(L'=50\) | 77.2% | 29.8% | Overly large optimization space hinders policy search |
| Retrieval: Cosine Similarity | 77.8% | 48.4% (JAMA) | Most robust metric across both language and math queries |
| Retrieval: Euclidean Distance | 75.9% | 47.1% (JAMA) | Degrades under high-dimensional representation geometry |
| Gating Threshold \(\tau=0.3\) | 76.0% | 46.2% (JAMA) | Loose filter admits low-quality noisy optimization traces |
| Gating Threshold \(\tau=0.9\) | 74.2% | 47.1% (JAMA) | Overly aggressive pruning starves the weaver of training data |
Key Findings¶
- Nighttime consolidation governs cross-domain transferability: Ablation confirms that nighttime consolidation contributes more heavily to broad generalization than daytime retrieval. In sequential streaming experiments on Gemma-3-12B (Figure 3), executing the initial nighttime consolidation on MATH tasks yielded an immediate \(+4.5\%\) jump on the completely unseen medical diagnostic benchmark JAMA, compared to a negligible \(+0.4\%\) bump from daytime episodic adaptation alone.
- Continual learning without catastrophic forgetting: Sequential evaluation across domains demonstrates remarkable stability. After processing two successive rounds of MMLU evaluations (Figure 3), Gemma-3-12B not only maintained its previously acquired mathematical proficiency on MATH-500 but actually exhibited a minor improvement from \(78.6\%\) to \(80.2\%\), validating that latent consolidation avoids catastrophic forgetting.
- Centroid convergence toward successful latent manifolds: Visualizations using t-SNE (Figure 5) and Euclidean distance tracking between cluster centroids (Tables 11/12) demonstrate that while raw base latents remain distant from successful trajectories (\(d=2317.4\) on Llama-3.2-3B + JAMA), nighttime-refined latents close the distance to \(1094.2\) in Iter 2 and \(834.1\) in Iter 3, proving that the weaver learns to seed inference directly into fertile solution spaces.
Highlights & Insights¶
- Neuroscience-grounded test-time adaptation: By formalizing the Complementary Learning Systems (CLS) hypothesis in continuous hidden representation space, the paper presents the first practical blueprint for test-time scaling that actually learns from its own problem-solving history.
- Momentum transfer avoids semantic collision: Shifting initial vectors by directional optimization vectors (\(\Delta z\)) rather than absolute coordinates guarantees that historical momentum is inherited without corrupting query-specific lexical constraints.
- Frugal resource footprint: Nighttime parameter distillation runs asynchronously and requires only \(1.1\) to \(1.3\) GPU hours per cycle on a compact 1.5B model, while during daytime inference it shrinks generated CoT answer lengths by \(5\%\) to \(8\%\) through more targeted reasoning.
Limitations & Future Work¶
- Test-time computation latency: The iterative policy gradient search (\(K=10\) steps, \(M=8\) rollouts) introduces measurable latency overhead, making deployment challenging in real-time, ultra-low-latency interactive systems.
- Vulnerability to self-reward collapse: In completely unsupervised environments, the optimization gradient is entirely coupled to the model's self-assessed reward. On open-ended, subjective, or highly deceptive queries, flawed self-evaluation could cause circular optimization into reward-hacked errors.
- Narrow evaluation scope: Experiments focus primarily on structured, verifiable reasoning domains (math, chemistry, physics, medicine). Extending the framework to multi-turn agentic environments and long-context dialogue remains open for exploration.
Related Work & Insights¶
- vs LatentSeek / SoftCoT: LatentSeek optimizes continuous vectors per query in isolation, resetting to a cold start upon every new question; LatentEvolve introduces an enduring dual-phase memory system, outperforming LatentSeek on MATH-500 by \(4.95\%\).
- vs TTRL (Test-Time Reinforcement Learning): TTRL updates the primary LLM parameters directly during inference, inducing substantial memory overhead and risking catastrophic forgetting; LatentEvolve keeps the foundational backbone strictly frozen and channels parameter updates entirely into a small auxiliary weaver.
- vs Verbal Self-Refine / Reflexion: Language-level reflection stores feedback as discrete text prompts within the context window, competing for context capacity and introducing lexical ambiguity; LatentEvolve maintains experience compactly in continuous latent space with higher expressivity and lower token overhead.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ [Pioneering integration of CLS cognitive theory into continuous latent test-time scaling with a dual-phase self-evolution cycle]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive validation across 8 benchmarks, 5 LLM families, detailed ablations, parameter sensitivities, and centroid distance analyses]
- Writing Quality: ⭐⭐⭐⭐⭐ [Extremely clear narrative structure, rigorous mathematical formalization, intuitive figures, and meticulous documentation]
- Value: ⭐⭐⭐⭐⭐ [Opens an exciting paradigm shift from static test-time compute scaling toward dynamic, lifelong self-evolving inference systems]