Skip to content

💡 LLM Reasoning

🧠 NeurIPS2026 · 111 paper notes

📌 Same area in other venues: 📷 CVPR2026 (16) · 🔬 ICLR2026 (241) · 💬 ACL2026 (82) · 🧪 ICML2026 (78) · 🤖 AAAI2026 (37) · 🧠 NeurIPS2025 (82)

🔥 Top topics: Reasoning ×71 · LLM ×23 · Alignment/RLHF ×4 · Medical Imaging ×2 · Agents ×2

A Mechanistic Analysis of Looped Reasoning Language Models

Through a rigorous mechanistic analysis of latent trajectories and attention dynamics, this paper reveals that recurrent Transformer layers converge to distinct cyclic fixed points, stabilizing attention patterns across recurrences and periodically replicating feedforward stages of inference within each recurrent block.

A Theoretical Framework for Self-Play Theorem Proving Algorithms

This paper introduces the first theoretical framework for LLM-based self-play theorem proving, proving that a prover-conjecturer system over a well-connected theorem graph achieves exponential growth in provable theorems, and designing a diffusion similarity-guided conjecturer via contrastive learning to resolve theorem clustering collapse.

A Theory of Online Learning with Autoregressive Chain-of-Thought Reasoning

This paper develops the theoretical foundations of online learning with autoregressive chain-of-thought reasoning, establishing an exact taxonomy of end-to-end mistake bound growth from constant to logarithmic \(\Theta(L(F)\log M)\) in generation horizon \(M\), while proving that full chain-of-thought feedback completely eliminates any dependence on \(M\).

Adaptive Generate-Rank-Verify: Inference-Time Search with Costly Verification

Addressing the compute-allocation dilemma between cheap reward proxies and expensive gold verifiers in LLM reasoning, this paper establishes a generative active search theoretical framework and introduces ADAP, an online shellwise adaptive policy that eliminates the need for offline difficulty calibration while matching 100% success at up to 5.5× lower cost with constant-factor optimality guarantees.

Advancing Entropy-Level Credit Assignment in RLVR via Proximal Entropy Policy Optimization

PEPO weights token updates in successful reasoning trajectories by uncertainty relative to their neighborhoods while preserving each trajectory's total weight, outperforming the compared GRPO and global-entropy baselines in the main mathematical reasoning experiments, with explicit limits on trend robustness and cross-task generalization.

An Enigma of Artificial Reason: Investigating the Production-Evaluation Gap in Large Reasoning Models

This paper uncovers a severe "production-evaluation gap" in Large Reasoning Models (LRMs): while achieving near-perfect accuracy when producing solutions from scratch, frontier models score as low as 48% when evaluating solutions with trivial reasoning flaws but valid answers (VAIR), fundamentally driven by an answer confirmation bias that overrides internal validity representations.

Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information

This paper demonstrates that privileged on-policy self-distillation inherently amounts to conditional pointwise mutual information (PMI), which systematically inflates shortcut tokens and penalizes deliberation; to resolve this, the authors propose Anti-Self-Distillation (AntiSD) by ascending Jensen-Shannon divergence under an entropy-triggered gate, achieving 2-10x training speedups and up to +11.5 pp accuracy gains over GRPO without external models.

Are Your Reasoning Models Reasoning or Guessing? A Mechanistic Analysis of Hierarchical Reasoning Models

Through a mechanistic analysis of latent-space trajectories in Hierarchical Reasoning Models (HRM), this paper reveals that HRM operates by "guessing" fixed points rather than continuous reasoning, and introduces data mixing, input perturbations, and model bootstrapping to elevate Sudoku-Extreme accuracy from 55.0% to 96.9%.

Ares: Adaptive Reasoning Effort Selection for Efficient LLM Agents

Ares introduces a lightweight per-step reasoning router that adaptively selects the minimum sufficient reasoning effort level for each step in multi-turn agent interactions, cutting reasoning tokens by up to 52.7% while preserving task success rates.

Asking the Right Questions: Improving Reasoning with Generated Stepping Stones

The paper introduces ARQ (Asking the Right Questions), a framework that decouples question generation from problem solving by producing intermediate stepping stones, and establishes stepping stone formulation as an optimizable post-training task via Monte Carlo expected reward estimation, SFT, and DPO.

Axiomatic World Modeling for Physics Reasoning

Addressing the critical failure mode of premise-level "modeling drift" where LLMs silently corrupt physical setups to force familiar textbook patterns, this paper proposes Reinforcement Learning with Axiomatic World Modeling (RLAWM). By decoupling reasoning into axiomatic world modeling and executable derivation, and jointly training with schema-bounded motif rewards (\(R_M\)) and executable physical unit-test checkers (\(R_S\)), RLAWM achieves substantial accuracy gains, mitigates diversity collapse, and transfers zero-shot across domains without inference-time overhead.

Beyond Right and Wrong: Evaluating Second-Order Social Reasoning in Large Language Models

Introduces a dual-dimensional framework and the multi-perspective NormReact benchmark (450 everyday norm violation scenarios) to evaluate second-order social metanorm reasoning in LLMs, revealing that current models construct a systematically more punitive social world and deteriorate sharply as social distance increases.

Beyond What Seems Necessary: Hidden Gains from Scaling Training-Time Reasoning Length

Under outcome-only supervision, scaling training-time reasoning length (e.g., token budgets in RL or loop counts in looped Transformers) continues to yield substantial out-of-distribution (OOD) performance gains even after in-distribution (ID) accuracy has saturated, driven by inductive bias reshaping and regularization-mediated shortcut suppression.

BFS-PO: Best-First Search for Large Reasoning Models

BFS-PO is a reinforcement learning algorithm for large reasoning models that guides online exploration using a best-first search strategy on maximum-entropy backtracking nodes, reducing chain-of-thought length while simultaneously boosting reasoning accuracy without requiring a process reward model.

Boosting Inference with Guided Reasoning: Stochastic Exploration for Recursive Models

To overcome the failure of tiny recursive models where deterministic latent trajectories become trapped in local attractor states, this paper formulates recursive inference as approximate Bayesian inference over latent trajectories, injecting stochastic perturbations to explore candidate paths and repurposing the pre-trained Q-head for online Feynman–Kac reweighting (SMC framework)—achieving zero-retraining accuracy gains on Sudoku-Extreme while introducing three label-free diagnostics for stability, alignment, and uncertainty.

CircuitSeer: Mining High-Quality Data by Probing Mathematical Reasoning Circuits in LLMs

CircuitSeer abandons heavy external model evaluations and heuristic rules by probing sparse mathematical reasoning circuits inside LLMs, measuring reasoning complexity via attention variance on problem inputs, and curating a compact 10% subset through soft sampling that outperforms full-dataset supervised fine-tuning.

CITE: Anytime-Valid Statistical Inference in LLM Self-Consistency

Addressing the challenge of unknown and growing answer category spaces in adaptive LLM self-consistency, this paper proposes CITE, an anytime-valid mode certification algorithm based on intersection-union testing with e-processes, which provably controls false certification error under arbitrary data-driven stopping rules with a category-set-size-free stopping time matching minimax lower bounds.

Cite What You Explore: Budget-Aware LLM Reasoning over Medical KGs with Verifiable Evidence

To address missing temporal dependencies in electronic health records and unconstrained exploration in medical knowledge graphs, this paper introduces BAR, a budget-aware LLM reasoning framework that refines raw knowledge graphs into quality-annotated evidence graphs, allocates patient-adaptive exploration budgets based on EHR prediction uncertainty, executes a plan-navigate-verify loop with dynamic revision, and optimizes exploration policies via paired gain reinforcement learning for high-accuracy, cost-effective, and fully verifiable clinical risk prediction.

Compute-Aligned Training: Optimizing for Test-Time Inference

Formalizes test-time inference strategies as differentiable operators over the base policy and derives closed-form marginal utility gradient reweighting across SFT and RL, substantially enhancing test-time scaling performance with zero extra training compute.

Confidence-Calibrated Inference Expansion for Evaluator-Guided Test-Time Reasoning

This paper models evaluator-guided adaptive reasoning as a compute-aware local controller with three actions (trust the current proposal, emit a conservative fallback, or buy additional expansion), achieving higher utility than strong fixed baselines while trimming online compute by 16%–30% across width and revision settings via action-specific calibration and local uncertainty penalties.

Consilience for Verifier-Free Test-Time Scaling

Addressing the failure of global confidence maximization on challenging problems—where models favor confidently wrong answers due to premature convergence—this paper proposes Consilience, a verifier-free selection metric that actively penalizes early confidence while rewarding terminal certainty within an isolated reasoning phase, yielding consistent gains on free-form coding and mathematical benchmarks.

Convex Compositional Reasoning Models

Addressing the fundamental issue where compositional energy-based models suffer from spurious local minima caused by non-convex landscape interactions, this paper proposes CCEM, parameterizing local factors via input-convex neural networks over tight convex relaxations to construct globally convex energy landscapes, replacing heuristic diffusion sampling with deterministic projected first-order optimization for zero-shot transfer across problem scales.

Correct but Unselectable: The Hidden Interface Tax in Multi-Candidate Reasoning

Addressing the bottleneck where small models produce correct reasoning traces that cannot be consumed by fixed selectors due to fragmentation or answer-mode mismatch, this paper formalizes the coverage–conversion decomposition and introduces Compatibility-Aware Candidate Construction (CACC), boosting final accuracy on GPQA Diamond from 5.6% to 19.2% without retraining.

Credit Assignment with Resets in Language Model Reasoning

This paper introduces reset-based reinforcement learning with verifiable rewards (RRPO and SRPO), enabling models to self-localize the first erroneous reasoning thought and resample counterfactual suffixes, provably and empirically overcoming the uniform credit assignment bottleneck in multi-step reasoning.

CurveRL: Principled Distribution-Aware Context Reweighting for LLM Reasoning

Formalizes prompt reweighting in Reinforcement Learning with Verified Rewards (RLVR) as context distribution control via functional derivatives of utility functionals, and proposes CurveRL, an adaptive quantile-space reweighting algorithm that leverages pass-rate ranking and density to consistently improve the Pareto frontier across single-shot accuracy (pass@1) and test-time search boundaries (pass@k).

Decoding the Critique Mechanism in Large Reasoning Models

This paper uncovers a "hidden self-correction" phenomenon where Large Reasoning Models (LRMs) silently recover the correct final answer despite uncorrected errors propagating across the entire chain-of-thought, and extracts an interpretable linear critique vector via mean difference that causally steers error detection and enhances test-time scaling without extra training.

Depth-Recurrent Attention Mixtures: Giving Latent Reasoning the Attention it Deserves

To tackle both the layer-size and hidden-size bottlenecks in depth-recurrent architectures, this paper proposes Dreamer, a modular single-layer framework that orthogonalizes sequence, depth, and expert attention, delivering 2–8× data efficiency gains under strictly matched FLOPs, parameters, and memory.

DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoning

To resolve the out-of-distribution generalization breakdown in looped Transformers caused by the geometric mismatch between continuous hidden states and discrete embeddings, DiscoLoop passes both a continuous hidden-state channel and a soft-decoded discrete-embedding channel across iterations, achieving near-perfect multi-hop composition and improving real-world language modeling.

Diversity Combining for Multi-Path LLM Reasoning

Formalizes multi-path LLM reasoning as diversity combining over wireless multipath channels, proving that pairwise correctness correlation limits the vote-level effective sample size to a finite ceiling of \(1/c\) and deriving a 4-path pilot Adaptive-K rule.

Diversity Combining for Multi-Path LLM Reasoning

The paper explains diminishing returns in multi-path reasoning through correctness correlation and the design effect, then estimates a fixed deployment budget from a labeled offline four-path pilot; across five model–task configurations, 4–10 paths retain 96%–103% of the binary majority-vote accuracy at 32 paths, although that evaluation metric is not the actual plurality-vote accuracy for open answers.

Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks

Through mechanistic interpretability of continuous-thought teacher-student distillation (CODI) on strictly sequential polynomial tasks, this paper reveals a dual-route late-fusion architecture on short horizons that degrades into a late-bottleneck shortcut on long horizons and completely collapses under algebraically incompressible prime-modulus regimes.

ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation

Addressing the hallucination and superficial pattern-matching flaws of Multimodal LLMs in electrocardiogram interpretation, this work introduces ECG-Reasoning-Benchmark—a multi-turn evaluation framework with over 6,400 samples across 17 core diagnoses—demonstrating that leading models achieve near-zero completion rates (<6%) in executing complete clinical deduction chains grounded in raw signals.

Efficient Reasoning via Constrained Optimization in Latent Space

This paper introduces a training-free constrained optimization framework that projects deviating reasoning hidden states back into an efficient reasoning convex polytope via multi-parametric quadratic programming, curbing overthinking loops and reducing token generation by 11.8%–52.8% while consistently improving accuracy.

Efficiently Representing Algorithms with Chain-of-Thought Transformers

Proves that Chain-of-Thought (CoT) transformers can directly simulate Word RAM textbook algorithms with only poly-logarithmic token overhead, completely eliminating the quadratic simulation bottleneck of Turing machines.

Entropy-Gated Latent Recursion

This paper introduces Entropy-Gated Latent Recursion (EGLR), a training-free test-time decoding method that re-applies the top-\(L\) decoder layers at high-uncertainty tokens, establishing the layer span \(L\) as a deterministic sampling axis complementary to temperature and creating an \(L \times T\) Cartesian rollout space that significantly boosts reasoning ceilings.

ExComm: Exploration-Stage Communication for Error-Resilient Agentic Test-Time Scaling

To tackle error propagation where intermediate factual missteps corrupt downstream decisions in agentic test-time scaling, ExComm introduces exploration-stage communication that audits cross-agent factual conflicts, applies tool-augmented verification via targeted soft belief updates, and enforces plan-level trajectory diversification.

ExpLang: Improved Exploration and Exploitation in LLM Reasoning with On-Policy Thinking Language Selection

ExpLang proposes an end-to-end LLM reinforcement learning post-training pipeline that uses supervised cold-starting and on-policy thinking language selection to broaden the action space via under-sampled languages during exploration and converge onto high-efficiency languages during exploitation, steadily outperforming English-only RL within the same training budget.

Externalized CPDAG Summaries Improve LLM Causal Deduction

Structured Thinking asks the same large language model to produce a format-constrained CPDAG summary before checking whether a causal hypothesis holds in every compatible DAG, raising Qwen3.5-27B primary-seed F1(YES) on Corr2Cause from 73.01 to 86.36 without formally guaranteeing graph validity or reasoning over the entire equivalence class.

Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning

By extracting explicit search trees from chain-of-thought traces in four-in-a-row and fitting cognitive computational models alongside surgical causal pruning, this paper demonstrates that LLMs plan myopically based on candidate breadth rather than acting on deep lookahead.

Facts Don’t Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation

This paper challenges the prevailing assumptions that Long-CoT distillation requires factually correct traces and vast prompt diversity, proving that reasoning gains stem from behavioral alignment with structured reasoning patterns and unveiling how trajectory complexity dictates the optimal training objective.

Foresight-over-Graph: Reasoning Beyond Local Horizons for Knowledge Base Question Answering

Addressing the critical flaw where hop-by-hop greedy pruning in existing graph agents prematurely discards answer-relevant paths due to myopic local horizons, FoG introduces far-to-near message feedback and a persistent memory subgraph, slashing LLM API calls and token consumption while yielding state-of-the-art KBQA accuracy.

Formal Conjectures: An Open and Evolving Benchmark for Verified Discovery in Mathematics

The paper introduces Formal Conjectures, an open and evolving benchmark of 2,615 mathematical statements formalized in Lean 4 and Mathlib (featuring 1,029 open research conjectures and 836 solved problems), utilizing an answer(sorry) mechanism and a multi-level statement auditing pipeline to establish an uncheatable, kernel-verified foundation for autonomous mathematical discovery.

From Noise to Diversity: Random Embedding Injection in LLM Reasoning

This paper introduces training-free Random Soft Prompts (RSP), which append isotropic Gaussian vectors fitted to embedding statistics to inputs, perturbing early attention to induce trajectory branching while relying on KV cache growth for natural attention dilution, thereby matching optimized soft prompts and significantly enhancing Pass@N diversity and RL exploration at zero training cost.

FunctionEvolve: Structure-Guided Symbolic Regression with LLMs

Addressing the issues of structure-blind search and fragile parameter optimization in LLM-driven equation discovery, FunctionEvolve builds an evolutionary architecture anchored on explicit Abstract Syntax Trees, uniting rule-based AST edits, LLM-guided local subtree refinements, and variable projection parameter optimization to achieve 82.9% SA@50 and 55.8% SA@1 on LLM-SRBench.

Geometric Latent Reasoning Induces Shorter Generations in LLMs

This paper introduces Geometric Latent Reasoning (GLR), which formulates latent reasoning as continuous path approximation within the pretrained token-embedding space. By predicting local directional updates via a lightweight transition head to replace redundant early discrete reasoning tokens, GLR intrinsically induces substantially shorter generation steps without explicit length penalties while retaining high accuracy.

Geometric Signatures of Reasoning: A Spectral Perspective on Task Hardness

The paper models LLM chain-of-thought reasoning as discrete geometric curves in hidden-state space, proves that flatter eigenvalue spectra of trajectory covariance yield higher effective dimensions representing harder tasks, and shows that kinematic features from just the first 20% of generated tokens reliably predict solution correctness.

Hide to Guide: Learning via Semantic Masking

Addressing the reward-hacking vulnerability where expert trajectories leak verifier targets in reinforcement learning with verifiable rewards (RLVR), this paper introduces Semantic Masked Expert Policy Optimization (SMEPO), which masks reward-relevant semantic spans along the critical path while preserving procedural scaffolding, converting reasoning from scratch into fill-in-the-blank guidance and outperforming vanilla GRPO across math, code, and search by up to 3.2 points with up to 4.2x faster convergence.

Hint Tuning: Less Data Makes Better Reasoners

Hint Tuning uses the corresponding instruct model as an empirical difficulty probe to determine the minimum effective reasoning prefix required to solve a problem, achieving 24–66% token reductions with only 1K self-annotated training samples while preserving competitive accuracy across mathematical, coding, and scientific reasoning tasks.

Hypothesis Generation and Updating in Large Language Models

Using Tenenbaum's classic number game, this paper evaluates LLMs across posterior prediction, hypothesis evaluation, and hypothesis generation probes, demonstrating that while LLM belief updates resemble a two-parameter Bayesian fit, they suffer from an implicit strong-sampling bias toward narrow hypotheses, a stark evaluation–generation gap, and failure to extrapolate to unobserved domains.

Illusory Pattern Perception Drives Spurious Inference in Large Language Models

This paper presents the first systematic study of illusory pattern perception in large language models, showing that models infer spurious correlations and causal narratives more strongly than humans, and explains this phenomenon through Sparse Autoencoder (SAE) feature shifts and internal signal steering.

Inside the Loop: A Mechanistic Study of Weight-Tied Transformers on Depth-Bound Algorithmic Tasks

By probing and causally intervening on a controlled directed permutation traversal task at matched effective depth, this paper shows that weight-tied looped transformers naturally learn an adaptive, step-by-step internal algorithm forming a decodable representation staircase across iterations, solving multi-hop algorithmic problems where identical untied transformers fail.

KV-PRM: Efficient Process Reward Modeling via KV-Cache Transfer for Multi-Agent Test-Time Scaling

KV-PRM eliminates redundant full-text re-encoding in process reward models by directly transferring the KV cache produced during generation and evaluating with a single verify token, slashing scoring complexity from \(\mathcal{O}(dL^2)\) to \(\mathcal{O}(dL)\) while achieving up to a 37\(\times\) latency speedup, 34\(\times\) memory reduction, and matching or superior reasoning accuracy.

Lattice Deduction Transformers

The paper introduces Lattice Deduction Transformers (LDT), which project recurrent transformer latents into an abstract interpretation lattice between forward passes to execute sound candidate elimination and conflict detection, achieving 100% accuracy and empirical soundness on Sudoku-Extreme and Snowflake Sudoku with an 800K-parameter model.

Learning Robust Reasoning through Guided Adversarial Self-Play

GASP introduces an adversarial self-play framework within a single model—pairing a polluter that generates locally coherent corruptions with an agent trained to detect and repair errors—augmented by in-distribution repair guidance to achieve dramatic gains in reasoning robustness and clean accuracy using only verifiable outcome rewards.

Listwise Policy Optimization: Group-based RLVR as Target-Projection on the LLM Response Simplex

This paper unifies group-based RLVR algorithms as implicit first-order reverse KL projections toward reward-weighted targets on the response simplex, and introduces Listwise Policy Optimization (LPO) to explicitly decouple target construction from divergence projection, achieving provable monotonic improvement and mitigating entropy collapse.

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

AutoTTS introduces an environment-driven framework that casts test-time scaling strategy design into code-level controller synthesis over an offline replay MDP, utilizing scalar beta parameterization and execution trace feedback to automatically discover inference allocation algorithms that outperform strong handcrafted baselines at minimal search cost.

Measuring Weak-to-Strong Legibility of Reasoning Models

This paper introduces an interactive "transfer utility" evaluation framework to measure the legibility of reasoning language model (RLM) chains of thought for weaker models, uncovering a striking negative correlation between top-tier task accuracy/efficiency and weak-to-strong pedagogical legibility.

Memorize Theorems, Not Instances: Probing SFT Generalization through Mathematical Reasoning

Addressing the core issue where Supervised Fine-Tuning (SFT) damages reasoning generalization, this paper shows failures stem from memorizing surface correlations rather than memorization itself, proposing Theorem-SFT to reorient supervision toward structured rule execution—improving LLaMA-3.2-3B on MATH-500 by +8.8% and demonstrating that updating only MLP layers matches full-parameter tuning.

MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning

Addressing the lack of cross-problem experience reuse and correctness-oriented selection in test-time reasoning, MILES constructs asymmetric modular memory units of sub-goal embeddings and sub-instruction texts, trains lightweight selection heads via one-vs-rest cross-entropy on confident MCTS trajectories, and steers reasoning on uncertain problems via coarse-to-fine search while keeping the underlying LLM fully frozen.

MindLoom: Composing Thought Modes for Frontier-Level Reasoning Data Synthesis

By conceptualizing reasoning difficulty as an accumulation of atomic knowledge-reasoning transformations ("thought modes"), MindLoom establishes a four-stage pipeline—reverse-engineering verified solutions, training embedding-based retrieval, and applying distribution-aligned compositional synthesis—consistently outperforming base models, trajectory distillation, and open-source corpora across nine STEM and competition-level math benchmarks.

Natural Synthesis: Outperforming Reactive Synthesis Tools with Large Reasoning Models

Natural Synthesis couples large reasoning models with symbolic model checkers in a counterexample-guided repair loop at the register-transfer level, solving 92.5% of SYNTCOMP benchmarks to outperform dedicated state-of-the-art symbolic tools, while generalizing to undecidable parameterized synthesis and natural-language specifications.

Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models

This paper introduces Cascaded Reinforcement Learning (Cascade RL) to develop general-purpose reasoning models, sequentially decoupling multi-domain RL training across human alignment, instruction following, math reasoning, competitive programming, and software engineering to enable an 8B unified model and a 14B dedicated thinking model to surpass their 671B DeepSeek-R1-0528 teacher model.

On the Overscaling Curse of Parallel Thinking: System Efficacy Contradicts Sample Efficiency

This paper formalizes and quantifies the "Overscaling Curse" in parallel thinking—where maximizing overall dataset accuracy forces a global budget that severely degrades sample-level budget utilization—and proposes LanBo, a latent budget predictor under the Pre-decoding Budget Adaptation (PreAda) paradigm that cuts latency and memory by over 50% while preserving dataset accuracy.

On the Token Value Inequality in Efficient Reasoning

The paper uncovers the non-uniform "Token Value Inequality" in Chain-of-Thought reasoning, introduces response-normalized log probability (NormLP) and the TokenProbe diagnostic signal using only the model's native log probabilities, and incorporates selective KL anchoring and reward shaping into GRPO to preserve core reasoning tokens while cutting overall token consumption by up to 76% without accuracy loss.

One-Layer Transformers Provably Learn In-Context K-Nearest Neighbor Prediction with Chain-of-Thought

This paper proves that one-layer transformers cannot perform in-context \(K\)-NN classification for odd \(K>1\) via direct inference, but can provably solve it with Chain-of-Thought via dynamic attention masking, with the construction fully learnable under single-step gradient descent.

OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping

OS-Pruner formulates chain-of-thought pruning as a sequential optimal stopping problem rather than heuristic classification, optimizing an explicit utility trading final-answer accuracy against token length while keeping the base reasoning model frozen, achieving a 20% to 60% reduction in generation length with minimal accuracy degradation.

Outcome-Based RL Provably Leads Transformers to Reason, but Only With the Right Data

Through mathematical proofs and empirical validations, this paper demonstrates that outcome-based reinforcement learning with sparse final-answer rewards drives Transformers toward efficient step-by-step Chain-of-Thought reasoning only when the training distribution includes sufficient simple examples; otherwise, policy gradient learning becomes exponentially intractable.

Parameter Exploration for RLVR via Variational Learning

This paper introduces Perturbed Parameter Policy Optimization (3PO), a family of parameter-space exploration methods for RLVR that samples policies from an adaptive variational posterior to overcome the rank-preserving limitations of action-space noise, rescuing zero-advantage groups and substantially boosting mathematical reasoning and code generation.

PORTool: Importance-Aware Policy Optimization with Rewarded Tree for Multi-Tool-Integrated Reasoning

Addressing the credit-assignment ambiguity and collateral unlearning in outcome-supervised multi-tool reasoning, PORTool generates prefix-sharing rewarded rollout trees with adaptive correctness-dominant step-wise importance aggregation, combining theoretically scaled fork-relative and trajectory-relative advantages to improve task accuracy while reducing tool-call steps.

Power Distribution Bridges Sampling, Self-Reward RL, and Self-Distillation

Unifying power sampling, self-reward KL-regularized reinforcement learning, and self-distillation through the sequence-level power distribution, this paper proposes power self-distillation to amortize expensive inference-time sampling into offline supervised fine-tuning.

Predicting and Improving Test-Time Scaling Laws via Reward Tail Guided Search

This paper proposes estimating the truncated Gaussian upper tail of reward distributions to reliably predict scaling laws for large sample budgets, and introduces Scaling-Law Guided (SLG) Search, a two-stage test-time algorithm that dynamically allocates compute to exploit high-potential intermediate trajectories, outperforming naive Best-of-\(N\) with polynomial compute amplification.

Probing the Trajectories of Reasoning Traces in Large Language Models

By truncating reasoning traces into deciles and probing induced choice distributions via forced early stopping, this work proves that test-time accuracy gains stem from instance-specific semantic reasoning rather than token length or structural form, while uncovering late-stage overconfidence and cognitive anchoring in cross-model transfer.

Provable Test-Time Scaling for Beam Search in LLM Reasoning

Addressing the exponential test-time compute scaling of sequence-level sampling methods in long-horizon LLM reasoning, this paper establishes the first theoretical test-time guarantees for beam search, uncovers a quadratic coverage lower bound for vanilla beam search, and proposes Confidence-Filtered Beam Search (CF-Beam) to achieve near-linear coverage dependence and polynomial scaling with reasoning horizon length.

Provable Test-Time Scaling for Beam Search in LLM Reasoning

With sampling access to next tokens but no full logits, this paper filters low-frequency tokens before beam pruning and, under alignment between prefix likelihood and correctness, reduces the worst-step coverage dependence of search samples from worst-case quadratic to nearly linear; accuracy on an LLM matrix-multiplication task increases from 37.2% to 38.8%.

Quantifying the Stability of Multi-Step Reasoning via Error Amplification

This paper theoretically proves that the accumulated inference error in autoregressive multi-step reasoning is governed exponentially by the product of input-space Jacobian spectral norms (an error amplification factor), and proposes a regularized training algorithm combining chain-of-thought length compression with quantization-aware training to drastically stabilize long-chain generation.

Rationale-Guided Policy Optimization: Learning to Reason with Adaptive Rationale Scaffolding

Addressing the acute reward sparsity and exploration bottlenecks in reinforcement learning with verifiable rewards (RLVR) on challenging reasoning problems, this paper introduces Rationale-Guided Policy Optimization (RGPO)—a framework using reference rationales merely as temporary scaffolds for online self-refinement, applying reward gating to distill verified superior model-generated trajectories back under the original unguided prompt distribution.

Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuning

Addressing the structural failure mode where explicit reasoning models degrade into direct-answer generators during standard downstream fine-tuning lacking reasoning traces, this paper formalizes "reasoning-trace collapse", develops the decoupled evaluation framework ThinkPack, and demonstrates that lightweight loss-masking strategies effectively preserve explicit reasoning without requiring teacher distillation.

Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuning

By evaluating answer correctness separately from the continued production of complete explicit reasoning, this paper shows that ordinary fine-tuning without reasoning traces can improve Chemistry accuracy while reducing valid reasoning to approximately zero, and that reasoning-region loss masking mitigates this structural degradation, with model- and task-dependent effects.

Reasoning with Sampling: Cutting at Decision Points

Addressing the challenge that base language models can elicit reasoning via power distribution sampling without training but suffer from sluggish MCMC mixing, this paper introduces Entropy-Cut Metropolis–Hastings (Entropy-Cut MH) to guide proposal cuts using positive next-token entropy jumps, provably cutting mixing time from sequence length \(\mathcal{O}(T)\) to decision depth \(\mathcal{O}(k)\) and outperforming existing samplers and compute-matched Best-of-N.

Reliable Chain-of-Thought via Prefix Consistency

This paper introduces Prefix Consistency, a test-time reliability signal that truncates reasoning chains partway through to regenerate continuations, exploiting the asymmetry where correct traces reproduce original answers far more reliably than incorrect traces to weight votes without requiring token log-probabilities.

ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search

ReST-RL introduces a unified policy–value reinforced self-training framework that reconnects policy optimization and value-guided search, reshaping trajectory distributions via state-localized restarts in ReST-GRPO and performing test-time tree search using a line-level value model trained on self-collected MCTS targets.

Rewarding Novel Deductions: Solver-guided Process Supervision for Logical Reasoning

This paper proposes Spring, a reinforcement learning framework with SMT solver-guided process supervision that verifies the formal validity, state consistency, and non-redundant novelty of intermediate reasoning steps to provide dense process rewards and contradiction penalties, significantly outperforming outcome-only baselines and neurosymbolic approaches across ZebraLogic, AR-LSAT, and Knights and Knaves.

SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance

Addressing exploration bias and compounding bias in sparse-reward long-horizon reasoning, SAGE incorporates algebraic sparsification and hyperbolic structural guidance during post-training, delivering up to an 8-fold improvement on complex symbolic reasoning with zero inference overhead.

SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance

SAGE uses two soft potentials—whether an operation addresses unresolved constraints and whether its resulting state approaches a target structure—to guide both candidate sampling and rewards during post-training, improving long-horizon reasoning while retaining ordinary inference-time decoding; Qwen3.5-35B mathematical average accuracy rises from 61.92% with GRPO to 64.86%.

SARL: Label-Free Reinforcement Learning by Rewarding Reasoning Topology

SARL introduces a label-free reinforcement learning framework that segments chain-of-thought traces into latent reasoning functions and constructs transition graphs, optimizing small-world clustering and shortest path length as intrinsic structural rewards to significantly enhance mathematical and open-ended reasoning without ground-truth answers or preference supervision.

Seirênes: Adversarial Self-Play with Evolving Distractions for LLM Reasoning

Seirênes introduces a parameter-shared adversarial self-play reinforcement learning framework where a single model co-evolves as an Adversary synthesizing plausible deceptive distractions and a Reasoner recovering the core problem logic under interference, converting contextual vulnerability into an adaptive training signal that substantially improves standalone mathematical reasoning and contextual robustness.

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning

Reveals that numerous recent mixed-policy LLM reasoning methods rely on severely deflated baselines caused by two silent bugs in distributed training frameworks (DeepSpeed CPU-offload optimizer and OpenRLHF loss aggregation), and proves that the corrected classic SFT-then-RL pipeline decisively outperforms all published mixed-policy alternatives with far fewer FLOPs.

Smart Picks in the Dark: Towards Efficient RLVR for Reasoning via Tracing Metacognitive Pivots

Addressing the challenge of selecting training samples for RLVR without ground-truth answers, this paper introduces PivotTrace, a framework that quantifies reasoning uncertainty by tracing metacognitive pivots in long-range attention dynamics and adaptively triages data into three subsets, surpassing fully supervised training using only 29.3% annotated samples and achieving 2.75x faster convergence.

Soft Token Alignment for Cross-Lingual Reasoning

Addressing late-stage semantic divergence caused by discrete vocabulary commitments in multilingual LLMs, this paper proposes SOLAR, an auxiliary SFT objective that aligns mean-pooled soft-token representations of reasoning trajectories to an English pivot, substantially boosting reasoning accuracy and cross-lingual consistency.

SR-GRPO: Stable Rank as an Intrinsic Geometric Reward for Large Language Model Alignment

This paper introduces stable rank of LLM final-layer hidden states as an unsupervised intrinsic geometric reward reflecting representational effective dimensionality, establishing SR-GRPO for reinforcement learning alignment without external human annotations, verifiers, or reward models.

State of Thought Enables Endogenous Reasoning

Proposes State of Thought (SoT), a reasoning paradigm that extracts a 4D dynamic-geometric internal state from frozen LLMs and drives a lightweight 582-parameter controller to dynamically sparsify historical evidence and adaptively trigger stopping, framing reasoning as a closed-loop, state-conditioned evidence-organization process.

Step-by-Step Optimization-like Reasoning in LLMs over Expanding Search Spaces

Introduces OPT*, a scalable auto-verifiable optimization task family that decouples reasoning difficulty from human annotation effort, and establishes complementary recipes—solver-guided online policy optimization (νPO) with rank-based reward shaping and structure-aware offline tree search (νMCTS/νBeamSearch)—to drastically enhance step-by-step optimization reasoning over expanding combinatorial spaces.

Structured Sparse Memory for Recurrent Reasoning

CoSE replaces a large task table with a factorized FiLM conditioning branch and rank-32 instance residuals, reducing task-memory parameters to roughly 1/15 while improving pass@2 in controlled ARC experiments; CHARM combines this memory with synthetic data, recurrent computation, and extended training to reach 84.0% / 46.7% on ARC-AGI-1 / 2 public evaluation, which cannot be attributed entirely to CoSE.

TALES: Text Adventure Learning Environment Suite

TALES establishes a unified benchmark spanning 5 classic text-adventure environments and 122 games under minimal scaffolding, revealing that while top LLMs succeed on synthetic tasks, they collapse to under 15% on human-authored interactive fiction.

Test-Time Compute Games

This paper models the strategic allocation of test-time compute (TTC) in the LLM-as-a-service market as a normal-form game, proves that the current pay-for-compute paradigm inevitably settles into an inefficient pure-strategy Nash equilibrium with a Price of Anarchy up to 1.19, and introduces a reverse second-price scoring auction mechanism that provably restores social welfare optimality.

Test Time Scaling with Diffusion Language Models via Reward Guided Stitching

Addressing the computational waste and high serial latency of trajectory-level search aggregation, this paper proposes a reward-guided stitching framework that explores diverse chains-of-thought using cheap parallel masked diffusion sampling, scores intermediate steps via a process reward model (PRM), and stitches high-confidence sub-derivations to condition a lightweight autoregressive solver for final answer recomputation, delivering superior reasoning accuracy and up to 1.8x latency reduction.

The Era of Agentic Organization: Learning to Organize with Language Models

This paper proposes an agentic organization paradigm and asynchronous thinking mechanism (AsyncThink), integrating Fork-Join actions directly into autoregressive text generation and training models via format SFT and concurrency-rewarded GRPO, achieving a 28% reduction in inference latency over parallel thinking while enhancing accuracy.

The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment

The paper introduces the Master Key Hypothesis along with Unlock—a training-free and label-free framework that extracts capability directions by contrasting activations in Source models and aligns them to Target models via low-rank linear transformations, eliciting complex reasoning in base models without gradient updates.

Thinking Inertia: LLMs Keep Thinking When Told Not To

This paper defines LLM "no-thinking" as an observable response-level behavior, establishes a three-level evaluation framework across Empty-Thinking Rate, Question–Pre-answer Relevance, and Explicit Inference Rate, and demonstrates that explicit reasoning persistently leaks under disabled switches and strict suppression instructions due to "Thinking Inertia", which is heavily governed by the task's answer space.

ThinkSafe: Self-Generated Safety Alignment for Reasoning Models

ThinkSafe formalizes safety realignment as a KL projection onto the safe simplex and proves that the student's own safety-filtered distribution is the unique KL-optimal target, proposing a self-generated alignment framework via refusal steering that eliminates external teacher distillation discrepancy and achieves superior safety-reasoning balance at an order of magnitude less compute than online RL.

TTCS: Test-Time Curriculum Synthesis for Self-Evolving

Addressing unreliable pseudo-labels and the lack of learnable samples in test-time reinforcement learning, TTCS introduces a co-evolving framework where a synthesizer dynamically generates capability-aligned curriculum variants to guide an online solver toward stable self-evolution.

TTCS: Test-Time Curriculum Synthesis for Self-Evolving

TTCS proposes a co-evolving test-time training framework that couples a synthesizer and a solver initialized from the same base model via iterative GRPO, dynamically synthesizing tractable curriculum variants aligned with the solver's capability frontier to turn noisy pseudo-labeling into stable self-evolution.

Verifier-Backed Hard Problem Generation for Mathematical Reasoning

VHG introduces an independent verifier into the setter-solver self-play loop to decouple problem validity from solver difficulty, eliminating reward hacking and generating valid, challenging mathematical problems for both solver RL training and benchmark evaluation.

What are Key Factors for Updates in RL for LLM Reasoning?

Uncovers the "gradient dominance reversal" mechanism driven by importance-sampling ratio variance across off-policy regimes in RLVR, and proposes Adaptive Clip Policy Optimization (ACPO) to dynamically calibrate bin-wise clipping boundaries, achieving superior reasoning performance and cross-regime stability.

What should post-training optimize? A test-time scaling law perspective

To bridge the objective mismatch and budget disparity between mean-reward post-training and deployment-time Best-of-\(N\) sampling, this paper introduces Tail-Extrapolated Advantage (TEA) and debiased Prefix-TEA, optimizing large-budget test-time performance using only small rollout groups via Gaussian upper-tail extrapolation.

When Form Changes but Logic Doesn’t: Building Logic-invariant LLMs through Logical Structure

To tackle LLMs' vulnerability to decision flips under equivalent reformulations and poor out-of-distribution logical generalization, this paper proposes LoGIST, a logical structure-guided reasoning framework that maps reasoning instances to canonical Reduced Ordered Binary Decision Diagrams (ROBDDs) and guides decoding via contrastive graph embeddings and gated cross-attention, cutting inconsistency rates nearly in half across diverse training paradigms.

When Form Changes but Logic Doesn't: Building Logic-invariant LLMs through Logical Structure

Addressing the core limitation where large language models are brittle to surface-level syntactic rewrites despite identical underlying semantics, LoGIST canonicalizes inputs into Reduced Ordered Binary Decision Diagrams (ROBDDs), learns topology-aware logic embeddings via contrastive pretraining, and conditions LLM decoding through gated cross-attention to halve logical inconsistency and boost generalization to unseen logical forms.

When Reasoning Meets Its Laws

This paper introduces the Laws of Reasoning (LoRe) framework, theoretically formalizing linear compute scaling and exponential accuracy decay with respect to problem complexity, evaluates models via LoRe-Bench, and introduces SFT-Compo to enforce compute compositionality, substantially improving general reasoning performance.

Why Deterministic PRM Guidance Underperforms in Discrete Diffusion Reasoning

This diagnostic study charges diffusion generation and reward scoring to a shared inference budget, finds that deterministic top-1 PRM guidance on Dream-7B trails independent sampling with a task-matched ORM, and locates the failures through candidate-pool, terminal-scoring, and readout controls.

Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression

Addressing the training instability, representation collapse, and opacity of continuous latent reasoning, Discrete Latent Reasoning (DLR) discretizes continuous reasoning states into compact discrete tokens via rendered CoT compression and stochastic vector quantization, enabling standard autoregressive LM modeling and process-reward RL with up to 20× token reduction.

World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry

Addressing poor generalization and action-following hallucinations in world models under scarce action-labeled interaction data, this paper proposes World Action Verifier (WAV), an asymmetric forward-inverse verification cycle that leverages video subgoal generation and sparse inverse dynamics to achieve sample-efficient self-improvement without expensive human demonstrations.