💬 LLM (Other)¶
🧠 NeurIPS2026 · 98 paper notes
📌 Same area in other venues: 🎞️ ECCV2026 (13) · 📷 CVPR2026 (3) · 🔬 ICLR2026 (55) · 💬 ACL2026 (61) · 🧪 ICML2026 (39) · 🤖 AAAI2026 (30)
🔥 Top topics: LLM ×25 · Diffusion Models ×19 · Image Restoration ×3 · Dialogue ×3 · Reasoning ×3
- Allocate Marginal Reviews to Borderline Papers Using LLM Comparative Ranking
-
Proposes allocating surplus reviewer capacity at large ML conferences specifically to borderline papers identified beforehand via LLM pairwise comparisons and Bradley–Terry ranking, improving decision reliability without increasing overall reviewer load.
- Causal Edit Serialization
-
Proposes Causal Edit Serialization (CES), integrating a lightweight localization head and dual-index RoPE into standard causal decoders to serialize explicit edit programs—covering insertion, deletion, replacement, copy, and move—directly within the causal token stream while preserving the causal attention mask.
- Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effects
-
This paper establishes the first generalized discoverable extraction framework for diffusion language models under arbitrary masking patterns and stochastic denoising trajectories, theoretically proves and empirically validates that sampling resolution monotonically governs verbatim memorization, and reveals that DLMs exhibit substantially lower PII leakage risk than autoregressive models under aligned settings.
- COMPLLLM: Fine-tuning LLMs to Discover Complementary Signals for Decision-making
-
ComplLLM introduces a decision-theoretic post-training framework that fine-tunes language models using best-attainable utility gain as rewards via SFT and GRPO, enabling LLMs to surface actionable, discrete complementary signals from unstructured text that improve upstream model recommendations.
- CompLLM: Fine-Tuning LLMs to Discover Complementary Signals for Decision-Making
-
CompLLM introduces a decision-theoretic post-training framework that fine-tunes LLMs using best-attainable payoff gain as a reward under GRPO, discovering discrete complementary signals in unstructured text that repair blind spots in upstream agent recommendations.
- Confidence-Based Decoding is Provably Efficient for Diffusion Language Models
-
This paper develops the first theoretical framework for confidence-based decoding in diffusion language models (DLMs), proving that an entropy sum-based adaptive unmasking strategy achieves \(\varepsilon\)-accurate sampling in KL divergence with an expected iteration complexity of \(\widetilde{O}(H(X_0)/\varepsilon)\), enabling automatic acceleration on low-entropy data without hyperparameter tuning.
- Conformal Language Modeling via Posterior Sampling
-
Reframes conformal language model hallucination control from post-hoc claim deletion to posterior sampling conditioned on high-confidence regions, incorporating an explicit abstention mixture prior and monotone risk envelopes to achieve rigorous finite-sample risk bounds alongside superior downstream generation utility.
- Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata
-
Addressing the failure of autoregressive constrained decoding in discrete diffusion language models caused by simultaneous multi-position denoising, this paper formulates finite automata as chain-structured graphical models and introduces an exact, tractable log-depth tree sampling algorithm that guarantees 100% constraint satisfaction with minimal inference overhead.
- Continuous Diffusion Scales Competitively with Discrete Diffusion for Language
-
This paper revisits likelihood-based continuous diffusion language models and introduces RePlaid by aligning the architecture with modern discrete diffusion baselines, proving that continuous diffusion scales on par with discrete diffusion and establishes a new state of the art among continuous models.
- Continuous Latent Diffusion Language Model
-
This paper introduces Cola DLM, a continuous latent diffusion language model that decouples text generation into global continuous latent prior transport via a block-causal DiT and local textual realization via a Text VAE decoder, escaping strict left-to-right sequential bottlenecks and exhibiting favorable scaling dynamics over competitive 2B baselines.
- Coupling Models for One-Step Discrete Generation
-
Coupling Models introduces a two-stage discrete generative paradigm that establishes a sampleable coupling between discrete sequences and standard Gaussian latents via likelihood-based normalizing flows, enabling parallel single-step sequence generation and direct terminal guidance.
- DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models
-
DACA-GRPO identifies the lack of temporal credit assignment and the severe mean-field likelihood bias in diffusion LLM reinforcement learning, introducing zero-cost Denoising Progress Scores (DPS) for token advantage modulation alongside Stratified Masking Likelihood (SML) to achieve consistent, plug-and-play gains across seven benchmarks.
- DAWN: Dependency-Aware Fast Inference for Diffusion LLMs
-
DAWN is a training-free dependency-aware fast inference framework for diffusion LLMs that purges attention sink biases to build a sparse dependency graph, pairing anchor-guided threshold relaxation with conflict-based greedy scheduling to deliver 1.80–8.06× speedups without quality loss.
- Dependency-Guided Parallel Decoding in Discrete Diffusion Language Models
-
To address the distributional mismatch caused by strongly dependent tokens in parallel unmasking for discrete diffusion language models, this paper proposes DEMASK, a lightweight dependency predictor that estimates pairwise conditional influence matrices in a single forward pass and greedily unmasks approximately independent positions with provable total variation error bounds, achieving a 1.7–2.2× speedup while matching or improving accuracy.
- Depth Exploration for LLM Decoding
-
Depth Exploration Decoding (DEX) eliminates the single-exit depth selection bottleneck and its harsh rollback penalties by exploring multiple candidate depths in parallel on an expansion lattice, validating and committing the exact final-depth token while collapsing stale branches to retain reusable state, achieving lossless acceleration with up to 1.8x throughput speedup.
- Diffusion-State Policy Optimization for Masked Diffusion Language Models
-
Addressing the coarse credit-assignment issue of terminal-only rewards in masked diffusion language models, this paper proposes DiSPO, a plug-in layer that branches from intermediate masked states using rollout-cached logits and updates only newly filled tokens, substantially boosting mathematical reasoning and planning without additional multi-step diffusion rollouts or optimizer steps.
- Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning
-
This paper introduces the next-token probability discrepancy (\(d_{\text{NTP}}\)) as an intrinsic alignment criterion for task vectors and develops Linear Task Vector (LTV), a closed-form ridge regression method that matches zero-shot inference latency while significantly outperforming existing task vector baselines and enabling cross-scale transfer.
- DMax: Aggressive Parallel Decoding for dLLMs
-
DMax mitigates cascading error accumulation in masked diffusion language models under aggressive parallel decoding by unifying on-policy uniform training with confidence-weighted soft parallel decoding, boosting tokens per forward pass (TPF) on GSM8K from 2.04 to 5.48 with zero loss in generation accuracy.
- Dominant-Layer ZO: A Single Layer Dominates Zeroth-Order Fine-Tuning of LLMs
-
Zeroth-order fine-tuning of large language models is sharply dominated by a single decoding layer that aligns with the first activation-outlier layer; updating only this single layer matches or outperforms full-model ZO while delivering up to a 4.52× end-to-end training speedup.
- Don't Lose Focus: Activation Steering via Key-Orthogonal Projections
-
Addressing the core tension where activation steering severely impairs LLM reasoning and retrieval utility by disrupting query-key matching, this paper proposes Steering via Key-Orthogonal Projections (SKOP), an approach that projects steering vectors onto the orthogonal complement of dominant focus-to-tail key differences for high-risk attention heads, cutting utility degradation by 5–7× while preserving over 95% of steering efficacy and enabling robust long-context activation steering.
- Dynamic Latent Routing
-
Addressing the core bottleneck where discrete latents in LLM post-training disrupt pre-trained sequence structure and require multi-stage warm-up, this work establishes General Dijkstra Search (GDS) theory and introduces Dynamic Latent Routing (DLR) to dynamically search and inject chunk-level continuous steering vectors into the residual stream within a single training stage, consistently outperforming supervised fine-tuning in low-data regimes.
- Efficient Scaling of LLM Training with Flexible Context Parallelism
-
Addressing severe load imbalance and communication redundancy caused by skewed sequence length distributions in long-context and multimodal LLM training, this paper proposes Flexible Context Parallelism (FCP), which eliminates powers-of-two constraints to support arbitrary integer parallelism degrees via ring attention and leverages a millisecond-level 2D dynamic programming scheduler to reconfigure communication groups adaptively per micro-batch, achieving up to 1.47× average training throughput gain and 2.24× speedup on highly unbalanced batches.
- Emergent Semantic Role Understanding in Language Models
-
By conducting linear probing and targeted causal neuron ablation on frozen autoregressive Transformers on QA-SRL across multiple scales, this paper proves that semantic role structure emerges from unsupervised pre-training alone, and reveals that its underlying neural implementation transitions from localized specialized circuits to distributed and cross-role interfering representations as model scale increases.
- Ensembling Language Models with Sequential Monte Carlo
-
Addressing the failure of local token-level probability averaging to sample from global string ensemble posteriors and bridge tokenizer mismatches, this paper introduces a unified \(f\)-ensemble framework and a byte-level Sequential Monte Carlo (SMC) algorithm, demonstrating that consensus-seeking operators with rigorous posterior sampling significantly outperform conventional baselines.
- Evaluating Whether LLMs Can Reliably Connect the DOTs?
-
This paper introduces a multi-domain narrative text infilling benchmark comprising 9,142 instances across four distinct narrative genres to evaluate 20 open-source large language models under varied instruction specificity and reasoning prompts. The empirical evaluation reveals that parameter scale is not a reliable predictor of infilling quality (Gemma-2-2B achieves the highest qualitative score of 4.02/5, outperforming models over ten times larger), explicit chain-of-thought reasoning yields only marginal improvements (+0.6%), and counterfactual reasoning significantly degrades narrative consistency.
- Fine-Tuning Improves Information Conveyance in Language Models
-
Proposes Canopy Entropy (\(\mathrm{CE}^{\star}\)) and a prompt-controlled length-entropy correlation metric, demonstrating that fine-tuning does not merely shrink the generation space, but fundamentally reorganizes uncertainty allocation to significantly boost semantic information conveyance efficiency.
- FlexTab: A Flexible Encoder-Decoder Architecture for In-Context Learning Across Diverse Tabular Tasks
-
FlexTab introduces a decoupled encoder-decoder architecture for tabular in-context learning that pairs a single, target-agnostic encoder with a modular suite of task-specific decoders, establishing state-of-the-art performance across classification, regression, anomaly detection, clustering, cross-table entity matching, and relational database predictions.
- Flow Map Language Models: One-step Language Modeling via Continuous Denoising
-
Addressing the severe quality collapse of discrete diffusion language models in the few-step regime caused by factorization error, this paper introduces Flow Language Models (FLM) over continuous one-hot token embeddings and distills them into Flow Map Language Models (FMLM) via a novel simplex-valued two-time denoiser, surpassing 8-step discrete diffusion generation in a single forward evaluation.
- Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap
-
Addressing the severe "Lost-in-Conversation" performance drop when instructions are revealed incrementally across multiple conversational turns, this paper introduces Found in Conversation (FiC). By first priming models with multi-turn SFT to defer and request clarification when information is incomplete, and subsequently employing View-Asymmetric Self-Distillation (VASD) to distill representations from an information-equivalent single-turn view into on-policy multi-turn trajectories, FiC recovers 92%–100% of single-turn performance without requiring any stronger external teacher.
- Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D
-
Reveals why frontier LLMs fail catastrophically on exact string copying due to induction-shortcut inductive bias in 1D positional encodings, and introduces 2D-RoPE (and adaptive Auto-2D-RoPE) to organize text into a 2D grid, achieving perfect length generalization up to hundreds of times the training context.
- Frozen Memory Is Not Enough: Rethinking External Memory as Extraction
-
This paper proposes a cross-model frozen-memory extraction protocol, demonstrating that an Engram-style external memory table remains universally portable across heterogeneous backbones and tokenizers when paired with a tokenizer-agnostic addressing scheme and an expressive target-side reader.
- GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Models
-
Guided Denoiser Self-Distillation (GDSD) reformulates reinforcement learning for diffusion large language models as likelihood-free denoiser self-distillation from an advantage-guided teacher, eliminating training-inference mismatch bias and partition functions via token-level logit centralization.
- How Do Language Models Compose Functions?
-
This paper investigates the latent mechanisms employed by feedforward large language models when solving two-hop factual and functional compositions \(g(f(x))\), revealing a duality between multi-step compositional computation and direct idiomatic retrieval that is strongly governed by linear translation geometry across embedding spaces.
- InfoSFT: Learn More and Forget Less with Information-Aware Token Weighting
-
Addressing the issues of overfitting and catastrophic forgetting in standard SFT caused by uniformly fitting low-likelihood offline data, InfoSFT derives a proximal policy update weighting rule that concentrates gradient signals on medium-confidence tokens, requiring only a one-line loss modification to simultaneously improve generalization and mitigate capability forgetting.
- LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
-
LangFlow bridges embedding-space continuous diffusion with Flow Matching via Bregman divergence, deriving a rigorous ODE-based perplexity bound and introducing an information-uniform Gumbel noise schedule alongside self-conditioning to rival and surpass state-of-the-art discrete diffusion language models.
- Language Model Memory and Memory Models for Language
-
Addressing the finding that autoregressive language model embeddings retain minimal input information and struggle with arbitrary retrieval, this paper introduces a parallelizable encoder-decoder memory model combining causal modeling with explicit copy objectives to enable efficient inference with high-fidelity memory retention.
- Learning When to Trust LLM Priors: A Validated Framework for Semantic Prior Integration
-
Addressing the vulnerability of supervised models to noisy, misspecified, or hallucinated LLM priors, Statsformer introduces a model-agnostic framework that injects semantic feature priors into heterogeneous statistical learners via monotone adapter families and calibrates aggregation weights using out-of-fold validation, formally guaranteeing performance no worse than prior-free baselines up to statistical estimation error.
- Learning When to Trust LLM Priors: A Validated Framework for Semantic Prior Integration
-
Addressing the vulnerability of supervised models to noisy, misspecified, or hallucinated LLM priors, Statsformer introduces a model-agnostic framework that injects semantic feature priors into heterogeneous statistical learners via monotone adapter families and calibrates aggregation weights using out-of-fold validation, formally guaranteeing performance no worse than prior-free baselines up to statistical estimation error.
- MAGE: All-[MASK] Block Already Knows Where to Look in Block Diffusion LLM
-
Addressing the severe recall degradation that existing dynamic estimators suffer under the per-block shared KV subset constraint in block diffusion LLMs, MAGE leverages training-induced All-[MASK] query anchoring to run a single exact attention pass at Step 1, asynchronously extracting Top-\(k\) KV indices that are frozen across the remaining denoising steps to achieve up to \(6.82\times\) end-to-end inference speedup with near-lossless accuracy.
- MemAudit: An Exact Package-Oracle Evaluation Protocol for Budgeted Long-Term LLM Memory Writing
-
MemAudit formalizes budgeted long-term memory writing as an exact finite submodular coverage problem across the intersection of a knapsack constraint and a partition matroid; by certifying package optima via branch-and-bound and MILP, it decouples write-time semantic compression from downstream retrieval and reader reasoning.
- MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels
-
Addressing the bottleneck of expensive and scarce ground-truth reward annotations in LLM reinforcement learning, MemReward builds a cross-domain heterogeneous experience memory graph connecting queries, thinking processes, and final answers, leveraging GNNs to propagate reward signals from 20% labeled rollouts to achieve over 97% of full-supervision Oracle performance.
- Meta-Learning Preferences for Multilingual LLM Alignment
-
This paper formulates multilingual LLM preference alignment as a task-level meta-learning adaptation problem, instantiating MAML-RLHF and MAML-DPO algorithms via first-order MAML. In an extremely low-resource regime with only 100 target-language preference pairs, it achieves up to a 28% win-rate gain over baselines and establishes the first theoretical convergence guarantees under Bradley-Terry preference losses.
- Minimally Invasive Steering of Language Models
-
To prevent unregularized activation steering from degrading output distributions during test-time alignment, this paper proposes Minimally Invasive Steering Vector Optimization (MISVO), which penalizes pre-logit interventions using a Fisher-quadratic regularizer derived from local KL geometry, achieving state-of-the-art test-time reward with analytic gradients while bypassing backpropagation through the transformer body.
- More is Not More: What Matters for Diversity in LLM Opinions
-
Through a factorial audit decoupling persona depth and interaction architecture across 7 chat models and 100 real open-ended queries, this work proves that persona detail yields sharply diminishing returns, interaction architectures explore complementary opinion spaces, and low-cost heuristics provide negligible gains.
- Mutable Transcripts: Mitigating Context Pollution through Editable Conversation State
-
The paper lets users rewrite the state of an entire conversation through natural language rather than append more corrective messages; a controlled study with 17 participants supports usability, and three representative cases show shorter retained transcripts, without establishing improvements in long-term task performance or total API cost.
- Natural Language Actor-Critic: Scalable Off-Policy Learning in Language Space
-
Natural Language Actor-Critic (NLAC) trains large language model policies via a generative natural language critic using a single-step language Bellman backup to learn compact future summaries off-policy, followed by self-refinement and distillation for scalable policy improvement without action enumeration.
- No Model Required: Text Entropy Rate Filtering Mitigates Iterative Fine-Tuning Collapse
-
This paper introduces a model-free training data filtering mechanism based on the non-parametric Kontoyiannis entropy rate estimator \(\hat{H}_K\), which measures sequence complexity purely through match-length statistics and successfully halts model collapse during iterative fine-tuning loops while outperforming logprob-based methods.
- On Data Engineering for Scaling LLM Terminal Capabilities
-
Addressing the scarcity and proprietary nature of terminal agent training data, this paper introduces Terminal-Task-Gen, a synthetic data pipeline yielding the Terminal-Corpus dataset, and develops Nemotron-Terminal models that outperform 480B-scale frontier baselines on Terminal-Bench 2.0 with only 32B parameters.
- On the Importance of Gating: Memorization vs. In-Context Learning in State Space Models
-
This paper demonstrates through theory and multi-scale experiments the double-edged sword of gating mechanisms in State Space Models (SSMs): strong gating biases the optimization dynamics toward in-weights memorization and severely delays in-context learning (ICL), whereas it remains essential for suppressing distractor noise during length generalization.
- On the Importance of Gating: Memorization vs. In-Context Learning in State Space Models
-
This paper proves theoretically and validates across synthetic-to-agentic benchmarks that the recurrent gating mechanism in state space models (SSMs) biases optimization dynamics toward an in-weights memorization shortcut that severely delays in-context learning, while demonstrating that controlled decay is fundamentally necessary to suppress distractor noise variance and unlock length generalization.
- On the Nature of Attention Sink that Shapes Decoding Strategy in Omni-LLMs
-
This paper systematically investigates attention sinks in Omni-LLMs, reveals their functional nature as a shared global bias, refutes the assumption that sink heads are purely redundant, and proposes OutRo—a training-free inference-time strategy that achieves consistent cross-modal reasoning gains across 7 video QA benchmarks with only 1.11× decoding latency overhead.
- On the Nature of Attention Sink that Shapes Decoding Strategy in Omni-LLMs
-
This paper presents the first systematic analysis of attention sink representations in Omnimodal LLMs, uncovering that sink value vectors serve as a shared global bias across token outputs, and introduces OutRo—a training-free, FlashAttention-compatible inference-time modulation method that incurs only 1.1× decoding overhead.
- One Threshold Does Not Fit All Languages: Language-Conditional Deferral for Reliable and Efficient Low-Resource Text Classification
-
The paper assigns per-language conformal thresholds to a single inexpensive multilingual classifier, addressing language-specific coverage gaps hidden by pooled calibration while exposing human-review costs; the guarantee concerns wrong automatic outputs over all traffic, not accuracy conditional on automatic output.
- PCBSchemaGen: Reward-Guided LLM Code Synthesis for Printed Circuit Boards (PCB) Schematic Design with Structured Verification
-
Addressing the absence of unit tests and golden references in PCB schematic design, PCBSchemaGen introduces a training-free inference-time framework that pairs a datasheet-induced knowledge graph with a deterministic 5-layer continuous-reward verifier and a Thompson Sampling arm-acquiring bandit, boosting the Pass@1 of an open-weight 31B model to 81.3% and zero-shot transferring across held-out commercial circuits.
- Plausibility Is Not Prediction: Contrastive Evidence for LLM-Based Cellular Perturbation Reasoning
-
Reveals that LLM-based virtual-cell models suffer from severe miscalibration and false-positive bias by confusing mechanistic plausibility with empirical prediction, and introduces CORE to organize contrastive same-gene support evidence from a harmonized knowledge graph to restore perturbation-specific discrimination.
- Pretraining Curricula Enable Selective Fine-tuning
-
This paper demonstrates that imbalanced pretraining curricula induce task-specialized, disentangled neural circuits in Transformers, substantially mitigating overrefusal and enabling highly selective behavioral suppression during downstream safety fine-tuning.
- Probing Persona-Dependent Preferences in Language Models
-
By training linear probes on residual-stream activations at end-of-turn tokens to predict revealed pairwise choice utilities, this paper identifies a genuine evaluative preference vector that causally controls task choices and transfers across qualitatively distinct prompted personas.
- ProgramBench: Can Language Models Rebuild Programs From Scratch?
-
Addressing the constraint that existing code benchmarks are confined to localized bug fixing or predefined skeleton completion, this paper introduces ProgramBench, an implementation-agnostic benchmark where agents must architect, implement, and build software from scratch given only a gold binary and documentation under strict no-internet isolation, evaluated via coverage-guided black-box behavioral test suites.
- Prompt–Activation Duality: Improving Activation Steering via Attention-Level Interventions
-
Addressing severe coherence collapse in multi-turn activation steering caused by KV-cache contamination, this paper introduces Gated Cropped Attention-Delta steering (GCAD), which injects cropped system-prompt attention signals into attention outputs with query-key compatibility gating to stabilize long-horizon trait control.
- Re-evaluating Confidence Remasking in Masked Diffusion Language Models
-
This paper rigorously re-evaluates representative training-free confidence remasking (WINO) in masked diffusion LLMs, demonstrating that it yields negligible gains over adaptive unmasking under standard short-block greedy decoding due to high flip-flop rates, while effectively correcting errors in stochastic settings at the expense of generation diversity.
- Reconsidering Positional Supervision in Masked Diffusion Language Model Training
-
Revealing that masked diffusion language models (MDLMs) are acutely vulnerable to minor positional shifts during iterative parallel decoding, this paper adapts alignment flexibility from non-autoregressive translation into SFT via a
<slack>blank token augmentation and a surface-form-preserving CTC-S auxiliary loss, achieving statistically significant improvements across four open-ended generation benchmarks without altering standard inference. - Reflective Prompted Policy Optimization: Trajectory-Grounded Revision and Salience Bias
-
Addressing the issue that black-box LLM policy search relies exclusively on scalar rewards without diagnostic signals, this paper proposes R2PO, a two-stage framework combining global parameter search with trajectory-grounded revision, which mitigates the identified "salience bias" and achieves faster, more stable policy optimization across ten RL environments.
- ReLoop: Structured Modeling and Behavioral Verification for Reliable LLM-Based Optimization
-
ReLoop integrates four-stage structured generation with solver-based parameter perturbation testing to detect and repair semantic "silent failures" in LLM-generated optimization formulations without requiring ground truth.
- Rethinking Language Model Scaling under Transferable Hypersphere Optimization
-
Addressing the absence of hyperparameter transfer laws and training instability under second-order matrix optimizers, this paper proposes HyperP (Hypersphere Parameterization) and SqrtGate on Frobenius hyperspheres, achieving zero-shot optimal learning rate transfer across model width, depth, token budgets, and MoE granularity while delivering 1.58x compute efficiency leverage at \(6\times 10^{21}\) FLOPs with non-increasing instability metrics.
- Rethinking LLM Fine-Tuning via Weight Space Reparameterization: Preserving Safety during Downstream Adaptation
-
To prevent downstream fine-tuning from overriding safety alignment, this paper proposes WSR-Tune, which constructs an orthogonal basis via activation covariance SVD to reparameterize weight matrices, concentrates diffuse safety representations into a compact set of coordinates to freeze, and updates only the complementary subspace to preserve safety while boosting task adaptation.
- Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior
-
Evaluating LLM psychometric self-reports against downstream choices across a \(2 \times 2 \times 2\) factorial framework reveals that self-reports predict behavior only under shared context and fine-grained behavioral anchoring, while cross-session generalization is task-specific and cannot be rescued by persona prompting.
- Rethinking Structured Generation: Can Graph-Based Reasoning Resolve Ambiguity?
-
Reformulates structured generation from a deterministic one-to-one mapping into an ambiguity-aware set-level reasoning problem, using candidate relational graphs, edge-aware GNN scoring, and a multi-valid training objective to eliminate gradient suppression on valid alternative interpretations.
- RVR: Retrieve-Verify-Retrieve for Comprehensive Question Answering
-
To overcome the bottleneck where single-pass retrievers and agentic multi-hop search systems fail to balance high answer coverage with low redundancy in open-domain multi-answer QA, this paper proposes Retrieve-Verify-Retrieve (RVR)—a multi-round framework that filters high-quality documents using an LLM verifier to condition subsequent retrieval and fine-tunes retrievers for complementary targets, drastically improving complete recall while cutting agent search latency.
- Self-Rewarding Sequential Monte Carlo for Masked Diffusion Language Models
-
Addressing the myopic path collapse of masked diffusion language models caused by greedy step-wise confidence remasking, this paper proposes Self-Rewarding Sequential Monte Carlo (SR-SMC), which rigorously derives trajectory-level cumulative confidence as particle importance weights to steer diffusion sampling toward globally high-confidence solutions without extra training or external reward verifiers.
- Self Driving Datasets: From 20 Million Papers to Nuanced Biomedical Knowledge at Scale
-
Addressing the high maintenance costs, literature lag, and critical lack of experimental context in manually curated biomedical databases, this paper introduces Starling—an autonomous multi-agent deep research system built on an entity-tagged 22.5M-paper PubMed corpus that iteratively optimizes retrieval probes, induces schemas, and extracts 6.3M highly accurate, context-rich records across six therapeutic tasks at under $0.001 per record.
- Simulating Human Memory with Language Models
-
Addressing the issue that frontier LLMs fail as realistic user simulators due to their superhuman verbatim memory, this paper constructs a 10-task cognitive memory benchmark and introduces Compactor—a 4-chunk capacity-constrained agent architecture that induces human-like memory retention and forgetting patterns.
- SLS-Bench: A Benchmark for Incident Log Summarization with Synthetic Observability Data
-
Addressing the lack of end-to-end benchmarks due to privacy and data collection hurdles in real-world IT outages, SLS-Bench introduces 90 diverse incident tasks grounded in real-world postmortem reports with synthetic multi-tenant log streams and reference summaries, evaluating 14 LLM agents across quality, latency, and cost dimensions.
- Sparse Layers are Critical to Scaling Looped Language Models
-
Through rigorous IsoFLOP scaling studies, this paper demonstrates that sparse Mixture-of-Experts (MoE) layers resolve the scaling deficit of looped language models via inter-loop routing divergence, while loop boundaries structurally provide superior training-free early-exit efficiency.
- Structuring Open-Ended NAS: Semi-Automated Design Knowledge Structuring with LLMs for Efficient Neural Architecture Search
-
Addresses the disordered knowledge pools and search inefficiency in LLM-driven open-ended neural architecture search by proposing a semi-automated model design attribute tree, coupled with FairNAD featuring fair idea sampling, Pareto-aware mutation, and multi-verifier feedback loops.
- TED: Text-Axis Evidence Decomposition for Prompted Anomaly Localization
-
Addressing the failure where CLIP-adapted anomaly detectors assign elevated scores to visually complex normal regions under domain shift, TED introduces a post-hoc evidence decomposition framework that compares query patch projections against source defect and hard-false-positive banks along the host text axis to achieve substantial pixel-level localization gains without target-domain training.
- The Expressivity Boundary of Probabilistic Circuits: A Comparison with Large Language Models
-
This paper introduces a unified autoregressive framework to theoretically and empirically characterize the expressivity gap between Probabilistic Circuits (PCs) and Transformers, attributing it to an output bottleneck rooted in probability-space convex combinations and a context-encoding bottleneck driven by fixed vtree routing.
- The Grounding Gap: How LLMs Anchor the Meaning of Abstract Concepts Differently from Humans
-
By replicating cognitive science property-generation and multi-dimensional rating paradigms across 21 frontier and open LLMs alongside SAE mechanistic probing, this work demonstrates that models over-rely on verbal associations while systematically underproducing emotional and internal-state properties during free generation, exposing a robust, cross-architecture "grounding gap" far below human ceilings.
- The Override Gap: A Magnitude Account of Knowledge Conflict Failure in Hypernetwork-Based Instant LLM Adaptation
-
Reveals that hypernetwork-based instant LLM adaptation (e.g., Doc-to-LoRA) systematically fails on knowledge conflicts due to insufficient adapter signal amplitude rather than representational errors, proposing training-free Selective Layer Boosting and Conflict-Aware Internalization to raise deep-conflict accuracy from 46.4% to 71.0%.
- The Silent Hyperparameter: Quantifying the Impact of Inference Backends on LLM Reproducibility
-
This paper systematically quantifies the impact of inference engines on LLM evaluation reproducibility, demonstrating that holding model weights, decoding parameters, and hardware strictly constant, swapping only the inference backend causes benchmark scores to shift by up to 16.6 percentage points due to hidden defaults and compounding numerical drift.
- Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action
-
This paper introduces NCP-ExploreToM, a benchmark assessing the capacity of LLMs to induce desired true and false belief states in other agents via multi-step physical planning and actions rather than conversational persuasion, revealing that GPT-5 completes ~80% of tasks and surpasses human performance while challenging the standard assumption that agentic tasks are strictly harder than question answering.
- Theory on Attention Dynamics for Out-of-Distribution In-Context Learning
-
This paper establishes the first theoretical framework characterizing out-of-distribution (OOD) generalization and finetuning forgetting for single-layer Softmax Transformers in in-context learning (ICL) by decomposing self-attention into \(\alpha\)-type (correct feature confidence) and \(\beta\)-type (incorrect feature interference) dynamic weights, deriving explicit geometric error expressions and phase-transition conditions for positive backward transfer versus catastrophic forgetting.
- Token Inflation: How Dishonest Providers Can Overcharge for Large Language Model Usage
-
Reveals a fundamental "trust paradox" in auditing per-token LLM billing, demonstrating that dishonest providers can exploit reliance on provider-controlled artifacts to inflate hidden reasoning tokens by up to 1469% at negligible compute cost while evading detection.
- Token Time Continuous Diffusion for Language Modeling
-
Token Time Continuous Diffusion (TTCD) introduces per-token local times and initial uncertainty-driven denoising schedules into continuous-space flow matching, eliminating the independent factorization error of discrete diffusion under high speedups and significantly outperforming prior baselines in few-step generation.
- Training-Free Task Vectors for LLM Behavioral Control
-
Introduces Training-Free Task Vectors (TFTVs), which map activation steering directions into rank-one weight-space edits using only forward-pass statistics and singular value decomposition, enabling persistent behavioral control—via addition, subtraction, and multi-trait composition—while strictly preserving general LLM utility.
- Training Optimal Large Diffusion Language Models
-
Introduces Quokka, the first comprehensive scaling law for diffusion language models across compute-constrained and data-constrained regimes, revealing that diffusion models are 2–5x more data-hungry than autoregressive models at equivalent FLOPs and modeling the U-shaped overfitting validation loss surface.
- Triage: Evaluating Prospective Metacognitive Control in LLMs under Resource Constraints
-
This paper introduces TRIAGE, the first benchmark framework for evaluating prospective metacognitive control in LLMs under finite token budgets, revealing that current models collapse when held to their self-declared budgets and that extended reasoning actively degrades sensitivity to unsolvable problems.
- Trust the Direction, Search the Step: Zero-and-First-Order Methods for LLM Fine-Tuning
-
The paper introduces the Zeroth-and-First-Order (ZFO) framework, which decouples high-dimensional optimization by taking update directions from a trusted first-order optimizer while performing two symmetric zeroth-order probes along that one-dimensional ray to construct local Taylor or Padé models for closed-form, adaptive step-size selection.
- TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Models
-
Addressing the computational intractability of exact log-likelihood in discrete diffusion and any-order autoregressive language models, this paper introduces the Tangent Upper Bound on Evidence (TUBE), which provides a finite-sample unbiased Monte Carlo upper bound estimator that, together with the ELBO, establishes a tight two-sided localization proving diffusion models still underperform standard autoregressive baselines in likelihood.
- Uncertainty Quantification for Large Language Diffusion Models
-
Addressing the mismatch between discrete diffusion generation and classical autoregressive uncertainty quantification, this paper introduces lightweight, zero-shot uncertainty signals derived from the denoising trajectory and proposes D-CoCoA, which matches strong multi-sample baselines while cutting computational overhead by up to 100x.
- Unlocking LLM Creativity in Science through Analogical Reasoning
-
Addressing the pervasive mode collapse of large language models in open-ended scientific problem-solving, this paper introduces an analogical reasoning (AR) framework based on cognitive structure-mapping theory that maps cross-domain object-relation topologies to unlock diverse, novel solutions, demonstrating substantial quantitative gains across four real-world biomedical benchmarks.
- Value Entanglement: Conflation Between Different Kinds of Good In (Some) Large Language Models
-
This paper uncovers pervasive "value entanglement" in large language models—where moral goodness is erroneously conflated with grammatical correctness and economic valuation—and demonstrates through residual stream geometry and directional ablation that this conflation is causal and rectifiable.
- When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction
-
Reveals a channel-transition account where LLMs lose multi-turn instruction following because attention to goal tokens decays and closes, forcing models to rely on residual stream representations and resulting in architecture-dependent, complexity-graded failure modes.
- When Should LLMs Be Less Specific? Selective Abstraction for Reliable Long-Form Text Generation
-
To tackle factual hallucinations arising from overly specific details in open-ended generation, this paper proposes Selective Abstraction (SA), a framework that decomposes text into atomic claims and replaces uncertain claims with higher-confidence, less specific abstractions, significantly curbing factual errors while preserving core information.
- Which Way Did It Move? Diagnosing and Overcoming Directional Motion Blindness in Video-LLMs
-
The paper reveals that current Video-LLMs suffer from "directional motion blindness" on elementary image-plane motion directions, diagnoses this as a "direction binding gap" where the internal linear signal fails to bind to verbal answer options, and introduces DeltaDirect, a training-only objective that predicts normalized 2-D motion vectors from adjacent-frame projector feature deltas with zero inference overhead.
- Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?
-
Introduces Who&When Pro, a large-scale failure attribution benchmark for AI agents built via deterministic warm-start prefix replay and controlled error injection, delivering 12,326 multimodal golden failure traces and benchmarking ten frontier LLMs across localization protocols.
- Why Are LLMs Confidently Wrong? Correcting Overconfident Errors via Causal Head Intervention
-
This paper proposes CoCHI (Class-oriented Causal Head Intervention), an inference-time calibration framework that decomposes model predictions into four correctness-confidence states, identifies and suppresses internal attention heads driving overconfident errors and noise via differentiable causal head gating, and achieves substantial reductions in expected calibration error with zero inference computational overhead while preserving accuracy.
- Within-Model vs Between-Prompt Variability in Large Language Models for Creative Tasks
-
Evaluating 12 LLMs across 10 creativity prompts with 100 repeated samples each (\(N=12{,}000\)) reveals that output quality (originality) is co-governed by model choice and prompts, whereas output quantity (fluency) is overwhelmingly dictated by intrinsic model behavior and intra-model sampling noise, empirically disproving single-sample (\(N=1\)) benchmark reliability.
- WorldSpeech: A Multilingual Speech Corpus from Around the World
-
To tackle the severe scarcity of paired audio-transcript data and alignment collapse in low-resource speech recognition, WorldSpeech introduces a 65k-hour ground-truth aligned speech corpus across 76 languages, accompanied by an iterative alignment refinement framework that substantially boosts data yield and ASR transcription accuracy.