📐 Optimization & Theory¶
🧠 NeurIPS2026 · 313 paper notes
📌 Same area in other venues: 🎞️ ECCV2026 (16) · 📷 CVPR2026 (23) · 🔬 ICLR2026 (225) · 🧪 ICML2026 (91) · 🤖 AAAI2026 (22) · 🧠 NeurIPS2025 (127)
🔥 Top topics: Adversarial Robustness ×13 · Diffusion Models ×12 · LLM ×8 · Continual Learning ×5 · Compression ×4
- A Biconvex Formulation for Stable Transport of Mixture Models with a Unique Solution
-
Optimal Mixture Transport (OMT) reformulates sample-to-sample optimal transport into a dual-entropy regularized, strictly biconvex optimization problem between subpopulation mixture models, proving a unique global minimizer in a single step with rigorous tilt-stability perturbation bounds and sample-size-independent complexity.
- A Differentiable Interior-Point Method in Single Precision
-
Addressing the severe ill-conditioning and arithmetic breakdowns of standard interior-point methods in low precision, this paper introduces a softplus-based implicit complementarity formulation that produces a spectrally bounded symmetric KKT linear system, enabling robust QP solving and high-fidelity differentiation natively in single precision (float32).
- A Kernel Nonconformity Score for Multivariate Conformal Prediction
-
Introduces the Multivariate Kernel Score (MKS), which lifts Mahalanobis distance to a reproducing kernel Hilbert space (RKHS), proves an equivalence to Gaussian process posterior variance with an exact MMD-KPCA decomposition, and achieves distribution-free finite-sample coverage while drastically compressing prediction region volumes.
- A Memory Efficient Unified Algorithm for Online Learning of Linear Dynamical Systems
-
Addressing high-dimensional linear dynamical systems with a few unstable or resonant modes, this paper introduces a unified online predictor whose memory footprint scales as \(\widetilde{O}(k)\) with the instability complexity \(k\) rather than the full state dimension \(d\), achieving sublinear squared regret without knowing the system matrices.
- A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems
-
By reverse-engineering the state-of-the-art dynamical systems foundation model DynaMix, this paper derives DynaBase—a minimal, interpretable two-parameter affine nearest-neighbor map that achieves competitive zero-shot attractor reconstruction at negligible computational cost while theoretically reconciling context parroting, flow tracking, and chaotic dynamics within a single mathematical framework.
- A Near-optimal SQ Lower Bound for Smoothed Agnostic Learning of Boolean Halfspaces
-
Under uniform marginals on the Boolean hypercube, this paper establishes a near-optimal Statistical Query (SQ) lower bound of \(n^{\Omega(\log(1+\sigma/\varepsilon^2)/\sigma)}\) for smoothed agnostic learning of halfspaces, matching the \(L^1\) polynomial regression upper bound via an explicit sine-Laguerre dual witness.
- A Novel Schur-Decomposition-Based Weight Projection Method for Stable State-Space Neural-Network Architectures
-
This paper introduces a novel backpropagation-compatible weight projection scheme based on the real Schur decomposition and a pre-factorized formulation for linear discrete-time state-space layers, guaranteeing asymptotic Schur stability with minimal parameter overhead and fast convergence.
- A Provably Convergent and Practical Algorithm for Gromov–Wasserstein Optimal Transport
-
Addressing the gap between exact-projection theory and approximate implementations in nonconvex Gromov–Wasserstein optimal transport, this paper proposes an inexact projected gradient (iPG) framework based on a verifiable primal feasibility residual condition, guaranteeing whole-sequence convergence to stationary points while delivering high sparsity and competitive graph alignment accuracy.
- A Reduction from Delayed to Immediate Feedback for Online Convex Optimization with Improved Guarantees
-
This paper introduces a unified continuous-time event model that decomposes delayed online convex optimization into non-delayed regret and prediction-drift penalties, dramatically improving the delay-dependent regret bound in bandit convex optimization from \(\mathcal{O}(\sqrt{T d_{\max}})\) and \(\mathcal{O}((T d_{\text{tot}})^{1/3})\) down to \(\mathcal{O}(\sqrt{d_{\text{tot}}})\).
- A Spectral Framework for Closed-Form Relative Density Estimation
-
This paper introduces an analytic spectral framework based on generalized eigenvalue decompositions that reformulates the variational estimation of operator-convex f-divergences (including KL) into an integral of weighted chi-squared divergences, yielding closed-form estimators for both divergences and relative log-density potentials purely from first- and second-order feature moments without iterative optimization or partition function evaluations.
- A Unified Approach for Computing Wasserstein Barycenters of Discrete and Continuous Probability Measures
-
This paper introduces FRBary, a primal mirror descent algorithm operating in Fisher-Rao geometry that utilizes KL divergence as the Bregman distance to compute exact, unregularized Wasserstein barycenters across discrete, continuous, and heterogeneous distributions with provable suboptimality guarantees.
- Achieving Better Local Regret Bound for Online Non-Convex Bilevel Optimization
-
For non-convex-strongly-convex online bilevel optimization (OBO), this paper proposes adaptive inner-loop tracking and novel window-averaged hypergradient algorithms that achieve optimal local regret bounds matching theoretical lower bounds in both standard and window-averaged settings, along with efficient fully single-loop variants.
- Achieving Directional-Stationarity from a Single Random Direction Step
-
This paper introduces an exploration-augmented framework that pairs any feasible base optimization oracle with a single-step randomized direction probe, proving that all accumulation points achieve strong directional stationarity (d-stationarity) almost surely under mild regularity, while strictly preserving the base method's convergence rate.
- Activation-Keyed Momentum: An Anisotropic Momentum Update via the Delta Rule
-
The paper introduces Activation-Keyed Momentum (AK-Momentum), treating linear-layer input activations as keys and backpropagated errors as values to update the momentum buffer via the canonical delta rule, achieving implicit input-side curvature correction without matrix inversion or persistent state overhead while saving up to 46.6% steps to reach AdamW's target validation loss in LLM pretraining.
- Active Learning of Conditional Generative Models via the Transport Neural Tangent Kernel
-
To tackle high-cost data acquisition in high-throughput scientific experiments with distributional outputs, this paper introduces the transport Neural Tangent Kernel (tNTK) to measure distribution sensitivity to model parameters, proves that it upper-bounds posterior Fréchet variance, and develops a greedy distribution-aware active learning strategy that minimizes posterior GP variance.
- Adam under Generalized Smoothness with Second-Moment-Type Stochastic Gradients
-
The paper analyzes finite-horizon calibrated Adam without bias correction under generalized smoothness and second-moment ABC noise, shows that the confidence exponent \(\delta^{-1/2}\) cannot generally be removed, and separates expectation upper bounds for \(p<1\) from worst-case lower bounds for a specified algorithm family when \(1\leq p<2\).
- Adaptive Calibration in Non-Stationary Environments
-
Addressing the gap between conservative adversarial online calibration and benign stationary performance, this paper develops an adaptive calibration framework parameterized by deviation \(C\) and stationary segment count \(K\), smoothly interpolating optimal i.i.d. and adversarial bounds across \(\ell_1\), \(\ell_2\), and pseudo KL calibration metrics.
- Adaptive Conditional Gradient Sliding: Projection-Free and Line-Search-Free Acceleration
-
For constrained convex optimization problems where projections are computationally prohibitive, this paper proposes the Adaptive Conditional Gradient Sliding (AdCGS) method, achieving optimal Nesterov \(\mathcal{O}(1/k^2)\) acceleration and linear convergence for locally strongly convex objectives without projections, line searches, or global smoothness parameters.
- Adaptive Delayed-Update Cyclic Algorithm for Variational Inequalities
-
For generalized Minty variational inequalities with monotone Lipschitz operators, this paper introduces ADUCA, the first parameter-free and line-search-free locally adaptive cyclic block coordinate method with a one-cycle delayed-update mechanism, establishing near-optimal \(\mathcal{O}(1/\epsilon)\) and \(\mathcal{O}(\log^2(1/\epsilon))\) oracle complexities.
- Adaptive Power Iteration Method for Differentially Private PCA
-
Under the standard privacy model where neighboring inputs differ by a single row measured in Euclidean norm, this paper introduces an adaptive filtering private power iteration method that achieves beyond-worst-case guarantees for low-coherence matrices without requiring any prior knowledge of the coherence parameter.
- Adaptively Incorporating Directional Hints into Zeroth-Order Optimization
-
Control-Variate Zeroth-Order Descent (CV-ZOD) incorporates low-dimensional directional hints as unbiased control variates into zeroth-order gradient estimation, pairing them with an adaptive step-size rule to seamlessly interpolate between first-order \(\mathcal{O}(1/T)\) and dimension-dependent \(\mathcal{O}(d/T)\) convergence rates on non-convex objectives.
- AdaptNC: Adaptive Nonconformity Scores for Conformal Prediction under Distribution Shift
-
To eliminate excessive conservatism caused by structural error distortion in dynamic robotic environments, AdaptNC introduces joint online adaptation of nonconformity score geometry and conformal thresholds, leveraging dynamic expert reweighting and a counterfactual replay mechanism to maintain valid coverage while substantially shrinking prediction region volumes.
- Adversary-Robust Learning from Fully Asynchronous Directional Derivative Estimates
-
FAR-SIGN proposes a fully asynchronous, signed directional projection framework with two-timescale tracking that eliminates sign bias and delivers adversary-resilient learning in parameter-server architectures without buffering or server-side reference data, matching near-optimal first-order and standard zeroth-order non-convex convergence rates.
- Affine-invariant Cubic Newton with Weak Learners
-
This paper extends Affine-Invariant Cubic Newton (AICN) to convex optimization with weak learners, showing that its update reduces to a damped weak-Newton step with an explicit stepsize driven by directional improvement, establishing a global \(\mathcal{O}(1/\overline{\Theta}_k^4 k^2)\) rate under semi-strong self-concordance and naturally covering coordinate descent.
- Algorithm for Contextual Queueing Bandits with Rate-Optimal Queue Length Regret
-
For contextual queueing bandits with unknown service rates, this paper proposes CQB-\(\eta\)-2, a three-phase algorithm that cuts off random exploration at a critical round and switches to pure UCB, improving queue length regret from \(\widetilde{\mathcal{O}}(T^{-1/4})\) to the rate-optimal \(\widetilde{\mathcal{O}}(T^{-1/2})\) and establishing a matching \(\Omega(T^{-1/2})\) minimax lower bound.
- Amortized Optimal Transport from Sliced Potentials
-
The paper uses inexpensive one-dimensional Kantorovich potentials as features, predicts original-space potentials with shared linear coefficients, and reconstructs approximate transport plans; RA-OT and OA-OT reduce training costs and support variable numbers of atoms, but are not universally fastest at inference or best in generation quality.
- Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models
-
Addressing the systemic underestimation of epistemic uncertainty in Gaussian Process Latent Variable Models (GPLVMs) caused by mean-field variational assumptions, this paper proposes Amortized Structured Stochastic Variational Inference (A-S-GPLVM), explicitly conditioning the latent posterior on reparameterized GP inducing variables while retaining closed-form kernel expectations and scalable minibatch optimization.
- AMUSE: Anytime Muon with Stable Gradient Evaluation
-
Analyzed through the river-valley loss landscape, AMUSE integrates Muon's rapid bulk subspace progress with the stabilizing effect of Schedule-Free iterate averaging via a time-varying interpolation schedule, completely eliminating learning rate decay schedules while pushing the performance-iteration Pareto frontier.
- Annealing in Variational Inference Mitigates Mode Collapse: A Theoretical Study of Gaussian Mixture Models
-
This paper develops a low-dimensional summary-statistics dynamical system for variational inference on bimodal Gaussian mixtures under simulated annealing, deriving a closed-form formula for the probability of mode collapse that identifies the annealing rate as the governing scaling parameter and qualitatively validates the findings on RealNVP normalizing flows.
- Approaching I/O-optimality for Approximate Attention
-
For near-linear approximate attention via polynomial expansion, this paper establishes comprehensive tight and nearly-tight I/O complexity bounds under a two-level memory hierarchy, demonstrating that data transfers scale almost-linearly with sequence length and dramatically breaking the quadratic I/O barrier of FlashAttention.
- Archimedean Copula Inference via Taylor-Mode AD
-
Introduced acopula, the first JAX-native framework for nested Archimedean copula inference, which leverages polynomial powering of Taylor-mode automatic differentiation outputs to eliminate hand-derived Bell polynomial tables and enable exact likelihood computation and parameter gradients under arbitrary per-variable right-censoring in polynomial time.
- ASTRA: ADMM-Accelerated Topology Reconfiguration for Dynamic Satellite Constellations
-
This paper proposes ASTRA, an adaptive satellite topology reconfiguration framework that derives a structured offline ADMM solver and closed-form online subroutines for OGD and OCG, proves strong convexity and logarithmic static regret for entry-wise non-zero utilities, and bounds dynamic regret under inexact inner iterations, achieving state-of-the-art connectivity quality with substantial speedups on real-world Starlink partial-deployment constellations.
- AsyncMesh: Fully Asynchronous Optimization for Data and Pipeline Parallelism
-
AsyncMesh establishes a fully asynchronous optimization framework across both data and pipeline parallelism axes in a 2D mesh, eliminating synchronization bubbles via weight look-ahead and exponential moving average (EMA) delay-corrected sparse averaging while matching fully synchronous convergence.
- AT-SKM-Net: An Accelerated Trainable Sampling Kaczmarz-Motzkin Framework for Linear Hard-Constraint Feasibility on Dynamic Graphs
-
To eliminate heavy projection and sampling overheads in dynamic graph linear hard-constrained optimization, AT-SKM-Net couples low-rank Cholesky updates with HGNN-guided active-set hybrid sampling, guaranteeing zero constraint violations while delivering 2.95× to 7.29× end-to-end projection speedups.
- Auditing Conformal Prediction for Language Models Under Inference-Time Model Shift
-
Formalizes inference-time model shift where mismatch in post-training inference configurations (quantization and temperature) breaks split-conformal score exchangeability despite fixed task distributions, demonstrating severe coverage degradation and establishing robust auditing baselines.
- Automated Kernel Discovery Towards Understanding High-Dimensional Bayesian Optimization
-
Kernel Discovery is an LLM-driven evolutionary framework for high-dimensional Bayesian optimization that decouples kernel search from raw numerical observations via a two-stage "mathematical formulation to executable code" pipeline, combined with a leave-one-out continuous ranked probability score (LOO-CRPS) criterion, achieving an average rank of 1.2 across 5 challenging benchmarks and uncovering the power of geometric and non-stationary kernels.
- Automated Reformulation of Robust Optimization via Memory-Augmented Large Language Models
-
Addressing the fundamental bottleneck in robust optimization (RO) where manually transforming uncertain models into deterministic counterparts is error-prone and labor-intensive, this paper introduces the first dedicated benchmark AutoRO-Bench alongside AutoREM, a tuning-free memory-augmented framework that autonomously synthesizes compact, non-redundant dual reformulation templates via unit-level experience decomposition and dual-check commit mechanisms, substantially outperforming existing memory baselines and expert prompts across diverse base LLMs and out-of-distribution benchmarks.
- B2P-Corr: Batch-to-Population Gradient Estimators for Non-Decomposable Correlation Losses
-
To tackle the severe \(O(1/B)\) gradient bias when optimizing non-decomposable correlation objectives (PCC/CCC) in stochastic mini-batch settings, B2P-Corr decouples moment estimation from backpropagation via stop-gradient pseudo-losses and lightweight global moment sketches, enabling asymptotically unbiased gradient tracking with constant \(O(B)\) cost.
- B3-PWL: GPU-Batched Branch-and-Bound for Piecewise-Linear Optimization with SOS2 Constraints
-
Addressing the sequential bottleneck of CPU simplex solvers in solving massive node LP relaxations, B3-PWL introduces the first GPU-batched branch-and-bound framework centered on a first-order primal-dual LP engine for SOS2 piecewise-linear optimization, delivering a 9.25× geometric-mean speedup over NVIDIA cuOpt via a custom B2-SpMV ragged sparse matrix kernel and a unified feasibility search module.
- Bayesian Optimization on Function Spaces via Sparse RKHS Manifolds
-
Addressing the expressivity bottleneck of prior Functional Bayesian Optimization (FBO) methods caused by fixed bases or post-hoc sparsification, this paper proposes L0MO (\(L^0\) Manifold Optimization), which jointly optimizes kernel support locations and linear coefficients over sparse stratified RKHS manifolds, delivering significantly higher representation power with compact candidate functions.
- Bayesian Optimization with Fisher Information Geometry: Gradient Bounds and Trust-Region Methods
-
The paper separates surrogate geometry from utility sensitivity through the pullback Fisher tensor of the posterior map, then replaces TuRBO's kernel-lengthscale weights with regularized local Fisher diagonal weights, achieving competitive SE-kernel benchmark performance without a regret or global convergence guarantee for outer Bayesian optimization.
- Benchmarking Optimizers for Large Language Model Pretraining
-
Through a standardized, controlled evaluation of 11 recent optimizers across four dense model scales, MoE architectures, varying batch sizes, and training durations (spending ~30,000 GPU hours across 2,900 trained models), this work establishes empirical Pareto frontiers and actionable hyperparameter guidelines for language model pretraining.
- Best Arm Identification for Bandits with Shifting Means
-
Under an adversarial non-stationary setting where latent common shifts perturb reward means while relative gaps remain invariant, this paper proves that classical GLRT and Track-and-Stop fail, and introduces Importance Weights for Shifting Means (ISM)—an algorithm achieving provable \(\delta\)-correctness and order-optimal sample complexity.
- Beyond Maximum Likelihood: Variational Inequality Estimation for Generalized Linear Models
-
This paper proposes an operator-based variational inequality (VI) framework for parameter estimation in generalized linear models (GLMs) and generalized additive models (GAMs), bypassing the non-convex, ill-conditioned, and non-smooth optimization landscapes of maximum likelihood estimation (MLE) under non-canonical links while establishing \(\mathcal{O}(N^{-1/2})\) statistical error bounds, asymptotic normality, and linear algorithmic convergence.
- Bidirectional Information Flow (BIF) - A Sample Efficient Hierarchical Gaussian Process for Bayesian Optimization
-
BIF constructs a soft parent prior from child Gaussian processes' acquisition maps and allocates real parent responses to children using uncertainty-aware weights for continual learning, improving reconstruction and learning trajectories in low-budget composite tasks; this feedback comprises biased pseudo-responses, not true subtask decomposition.
- Blocked Gibbs Meets Diffusion Transformers: Unsupervised Learning for Constraint Optimization
-
To resolve the fundamental mismatch between standard global continuous Gaussian diffusion and the need for localized, surgical edits in constraint optimization, BloGDiT introduces a Blocked Gibbs Diffusion Transformer that leverages annealed block resampling and adaptive mask selection to achieve state-of-the-art unsupervised constraint solving.
- BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability
-
Addressing standard Bayesian optimization's tendency to perturb weakly relevant parameters and yield dense, uninterpretable configurations, BONSAI introduces a lightweight candidate-pruning mechanism that greedily resets low-impact deviations back to default values under an acquisition-gap budget, preserving competitive optimization performance at minimal runtime overhead (1.5x) with sublinear regret and support recovery guarantees.
- Born-Preconditioned Learning for Iterative Neural PDE Solvers
-
Addressing the challenges of high-frequency wave simulation and ill-conditioned linear systems in heterogeneous media, this paper introduces Neural Preconditioned Born Series (NPBS), which identifies the Convergent Born Series as shifted-Laplacian left preconditioning and trains a residual-to-correction neural operator under a Green-operator-induced metric-matched loss, significantly reducing iteration counts and wall-clock solve time.
- BROS: Bias-Corrected Randomized Subspaces for Memory-Efficient Single-Loop Bilevel Optimization
-
To tackle the severe memory footprint caused by Hessian-vector products and auxiliary variables in large-scale neural network bilevel optimization, BROS updates lower-level and auxiliary variables in randomized subspaces with a layerwise Rademacher bi-probe correction, matching the optimal \(\mathcal{O}(\varepsilon^{-2})\) sample complexity while reducing peak training memory by up to 44.9%.
- Calibrating Generative Models to Feature Distributions with MMD Finetuning
-
This paper introduces kernel Calibrating Generative Models (kCGM), a post-training framework that aligns autoregressive and diffusion generative models to target black-box feature distributions by minimizing an unbiased kernel Maximum Mean Discrepancy (MMD) estimator paired with an \(O(M^2)\) coupled leave-one-out score-function gradient baseline (MMD-LOO) and KL regularization, substantially outperforming mean-matching CGM and direct supervised fine-tuning across small-molecule antibiotics, protein secondary structures, and regulatory DNA sequences.
- Capacity-Constrained Online Convex Optimization with Delayed Feedback
-
This paper investigates online convex optimization and bandit convex optimization under a hard tracking capacity constraint \(C\), reducing the problem via proxy-delay scheduling to a novel delayed-weighted framework and proving near-optimal regret with capacity as small as \(C=\Omega(\ln T)\).
- Certification-Enhanced Generalization Bounds
-
By recasting algorithmic stability as a formal verification specification via Abstract Gradient Training (AGT) and introducing a novel smooth sensitivity concentration inequality, this work delivers the first non-vacuous, sound, and tight generalization bounds without analytical assumptions on model convexity or smoothness.
- Characterizing the Generalization Error of Random Feature Regression with Arbitrary Data-Augmentation
-
This paper establishes the first exact deterministic equivalent for the test error of random feature ridge regression with arbitrary, potentially biased, and distribution-altering data augmentation in the high-dimensional proportional regime, providing explicit non-asymptotic bounds on the approximation error.
- Christoffel-DPS: Optimal Sensor Placement in Diffusion Posterior Sampling for Arbitrary Distributions
-
Addressing state estimation on complex non-Gaussian distributions, this work proposes the first distribution-free optimal sensor placement framework based on Christoffel functions, derives non-asymptotic recovery bounds, and introduces offline and online Christoffel-DPS algorithms that cut required sensor budgets by nearly half.
- Co-Optimization for Adaptive Conformal Prediction
-
CoCP introduces a co-optimization framework for adaptive conformal prediction that couples center translation and radius scaling via folded quantile regression and a boundary-local soft-coverage objective, asymptotically recovering the shortest highest-density intervals while rigorously preserving finite-sample marginal coverage.
- Colored Noise Diffusion Sampling
-
Addressing the inefficiency of conventional diffusion SDE solvers that blindly inject uniform white noise and waste stochastic energy, Colored Noise Sampling (CNS) dynamically routes noise variance to unresolved frequency bands using a structural progression matrix, achieving plug-and-play generation gains without model retraining while strictly conserving global variance.
- Commutator Memory: Sparse, Path-Local Reading and Steering in Language Models
-
The paper decomposes the noncommutative residue of two training updates into a signed token readout, localizing and intervening on training-order differences under local SGD conditions; targeted interventions close the held-out loss gap by a median 32.0% on Qwen-3-4B, while paired-endpoint order assignment reaches 66/72 = 91.7%.
- Compatible Likelihoods for Flow Matching on Manifolds
-
This paper reveals that endpoint-likelihood variational flow matching on manifolds (RG-VFM) suffers from an incompatibility flaw where its population minimizer fails to recover the true marginal velocity field, and proposes an endpoint-space exponential family objective with the conditional velocity as the sufficient statistic, inducing an inverse Fisher information preconditioned regression loss that achieves both compatibility and compact support-awareness while substantially improving likelihood and sample quality across flat tori, spherical geospatial data, and high-dimensional human poses.
- Compute Efficiency and Serial Runtime Tradeoffs for Stochastic Momentum Methods
-
For consistent linear regression with Gaussian covariates, this paper proves discrete-time finite-dimensional lower bounds on batch-size scaling for Heavy Ball (HB) and Accelerated SGD (ASGD), showing that HB does not improve the compute efficiency frontier over SGD but extends its linear scaling window by a factor of \(\sqrt{\kappa}\), whereas ASGD improves small-batch compute efficiency but trades it off for serial runtime as batch size grows.
- Concept Modulation Models: A Unified Framework for Identifiability and Extrapolation
-
This paper introduces concept modulation models (CMMs) with structure \(A \to \Lambda \to C \to X\), proving that feature equivalence on observed attributes forces a latent concept transition that preserves log-density ratio attribute potentials, which in turn yields unified algebraic criteria for both latent identifiability and unseen attribute extrapolation.
- Conditioning Gaussian Processes on Almost Anything
-
This paper establishes an exact mathematical equivalence between Gaussian processes and linear diffusion models/flow matching, proposing FlowGP to perform conditional GP inference under non-linear physics and LLM text constraints via an ODE with Monte Carlo guidance.
- Conformal Prediction with Paraphrase-Aware Scoring for LLM Uncertainty Quantification
-
To tackle predictive confidence volatility caused by meaning-preserving prompt rewordings in large language models, this paper proposes paraphrase-aware nonconformity scores (PA Score) that aggregate predictions from a calibrated hidden-state proxy model across paraphrases, reducing prediction set sizes by 2x to 4x while preserving rigorous conformal coverage guarantees.
- Conservation Laws for Diffusion Models
-
Building on generalized extrinsic information transfer (GEXIT) functions from coding theory, this paper establishes unified mismatched conservation laws showing that the data–model cross-entropy for discrete and continuous diffusion over memoryless degraded channels equals the path integral of local marginal-posterior derivatives, simplifying training to channel-tilted extrinsic-prior cross-entropy minimization and revealing substantial channel-dependent finite-capacity gaps across Markov sources, text8, and CIFAR-10.
- Consistency Regularised Gradient Flows for Inverse Problems
-
This paper introduces Consistency-regularised Wasserstein Gradient Flow (CWGF), a unified Euclidean-Wasserstein gradient flow framework that jointly optimizes text prompt embeddings and performs latent posterior sampling with Latent Consistency Models (LCMs), achieving state-of-the-art image restoration in just 16 NFEs without backpropagation through the autoencoder.
- Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives
-
Addressing the lack of rigorous constraint and regularization support in modern spectral optimizers such as Muon and Scion, this paper introduces a unified stochastic proximal preconditioned gradient framework with anisotropic proximal operators, proving for the first time adaptive sublinear convergence under heavy-tailed noise and general nonconvex constraints while formalizing polynomial iterations like Polar Express as nonlinear preconditioners.
- Continuous-depth Deep Gaussian Processes
-
Reframing deep Gaussian processes (DGPs) from discrete hierarchical compositions into latent stochastic differential equations (SDEs) driven by GP-based drift priors (CDGP and FlowDGP), eliminating the depth-induced sensitivity pathologies of discrete layer stacking.
- Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusion
-
Addressing the severe inference bottleneck in discrete diffusion reward alignment caused by costly Monte Carlo estimation of optimal twist functions, this paper proposes Contrastive Distribution Matching (CDM) to amortize Twisted Sequential Monte Carlo (SMC) into a single forward pass (<5% compute overhead) using forward KL divergence and closed-form forward diffusion kernels, establishing superior Pareto frontiers across biological sequence design and diffusion LLM alignment.
- Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit
-
This paper develops the first rigorous convergence analysis for the regularized Newton method on overparameterized neural networks, proving that the training dynamics converge in probability to a deterministic limit governed by a Newton Neural Tangent Kernel (NNTK) that eliminates the spectral bias of gradient descent and enables uniform exponential convergence across all frequency components.
- Corrected Integrated Laplace Approximation for Bayesian Inference in Latent Gaussian Models
-
To overcome the unremovable posterior approximation error of integrated Laplace approximation (ILA) in latent Gaussian models, this paper proposes an importance sampling correction scheme (PM-ADLA, QMC-ADLA, and RQMC-ADLA) coupled with reverse-mode automatic differentiation, achieving asymptotically exact marginal inference under Hamiltonian Monte Carlo.
- Counterfactual Online Conformal Prediction Under Adaptive Logging
-
Addressing the endogenous distribution shift induced by adaptive logging in online decision-making, this paper proposes Propensity-Weighted Online Conformal Prediction (PW-OCP) and its doubly robust variant (DR-OCP), establishing minimax-optimal counterfactual coverage bounds and mitigating downstream decision regret.
- CRUMB: Efficient Prior Fitted Network Inference via Distributionally Matched Context Batching
-
To eliminate the prohibitive quadratic attention bottleneck of Prior-Fitted Networks (PFNs) on large tabular training sets while avoiding the unbatchable overhead of per-query nearest-neighbour retrieval, CRUMB clusters test queries and retrieves small, distributionally matched training contexts via greedy MMD minimisation, achieving batched inference throughput and superior resilience to covariate drift.
- Cumulative-Goodness Free-Riding in Forward-Forward Networks: Real, Repairable, but Not Accuracy-Dominant
-
The paper proves that cumulative goodness exactly attenuates deep blocks' local discrimination gradients on examples already separated upstream, and repairs block health through history removal, hardness gating, and gradient compensation, but finds no accuracy-dominant benefit from these repairs; MGC even reduces CIFAR-100 Stage-1 single-crop accuracy by 1.05 percentage points.
- Curvature Beyond Positivity: Greedy Guarantees for Arbitrary Submodular Functions
-
To tackle submodular optimization under cost penalties that render the objective non-monotone and potentially negative, this paper extends the classical Conforti–Cornuéjols curvature to arbitrary submodular functions and develops a greedy algorithm with pruning that achieves the first multiplicative approximation ratio \((1-e^{-\bar{c}_g})/\bar{c}_g\) beyond monotonicity and non-negativity.
- Curvature-Dependent Lower Bounds for Frank–Wolfe
-
By analyzing the contraction dynamics along an invariant manifold of a 2D \(\ell_p\) ball projection problem, this paper proves matching dimension-independent \(\Omega(T^{-p/(p-1)})\) lower bounds for Frank–Wolfe with exact line search or short steps on \(p\)-uniformly convex sets, resolving a key tightness open problem.
- Debiasing Random Oblique Projections for Subsampled OLS and Fast CUR in High Dimensions
-
Addressing the systematic statistical bias caused by the nonlinear pseudo-inversion in random sampling-induced oblique projections, this paper develops a sharp non-asymptotic bias-variance characterization in high dimensions, designs an adaptive diagonal debiasing framework, and demonstrates provable error reductions in subsampled OLS and fast CUR decomposition without variance penalty.
- Decoupled Descent: Exact Test Error Tracking Via Approximate Message Passing
-
Built on Approximate Message Passing (AMP) theory, this paper introduces Decoupled Descent (DD), a training algorithm that incorporates dual Onsager correction terms into pre-activations and parameter gradients to cancel data-reuse bias in full-batch training, rigorously establishing an asymptotic "train-test identity" on high-dimensional Gaussian mixture models and enabling zero-cost validation with 100% data utilization.
- Decoupled Optimization for Teacher–Student Semi-Supervised Learning via a Pioneer Student
-
Addressing the fitting–generalization dilemma and label-driven trap caused by parameter coupling in semi-supervised learning, this paper proposes Pioneer Student (PiS)—an auxiliary branch operating in an isolated parameter space that decouples optimization and transfers accumulated robust knowledge via periodic hard parameter overwrites, delivering consistent performance gains and 3-4× convergence speedups.
- DEG: Diffusion-Enhanced GFlowNet for Solving Vehicle Routing Problems
-
One-Sentence Summary¶
- Delve into the Applicability of Advanced Optimizers for Multi-Task Learning
-
This paper demonstrates that exponential moving average (EMA) momentum in advanced optimizers severely dilutes the instant gradient de-conflicting in multi-task learning, and proposes APT—featuring curvature-guided adaptive momentum and light direction preservation—to fully harmonize advanced optimizers with multi-task optimization dynamics.
- Denoising Implicit Variational Inference
-
Addressing the intractable marginal score bottleneck in semi-implicit variational inference, this paper introduces Denoising Implicit Variational Inference (DIVI), which learns the marginal score directly via denoising score matching as a plug-in estimator in pathwise ELBO gradients, bypassing reverse-conditional sampling with provable inexact convergence.
- density-ratio losses for post-hoc learning to defer
-
This paper formulates post-hoc learning to defer (L2D) through the lens of divergence-regularized ideal distributions, deriving an unbiased joint DR CPE loss via a reduction from density-ratio estimation to class-probability estimation that enables flexible test-time deferral thresholding without retraining.
- Diagonalizing the Softmax: Hadamard Initialization for Tractable Cross-Entropy Dynamics
-
By proving that Sylvester-Hadamard initialization precisely diagonalizes the non-linear softmax operator, this paper extends exact learning dynamics to multi-class non-convex cross-entropy optimization, establishing the first rigorous proof of global convergence to neural collapse via an explicit KL divergence Lyapunov function.
- Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimization
-
This paper introduces the Wasserstein Convex-Concave Procedure (WCCCP) to optimize non-geodesically-convex functionals such as Maximum Mean Discrepancy (MMD) and Energy Distance (ED) on probability measure spaces, developing explicit kernel DC decompositions that yield fast, stable particle convergence and provable sublinear rates.
- Differentiable Knapsack and Top-k Operators via Dynamic Programming
-
Unifies 0/1 Knapsack and Top-\(k\) subset selection as dynamic programs, smoothing the Bellman recursion max operators to derive differentiable relaxations with efficient VJPs, permutation equivariance characterization, and sparse autoregressive sampling.
- Differentiable Systematic Resampling for Variational Sequential Monte Carlo
-
To eliminate the non-differentiability bottleneck of systematic resampling in variational sequential Monte Carlo (VSMC), this paper proposes Differentiable Systematic Resampling (DSR), which smooths the cumulative distribution function (CDF) indicator step via temperature-controlled sigmoids and incorporates Morton space-filling curve sorting to preserve low-variance banded structure while enabling full gradient flow and exponential convergence.
- Diffusion Path Samplers via Sequential Monte Carlo
-
Addressing the challenge of sampling from unnormalized target densities, this paper introduces Diffusion Path Sequential Monte Carlo (DPSMC), which leverages SMC to track auxiliary conditional posteriors along diffusion paths, producing unbiased marginal density and variance-reduced score estimates.
- DisCoMBO: Steering Expert-in-the-Loop Black Box Optimization via Distributional Conformance
-
To overcome the absence of epistemic uncertainty and prior conformance checks in Probabilistic Circuits (PCs) for black-box optimization, this paper proposes the Distributional Conformance Score (DisCo)—which is order-isomorphic to symmetric kernels—and presents DisCoMBO, a zero-regret expert-in-the-loop framework with linear-time inference.
- Discrete Langevin-Inspired Posterior Sampling
-
This paper introduces \(\Delta\)LPS, a training-free discrete Langevin-inspired posterior sampler that uses continuous likelihood gradients to construct factorized categorical proposal distributions over valid tokens, enabling efficient parallel updates and accurate reconstruction without continuous relaxations.
- Distributed Online Convex Optimization with Compressed Communication: Optimal Regret and Applications
-
Addressing the long-standing divergence issue caused by the coupling between compression and projection errors in constrained distributed online convex optimization, this paper integrates bidirectional error feedback into the FTRL framework and devises a blocked online fast compression scheme, establishing the first matching optimal regret bounds and offline non-smooth convergence rates.
- Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
-
Addressing the long-standing consensus that asynchronous SGD is necessary to eliminate computation bubbles in heterogeneous environments, this paper proves that standard Synchronous SGD and its robust backup-worker variant \(m\)-Synchronous SGD achieve near-optimal time complexities (up to logarithmic factors) under fixed, sub-exponential random, and adversarial partial participation regimes.
- DRIVE: Fine-tuning via Data Contribution- and Diversity-aware Weighting with Prior Regularization
-
DRIVE establishes a unified differentiable data weighting framework that decouples anti-forgetting from new-task adaptation via Shapley additivity, incorporates a covariance-determinant diversity term, and stabilizes weights with a KL prior, achieving an optimal stability-plasticity trade-off under data-oblivious fine-tuning.
- Dynamic k-center Clustering with Lifetimes
-
This paper introduces the dynamic setting with lifetimes bridging sliding-window and fully dynamic models, developing an almost-tight deterministic \((2+\varepsilon)\)-approximation algorithm with \(\widetilde{O}(k/\varepsilon)\) amortized update time and a space-efficient \((6+\varepsilon)\)-approximation algorithm with worst-case \(\widetilde{O}(k/\varepsilon)\) update time and sublinear working memory for \(H\)-ordered streams.
- Dynamic Regret in Online Convex Optimization with Indicator Switching Costs
-
The paper aggregates distributions from randomized lazy FTRL learners restarted at dyadic scales through a movement-aware meta-learner, uses maximal coupling to control actual action changes, and obtains near-optimal expected dynamic regret for piecewise-constant comparators alongside a separate guarantee for comparators with small path length.
- E-GEO: A Testbed for Generative Engine Optimization in E-Commerce
-
This paper introduces E-GEO, the first benchmark dedicated to Generative Engine Optimization in e-commerce, and designs a reflection-driven prompt meta-optimizer that consistently improves cross-engine product rankings while converging toward factual, intent-aligned structure under an adversarial defense.
- Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD
-
Reveals that mini-batch SGD under MSE loss does not stabilize at the full-batch Hessian top eigenvalue \(\lambda_{\max} \approx 2/\eta\), but instead enters an "Edge of Stochastic Stability" (EoSS) governed by Batch Sharpness (the expected Rayleigh quotient of mini-batch Hessians along their gradients) saturating at \(2/\eta\), explaining why smaller batch sizes intrinsically select flatter minima.
- EDISCO: Equivariant Discrete Diffusion for Euclidean Combinatorial Optimization
-
EDISCO establishes the first discrete diffusion model for Euclidean combinatorial optimization with an exact \(E(2)\)-invariant generative distribution over node-index solutions, outperforming prior neural solvers on synthetic TSPs up to 10,000 nodes and CVRPs up to 2,000 customers while requiring only 33%–50% of the training instances.
- Efficient Pre-Training with Token Superposition
-
Token-Superposition Training (TST) boosts pre-training raw token throughput at constant per-step FLOPs by averaging contiguous input embeddings into bags and predicting future bags via multi-hot cross-entropy, achieving a 2.5× pre-training speedup at the 10B MoE scale without modifying inference architecture.
- Efficient Transferable Optimal Transport via Min-Sliced Transport Plans
-
Addressing the computational bottleneck and lack of solution reusability in standard optimal transport, this paper introduces the transferable min-Sliced Transport Plan (min-STP) framework powered by LapSum differentiable sorting and symmetric gradient flows, establishing stability and transferability guarantees alongside finite-sample mini-batch statistical bounds for efficient amortized 3D point cloud alignment and flow-based generation.
- EnerGNN: Learning Optimization-Compatible Energy Functions for Exact Constrained Combinatorial Inference
-
EnerGNN learns an optimization-compatible piecewise linear energy function over complete binary configurations that admits an exact mixed integer programming (MIP) formulation, enabling global energy minimization under explicit hard constraints and achieving near-optimal solutions with 100% feasibility.
- Ensemble Modeling for Time Series Forecasting: an Adaptive Robust Optimization Approach
-
This paper introduces an Adaptive Robust Optimization (ARO) framework for dynamic linear time series ensemble modeling, adjusting base model weights through an affine decision rule driven by past forecast errors and reformulating the adversarial min-max problem into an equivalent tractable regularized regression, significantly mitigating prediction error and tail risk.
- Escaping Parameter Space: Tight Generalization Bounds via Representation Quality
-
Escaping traditional parameter-space complexity measures and weight perturbations, this paper establishes tight, non-vacuous generalization certificates for deterministic neural networks based directly on representation quality in the penultimate feature space via cross-fitting.
- Estimating and Orthogonalizing Unknown Pre-training Gradients for Continual Fine-tuning of Large Language Models
-
EoupCT uses a frozen pretrained model and learnable soft prompts to construct differentiable knowledge proxies vulnerable to new-task updates, then combines distillation with gradient projection to protect historical tasks and general capabilities; experiments on SuperNI/MMLU subsets across six models improve retention, but its “first-order-only” and “absolute zero forgetting” claims require qualification.
- EvoOptiGraph: Weakness-Driven Coevolution via Graph-Based Structural Generation for Optimization Modeling
-
EvoOptiGraph represents mixed-integer linear programming (MILP) problems as attributed bipartite graphs for validity-preserving evolutionary generation, diagnoses model blind spots via interpretable graph features, and trains the model via solver-verifiable reinforcement learning (RLVR), establishing a closed-loop coevolution framework between data and policy.
- EvoOptiGraph: Weakness-Driven Coevolution via Graph-Based Structural Generation for Hard Optimization Problems
-
Addressing the lack of structural diversity and the static decoupling between data synthesis and model learning in natural-language-to-optimization modeling, EvoOptiGraph represents mixed-integer linear programs as attributed bipartite graphs for validity-preserving genetic evolution, diagnoses model failures via tree-based SHAP feature attribution, and closes the loop through verifiable reward reinforcement learning (RLVR), achieving a state-of-the-art 78.1% macro-average Pass@1 on six OR benchmarks using an 8B backbone.
- Extended Wasserstein-GAN Approach to Causal Distribution Learning: Density-Free Estimation and Minimax Optimality
-
The paper proposes GANICE, an extended Wasserstein GAN framework that formulates causal conditional distribution learning through diagonal transport couplings, eliminates the need for density-ratio and propensity-score estimation via cell-normalized stratification, and establishes minimax-optimal convergence rates under both discrete and anisotropic continuous conditioning.
- Factor Augmented High-Dimensional SGD
-
Factor-Augmented SGD (FSGD) dynamically extracts latent factors via streaming Oja's algorithm and optimizes model parameters via mini-batch SGD on the induced factor space, establishing the first \(\ell^s\)-moment convergence guarantees that rigorously characterize the propagation of latent factor estimation error and idiosyncratic noise.
- Factorized Gradients for Scalable Highly-expressive Parametric Diffeomorphisms
-
This paper introduces FG-CPAB, which decouples spatial trajectory integration from parameter-space projection via discrete-adjoint moment accumulation, reducing the backward gradient complexity of continuous piecewise-affine (CPAB) diffeomorphisms from \(\mathcal{O}(dTN)\) to \(\mathcal{O}(TN + dC)\), achieving up to four orders of magnitude speedups and over \(500\times\) peak memory reduction while making fine-tessellation CPAB practical for large-scale 2D and 3D spatial modeling.
- Fast and Stable Gradient Approximation for Bilinear Forms of Hermitian Matrix Functions
-
Addressing the numerical instability and high computation costs of differentiating Lanczos-approximated bilinear forms of Hermitian matrix functions, this paper proposes a forward-only gradient estimator that drops basis-variation terms, proves the error is strictly bounded by the Lanczos residual coefficient \(\beta_m\), and achieves unconditional stability without reorthogonalization.
- Fast Gauss-Newton for Multiclass Cross-Entropy
-
Addressing the severe computational bottleneck of standard multiclass softmax generalized Gauss-Newton (GGN) curvature when class counts grow, this paper proposes Fast Gauss-Newton (FGN), which algebraically decomposes the GGN into a rank-one true-vs-rest margin term and a positive semidefinite competitor covariance, enabling an exact-loss, exact-gradient, matrix-free conjugate gradient row-space solver that scales with batch size rather than class dimension.
- Faster Rates for Federated Variational Inequalities
-
Addressing the gap between federated variational inequality rates and convex optimization bounds, this paper establishes tighter convergence guarantees for Local Extra SGD via refined client drift analysis, and introduces LIPPAX (with its smoothed variant SLIPPAX) to eliminate operator-expansion drift, achieving the LSGD-optimal rate across multiple regimes.
- Federated Learning by Utility-Constrained Stochastic Aggregation for Improving Rational Participation
-
To prevent the attrition of self-interested clients in heterogeneous cross-silo federated learning, FedUCA reformulates server aggregation into a convex feasibility optimization under individual rationality constraints, leveraging Dirichlet-randomized candidate aggregation strategies to extract a Jensen's gap utility surplus that sustains collaboration without external monetary incentives.
- FedRSPO+: A Heterogeneity-Aware Algorithm for Decision-Focused Federated Learning
-
Addressing the challenge of discontinuous extreme-point jumps in optimal solution maps caused by heterogeneous downstream objectives and polyhedral constraints in cross-silo collaborative decision settings, this paper proposes FedRSPO+, a decision-focused federated learning framework based on a Moreau-smoothed surrogate loss that provides heterogeneity bounds without strong convexity that vanish at homogeneity and achieves significantly superior empirical decision regret.
- FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning
-
To eliminate the "flatness incompatibility" induced by cross-device data heterogeneity and the intra-device friendly adversary phenomenon, FedVSSAM constructs a server-maintained variance-suppressed adjusted direction and consistently anchors local flatness search, local descent, and global aggregation to it—attaining superior global generalization and non-convex linear speedup without uploading auxiliary uplink vectors.
- Finding Simple Proofs for First-Order Optimization
-
To address the opacity of machine-generated convergence certificates, this paper formulates the search for simpler proof structures as a second-stage sparse optimization and semidefinite programming (SDP) problem over Lagrangian dual certificates, successfully pruning redundant inequalities and recovering compact Lyapunov functions.
- Finite-Sample Performance of Gradient Descent in Logistic Regression with Gaussian Design
-
Under well-specified Gaussian logistic regression, the paper combines population curvature analysis with approximate invertibility of the empirical gradient to prove linear convergence of gradient descent to a statistical-error neighborhood, and obtains a sharper high-dimensional error bound by estimating direction and norm separately; large-step acceleration is only local, and the stated near-optimal regime contains a condition discrepancy that must be retained.
- Flash-KMeans: Fast and Memory-Efficient Exact K-Means
-
Addressing the severe HBM IO bottleneck in the assignment stage and hardware atomic write contention in the centroid update stage of Lloyd's k-means on modern GPUs, Flash-KMeans introduces FlashAssign and Sort-Inverse Update to deliver up to 17.9x end-to-end speedups and out-of-core scalability to one billion points without any mathematical approximation.
- Flash-SD-KDE: Accelerating SD-KDE with Tensor Cores
-
Flash-SD-KDE tackles the prohibitive computational bottleneck of score-debiased kernel density estimation by algebraically reformulating the empirical score and density evaluation into high-throughput Tensor Core GEMMs with streaming tile accumulation, enabling scalable debiased estimation on millions of samples.
- Flow Annealing Posterior Sampling for Function-Space Regression and Inverse Problems
-
Flow Annealing Posterior Sampling (FLAPS) is the first function-space posterior sampling framework that leverages pretrained operator flow-matching priors, combining deterministic flow transport, offline low-rank covariance-preconditioned Langevin correction, and re-bridging to unify sparse regression and PDE inverse problems without explicit prior-density evaluation.
- Flow Perturbation++: Multi-Step Unbiased Jacobian Estimation for High-Dimensional Boltzmann Sampling
-
Flow Perturbation++ (FP++) leverages the semigroup property of probability-flow ODEs to factorize the global Jacobian determinant into local step determinants, applying independent unit-sphere perturbations per step to deliver an unbiased, low-variance Jacobian estimator that achieves high-fidelity equilibrium sampling on 1000D GMM and all-atom Chignolin within a Sequential Monte Carlo framework.
- Flow-Transformed Implicit Processes for Function-Space Variational Inference
-
The paper introduces Flow-Transformed Implicit Processes (FTIP), replacing the Gaussian coefficient posterior in Variational Implicit Processes with an invertible normalizing flow, thereby inducing expressive non-Gaussian function-space posteriors that capture multimodal and asymmetric predictive uncertainty without requiring network Jacobians.
- FluxLite: Inference-Time Proposal Control for Discrete Diffusion Models
-
FluxLite jointly adjusts sparse jump rates and compensating weights without retraining a discrete diffusion model, preserving the target marginal-distribution path through graph divergence while mitigating particle-weight degeneracy with the local HEU rule or a small nonnegative quadratic program, D-VCG.
- From Cursed to Competitive: Closing the ZO–FO Gap via Input-to-State Stability
-
By framing Gaussian zeroth-order optimization as a discrete-time dynamical system with bounded input perturbations, this paper leverages input-to-state stability (ISS) to prove that ZO algorithms in expectation match the exact dimension-free transient decay rates of first-order methods, with problem dimension affecting only the controllable steady-state neighborhood radius.
- From Generic to Dedicated: A Novel Optimizer for Online Continual Learning
-
Addressing the failure of generic optimizers dominated by instantaneous sample distributions in Online Continual Learning (OCL), this paper proposes SPIN, the first dedicated optimizer that decouples plasticity and stability via parameter-space interpolation across dual iterate sequences, theoretically providing implicit Tikhonov-regularized Hessian damping to consistently outperform SGD and AdamW across diverse OCL algorithms.
- From Inexact Gradients to Byzantine Robustness: Acceleration and Optimization under Similarity
-
This paper casts Byzantine-robust distributed learning with data heterogeneity into a first-order \((\zeta^2, \alpha)\)-inexact gradient oracle framework, establishes the tightness of this reduction against the theoretical Byzantine lower bound, and develops both the first Byzantine-resilient Nesterov accelerated scheme achieving an \(\mathcal{O}(\sqrt{\kappa}\log(1/\varepsilon))\) rate and a similarity-preconditioned proximal method (PIGS) that slashes communication complexity to \(\mathcal{O}(\frac{\Delta}{\mu}\log(1/\varepsilon))\).
- From SGD to Muon: Adaptive Optimization via Schatten-p Norms
-
The paper unifies SGDM and Muon into a continuous family of Schatten-\((p+1)\) norm constrained descent updates under Linear Minimization Oracle (LMO) theory, derives a closed-form first-order proxy criterion for layer-wise optimal \(p^*\), and integrates \(p\)-dependent second-order preconditioning alongside a hardware-efficient Taylor-Horner polar approximation to achieve scalable adaptive spectral optimization.
- GauS: Differentiable Scheduling Optimization via Gaussian Reparameterization
-
GauS models operator scheduling time steps as continuous 1D Gaussian random variables to preserve intrinsic temporal ordinal proximity and compress the parameter space down to \(O(|V|)\), enabling fully GPU-vectorized, end-to-end differentiable optimization for both regular and pipelined modulo scheduling under augmented Lagrangian methods.
- Generalization Analysis of Biased Stochastic Gradient Methods for Minimax Problems
-
This paper establishes the first theoretical framework connecting algorithmic stability and generalization for biased stochastic gradient methods in minimax optimization, relaxing the restrictive bounded gradient assumption to a bounded second moment condition, and deriving minimax-optimal \(\mathcal{O}(1/\sqrt{n})\) excess risk bounds for both Zeroth-order and Clipped SGDA.
- Global Convergence of Four-Layer Matrix Factorization under Random Initialization
-
For four-layer matrix factorization (deep linear networks), this paper proves that randomly initialized gradient descent with a standard balancing regularizer achieves polynomial-time global convergence to zero loss with high probability over complex initializations and with probability close to 1/2 over real initializations.
- GPU-Accelerated Synthesis of Mixed-Boolean Arithmetic: Beyond Caching
-
Addressing the memory explosion of observational equivalence caches in quantitative domains, Simba introduces a cache-free GPU bottom-up synthesizer using bijective RPN decoding and locally consistent warp scheduling, achieving exact synthesis of MBA expressions up to size 16.
- Greedy Alignment Principle for Optimizer Selection
-
By modeling first-order optimizers as causal filters operating on gradient streams, this paper proves that maximizing the expected instantaneous loss drop rate is mathematically equivalent to the Hilbert inner product between the optimizer filter and gradient autocorrelation, establishing closed-form dynamic momentum selection rules for SGD and Adam without hyperparameter sweeps.
- Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes
-
The paper proves that periodic slingshot loss spikes in unregularized long-term training stem from floating-point absorption errors breaking the cross-entropy gradient zero-sum constraint, triggering an anti-parallel exponential explosion of classifier and feature means (Numerical Feature Inflation) that ultimately erupts under amplified Adam step sizes.
- Group Distributionally Robust Optimization with Flexible Sample Queries
-
By casting group distributionally robust optimization as a two-player game between online convex optimization and non-oblivious prediction with limited advice, this paper proposes a unified FTRL framework supporting dynamic sample queries, establishes the first high-probability regret bound for non-oblivious PLA, and sharpens the GDRO anytime optimization error by eliminating logarithmic time factors.
- Guaranteed Noisy CP Tensor Recovery via Riemannian Optimization on the Segre Manifold
-
Formulating noisy low-CP-rank tensor recovery as an optimization problem over the Segre manifold of rank-one tensors, this paper develops structure-preserving Riemannian Gradient Descent (RGD) and Riemannian Gauss-Newton (RGN) algorithms, proving local linear and local quadratic convergence guarantees up to the statistical noise floor.
- How I Learned to Stop Worrying and Love Stopgrads: Stationarity, Convergence, and Acceleration in Implicit Models
-
This paper grounds stopgrad training in variational calculus and introduces the Stopgrad Regression Principle, proving that the true flow map is the unique stationary point for MeanFlow, deriving memory-halving Slim ESD/LSD objectives without second-order derivatives, and establishing closed-form exponential convergence of functional semi-gradient flows via time-composition.
- HyCO: A Hybrid Neural Solver for Combinatorial Optimization
-
HyCO reconciles the complementary error dynamics of sequential RL (horizon-wise compounding errors) and global diffusion models (subspace-dimension-governed regret) through a unified marginal-regret framework, proving the existence of a unique optimal handover point and executing adaptive RL-to-DM triggering with dual-track completion to consistently outperform standalone solvers.
- Hypernetworks for Dynamic Feature Selection
-
Addressing the capacity bottlenecks and gradient conflicts of single shared predictors in dynamic feature selection, Hyper-DFS leverages a Set Transformer to map feature observation statuses into a smooth geometry and employs hypernetworks to generate subset-specific primary classifier parameters on demand.
- Improved Regret Analysis for Parallel Gaussian Process Bandit Optimization
-
For parallel and delayed-feedback Gaussian process bandit optimization, this paper proposes an improved regret analysis technique based on variance-threshold potential counting that decouples batch size/delay \(Q\) into an additive term for batched Thompson sampling (GP-BTS) without any initial uncertainty sampling phase, and establishes substantially tighter sublinear and polylogarithmic regret bounds in the noiseless setting.
- Inexact Bregman Sparse Newton Method for Efficient Optimal Transport
-
Addressing the excessive computational cost and numerical instability of solving exact and entropy-regularized optimal transport at scale, this paper proposes the Inexact Bregman Sparse Newton (IBSN) framework, combining semi-dual dimension reduction, positive-definite Hessian sparsification, and verifiable inexact stopping criteria to achieve fast superlinear convergence to the exact optimal plan.
- Interpretable Relational Inference with LLM-Guided Symbolic Dynamics Modeling
-
COSINE presents a differentiable framework that co-optimizes latent interaction graphs and sparse symbolic dynamics, incorporating an outer-loop LLM to adaptively evolve the basis library for robust relational inference and mechanistic equation discovery without template priors.
- Interpreting Neural Combinatorial Optimization via Evolving Programmatic Bottlenecks
-
This paper introduces Evolving Programmatic Bottlenecks (EPB), the first interpretability framework for neural combinatorial optimization (NCO) that distills black-box neural policies into dynamically routed portfolios of human-readable heuristic programs, achieving near-lossless approximation while significantly boosting out-of-distribution generalization.
- Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds
-
Addressing the fundamental difficulties of extending spectral-norm optimizers to manifold settings, this paper introduces Intrinsic Muon (iMuon), which canonically lifts unitarily invariant norms to tangent spaces via the Riemannian metric, simultaneously resolving quotient symmetry breaking and constraint coupling to yield exact closed-form updates with dimension-only convergence rates across multiple matrix manifolds.
- Jacobian Descent for Multi-Objective Optimization
-
The paper formalizes Jacobian descent (JD) for vector-valued objective optimization, introduces the non-conflicting projection aggregator \(\mathcal{A}_{\text{UPGrad}}\) with provable convergence to the Pareto front under smooth convex objectives, and reduces memory complexity from \(\mathcal{O}(m \times n)\) to \(\mathcal{O}(n + m^2)\) via Gramian reverse accumulation, establishing the instance-wise risk minimization (IWRM) learning paradigm.
- Kernel-based guarantees for nonlinear parametric models in Bayesian optimization
-
Addressing the lack of theoretical guarantees for nonlinear parametric models in adaptive data collection and likelihood-free Bayesian optimization, this paper leverages parameter-space kernels to induce RKHS geometries over model classes, deriving GP-style confidence bounds and establishing the first sublinear cumulative regret guarantee for randomized nonlinear policies under approximate Thompson sampling.
- Last-Iterate Convergence of Single-Loop Stochastic Methods for Constrained Convex-Concave Minimax Problems
-
To tackle the fundamental non-convergence of last iterates in vanilla stochastic extragradient (S-EG) and optimistic gradient descent-ascent (S-OGDA) methods, this paper introduces a quadratic perturbation framework that achieves non-asymptotic last-iterate convergence on general closed convex sets under standard bounded-variance oracles (\(\mathcal{O}(T^{-1/4})\) for restricted primal-dual gap when the horizon is known, \(\mathcal{O}(t^{-1/5})\) anytime, and \(\mathcal{O}(t^{-1/4})\) anytime gradient-norm in unconstrained settings).
- Layerwise LQR for Geometry-Aware Optimization of Deep Networks
-
Establishes an exact mathematical equivalence between divergence-induced steepest descent and a layerwise finite-horizon Linear Quadratic Regulator (LQR), and proposes a scalable relaxation that directly optimizes structured inverse preconditioners without explicit curvature matrix formation or inversion.
- Leaner Transformers Can Easily Learn to Cluster
-
This paper proposes a compact binary token embedding scheme that reduces the transformer embedding dimension for emulating Lloyd's clustering algorithm from \(d+k\) to \(d+\lceil\log_2 k\rceil\), providing comprehensive theoretical and empirical analyses of convergence and generalization under smooth surrogate objectives.
- Learning-Augmented Approximation for Unrelated-Machines Makespan Scheduling
-
This paper develops the first learning-augmented approximation algorithm for unrelated parallel machines makespan scheduling (\(R\|C_{\max}\)), leveraging heavy-job predictions within a localized search and adaptive residual LP rounding scheme to achieve a polynomial-time \((1+\varepsilon)\mathrm{OPT}\) makespan under accurate predictions while degrading smoothly to the classical \(2\mathrm{OPT}\) baseline under arbitrary prediction errors.
- Learning-Augmented Coordination Mechanisms
-
This paper introduces the learning-augmented framework into coordination mechanism design for atomic and non-atomic congestion games with polynomial latencies, constructing a potential-based threshold mechanism tuned by a confidence parameter that achieves near-optimal consistency under accurate demand predictions without sacrificing worst-case robustness guarantees.
- Learning-Augmented Online Scheduling with Parsimonious Preemption
-
This paper introduces the first learning-augmented online scheduling framework with parsimonious preemption, establishing \(O(1)\)-competitive ratios for single, unrelated, and malleable machines while bounding preemption overhead to scale logarithmically with prediction errors.
- Learning-Augmented Streaming Algorithms for Approximating Boolean Max-CSPs
-
By treating noisy oracle predictions as the output of a binary symmetric channel and inverting the linear noise map via Gaussian elimination, this paper achieves a single-pass \((1-\eta)\)-approximation for Boolean Max-CSPs in space independent of \(n\) (or poly-logarithmic in dynamic streams), circumventing longstanding unconditional streaming barriers.
- Learning Augmented Exact Exponential Algorithms
-
This paper initiates the study of learning-augmented exact exponential-time algorithms, proving that a noisy predictor with pairwise-independent errors that is only marginally better than random guessing provably shaves off an \(\Omega(\varepsilon^2)\) factor from the exponential base of subset selection problems and circumvents the SETH barrier.
- Learning Orthogonal Multi-Index Models Beyond Small Initialization: Incremental Learning, Competitive Dynamics and Symmetry
-
Proves that polynomial-width two-layer neural networks under standard initialization learn orthogonal multi-index models via incremental Hermite-ordered loss reduction, driven by competitive parameter mass reallocation and analyzed via a finite-width symmetrization framework.
- Learning Pareto Stationary Fronts via Single-Pass Backpropagation
-
This paper introduces MOSEL, a framework that reformulates a posteriori Pareto front learning into a Stackelberg game between a representation network (follower) and a preference alignment network (leader), achieving Pareto stationary front recovery via timescale-separated single-pass backpropagation at standard single-objective training speed.
- Learning Polyhedral Conformal Sets for Robust Optimization
-
Addressing the severe over-conservativeness caused by task-agnostic uncertainty sets in robust optimization, this paper proposes a decision-aware conformal framework that learns polyhedral set geometry directly tailored to downstream worst-case loss, restoring finite-sample coverage validity through independent post-selection recalibration.
- Learning Provable Neural Network Observer for Uncertain Dynamical Systems
-
Addressing the computational scalability bottleneck of enforcing Lyapunov linear matrix inequality (LMI) constraints on neural network observers for uncertain dynamical systems, this paper introduces a two-stage framework decoupling point-guided Lyapunov pre-training and lightweight LMI fine-tuning, delivering 2.5× training acceleration while strictly guaranteeing global stability certificates and robust disturbance rejection.
- Learning Rate Transfer for Hybrid Transformer-SSM Architectures
-
This paper studies learning rate scaling for hybrid Transformer-SSM architectures, demonstrating that naive maximal update parameterization (Naïve μP) under AdamW achieves near-zero transfer gaps across widths and depths up to billion-parameter scale without requiring custom SSM-specific scaling rules.
- Learning Scenario Reduction for Two-Stage Robust Optimization with Discrete Uncertainty
-
To overcome the prohibitive computational overhead of two-stage robust optimization under discrete uncertainty, this paper introduces a problem-driven sequential lookahead heuristic PRISE to evaluate marginal gains and builds a GNN-Transformer surrogate NeurPRISE that performs one-pass scenario ranking, delivering 77–200× speedup while matching oracle solution quality.
- Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unity
-
This paper introduces a constructive approximation framework via Softmax Partition of Unity (POU), proving that a shallow Transformer with only two encoder blocks, standard Softmax dot-product attention, and sinusoidal positional encodings achieves optimal \(\mathcal{O}(\varepsilon^{-d/\alpha})\) approximation complexity and near minimax-optimal generalization rates \(\mathcal{O}(n^{-\frac{2\alpha}{2\alpha+d}}\log n)\), natively overcoming the curse of dimensionality on Riemannian manifolds.
- Learning to Solve Compositional Geometry Routing Problems
-
Addressing the Compositional Geometry Routing Problem (CGRP) characterized by mixed point, line, and area tasks with massive redundant action spaces, DiCon provides a plug-and-play framework combining differential attention for candidate action denoising with instance- and intra-task contrastive learning, achieving robust representation and superior zero-shot compositional generalization.
- LINC: Decoupling Local Consequence Scoring from Hidden Matching in Constructive Neural Routing
-
To resolve the bottleneck where constructive neural combinatorial optimization solvers force hidden states to implicitly reconstruct transition arithmetic, LINC explicitly decouples deterministic local transition consequences from latent global matching, employing a shared linear comparator and feasible-set state modulation to achieve highly effective, scale-robust routing.
- Literati: Towards Anytime Optimal Shape Generalized Trees via AO*
-
This paper proposes Literati, the first globally optimal Shape Generalized Tree induction algorithm, which jointly optimizes tree topology and univariate shape function changepoints over an AND/OR graph using an enhanced AO* search with an inadmissible secondary heuristic and round-robin exploration for anytime optimality.
- Local linear convergence of gradient methods for overparameterized Gaussian mixture models
-
Addressing the severe local slowdown caused by singular Fisher Information Matrices in overparameterized Gaussian mixture models, this paper uncovers an underlying quartic "ravine" geometry in the KL loss landscape and introduces a phase-switching GDPolyak algorithm that alternates standard gradient steps with long Polyak steps, establishing and achieving local linear convergence while retaining global recovery guarantees.
- Local linear convergence of gradient methods for overparameterized Gaussian mixtures
-
Addressing the severe local sublinear convergence slowdown of gradient EM in overparameterized Gaussian mixture models caused by Fisher information singularity, this paper uncovers a fourth-order growth ravine manifold in the KL loss and designs a phase-switching first-order method interleaving gradient steps with Polyak steps that provably achieves local linear convergence.
- Log-Averaged Mirror Prox for Fast, Large-Scale Optimal Transport in Linear Space
-
Log-Averaged Mirror Prox (LAMP) implements primal-dual mirror prox entirely in the dual space via weighted logarithmic averages, slashing storage from \(\mathcal{O}(nm)\) to \(\mathcal{O}(n+m)\) while retaining last-iterate \(\widetilde{\mathcal{O}}(nm\varepsilon^{-1})\) complexity and enabling fast, dense GPU-accelerated reductions on massive problems.
- MAdam: Metric-Aware Multi-Objective Adam
-
Reveals two systematic mismatches (weight marginalization and Euclidean geometric distortion) caused by coupling MOO solvers with Adam, and proposes MAdam, a plug-and-play wrapper that preconditions reconciled directions using an online-estimated preference-conditioned Fisher matrix so that updates follow the true manifold metric, consistently outperforming Adam across MTL, Pareto front tracing, PINNs, and medical imaging.
- Manifold Sampling via Entropy Maximization
-
Addressing the fundamental challenge where local MCMC samplers fail to bridge infeasibility barriers and allocate probability mass across disconnected components on constrained manifolds, MASEM introduces a modular resampling scheme based on \(k\)-nearest neighbor density estimation to maximize sample entropy, proving an exponential contraction of KL divergence to the uniform target at a geometric rate in the mean field.
- Maximin Robust Bayesian Experimental Design
-
This paper formulates Bayesian experimental design under model misspecification as a max-min game between the experimenter and an adversarial nature, establishing Sibson's \(\alpha\)-mutual information as the robust expected information gain and optimizing randomized design policies via a PAC-Bayes framework to control finite-sample Monte Carlo errors.
- Maxitive Donsker-Varadhan Formulation for Possibilistic Variational Inference
-
This paper establishes the maxitive analogue of the classical Donsker-Varadhan variational duality in possibility theory, introduces dual consistency bounds (CBO) built on max-relative entropy, and derives the CBOpt family of neural network optimizers that achieve competitive in-domain and out-of-domain performance without test-time sampling.
- Mechanistic Insights into LoRA: Layer Sparsity for Adaptive Fine-Tuning via Path Patching
-
By employing path patching causal analysis to uncover highly non-uniform and task-specific layer contributions in LoRA, this paper introduces Adaptive-LoRA, which uses frozen-base zero-ablation to calibrate critical layers, outperforming full-layer baselines while adapting only 30% of layers with a 3.5× parameter reduction.
- Mildly Overparameterized ReLU Networks on Orthogonal Data: Incremental Learning Dynamics and Edge of Stability
-
This paper proves that under small initialization, gradient flow on mildly overparameterized two-layer ReLU networks with orthogonal training data converges to a discrete saddle-to-saddle jump process, providing the first rigorous proof of incremental feature learning with one-by-one neuron activations in nonlinear networks and demonstrating that the parameter squared \(\ell_2\)-norm of the learned interpolator scales as \(\mathcal{O}(\sqrt{n})\), matching the minimal \(\ell_2\)-norm interpolator up to an absolute constant factor.
- Mildly Overparameterized ReLU Networks on Orthogonal Data: Incremental Learning and Implicit Bias
-
This paper provides the first rigorous theoretical characterization of gradient flow dynamics for two-layer ReLU networks from small initialization on orthogonal data with mild overparameterization (\(m \gtrsim \log n\)), proving weak convergence to a discrete saddle-to-saddle jump process where each saddle activates a single neuron, and showing that the parameter squared \(\ell_2\)-norm of the learned interpolator is within a constant factor of the minimal \(\ell_2\)-norm interpolator (\(\mathcal{O}(\sqrt{n})\)).
- Min-Max Optimization Requires Exponentially Many Queries
-
The paper proves that finding an \(\varepsilon\)-approximate first-order stationary point of a smooth nonconvex-nonconcave min-max optimization problem over independent product domains requires exponentially many queries (\(2^{d^{\Omega(1)}}\), even when \(\varepsilon = 1/\text{poly}(d)\)), establishing an unconditional black-box query lower bound under product constraints and ruling out polynomial query complexity for arbitrary constant-order optimization algorithms.
- Minimax Optimal Early-Stopped Gradient Descent for Gaussian Mixture Classification
-
This paper proves that for Gaussian mixture classification with label-flipping noise, gradient descent (GD) on the logistic loss stopped at an oracle stopping time achieves the minimax-optimal excess zero-one risk under fast and continuously decaying (FCD) covariance spectra, whereas all linear interpolating classifiers require exponentially more samples.
- Minimax Optimal Estimation of Transport-Growth Pairs in Unbalanced Optimal Transport
-
This paper establishes the first minimax optimal estimation theory for Monge-type unbalanced optimal transport (UOT) by defining the estimand as a joint transport-growth pair, designing plan-based and boundary-adapted kernel estimators, and analyzing them via a novel value-based objective stability reduction.
- Minimax Private Estimation of Smooth Optimal-Transport Maps
-
This paper investigates differentially private estimation of smooth optimal transport maps, introducing plug-in estimators based on compactly supported wavelets and quantile regression that achieve matching (near) minimax optimal convergence rates under both central and local differential privacy.
- Mirror Descent-Ascent for mean-field min-max problems
-
This paper studies mirror descent-ascent (MDA) over the space of probability measures and proves via infinite-dimensional dual space analysis that alternating play strictly accelerates the convergence rate to \(\mathcal{O}(N^{-2/3})\) compared to the \(\mathcal{O}(N^{-1/2})\) rate of simultaneous play.
- Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialisation Sensitivity
-
The paper proves that Mirror Descent under non-quadratic regularizers can be exponentially more sensitive to initial perturbations than standard Gradient Descent even for smooth convex objectives and well-conditioned regularizers, analyzes high-dimensional simplex entropic instability, and shows that anchoring to a fixed reference point stabilizes the dynamics to \(\mathcal{O}(1/T)\) sensitivity.
- Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity
-
The paper reveals that mirror descent (MD) under non-quadratic geometries exhibits an exponential separation in initialization sensitivity compared to Euclidean gradient descent: even for convex smooth objectives and well-conditioned regularizers, initial perturbations can be amplified exponentially in the number of iterations; introducing a fixed-anchor Bregman regularization successfully restores \(O(1/T)\) stability.
- Most ReLU Networks Admit Identifiable Parameters
-
By introducing weighted polyhedral complexes and directed dependency graphs, this paper proves that every deep ReLU architecture with input and hidden layer widths at least two admits a non-empty open set of identifiable parameters, establishes their exact functional dimension, and shows that minimal functional representations can still harbor non-trivial continuous parameter redundancies.
- Move on Muon: A Hamiltonian Probability Gradient Flow Perspective of Muon Optimizer
-
Regularizes idealized Muon orthogonalization as a mirror step induced by smooth Fenchel-dual smoothing of the nuclear norm, lifts the dynamics to matrix probability measures to establish a damped Hamiltonian probability flow with an exact dissipation identity, and proves McKean-Vlasov mean-field limits, propagation of chaos, and exponential convergence.
- Multi-Marginal Couplings for Metropolis--Hastings
-
Addressing the fixed-partner rigidity and exponential high-dimensional degradation when extending pairwise MCMC couplings to multiple chains, this paper introduces a joint multi-marginal coupling framework via a shared marked Poisson process, incorporates an adaptive mixture barycenter proposal that eliminates dimension-dependent complexity, and proves that coupling in the product-augmented proposal-acceptance space strictly outperforms direct kernel coupling, reducing grand meeting times by up to 50%.
- Muon Does Not Converge on Convex Lipschitz Functions
-
The paper proves that the spectral descent optimizer Muon fails to converge on convex Lipschitz objectives even with decaying stepsize schedules, shows that it reduces to signed momentum on diagonal matrices, and demonstrates that while error feedback restores theoretical convergence, this theoretical fix substantially degrades empirical performance in real-world vision and language model pretraining.
- Muon Dynamics as a Spectral Wasserstein Flow
-
This paper establishes a rigorous infinite-width mean-field mathematical framework for spectrally normalized optimizers such as Muon by introducing monotone spectral gauges on positive semidefinite cones to define Spectral Wasserstein distances, proving that the normalized mean-field PDE is fundamentally a curve of maximal slope gradient flow under this transport metric.
- Nash Social Welfare for Multi Armed Bandits: Trajectory-wise Expected and High Probability Regret
-
Addressing the limitation that ensemble Nash regret averages across paths before applying the geometric mean and misses trajectory-level fairness, this paper establishes trajectory-wise expected and high probability Nash regret metrics and designs a two-phase RR-NCB algorithm achieving optimal \(\widetilde{\mathcal{O}}(\sqrt{k/T})\) scaling.
- Near-Linear Time Generalized Sinkhorn Algorithms for Bounded Genus Graphs
-
The paper presents GenusSink, the first approximate generalized Sinkhorn algorithm for geodesic shortest-path distance costs on bounded genus graphs (e.g., planar graphs and 3D surface meshes) that achieves near-linear time in preprocessing, iteration steps, and transport plan querying, while maintaining near-linear memory and numerical precision.
- Near-Optimal Decentralized Stochastic Convex Optimization over Networks
-
The paper investigates decentralized stochastic smooth convex optimization over nonblocking gossip networks, proposing the doubly accelerated algorithm DDA-SGD that interleaves local minibatching with accelerated gossip via momentum-compensated delayed queries, improving the critical parallelism threshold for retaining the centralized statistical rate from \(M \lesssim \rho\sqrt{N}\) to a near-optimal \(M \lesssim \sqrt{\rho} N^{3/4}\) with only logarithmic dependence on optimum-local data heterogeneity.
- Near-Optimal Regret in Adversarial Kernel Bandits
-
Addressing the adversarial kernel bandit problem in potentially infinite-dimensional reproducing kernel Hilbert spaces (RKHS), this paper introduces an exponential-weights algorithm with a regularized loss estimator and an explicit debiasing correction, removing the rank-one adversary restriction while achieving a near-optimal regret bound that matches known stochastic kernel bandit lower bounds.
- Nearest-Neighbor Radii under Dependent Sampling
-
Establishes the first systematic geometric theory of nearest-neighbor radii under strong mixing dependent sampling, proving distribution-free almost sure convergence under polynomial mixing and sharp non-asymptotic moment bounds \(\mathbb{E}[R_{n,k}(x)^p] \asymp (k/n)^{p/s}\) under geometric mixing, revealing that weak dependence alters only local concentration constants while preserving the canonical scale governed by the local intrinsic dimension \(s\).
- Neural Compression of Long ADMM Trajectory for Multiparametric Quadratic Program
-
To tackle the computational bottleneck of thousands of iterative updates in large-scale multiparametric quadratic programming, this paper introduces TQPNet, a self-supervised neural compression framework that derives a low-dimensional primal-dual ADMM(ReLU) operator and couples a deep MLP(ReLU) with unrolled refinement layers, enabling ultra-fast batch-parallel solving without solver-generated labels and with adaptive online residual certification.
- Neural Galerkin Normalizing Flows for Bayesian Inference of Diffusions with Inaccessible Boundaries
-
For continuous-time stochastic diffusion processes characterized by inaccessible boundaries and vanishing diffusion matrices, this paper proposes a surrogate transition probability density function based on bounded truncated-Gaussian mixture normalizing flows and Neural Galerkin temporal projection, eliminating expensive online diffusion-bridge simulation and real-time Fokker-Planck PDE solves to enable scalable, high-accuracy MCMC Bayesian parameter inference.
- Neural Slack Variables for Shape Constraints
-
This paper proposes Neural Slack Variables, a deep learning-native primal-side soft constrained optimization framework that reformulates functional inequality constraints into regression matching between a primary network and an architecturally non-negative auxiliary network with scale-balancing and inverse hyperbolic sine stabilization, eliminating the constraint drifting failure mode in penalty and Lagrangian methods to achieve zero violations and high-precision data fitting across monotonicity, convexity, and financial no-arbitrage surfaces.
- Non-Linear Pricing Restores Tractability for a Data Seller
-
With known linear buyer valuations and budgets and separate pricing for each dataset, allowing nonlinear prices turns the previously APX-hard optimal linear-pricing problem into a polynomial-time LP and guarantees an optimum with at most as many total kinks as buyers; instances constructed from California Housing also exhibit revenue ratios of at most 1.1 and at most 10 total kinks.
- Nonconvex Decentralized Stochastic Bilevel Optimization under Heavy-Tailed Noise
-
To address nonconvex decentralized bilevel optimization where lower-level strong convexity fails and gradient noise exhibits heavy tails, this paper proposes the first normalized stochastic variance-reduced bilevel gradient descent-ascent algorithm without gradient clipping (D-NSVRGDA), establishing provable convergence guarantees with linear speedup.
- On the Last-Iterate Convergence of Clipped Gradient Methods
-
Under heavy-tailed gradient noise, this work establishes the first optimal high-probability last-iterate convergence rates for Clipped-SGD in both convex and strongly star-convex stochastic optimization, closing a fundamental theoretical gap.
- Online Learning on Hidden-Convex Losses via Algorithmic Equivalence: Optimal Regret, Geometric Barrier, and Bandit Feedback
-
By establishing an optimality-condition-driven \(\mathcal{O}(\eta^2)\) discrete coupling between OGD and ghost OMD, this paper proves that OGD on unknown hidden-convex losses achieves the minimax-optimal \(\mathcal{O}(\sqrt{T})\) full-information regret and \(\mathcal{O}(T^{3/4})\) one-point bandit regret, while characterizing a non-conservative geometric barrier that inflicts an \(\Omega(T)\) linear lower bound when Hessian compatibility fails.
- Online Learning via Learned Latent Bayesian Tracking
-
The paper learns a low-dimensional parameter-generating space and dynamical prior offline, performs one latent extended Kalman filtering update per labeled sample online, and reconstructs prediction parameters, improving adaptation under limited supervision and computational constraints in image classification and time-varying wireless reception.
- Online Localized Conformal Prediction
-
Addressing the dual challenges of temporal distribution shift and covariate heterogeneity in sequential data streams, this paper introduces Online Localized Conformal Prediction (OLCP) and its constrained expert aggregation variant OLCP-Hedge, achieving provable long-run coverage validity while producing significantly narrower prediction intervals than existing online conformal baselines.
- Online Min-Max Optimization: From Individual Regrets to Cumulative Saddle Points
-
Addressing the fundamental gap where minimizing individual regrets fails to converge to the cumulative saddle point in time-varying zero-sum games, this paper introduces the static duality gap and dynamic saddle point regret, establishes logarithmic regret bounds and saddle-point convergence for OGDA and OMMNS, and proves an impossibility theorem across strongly convex-strongly concave and min-max exponentially concave functions.
- Optimal Dimension-Free Sampling for Regularized Classification
-
For regularized classification across a wide family of Lipschitz losses (logistic, sigmoid, hinge, ReLU), this paper establishes tight dimension-free \((1\pm\varepsilon)\)-relative error sampling bounds by combining simple norm-mixed sampling with empirical process higher moments and metric entropy chaining, breaking the previous cubic barrier down to optimal linear \(\tilde{\Theta}(k/\varepsilon^2)\) and quadratic \(\Theta(k^2/\varepsilon^2)\) bounds alongside matching lower bounds and impossibility results.
- Optimal Hidden-Target Learning for Online Inventory Optimization on General Convex Sets
-
For online inventory optimization (OIO) under general bounded convex capacity sets, this paper proves that the simple hidden-target projection algorithm achieves minimax optimal regret: by introducing the norm alignment principle, state-dependent carryover feasibility is reduced to a one-dimensional Lindley queue control, sharpening the regret dependence on the common-demand probability to the optimal \(\widetilde{O}(\sqrt{T/\mu})\) alongside matching lower bounds and extensions to strongly convex and dynamic environments.
- Optimal In-context Adaptivity and Distributional Robustness of Transformers
-
This paper proves that large Transformers pretrained on a mixture of multi-task regression problems achieve optimal in-context prediction rates adaptive to task difficulty (smoothness and effective dimension), while maintaining rigorous distributional robustness against test shifts within a chi-squared divergence ball.
- Optimal Learning-Augmented Algorithm for Online Bidding
-
This paper introduces the novel analytical framework of bidding profiles, completely resolves the robustness-consistency trade-off for randomized learning-augmented online bidding via delayed differential equations, proves its strict Pareto optimality across all parameter regimes, and demonstrates substantial improvements in linear search.
- Optimal Rates for Pure \(\varepsilon\)-Differentially Private Stochastic Convex Optimization with Heavy Tails
-
The paper resolves the open problem of stochastic convex optimization with heavy-tailed gradients under pure \(\varepsilon\)-differential privacy, establishing a minimax optimal excess risk rate in high-probability polynomial time via localized Lipschitz extensions and joint convex reformulation, complemented by a nearly matching high-probability lower bound.
- Optimal Recalibration of an Online Predictor
-
Addressing the fundamental tension between empirical loss minimization and probability calibration in sequential prediction, this paper proposes an asymmetric imbalanced Blackwell approachability framework and a quadratic audit mechanism to achieve minimax-optimal \((\varepsilon, \varepsilon^2)\) online recalibration and simultaneous \(\varepsilon\)-calibration with \(\varepsilon^2\)-calibeating in \(\widetilde{\mathcal{O}}(\varepsilon^{-3})\) rounds for any Lipschitz proper loss.
- Optimistic Dual Averaging Unifies Modern Optimizers
-
This paper introduces SODA, a generalized Optimistic Dual Averaging framework that unifies modern first-order optimizers like Adam, Lion, Muon, and NAdam, revealing that weight decay acts as primal iterate averaging and yielding a hyperparameter-free \(1/(k+2)\) schedule wrapper.
- Optimizing Computational-Statistical Runtime for Wasserstein Distance Estimation
-
This paper introduces a Sample-Sketch-Solve paradigm for estimating squared Wasserstein distance between continuous distributions by projecting empirical samples onto Cartesian grid cell centers and solving an equivalent multipartite min-cost flow problem, achieving an optimal \(\tilde{O}(\varepsilon^{-2})\) computational-statistical runtime matching the statistical lower bound for smooth distributions in \(d=2\).
- Overcoming Rank Collapse in Feedback Alignment
-
Reveals that the failure of feedback alignment (FA) in deeper networks is driven by rank collapse of gradients and update trajectories, and demonstrates that orthogonalizing parameter updates via the Muon optimizer alongside activity normalization restores effective gradient dimensionality, scaling pure FA to ResNet-18 across visual benchmarks.
- Pairwise AUC Optimization Needs Corrective Power: A Unified View
-
The paper introduces the Corrective Power framework to explain why direct pairwise AUC surrogate optimization frequently stalls in bad regions, proving that degenerate U-statistic pairwise gradients collapse first-order projected noise variance from \(\mathcal{O}(B^{-1})\) to \(\mathcal{O}(B^{-2})\), and providing a unified theoretical foundation for the superiority of squared-hinge loss, the role of cross-entropy warm-up, and the benefit of pretrained parameter-efficient fine-tuning.
- Parameterized Complexity of Stationarity Testing for Piecewise-Affine Functions and Shallow CNN Losses
-
This paper establishes the parameterized complexity landscape of testing approximate first-order stationarity at a prescribed point for continuous piecewise-affine functions and shallow ReLU CNN losses, demonstrating XP tractability in fixed dimension alongside W[1]-hardness and ETH-tight lower bounds for complementary polarities.
- Path-Guided Flow Matching for Dataset Distillation
-
PGFM introduces the first flow matching framework for generative dataset distillation, combining latent-space deterministic ODE sampling with lightweight, trust-region-bounded prototype path guidance to achieve superior mode coverage and 7.6× higher synthesis efficiency than diffusion baselines.
- Penalty-Based First-Order Methods for Bilevel Optimization with Minimax and Constrained Lower-Level Problems
-
Focusing on a challenging class of bilevel optimization problems where both upper and lower levels have minimax structures, this paper proposes the first penalty-based proximal first-order method based on lower-level duality gaps, achieving \(\tilde{O}(\epsilon^{-4})\) deterministic and \(\tilde{O}(\epsilon^{-9})\) stochastic \(\epsilon\)-KKT complexity, while improving the complexity of convex-constrained lower-level bilevel optimization from \(\tilde{O}(\epsilon^{-7})\) to \(\tilde{O}(\epsilon^{-4})\).
- Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration
-
For stochastic quadratic optimization in the interpolation regime, this paper develops a general theory of classical momentum acceleration (covering both heavy ball and Nesterov momentum), proving that convergence scales linearly with mini-batch size \(m\) up to an accelerated \(\mathcal{O}(\sqrt{\kappa_f})\) saturation point, thereby theoretically establishing that classical momentum enables perfect parallelization for mini-batch SGD under minimal noise assumptions.
- Phase Kernel Lifts Capacity of Dense Associative Memory
-
Addressing the severe attractor basin overlap and spurious state breakdown of Modern Hopfield Networks under linear kernels at high memory loads, this paper develops a theoretical framework of separable strongly convex nonlinear phase kernels, instantiated as S-Hop, that guarantees CCCP monotone energy descent while achieving an over \(50\times\) critical retrieval load gain on ImageNet.
- Phases of Muon: When Muon Eclipses SignSGD
-
This paper derives exact deterministic risk dynamics for stochastic spectral optimizers (SignSVD/Muon) and coordinate-wise sign methods (SignSGD/Adam) via random matrix theory on high-dimensional matrix linear regression, proving square-root spectral preconditioning at large batch sizes, critical resolution transitions, and a three-phase diagram under power-law data anisotropy.
- Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
-
To resolve weight spectrum drift and activation explosion in large language model training, this paper proposes Pion, an optimizer derived from orthogonal equivalence transformations that modulates weight geometry via left and right orthogonal Lie group rotations while strictly preserving singular values, achieving stable and efficient training even in normalization-free and 200-layer ultra-deep networks.
- Polynomial-Time Algorithm for Thiele Voting Rules with Voter Interval Preferences
-
Resolving a 10-year-old open problem in computational social choice, this paper proves a structural concavity theorem for interval families under Thiele voting scores and establishes the first polynomial-time exact algorithm for the Voter Interval domain via Lagrangian relaxation and totally unimodular integer linear programming.
- Polynomial-Time Robust Multiclass Linear Classification under Gaussian Marginals
-
Addressing the open challenge of agnostically learning multiclass linear classifiers under Gaussian marginals and adversarial noise, this work establishes an inherent super-polynomial failure of the standard multiclass perceptron even without noise, and designs the first fully polynomial-time robust learning framework via pairwise improper tournaments and soft-truncation localization achieving dimension-independent near-linear error guarantees.
- Pooling Versus Ensembling for Ridge Regression Under Covariate Shift
-
Under a high-dimensional proportional asymptotic regime with covariate shift across clusters, this paper proves that an optimally tuned pooled ridge regression predictor achieves strictly smaller asymptotic prediction risk than any convex ensemble of individually tuned cluster predictors, with the performance gap governed entirely by the aspect ratio rather than cluster geometry.
- Position: Adopt Constraints Over Fixed Penalties in Deep Learning
-
Challenging the standard deep learning practice of replacing non-negotiable requirements with fixed weighted penalties, this paper demonstrates that scalarization fails geometrically on non-convex objectives, alters problem definitions, and induces severe tuning burdens, advocating for native primal-dual Lagrangian constraint formulations.
- Posterior Inference in Latent Space for Scalable Constrained Black-box Optimization
-
To tackle high-dimensional black-box optimization under black-box constraints where feasible regions are elusive and penalty landscapes are severely disconnected, CiBO frames candidate selection as posterior inference in the smooth latent space of flow-based models, utilizing an outsourced diffusion sampler trained with trajectory balance for sample-efficient exploration.
- Predicting Plasticity in Deep Continual Learning: A Theoretical Perspective
-
By proving via theoretical counterexamples that widely adopted structural diagnostics (such as representation rank and eNTK rank) can fail to predict plasticity loss, this paper introduces Optimization Readiness—a metric combining normalized gradient strength and signal-to-noise reliability that theoretically lower-bounds one-step optimization gain and reliably predicts future trainability.
- Provably Reliable Classifier Guidance via Cross-Entropy Control
-
This paper establishes the first quantitative theoretical link between classifier training via cross-entropy loss (conditional KL divergence) and diffusion guidance alignment, proving that while vanishing classification error alone permits divergent guidance via high-frequency oscillations, matching second-order smoothness guarantees a rate-optimal \(\widetilde{\mathcal{O}}(d\varepsilon)\) guidance MSE from an \(\mathcal{O}(\varepsilon^2)\) conditional KL divergence.
- Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime
-
This paper rigorously characterizes the inference dynamics of deep transformers via continuous-time mean-field continuity equations, establishing the first non-asymptotic Wasserstein convergence bound toward the push-forward projection \(\Pi_\sharp \rho_0\) on the dominant eigenspace of \(VB^\top\) within \(t \lesssim \mathcal{O}(\log \beta)\), and uncovering a terminal phase transition governed by the spectrum of \(V\).
- Quantitative Local Convergence of Mean-Field Stein Variational Gradient Flow
-
This paper establishes the first explicit quantitative local polynomial convergence rates in strong \(L^2\)-norm for the continuous-time mean-field Stein Variational Gradient Flow (SVGF) with Riesz interaction kernels on the torus, proving tightness for uniform targets.
- QuanVI: Score-based Variational Inference via Quantum Maximally Mixed States
-
QuanVI replaces EigenVI's single-eigenvector solution with a maximally mixed state on a low-energy subspace and compresses the density operator through local quantum tensor networks, scaling to 100-dimensional chain-structured synthetic targets while remaining limited by local-window capacity and cost.
- Query-Limited Community Recovery in Stochastic Block Models
-
This paper investigates exact community recovery in the two-community stochastic block model under limited, noisy neighborhood oracle access, establishing that adaptive querying breaks the linear-budget constant barrier in the oracle-only setting and achieves a strict sublinear-query adaptivity gap with an auxiliary subsampled graph.
- Query Lower Bounds for Diffusion Sampling
-
Establishes the first score query complexity lower bound of \(\widetilde{\Omega}(\sqrt{d})\) for diffusion sampling under polynomial-accuracy oracles, proving that any sampler within polynomial total budget must search over at least \(\widetilde{\Omega}(\sqrt{d})\) distinct noise levels.
- Reaching a Consensus in Predictive Loops
-
This paper introduces a minimal predictive loop model that formalizes the co-evolution of network opinion dynamics (Friedkin-Johnsen model) and performative machine learning retraining, establishing both theoretically and empirically that standard predictive objectives act as a homogenizing force that drives networks toward consensus even under conditions where stubborn agents maintain persistent disagreement in classical opinion models.
- Reflected Schrödinger Bridge Matching
-
This paper introduces Reflected Schrödinger Bridge Matching (RSBM), extending the partially simulation-free iterative Markovian fitting framework to reflected Brownian motion on the unit hypercube through coordinate-separable score approximations and a two-step exact bridge midpoint sampler, strictly eliminating out-of-bounds artifacts in bounded image generation with negligible computational overhead.
- Regularized Large Neighborhood Search
-
Addressing the computational bottleneck of requiring global exact solvers in neural combinatorial optimization layers, this paper introduces Regularized Large Neighborhood Search (RLNS), which converts local subproblem re-optimizations into a rejection-free MCMC sampler via smoothed maximum operators, rigorously proves exact equivalence to block Gibbs sampling under entropic regularization, and smoothly interpolates between pseudolikelihood and maximum likelihood estimation via search iterations.
- Remember with Confidence: Uncertainty Quantification for Spatio-temporal Memory with Probabilistic Guarantees
-
To tackle noisy multi-view VLM descriptions and silent hallucinations in embodied 4D spatio-temporal memory, this work introduces an object-level semantic uncertainty metric based on the log-determinant of a weighted semantic scatter matrix and proves a probabilistic guarantee under first-order stochastic dominance linking view-quality heuristics to uncertainty reduction, enabling budget-constrained active refinement and stage-aware multi-view caption fusion.
- Rennala NSGD: Asynchronous Stochastic Optimization Beyond Euclidean Geometry
-
Rennala NSGD introduces a semi-asynchronous distributed SGD framework for smooth non-convex optimization in general normed spaces, deriving high-probability oracle complexity bounds under dual-norm light-tailed noise via \(\kappa\)-regularity and establishing minimax-optimal wall-clock time complexity under computational worker heterogeneity.
- Replicable Constrained Bandits
-
This paper initiates the formal study of algorithmic replicability in constrained stochastic multi-armed bandits, proposing the DEBORA algorithmic framework that matches the \(\widetilde{\mathcal{O}}(\sqrt{T})\) regret and constraint violation bounds of non-replicable state-of-the-art baselines under both soft and hard constraints.
- Resilient Byzantine Agreement with Predictions
-
This paper initiates the formal study of algorithmic resilience in prediction-augmented Byzantine agreement, establishing optimal consistency-robustness trade-offs and full smoothness characterizations for both non-authenticated and authenticated communication settings alongside matching impossibility results.
- Rethinking Forward Processes for Score-Based Nonlinear Data Assimilation in High Dimensions
-
Addressing the failure of existing score-based filters under spatially sparse and channel-coupled nonlinear observations, this paper introduces a Measurement-Aware Score-based Filter (MASF) that interpolates states directly toward measurement space, deriving exact and Gaussian-surrogate likelihood scores via Markovian projection to achieve superior accuracy and up to 28.2x speedup on \(10^5\)-dimensional Kolmogorov turbulence.
- Rethinking Neural Nonlinearity as Gating
-
Neural nonlinearity is reformulated as input-conditioned pre-MAC threshold gating, proving standard activations and Softmax are gating instances, enabling lossless zero-shot network conversion and unified in-memory computing.
- Retrieval-Centric Deep Learning in Growing Nonparametric Neural Networks
-
This paper introduces the retrieval-centric deep learning (RCDL) framework, which replaces the fixed-size weight matrix of deep neural network layers with a growing key-value database that stores input activations and error gradients during training, retrieving and recombining them via non-linear RBF or weighted softmax kernel attention at inference time under principled functional gradient descent learning rules.
- Revisiting Decentralized Online Convex Optimization with Compressed Communication
-
Addressing the communication bottleneck and projection-coupling dilemma in decentralized online convex optimization, this work proposes the first FTRL-type algorithms with compressed communication, elegantly bypassing feasibility projection errors via dual consensus and simultaneously achieving superior regret bounds with sublinear communication rounds under bandit feedback.
- Robust Importance Sampling for Rare Events via Constrained Gaussian Mixtures
-
To resolve the twin bottlenecks of optimization cold-start and proposal variance explosion in ultra-rare event probability estimation, this paper introduces a two-stage importance sampling framework that decouples coverage-first initialization from constrained Gaussian mixture refinement enforcing strict covariance floor bounds \(\boldsymbol{\Sigma}_k \succ \frac{1}{2}\boldsymbol{\Sigma}\), theoretically guaranteeing finite estimator variance while delivering orders-of-magnitude empirical variance reduction across multimodal, high-dimensional, and deep neural network perturbation benchmarks.
- Robust Statistical Estimators with Bounded Empirical Sensitivity
-
Addressing the inability of classical robust statistics to capture an estimator's internal drift under adversarial contamination, this paper introduces the notion of empirical sensitivity, proves a tight lower bound of \(\Omega(\eta + \sqrt{\eta d / n})\) for MSE-optimal Gaussian mean estimators, and matches it up to logarithmic factors via robust empirical mean recovery.
- SAG-Sep: Sparse Augmented Graphs and Onion-Guided Search for Rounded Capacity Cut Separation
-
Addressing the prohibitive computational overhead of exact rounded capacity cut (RCC) separation and the slow inference of recursive graph coarsening in neural separators for large-scale CVRP, SAG-Sep introduces a sparse augmented graph representation with single-pass transformer encoding and a training-free Onion-Sweep decoder, accelerating neural separation roughly 30-fold while significantly tightening root relaxation bounds.
- Sample-Efficient Optimization over Generative Priors via Coarse Learnability
-
Addressing the lack of finite-sample guarantees for model-based optimization with expressive generative priors, this paper introduces the "coarse learnability" coverage assumption and proposes ALDrIFT—an annealed MBO algorithm with Metropolis-Hastings correction—achieving polynomial sample complexity and proving a fast \(\widetilde{O}(\log 1/\varepsilon)\) rate for non-convex objectives bounded by quadratic envelopes.
- Scalable Derivative Gaussian Processes via Exact Gradient Reduction
-
To eliminate the prohibitive \(\mathcal{O}(n^3 d^3)\) computational bottleneck of derivative Gaussian processes, Tera proves that orthogonal gradient components under stationary kernels are conditionally independent of scalar target values, reducing local gradients to at most \(m^2\) directional statistics and decoupling the local solve complexity into dimension-free \(\mathcal{O}(m^6)\) operations via Vecchia approximations.
- Scaling Full Conformal Image Classifiers
-
Addressing the computational explosion of Full Conformal Prediction (FCP) caused by candidate-specific model refits in thousand-class classification, this paper introduces Targeted Full Conformal Prediction (T-FCP) with zero-shot VLM pruning and Stabilized Online LDA (SO-LDA), slashing ImageNet per-image latency from 500s to 30ms while preserving rigorous statistical coverage guarantees.
- Search at the Cost of Sampling: Nearly-Instant Latent Space Bayesian Optimization
-
By restricting linear surrogate modeling to the hyperspherical thin shell of generative latent spaces, this paper derives nearly closed-form, one-dimensional root-finding solutions for model fitting and acquisition optimization, achieving over a 100x speedup while matching or outperforming Gaussian process baselines.
- Select-then-differentiate: Solving Bilevel Optimization with Manifold Lower-level Solution Sets
-
Addressing the challenge where overparameterized and nonconvex neural lower-level problems yield non-isolated solution manifolds rather than singletons, this paper introduces the "select-then-differentiate" framework, proving that uniqueness of the optimistic selection is sufficient for hyper-objective differentiability, deriving an explicit pseudoinverse-based hyper-gradient formula, and developing the HG-MS algorithm whose oracle complexity depends on the intrinsic dimension of the solution manifold.
- Select Smarter, Not More: Prompt-Aware Evaluation Scheduling with Submodular Guarantees
-
By reformulating automatic prompt optimization evaluation as an online computerized adaptive testing problem, this paper introduces POES—a monotone submodular scheduling framework integrating IRT-based discrimination, facility-location coverage, and switching-cost-aware warm-start updates with provable \((1-1/e)\) approximation and bounded-drift guarantees, reducing evaluation tokens by 35%–60% while substantially outperforming static and heuristic baselines.
- Self-Organized Conformal Prediction: Reducing Regional Coverage Gaps with Unsupervised Group Discovery
-
Self-Organized Conformal Prediction (SOCP) introduces an unsupervised Self-Organizing Map (SOM) calibration framework that discovers feature-space geometry without group labels, retrieving local calibration buffers via three flexible regimes to reduce regional weighted coverage gaps by an average of 14.8% across ten benchmarks without altering base models or nonconformity scores.
- Semiparametrically Efficient Inference for Kernel Measures of Noise Heterogeneity
-
To tackle spurious dependence between covariates and residuals induced by first-stage nonparametric regression errors, this paper develops a Hilbert-valued one-step debiased estimator of the kernel cross-covariance operator that circumvents scalar HSIC degeneracy at the null, enabling calibrated goodness-of-fit testing and semiparametrically efficient confidence intervals.
- SGD in Multiclass Logistic Regression: Sequential Learning and Scaling Laws
-
For high-dimensional Gaussian mixture models with divergingly many classes and power-law priors, this paper analytically proves the sequential learning mechanism and three-phase cross-entropy scaling laws under multiclass logistic regression, establishing compute-optimal trade-offs under PCA dimension reduction.
- SGD Provably Prioritizes a Shortcut Spurious Feature in the XOR Model
-
This paper provides the first end-to-end theoretical characterization of online minibatch SGD dynamics in two-layer ReLU networks on an XOR target with a linear shortcut, rigorously proving that the spurious feature grows exponentially fast and exponentially suppresses the quadratic signal feature through classification margins.
- Sharp Feature-Learning Transitions and Bayes-Optimal Neural Scaling Laws in Extensive-Width Networks
-
In the extensive-width regime (\(k=\Theta(d)\)) with hierarchical features, this paper derives closed-form fixed-point equations for Bayes-optimal generalization error and feature overlaps via leave-one-out cavity decoupling, revealing a sequential staircase of first-order phase transitions and unifying neural scaling laws under an effective width \(k_c\).
- Sharpness of Minima in Deep Matrix Factorization
-
This paper establishes the first closed-form expression for the maximum eigenvalue of the Hessian at any global minimum in deep matrix factorization and deep linear neural networks, proves that flat minima are fundamentally characterized by spectral-norm balancing rather than Frobenius-norm balancing, and provides the first empirical characterization of dynamical escape phenomena in deep matrix factorization.
- Signal-Adaptive Trust Regions for Gradient-Free Optimization of Recurrent Spiking Neural Networks
-
Addressing the vulnerability of recurrent spiking neural networks (RSNNs) in gradient-free reinforcement learning where finite population noise triggers catastrophic distributional shifts, this paper introduces Signal-Adaptive Trust Regions (SATR) that couple coordinate-wise KL divergence bounds to empirical signal energy, yielding a closed-form natural gradient update for Bernoulli connectivity alongside a bitset acceleration engine that achieves superior stability and compute throughput on continuous control benchmarks.
- Simple Extensions of Single-Objective Acquisition Functions and Hedge Strategies for Multi-Objective Bayesian Optimization
-
The paper generates candidates through Pareto search over a vector of single-objective acquisitions, selects queries using posterior-mean hypervolume, and adaptively chooses acquisitions with MO-Hedge; results are competitive on nine benchmarks, but the runtime advantage is mainly evident in batch mode rather than uniformly across settings.
- SimpleEvol: An Agent-Loop Framework for LLM-Driven Automated Heuristic Design with Minimal Human Priors
-
SimpleEvol replaces elaborate population and evolutionary-operator orchestration with one “generate code—execute and evaluate—compress memory” search trajectory, obtaining the highest TSP/CVRP intelligence-conversion regression slopes across ten LLMs and lower test gaps at every evaluated size with GPT-5-mini, without establishing that fewer priors necessarily improve performance.
- Sinkhorn Based Associative Memory Retrieval Using Spherical Hellinger Kantorovich Dynamics
-
This paper extends dense associative memory to empirical probability measures (weighted point clouds) by minimizing a log-sum-exp energy built from the debiased Sinkhorn divergence via spherical Hellinger-Kantorovich gradient dynamics, simultaneously updating support positions through barycentric transport and simplex weights through multiplicative reaction with exponential storage capacity.
- Skip the Hessian, Keep the Rates: Globalized Semismooth Newton with Lazy Hessian Updates
-
Addressing widespread nonsmooth derivatives in machine learning optimization, this paper proposes GLAd-SSN, the first semismooth Newton method that enables periodic reuse of stale Hessians (lazy Hessian updates) while guaranteeing nonasymptotic global convergence rates and local superlinear acceleration without requiring second-order differentiability.
- Slower Generalization, Faster Memorization: A Sweet Spot in Algorithmic Learning
-
Reveals that in structured-output algorithmic learning tasks, validation exact-match convergence is fastest at an intermediate dataset size (a dataset-size sweet spot), overturning the intuition that more data monotonically accelerates post-threshold convergence, and proves that partial rule learning accelerates training set fitting.
- Slowly Annealed Langevin Dynamics: Theory and Applications to Training-Free Guided Generation
-
This paper establishes a non-asymptotic reverse-KL convergence theory for Slowly Annealed Langevin Dynamics (SALD) via differential inequalities, and introduces Velocity-Aware SALD (VA-SALD) to decouple pretrained transport velocities from guide-induced residuals for provably stable training-free guided sampling.
- SMOG: Scalable Meta-Learning for Multi-Objective Bayesian Optimization
-
Addressing the challenges of meta-task uncertainty propagation and cross-objective correlation in multi-objective Bayesian optimization, SMOG proposes a structured multi-output Gaussian process prior that delivers a closed-form target prior with linear scaling in meta-tasks while capturing inter-objective dependencies.
- Smoothed Score Queries and the Complexity of Sampling
-
This paper demonstrates that for high-dimensional Gaussian sampling, smoothed-score queries collapse the condition-number query complexity from the classical first-order barrier of \(O(\sqrt{\kappa})\) to logarithmic \(\mathrm{polylog}(\kappa)\), while establishing nearly matching information-theoretic lower bounds via reverse Shannon channel synthesis.
- Sobolev Regularized MMD Gradient Flow
-
To overcome the non-convexity and sluggish convergence of Maximum Mean Discrepancy (MMD) and Kernel Stein Discrepancy (KSD) Wasserstein gradient flows, this paper proposes Sobolev-regularized MMD (SrMMD) flow with a witness gradient penalty, establishing provable global exponential convergence independent of target isoperimetric constants for both unnormalized sampling and generative modeling.
- SplineFlow: Flow Matching for Dynamical Systems with B-Spline Interpolants
-
SplineFlow constructs smooth, locally stable conditional probability paths via Cox-de Boor B-spline interpolation, eliminating the trajectory curvature distortion of linear flow matching while achieving over 100x speedup over adjoint-based Neural ODEs with provable derivative error bounds.
- SS-eSOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning
-
SS-eSOAP augments Kronecker-factored SOAP preconditioning with a scalar secant-energy correction and adaptive eigenbasis triggers paired with variance-state downscaling, achieving the lowest final PDE residual on six of eight benchmarks and accelerating stiff PINN training.
- Stochastic Dynamic Barrier Perturbed Gradient Methods for Nonconvex Simple Bilevel Optimization
-
Addressing stochastic simple bilevel optimization where both upper- and lower-level objectives are smooth and nonconvex, this paper proposes dual-perturbed and penalty-regularized dynamic barrier algorithms (SDBPG, PR-SDBPG, VR-PR-SDBPG), establishing the first explicit sample complexity guarantees for achieving \((\epsilon_f, \epsilon_g)\)-stationarity.
- Stochastic Grouping Conformal Prediction for Effective Subgroup Reliability
-
To overcome the limitations of standard conformal prediction—which guarantees coverage only on population averages while masking severe miscoverage on clinical subgroups—and existing subgroup calibration methods that inflate set sizes and suffer from small-sample instability, this paper proposes Stochastic Grouping Conformal Prediction (SGCP). By modeling a sample-adaptive local score law as a convex mixture of latent calibration components via posterior grouping, SGCP minimizes subgroup coverage disparity without requiring sensitive attributes or sacrificing prediction set efficiency.
- Stochastic Zeroth-Order Optimization Under Heavy-Tailed Noise
-
To tackle heavy-tailed sample-gradient noise with potentially infinite variance in modern machine learning, this paper proposes RSC-ZO and its momentum variant—two-point zeroth-order methods using shared randomness and scalar clipping before aggregation—establishing the first high-probability convergence guarantees for smooth nonconvex objectives under weak-\(L_p\) noise with optimal \(\widetilde{O}(d^{\frac{p}{2(p-1)}}\varepsilon^{-\frac{3p-2}{p-1}})\) query complexity.
- Sublinear Time Quantum Sensitivity Sampling
-
Addressing large-scale data summarization and approximation, this paper presents a unified quantum sensitivity sampling framework based on Grover search and recursive halving that constructs coresets for clustering, regression, and low-rank approximation in sublinear time without depending on data-specific parameters like condition numbers or matrix norms.
- Submodular Benchmark Selection
-
Formalizes LLM benchmark subset selection as submodular optimization under a multivariate Gaussian model, demonstrating that greedy mutual information selection vastly outperforms greedy entropy (pivoted Cholesky) for small subsets by directly capturing coupling with unselected benchmarks, achieving imputation \(R^2 \ge 0.91\) with only 5 MMLU subjects.
- Super-Level-Set Regression: Conditional Quantiles via Volume Minimization
-
Addresses the two-step plug-in limitation of multivariate conditional quantile region estimation by proposing Super-Level-Set Regression (SLS), which uncouples the volume objective from the model's internal conditional quantiles via a shrinking probability window surrogate, enabling direct end-to-end geometric optimization of complex multimodal highest density regions.
- Supervised Distributional Reduction via Optimal Transport and Dependence Maximization
-
The paper proposes Supervised Distributional Reduction (SDR), an optimal transport framework combining barycentric Fused Gromov-Wasserstein with explicit Centered Kernel Alignment (CKA) to bypass the supervision bottleneck, enabling stable RKHS out-of-sample extension and data-dependent non-stationary kernel construction for Gaussian Processes.
- SURF: Steering the Scalarization Weight to Uniformly Traverse the Pareto Front
-
The paper uncovers the geometric mismatch where uniform scalarization weight sampling induces non-uniform Pareto front coverage, and proposes SURF (Sampling Uniformly along the Pareto Front) based on arc-length cumulative distribution function (CDF) inversion, achieving uniform Pareto front traversal with an \(\mathcal{O}(N^{-2})\) discretization floor while preserving the inner single-objective solver untouched.
- Surprises in Proper Positive-Only Learning
-
This paper resolves the long-standing open question of characterizing proper PAC learning from positive-only examples by proving that a concept class is properly learnable if and only if it has finite VC dimension and satisfies uniform exterior separability, while establishing fundamental separations between proper vs. improper learning, deterministic vs. randomized learning, and ERM vs. non-ERM learners.
- Symmetry Guarantees Statistic Recovery in Variational Inference
-
This paper develops a unified measure-theoretic framework of symmetry-induced statistic recovery in variational inference, proving that when the target is invariant under a transformation group and the variational family is stable under it, any unique minimiser inherits the target's invariance and recovers all symmetry-determined statistics, unifying Euclidean location-scale guarantees and establishing novel axis recovery theorems for von Mises-Fisher families on the sphere.
- TagBO: LLM-Driven Task-Aware Graph Bayesian Optimization for Scientific Discovery
-
To resolve the fundamental misalignment between task-agnostic structural descriptors and target objective behavior in scientific discovery, TagBO extracts coarse-grained ordinal relational priors from LLM reasoning and adaptively fuses them with descriptor-based structural graphs via multi-gradient descent, constructing an optimization-friendly joint manifold that drastically accelerates Bayesian optimization under scarce experimental budgets.
- Taking the Road Less Scheduled with Adaptive Polyak Steps
-
This paper derives closed-form, adaptive Polyak-type step sizes for Schedule-Free SGD and Adam that compute iteration-wise learning rates from loss values and gradient norms, eliminating both the preset training horizon and base learning rate tuning.
- Teacher-Aware Evolution of Heuristic Programs from Learned Optimization Policies
-
By querying independently trained black-box neural optimization policies on states visited by candidate heuristic programs to extract local action preferences, this framework guides LLM reflection and Pareto evolution to discover high-performing, interpretable static heuristics without requiring any neural network inference at deployment.
- Tensorion: A Tensor-Aware Generalization of the Muon Optimizer
-
Tensorion introduces a tensor-aware optimizer that generalizes Muon's spectral norm constrained descent from matrices to higher-order tensors via a tractable dual nuclear norm relaxation and a dimension-balanced unfolding heuristic, achieving superior convergence and accuracy over SGD and AdamW across CNN and Vision Transformer architectures.
- Test time training enhances in-context learning of nonlinear functions
-
This paper proves that single-layer Transformers equipped with test-time training (TTT) can effectively learn dynamic nonlinear single-index models in-context, overcoming fundamental limitations of standard in-context learning regarding static link functions and non-vanishing asymptotic bias.
- The Implicit Bias of Hyperbolic Representation Learning for Multiclass Data: A Busemann Risk Perspective
-
Through a large-radius radial-Busemann decomposition of hyperbolic distance, this paper establishes the first rigorous asymptotic theory of Riemannian gradient flow under permutation-equivariant relative-margin (PERM) multiclass losses, revealing a radial escape/repulsion dichotomy and directional convergence toward critical points of the Busemann risk.
- The Multi-Block DC Function Class: Theory, Algorithms, and Applications
-
Addressing the exponential representation blowup and non-constructive nature of classical difference-of-convex (DC) programming in block-structured models, this paper introduces the Multi-Block DC (BDC) function class, proves an exponential separation in monomial representation complexity over global DC, provides the first explicit layer-wise constructive decomposition for deep ReLU networks, and establishes non-asymptotic convergence guarantees under generalized smoothness and stochastic settings.
- The Query Complexity of Local Search in Rounds on General Graphs
-
This paper provides the first comprehensive characterization of the query complexity of local search in at most \(t\) rounds on general graphs, establishing a deterministic upper bound \(O(t n^{1/t} (s\Delta)^{1-1/t})\) based on the separation number \(s\) and maximum degree \(\Delta\), a universal randomized lower bound \(\Omega(t n^{1/t} - t)\) holding for any connected graph, and a sublinear \(O(n/\log n)\) bound for high-expansion graphs using randomized parallel steepest descent.
- The Sign Code: The Hidden Binary Nature of Deep Networks
-
This paper uncovers the hidden binary nature of deep networks: for all sign-preserving activations (ReLU, SiLU, GELU, tanh), neuron activations decompose exactly into binary signs and non-negative magnitudes; in the overparameterized regime, a weight-free nearest-neighbor classifier on discrete sign codes alone matches full-model accuracy, and causal interventions prove that forward classification is dominated by sign routing while remaining remarkably robust to magnitude noise.
- The Symmetries of Three-Layer ReLU Networks
-
This paper develops a geometric framework based on tropical geometry and balanced polyhedral complexes to provide the first complete characterization of generic parameter fibers for three-layer bottleneck ReLU networks (\(n_1 \le d\)), revealing novel continuous translation and discrete sign-flipping symmetries induced by layer composition, establishing a polynomial-time equivalence algorithm, and determining their impact on gradient flow conservation laws.
- The two clocks and the innovation window: When and how generative models learn rules
-
By studying controlled synthetic rule tasks, this paper reveals two decoupled timescales in generative model training—the rule-learning clock \(\tau_{\text{rule}}\) and the memorization clock \(\tau_{\text{mem}}\)—whose temporal gap defines an "innovation window" for novel rule-valid generation, supported by energy-based and attractor landscape analyses.
- TILT: Target-Induced Loss Tilting under Covariate Shift
-
This paper introduces Target-Induced Loss Tilting (TILT) for unsupervised domain adaptation under covariate shift, which decomposes the source predictor into a deployable model and an auxiliary residual component penalized on unlabeled target data, achieving an exact relative reweighting excess risk criterion without estimating or clipping potentially unbounded density ratios.
- Towards Understanding Momentum Acceleration in River-Valley Loss Landscape
-
This paper develops a non-asymptotic dynamical theory for heavy-ball momentum in non-convex "river-valley" loss landscapes, demonstrating that momentum accelerates optimization not by directly boosting velocity along the flat river, but by damping transverse oscillations in the sharp valley directions to enlarge the maximum tolerable learning rate by approximately \(\frac{1+\beta}{1-\beta}\).
- Towards Understanding the Power and Limits of the Muon Optimizer: A River-Valley Perspective
-
This paper develops a trajectory-level theory via a mixed-spiked matrix sensing model and a river-valley landscape perspective, revealing that Muon achieves rapid linear progress along information-bearing river directions early on but inherently suffers from step-size contraction and oscillations near the optimum due to stripping scale information, which motivates a two-stage "Muon exploration \(\to\) AdamW refinement" optimization paradigm.
- Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers
-
Establishes a mean-field theory of noisy Transformers with trained feedforward networks, rigorously proving that learning reshapes layerwise token dynamics into a three-phase structure consisting of initial clustering, an interior turnpike plateau, and a terminal gradient-driven escape.
- Training for the Model You Return: Improving Optimization for Iterate-Averaged Language Models
-
By formalizing optimizer design for pipelines that return iterate-averaged models as a continuous-time stochastic optimal control problem, this paper derives a state-feedback control law that pulls live parameters toward their EMA, resulting in PACE—a lightweight AdamW wrapper that outperforms standard AdamW with decay schedules in LLM pretraining and fine-tuning even under a constant learning rate.
- Training with (Swap) Regret Loss in a Single-Layer Self-Attention Model: A Case Study on the Probability Simplex
-
This paper proves that training single-layer self-attention models with (swap) regret loss directly on probability-simplex policies end-to-end yields stationary points whose forward passes precisely implement smoothed fictitious play with a \(\widetilde{\Theta}(1/\sqrt{T})\) stepsize (guaranteeing external no-regret and coarse correlated equilibrium) and the Blum–Mansour algorithm via a multi-head fixed-point architecture (guaranteeing no-swap-regret and correlated equilibrium).
- Tree-Structured Synergy of Large Language Models and Bayesian Optimization for Efficient CASH
-
Addressing the high-dimensional hierarchical CASH problem, LB-MCTS models optimization as a Monte Carlo Tree Search over structured trajectories, seamlessly unifying LLM semantic priors and Bayesian optimization numerical search through shared tree states, Selective Tuning Memory, and reliability-aware dynamic proposer selection to achieve state-of-the-art sample efficiency across 104 AMLB datasets.
- Two-Level Softmax Sampling Done Right: Correcting Bias from Size Imbalance and Dispersion
-
Addressing systematic sampling distortions in Two-Level Softmax (2LS) induced by cluster size imbalance and intra-cluster similarity dispersion, this work introduces S-2LS and SD-2LS, reducing approximation error by orders of magnitude while preserving sublinear complexity.
- Two Speeds of Learning: A Representation-Readout Decomposition of Grokking and Double Descent
-
Decomposing a neural network into a nonlinear encoder and a linear readout, this paper introduces a top-down diagnostic framework based on critical dimension, manifold geometry, linear probing, and last-layer NTK alignment, showing that grokking and epoch-wise double descent stem from a speed imbalance between continuous representation learning and train-biased readout calibration while identifying spurious generalization modes.
- Understanding Generalization Requires Universal Induction
-
This paper demonstrates that classical statistical learning theory and its meta-learning extensions are fundamentally constrained by No Free Lunch theorems, and establishes that relativized Solomonoff induction anchored to physical information vantage points offers the only non-vacuous theoretical foundation for general-purpose AI generalization via algorithmic mutual information conservation.
- Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization
-
This paper establishes the theoretical equivalence between gradient orthogonalization (e.g., Muon) and non-Euclidean trust-region optimization under the matrix spectral norm, providing unified stochastic convergence guarantees with momentum, weight decay, and extrapolation across non-convex, star-convex, and second-order smooth settings.
- Understanding Sample Efficiency in Predictive Coding
-
This paper theoretically explains why Predictive Coding (PC) achieves superior sample efficiency over Backpropagation (BP) using a "target alignment" metric, proving that equilibrium inference in PC naturally forms a preconditioning matrix that cancels weight-induced interference, and derives exact rescaling conditions for optimal interference-free updates.
- Understanding Schedule-Free Methods in Nonconvex Optimization: Rate Guarantees and Escaping Saddles
-
By establishing continuous-time limiting ODEs and discrete Lyapunov potentials, this paper proves that standard Schedule-Free GD and SGD achieve optimal first-order worst-case convergence rates on smooth nonconvex objectives and almost surely avoid strict saddles under a one-time perturbation.
- Understanding the Curse of Unrolling
-
This paper provides the first non-asymptotic analysis of the "curse of unrolling"—where derivative iterates transiently diverge from the true Jacobian—demonstrating that early iteration errors create an \(\mathcal{O}(k\rho^k)\) transient growth term, and proving that explicit truncation and warm-starting effectively eliminate this curse under a fixed computational budget.
- Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks
-
This paper proves non-asymptotic, uniform-in-time weak propagation-of-chaos bounds for shallow neural networks in the feature-learning regime under deterministic, noiseless Wasserstein gradient flow, guaranteeing that an excess MSE of \(\epsilon\) is achievable with \(\operatorname{poly}(d/\epsilon)\) neurons, training samples, and gradient descent steps whenever the mean-field loss decays faster than \(t^{-2}\).
- Universal Adaptive Proximal Gradient Methods via Gradient Mapping Accumulation
-
The paper proposes a universal adaptive proximal gradient method (and an accelerated convex variant) that accumulates historical gradient mapping norms in the step-size denominator, achieving near-optimal convergence rates across nonconvex smooth, convex nonsmooth, and convex smooth composite objectives without requiring any prior knowledge of problem parameters.
- Universal Approximation Theorems for Dynamical Systems with Infinite-Time Horizon Guarantees
-
Addressing the longstanding limitation that dynamical system approximation guarantees are restricted to finite horizons or monostable fading-memory systems, this work establishes the first universal approximation framework for multistable continuous-time dynamical systems over an infinite-time horizon \([0, \infty)\) using a decoupled \((\varepsilon, \delta)\)-closeness metric, proving that finite-size Neural ODEs achieve uniform trajectory fidelity across fixed points, limit cycles, and continuous attractors.
- Unpaired Canonical Correlation Analysis
-
Addressing the fundamental bottleneck of Canonical Correlation Analysis requiring strictly paired data, this paper establishes the theoretical foundations of Unpaired CCA (UCCA) by proving that whitened kernel Quadratic Assignment Problem (QAP) is equivalent to a Frobenius relaxation of maximum correlation pairing, and demonstrates that a practical anchor-based QAP matching algorithm recovers latent correlations close to the paired CCA upper bound.
- Variance Reduction for Expectations with Diffusion Teachers
-
Addressing the high variance and computational cost of Monte Carlo gradient estimation when using frozen diffusion models as teachers in SDS optimization, one-step distillation, and data attribution, CARV introduces compute-aware amortized re-noising, explicit weight-driven timestep importance sampling, and stratified inverse-CDF sampling, achieving a 2-3x effective compute multiplier without altering the underlying objective.
- Very Fast Bayesian Additive Regression Trees on GPU
-
Addressing the computational bottleneck of Bayesian Additive Regression Trees (BART) caused by dynamic tree structures and sequential MCMC sampling on CPU, this paper introduces a fixed-capacity binary heap matrix representation and a branchless vectorized algorithm implemented in JAX, achieving up to 200× speedup on GPU while preserving exact Bayesian inference and rivaling XGBoost in execution time.
- Wavelet Flow Matching for Multi-Scale Physics Emulation
-
Wavelet Flow Matching (WFM) formulates optimal-transport conditional flow matching directly within a multi-scale discrete wavelet space, leveraging a hierarchical U-Net to jointly predict flow velocities across sub-bands and achieving stable, high-fidelity autoregressive physical dynamics without learned autoencoders.
- When and Why SignSGD Outperforms SGD: A Theoretical Study Based on \(\ell_1\)-norm Lower Bounds
-
The paper circumvents the classical obstacle of SGD's minimax optimality in \(\ell_2\) Euclidean geometry by establishing an \(\ell_\infty\)-smoothness and separable noise theoretical framework, proving that SignSGD achieves up to a \(d\)-fold dimension complexity speedup over SGD under sparse noise, and seamlessly extending this tight bound to the matrix Muon optimizer via SVD dynamical isomorphism.
- When Does Trimming Help Conformal Prediction? A Retained-Law Diagnostic under Calibration Contamination
-
Reframes outlier trimming in calibration sets as conditioning onto an induced retained law, establishing an exact finite-sample coverage identity and diagnostic that disentangles clean-side covariance distortion from dirty retention penalties.
- When One Leak Pays Forever: Context Binding and the Price of Deterring Collusion
-
The paper characterizes the exact deterrence threshold through the future value reachable by one terminal leak: genuine context isolation requires exposure covering only one round, whereas long-lived reuse can multiply the requirement by \(T\) over an undiscounted finite horizon.
- Zeroth-Order Sharpness-Aware Learning with Exponential Tilting
-
This paper proposes ZEST, a gradient-free sharpness-aware optimization algorithm using exponential tilting that bridges average-loss smoothing and worst-case min-max SAM, maintaining the \(2|x|\) memory efficiency of vanilla zeroth-order methods while penalizing high-curvature directions to achieve superior generalization in LLM fine-tuning.
- Hierarchical Conformal Classification
-
Hierarchical Conformal Classification (HCC) incorporates class taxonomies modeled as directed acyclic graphs (DAGs) into conformal prediction, formulating prediction set generation as a constrained optimization problem that prunes the search space via non-overlapping leaf covers to produce compact, user-aligned prediction sets with guaranteed finite-sample coverage.
- SpecDrop: Parameter-Free Category-Conditioned Routing for Modular Specialization
-
SpecDrop shapes modular specialization through a fixed category-to-branch assignment, nonzero cross-category soft weights, and fixed-denominator merging, outperforming label-free matched controls on vision tasks with trusted inference-time category labels, while label-aware dense logit masking is more accurate and routing gains on fuzzy text partitions are near zero.