Skip to content

🎮 Reinforcement Learning

🧠 NeurIPS2026 · 237 paper notes

📌 Same area in other venues: 🎞️ ECCV2026 (12) · 📷 CVPR2026 (25) · 🔬 ICLR2026 (400) · 💬 ACL2026 (46) · 🧪 ICML2026 (110) · 🤖 AAAI2026 (58)

🔥 Top topics: Reinforcement Learning ×94 · Agents ×12 · Adversarial Robustness ×7 · LLM ×4 · Reasoning ×4

A Distribution Mapping Approach to Counterfactually Fair Reinforcement Learning

To eliminate sequential discrimination in high-stakes reinforcement learning without relying on restrictive additive noise assumptions, this paper develops a sequential marginal distribution mapping preprocessing framework (CFSMDM) based on conditional quantile alignment, establishing theoretical finite-sample bounds on suboptimality gap and counterfactual unfairness.

A Geometric Perspective on Reward Function Updates in Inverse Reinforcement Learning

This paper formulates an occupancy-space geometric view of inverse reinforcement learning, proving that moving along the e-geodesic toward the optimal occupancy is equivalent to preconditioning the reward dual gradient with the integral Fisher Information Matrix, which reduces to a computation-free functional convex interpolation under Mirror Descent.

A Single-Sample Polylogarithmic Regret Bound for Nonstationary Online Linear Programming

Addressing nonstationary Online Linear Programming under extreme data scarcity (a single sample per arrival distribution), this paper proposes a dynamic programming-inspired dual re-solving algorithm that leverages a deterministic \(\delta\)-path and uniform dual convergence to establish an \(\mathcal{O}(\log^2 n)\) polylogarithmic regret bound in the large-resource regime.

A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Sets

This paper establishes the uniqueness and Lipschitz continuity of optimal objective vectors under Smooth Tchebycheff scalarization, deriving an \(\mathcal{O}(1/k)\)-convergent Concave Mirror Descent Policy Iteration (CMDPI) algorithm in occupancy space and its deep actor-critic extensions for dense Pareto-front coverage.

Absolute State-wise Constrained Policy Optimization: High-Probability State-wise Constraints Satisfaction

Addressing the vulnerability of traditional safe RL methods that only enforce constraints in expectation and fail under long-tail violations, this paper proposes Absolute State-wise Constrained Policy Optimization (ASCPO)—the first model-free policy search algorithm that guarantees high-probability state-wise safety constraint satisfaction without assuming known system dynamics, achieved via rigorous non-discounted upper bounds on both the expectation and variance of maximum state-wise costs.

Achieving Optimal Static and Dynamic Regret Simultaneously in Bandits with Deterministic Losses

This paper demonstrates that in oblivious adversarial multi-armed bandits with deterministic losses and known switch count \(S\), one can simultaneously achieve optimal static regret \(\widetilde{O}(\sqrt{AT})\) and optimal dynamic regret \(\widetilde{O}(\sqrt{SAT})\) by leveraging negative static regret and Blackwell approachability.

Active Context Selection Improves Simple Regret in Contextual Bandits

For contextual multi-armed bandits with finite contexts evaluated by context-weighted simple regret, this paper proves that active subpopulation selection yields up to a \(\Theta(k^{1/4})\) gain over passive sampling, derives the optimal power-law allocation \(q_j \propto p_j^{2/3}\) along with a threshold water-filling rule under budget constraints, and develops the EETC algorithm that asymptotically matches the known-distribution fully active rate without prior knowledge of context frequencies.

Actor-Accelerated Policy Dual Averaging for Reinforcement Learning in Continuous Action Spaces

Addressing the computational bottleneck where Policy Dual Averaging (PDA) requires solving a costly optimization sub-problem per continuous action selection, this paper introduces Actor-Accelerated PDA, which approximates sub-problem minimizers via a parameterized policy network and preserves global convergence guarantees while outperforming standard on-policy baselines like PPO.

Advancing Entropy-Level Credit Assignment in RLVR via Proximal Entropy Policy Optimization

AETDICE: Unified Framework and Offline Optimization for Nonlinear Multi-Objective RL

Addressing the long-standing bifurcation between Scalarized Expected Return (SER) and Expected Scalarized Return (ESR) in nonlinear multi-objective RL, this paper introduces the unified Aggregation–Expectation–Transformation (AET) framework and develops AETDICE, the first tractable offline optimization algorithm supporting both trajectory-level nonlinear transformations and expectation-level concave aggregations.

AffectGPT-RL: Revealing Roles of Reinforcement Learning in Open-Vocabulary Emotion Recognition

Addressing the misalignment between traditional token-level cross-entropy loss and non-differentiable Emotion Wheel metrics in open-vocabulary multimodal emotion recognition, AffectGPT-RL pioneers a reinforcement learning framework using Group Relative Policy Optimization, designs multifaceted rewards with length penalties to curb reward hacking, and achieves state-of-the-art results across diverse affective benchmarks.

Aligning Flow Map Policies with Optimal \(Q\)-Guidance

This paper introduces flow map policies and theoretically derives an analytically optimal closed-form learning target (FMQ) for online adaptation under critic trust-region guidance, alongside an inference-time search algorithm (QGBS), achieving state-of-the-art offline-to-online control with one-step action generation.

Almost Sure Convergence Rates of Stochastic Approximation and Reinforcement Learning via a Poisson-Moreau Drift

This paper develops a novel Poisson-Moreau drift Lyapunov function that directly cancels the conditional bias of Markovian noise on individual sample paths, establishing near-optimal almost sure convergence rates for contractive stochastic approximation and off-policy reinforcement learning under non-Euclidean norms.

AlphaPareto: Formulaic Alpha Discovery with LLM-Guided Multi-Objective Reinforcement Learning

AlphaPareto encodes the current alpha pool with a frozen LLM, conditions token-by-token MaskPPO formula search on existing signals, and uses adaptive multi-objective rewards for predictive power, ranking stability, perturbation robustness, and diversity, achieving out-of-sample ICs of 3.92%, 5.70%, and 10.10% on CSI300, CSI800, and the full market.

Amortized Molecular Optimization via Group Relative Policy Optimization

Addressing the training instability caused by heterogeneous starting-structure difficulty in structurally constrained molecular design, AMORTIX introduces a Graph Transformer policy trained with Group Relative Policy Optimization (GRPO) to generate optimized derivatives in a single forward pass with zero inference-time oracle calls.

Anatomy of Off-Policy Policy Gradient: Importance Sampling, KL Regularization, and Baselines

This paper provides a unified theoretical anatomy of off-policy policy gradient methods for LLM post-training, proving that consistency requires either exponential importance sampling or KL regularization with an optimal soft value baseline, and establishing that a constant baseline shift implicitly implements pessimism via an offline game-theoretic formulation.

Auction-Based Online Policy Adaptation for Evolving Objectives

Addressing multi-objective reinforcement learning under dynamically appearing and disappearing goals, this paper proposes a modular auction-based framework where decentralized selfish policies submit urgency-calibrated bids to win execution rights, enabling instant, zero-retraining adaptation and transparent trade-offs.

Augmented Lagrangian Method for Last-Iterate Convergence for Constrained MDPs

Addressing the memory overhead and safety hazards of mixture policies in primal-dual constrained MDPs, this paper proposes an inexact Augmented Lagrangian Method (ALM) policy optimization framework that achieves provable global last-iterate convergence with \(\mathcal{O}(1/\varepsilon^6)\) gradient evaluation complexity under both tabular and log-linear settings while outperforming standard baselines on continuous control benchmarks.

Bandits via Additive Quantized Representations

Repurposing Residual Quantization (RQ) codebooks as a nonlinear representation layer for contextual bandits, combined with residual-cascade gradient boosting and shadow-model online depth adaptation, achieves competitive regret against doubling-retrain XGBoost and neural bandits with strictly bounded \(\mathcal{O}(1)\) memory.

Batched Stochastic Linear Bandits with 1-Bit Communication Constraints

This paper investigates stochastic linear bandits under fixed-size batching (\(B\)) and a 1-bit per-batch uplink bottleneck, establishes an unavoidable \(\Omega(B\min\{d,\log|\mathcal{A}|\})\) communication lower bound even without noise, and develops two phased-elimination algorithms matching the minimax lower bound across broad regimes.

Bayes-Sufficient Compression Is Not Enough: How Does Communication Help Multi-Agent Systems?

This paper establishes a receiver-relative bounded coordination framework for multi-agent LLM systems, proving that Bayes-sufficient compression is insufficient to guarantee receiver usability, decomposing coordination loss into externalization, absorption, and action closure, and introducing an adaptive inference-time protocol selector that substantially improves the accuracy-cost frontier.

Bayesian Decision Making observing an Expert

Addressing multi-armed bandit decision-making in the presence of an expert, this paper proposes integrating the expert's optimal outcome distribution into the learner's prior via consistent Bayesian updates, proves that offline expert data tightens Thompson Sampling's regret bound by the mutual information between the expert data and the optimal action, and develops an active information-directed routing rule alongside expert trustworthiness modeling.

Bayesian Decision Making observing an Expert

This paper establishes a Bayesian decision-making framework for learners observing an expert's outcomes, proving that offline expert pretraining tightens Thompson Sampling regret bounds by the mutual information \(\mathbb{I}_0(A^*; D_N^*)\), and introducing an online maximum mutual information routing rule along with a dynamic trust inference mechanism against imperfect experts.

Bellman Residual Minimization for Control: Geometry, Stationarity, and Convergence

This paper establishes foundational geometric properties, Clarke subdifferentials, and convergence guarantees for control Bellman residual (CBR) and soft control Bellman residual (SCBR) minimization in policy optimization, proving that stationary points satisfy oblique projected control Bellman equations and circumventing the divergence pitfalls of projected value iteration.

Bench-MFG: A Benchmark Suite for Learning in Stationary Mean Field Games

To resolve the chronic lack of standardized evaluation protocols in Mean Field Games (MFGs), Bench-MFG introduces a comprehensive benchmark covering five fundamental theoretical game classes, procedural synthetic generation via MF-Garnet, and a black-box exploitability minimizer (MF-PSO), achieving ~2000x simulation speedup via JAX.

Best Arm Identification in Generalized Linear Bandits via Hybrid Feedback

This paper investigates fixed-confidence best arm identification in generalized linear bandits under hybrid feedback by constructing self-concordant likelihood-ratio confidence ellipsoids that unify absolute and relative observations, and designing a minimax-optimal Track-and-Stop policy (HyTS-GLB) with cost-aware extensions.

Beyond One-Size-Fits-All: Diagnosis-Driven Online Reinforcement Learning with Offline Priors

Addressing the core dilemma that benchmark rankings fail to guide the utilization of heterogeneous offline priors, this paper conceptualizes offline priors as "bounded commitments" whose validity boundaries are intrinsically uncertain, advocating a paradigm shift from searching for universally optimal methods to diagnosis-driven tension management based on prior-deployment match assessment and dynamic monitoring.

Block-R1: Rethinking the Role of Block Size in Multi-domain Reinforcement Learning for Diffusion Large Language Models

This paper formally defines the domain block size conflict in multi-domain RL post-training for diffusion LLMs, presenting the Block-R1 benchmark and the 41K dataset that allocates sample-level best-improved block sizes to break rigid decoding constraints and boost cross-domain generalization.

Block-R1: Rethinking the Role of Block Size in Multi-Domain Reinforcement Learning

Addressing the severe cross-domain degradation and performance collapse in multi-domain reinforcement learning (RL) for diffusion large language models (dLLMs) caused by rigid unified block sizes, this paper presents Block-R1, which identifies sample-level best-improved training block sizes via a teacher-student framework and constructs Block-R1-41K to achieve block-conditioned policy optimization with substantial generalization gains.

Breaking the Bias Barrier in Concave Multi-Objective Reinforcement Learning

Addressing the persistent gradient bias in concave-scalarized multi-objective reinforcement learning that previously restricted policy-gradient sample complexity to \(\widetilde{\mathcal{O}}(\epsilon^{-4})\), this paper introduces a natural policy gradient algorithm equipped with a multi-level Monte Carlo (MLMC) estimator to achieve the optimal \(\widetilde{\mathcal{O}}(\epsilon^{-2})\) rate, and demonstrates that second-order smooth scalarizations allow vanilla NPG to achieve the same rate via automatic bias cancellation.

Breaking the Grid: Distance-Guided Reinforcement Learning in Large Discrete Action Spaces

Addressing the curse of dimensionality and catastrophic gradient variance explosion in action spaces with up to \(10^{20}\) candidates, this paper introduces Distance-Guided Reinforcement Learning (DGRL), which combines Chebyshev-constrained Sampled Dynamic Neighborhoods (SDN) for linear-time volumetric exploration with Distance-Based Updates (DBU) that decouple gradient variance from action space cardinality.

CAFE: Causally-Guided Automated Feature Engineering with Multi-Agent Reinforcement Learning

CAFE casts automated feature engineering as a causally-guided sequential decision process, combining sparse causal graph discovery as a soft prior with cascading multi-agent deep Q-learning to achieve superior predictive accuracy and distribution robustness across 15 benchmark datasets.

Can In-Context Learning Support Intrinsic Curiosity?

This paper investigates whether emergent in-context learning (ICL) in sequence models can serve as update-free predictive world models to drive intrinsic curiosity, proving that unbiased estimation of Bayesian information gain (BIG) is impossible in general BAMDPs, while establishing that a novel cumulative future prediction reward \(r^{\mathrm{sum}}\) asymptotically recovers BIG and resists the noisy TV trap in Bayesian Experimental Design (BED).

Chess-World-Model: A 10M-Game Benchmark for Exact State Tracking from Chess Move Sequences

Constructs Chess-World-Model, a large-scale deterministic state-tracking benchmark spanning 10 million real games and uniformly random legal play, revealing that linear RNNs with expressive transition operators markedly outperform Transformers under small scales and distribution shifts, and demonstrating how model scale conceals internal transition failures.

CIG: Exploration via Conditional Information Gain

Addressing the dichotomy where lifelong rewards neglect within-rollout redundancy and episodic bonuses ignore lifetime progress, CIG derives a tractable trajectory-level information gain surrogate via trace reduction and Cholesky factorization, yielding causal per-step intrinsic rewards that excel across 12 diverse tasks under clean and noisy-distractor settings.

Compositional Reasoning in Language Models Under Reinforcement Learning Post-Training

This paper introduces a dependency-graph framework with three hierarchical complexity levels to formalize compositional reasoning, proving and empirically validating a fundamental decomposed-to-composed asymmetry where RL on decomposed skills fails to transfer to compound tasks, whereas compound-task RL readily transfers back to atomic skills.

Conformal Selective Acting: Anytime-Valid Risk Control for RLVR-Trained LLMs

For locally hosted specialist LLMs fine-tuned via reinforcement learning with verifiable rewards (RLVR) under adaptive online updates, this paper introduces Conformal Selective Acting (CSA), an anytime-valid test-supermartingale e-process wrapper over a Bonferroni threshold grid that guarantees pathwise selective risk control without exchangeability.

Context Value Informed In-Context Reinforcement Learning

Addressing the brittle and unstable test-time adaptation caused by high-variance Monte-Carlo return feedback in existing in-context reinforcement learning (ICRL), CV-ICRL introduces an explicit context value head and writes its predictions back into the context sequence, enabling low-variance temporal-difference credit assignment and substantially stabilizing cross-task adaptation.

Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing

To tackle high exploration costs and inter-model performance correlations in online LLM routing under bandit feedback, this paper proposes correlation-aware contextual bandits with surrogate rewards (CABS), introducing a decoupled prediction-mixing architecture (CABS-D) that achieves a best-of-both-worlds regret guarantee—matching standard bandits in the worst case while scaling inversely with surrogate observations under informative signals.

DaCe-DT: Data-Centric Offline Multi-Task Reinforcement Learning via Adaptive Prompts and Trajectory Correction for Heterogeneous Tasks

Addressing the core bottlenecks of context length mismatch, semantic irrelevance of prompt segments, and fragmented suboptimal trajectories in offline multi-task RL, DaCe-DT introduces a data-centric framework combining length-gated prompt masking, retrieval-augmented prompt construction, and value-adaptive return calibration, achieving state-of-the-art performance on Meta-World.

DARLING: Detection Augmented Reinforcement Learning with Non-Stationary Guarantees

Addressing model-free reinforcement learning in piecewise stationary finite-horizon MDPs without prior knowledge of changes, this paper establishes the first minimax dynamic regret lower bounds for tabular and linear MDPs and proposes DARLING, a modular detection-restart wrapper that explicitly monitors reward and transition shifts via decoupled uniform probing to achieve the first minimax-optimal dynamic regret in both settings.

Decision-Focused Learning in MDPs: An Occupancy Measure Approach

This paper reformulates MDPs with unknown dynamics into an occupancy-measure linear programming layer, derives closed-form gradients by inverting a small \(|\mathcal{S}| \times |\mathcal{S}|\) basis matrix, and addresses dynamics mismatch and scalability via an augmented Lagrangian surrogate, random row sketching, and learnable soft state aggregation.

Decoupling Time and Risk: Risk-Sensitive Reinforcement Learning with General Discounting

This paper extends stock-augmented distributional reinforcement learning to arbitrary non-exponential discount functions and optimized certainty equivalent (OCE) risk measures, establishing time-consistent dynamic programming algorithms and performance error bounds while resolving theoretical inconsistencies in prior multi-horizon methods.

Deep Double Q-learning

Deep Double Q-learning (DDQL) fully adapts classical Double Q-learning to deep RL by explicitly training two Q-functions via reciprocal bootstrapping, dataset partitioning, shared trunk representations, and a halved replay ratio (1/8), reducing overestimation and beating Double DQN on 47 out of 57 Atari 2600 games.

DeFlow: Decoupling Behavior-Prior Modeling and Value Maximization for Offline Policy Extraction

DeFlow decouples behavior distribution modeling from Q-value maximization in offline RL by preserving an expressive multi-step flow model as a behavior prior to avoid backpropagation through ODE solvers, while training an action-conditioned lightweight residual module under an adaptive trust-region penalty for local policy improvement.

Delayed Homomorphic Reinforcement Learning for Environments with Delayed Feedback

This paper introduces Delayed Homomorphic Reinforcement Learning (DHRL) and its continuous actor-critic algorithm D2HPG, which define a belief-equivalence relation over the augmented state space to collapse control-redundant states, eliminating the exponential sample complexity burden of delayed RL while preserving optimality under deterministic dynamics and significantly outperforming strong baselines under long delays.

Delightful Distributed Policy Gradient

Addressing the core issue where high-surprisal failure updates from stale actors, bugs, and mismatched inference dominate distributed policy gradients, this paper proposes the Delightful Policy Gradient (DG), which gates updates via delight—the product of advantage and surprisal—to suppress perpendicular failure noise while preserving rare discoveries without behavior probabilities.

Differentiable Belief-based Opponent Shaping

Addressing the computational intractability of parameter-space opponent shaping and the brittleness of heuristic deception objectives, D-BOS differentiates through \(k\)-step softmax-Bayes belief dynamics to directly shape observers' low-dimensional posterior beliefs driven strictly by task rewards.

DiPO: Disentangled Perplexity Policy Optimization for Fine-grained Exploration-Exploitation Trade-Off

To tackle advantage degradation on extreme sample groups and ineffective exploration-exploitation in RLVR, DiPO introduces perplexity space disentangling to dynamically partition exploration and exploitation subspaces, coupled with a bidirectional reward reallocation mechanism that orthogonalizes exploration guidance and minimally perturbs verification rewards.

Direction-Aware Offline-to-Online Learning in Linear Contextual Bandits

Addressing the limitation of conventional isotropic Euclidean bias bounds in offline-to-online contextual bandits, this paper proposes a directional bias certificate and a dual-branch algorithm, Ellipsoidal-MINUCB, that accelerates learning along aligned directions while providing provable online safety and data-driven adaptivity via parallel-sum geometry.

Direction-Aware Offline-to-Online Learning in Linear Contextual Bandits

Addressing the directional mismatch between offline historical logs and live online rewards in linear contextual bandits, this paper proposes an anisotropic directional bias certificate and the Ellipsoidal-MINUCB algorithm (with both fixed and data-driven adaptive variants), safely accelerating online exploration along well-covered, low-bias feature directions while maintaining worst-case online fallback guarantees.

Discovering What You Can Control: Interventional Boundary Discovery for Reinforcement Learning

To tackle high-dimensional distractors confounded with true state dynamics, Interventional Boundary Discovery (IBD) uses randomized action interventions to sever non-causal confounding paths and applies per-dimension two-sample tests with FDR control to construct an exact control boundary mask, achieving near-oracle performance across 12 continuous control tasks.

Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Sampling

To tackle imbalanced learning caused by rigid uniform interaction allocation in multi-task reinforcement learning, this paper presents Distributionally Robust Adaptive Task Sampling (DRATS), which formalizes MTRL as a feasibility minimax problem to adaptively steer data collection toward tasks furthest from their target returns with provable convergence guarantees.

Do LLMs Learn from Rewards in Context? : Rethinking the role of reward in In-Context RL

Controlled experiments across six models and four benchmarks show that direct in-context reinforcement learning (Direct ICRL) is driven by imitating the surface form and distribution of same-task trajectories rather than by reward semantics; reward modulation accounts for only ~14% of the trajectory exposure effect, and flipping or randomizing rewards leaves improvement curves largely intact, indicating that direct ICRL is an in-context learning (ICL) phenomenon rather than genuine inference-time RL.

does this gradient spark joy

Introduces the Kondo gate based on delight (the product of advantage and surprisal) and compute price, executing backward passes only on high-learning-value samples in reinforcement learning to preserve training quality while dramatically slashing backpropagation costs.

domination-avoiding learning agents cannot collude

This paper formally proves that no-external-regret learning agents can sustain arbitrary collusive pricing in Bertrand Logit duopolies, introduces the broader and robust class of Domination-Avoiding (DA) learning agents, and mathematically guarantees that DA agents provably eliminate iteratively dominated strategies and asymptotically converge to the competitive Nash equilibrium window.

Don't Let Gains FADE: Breaking Down Policy Gradient Weights in RL

This paper introduces an orthogonal policy gradient weight decomposition framework across sign and difficulty axes, analyzes the mechanisms of entropy and rank-1 update collapse, and proposes FADE, an adaptive advantage scheduler that preserves diversity while accelerating LLM reasoning post-training.

DRILL: Training World Models to Improve Policies, Not Predict Pixels

DRILL introduces a bilevel reinforcement learning framework that shifts the training objective of video world models from one-step pixel prediction to downstream policy return, bridging the gap between visual fidelity and control utility via implicit meta-gradients and signed visitation-weighted surrogates.

DROGO: Default Representation Objective via Graph Optimization in Reinforcement Learning

Addressing the computational bottleneck where obtaining the default representation (DR) principal eigenvector requires full-matrix estimation and eigendecomposition, DROGO introduces a log-space graph drawing objective optimized via Riemannian natural gradient with a single terminal anchor constraint, enabling end-to-end neural network approximation directly from transitions.

DRTriton: Large-Scale Synthetic Data Driven Reinforcement Learning for Triton Kernel Generation

DRTriton establishes a scalable framework that synthesizes shape-valid PyTorch programs via the CSP-DAG algorithm and employs curriculum reinforcement learning with decoupled correctness and speed rewards (DRPO), enabling a 7B language model to achieve speedups on 92% of KernelBench Level 2 tasks.

DUET: Optimize Token-Budget Allocation for Reinforcement Learning with Verifiable Rewards

DUET coordinates prompt-level rollout allocation and within-rollout early stopping under a shared token budget, using cost-weighted Neyman allocation with an online variance surrogate and marker-gated abort with importance reweighting, beating full-budget GRPO at half the compute with a 2.51× wall-clock speedup.

Dyna-Style Safety Augmented Reinforcement Learning: Staying Safe in the Face of Uncertainty

Proposes Dyna-SAuR, a model-based safe RL algorithm that jointly learns a control policy and a safety filter within a probabilistic ensemble dynamics model, cutting training-time safety violations by over two orders of magnitude via unit-hyperball hyperplane parameterization and data shaping.

ECHO-2: A Large-Scale Distributed Rollout Framework for Cost-Efficient Reinforcement Learning

ECHO-2 reduces LLM reinforcement learning post-training costs by disaggregating centralized policy learning from a distributed, wide-area pool of heterogeneous rollout workers, leveraging a bounded staleness execution model, an overlap-based closed-form capacity provisioning rule, and peer-assisted pipelined broadcast.

Effectiveness of Curriculum Learning Depends on Reward Sparsity and Competing Optima

Through an analytically tractable minimal model of open-loop target interception policy learning, this paper demonstrates that curriculum learning's profound utility in RL stems from overcoming exponentially slow convergence under sparse rewards and leveraging timely bifurcations to steer agents away from trivial quitting optima.

Efficient Lookahead Encoding and Abstracted Width for Learning General Policies in Classical Planning

To overcome severe memory explosions and combinatorial atom growth in iterated-width policies, this paper introduces an Aggregated-Delta (AD) search-tree encoding that scores all successors in a single forward pass, coupled with Abstracted Iterated Width (AIW) that compresses novelty checks to scale linearly with objects, achieving state-of-the-art results on IPC 2023 and outperforming LAMA.

Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control

This paper establishes a token-level theoretical framework of entropy mechanics in reinforcement learning with verifiable rewards (RLVR) via first-order logit perturbation, revealing a structural asymmetry where reinforcing high-probability tokens triggers entropy contraction, and introduces Polarity-Aware Policy Optimization (Papo) to adaptively reweight token advantages based on online entropy trajectories.

EpicWorldModel: Exploration-driven Planning with Latent World Models

Addressing the issue where deterministic JEPA world models hallucinate unobserved futures under occlusion, EpicWorldModel introduces a stochastic latent dynamics model trained via conditional flow matching and leverages multi-sample predictive variance as an information-theoretic exploration signal within CEM planning, achieving substantial success-rate gains across visual navigation and manipulation benchmarks.

Equivariant Reinforcement Learning for Clifford Quantum Circuit Synthesis

Addressing Clifford quantum circuit synthesis on fully connected architectures, this paper formulates the task as reverse symplectic reduction to the identity and develops a size-agnostic, qubit-permutation-equivariant graph reinforcement learning framework that sets a new optimum-recovery record on 6-qubit benchmarks and zero-shot generalizes up to 30 qubits.

Ergodic Risk Measures: Towards a Risk-Aware Foundation for Continual Reinforcement Learning

This work establishes the first formal theoretical foundation for risk-aware continual reinforcement learning, demonstrating that conventional static and nested risk measures violate continual learning axioms and proposing ergodic risk measures as a mathematically sound, plasticity-preserving framework.

Escaping Path Mirages in Offline Goal-Conditioned Reinforcement Learning

Addressing the "path mirage" failure mode where offline goal-conditioned RL agents overcommit to deceptively successful stochastic trajectories, this paper proposes Branch-aware Graph Planning (BGP)—which identifies branching landmarks via multi-step predictive uncertainty, prunes risky branches via goal-conditioned downstream edge evaluation, and stabilizes value propagation through segmented learning, achieving dramatic performance gains on complex OGBench tasks.

ESSAM: A Novel Competitive Evolution Strategies Approach to Reinforcement Learning for Memory Efficient LLMs Fine-Tuning

Addressing the extreme GPU memory consumption of reinforcement learning fine-tuning for large language models, this paper proposes ESSAM, a zeroth-order full-parameter fine-tuning framework tightly integrating Evolution Strategies with Sharpness-Aware Maximization, matching PPO and GRPO on mathematical reasoning tasks while operating with inference-level memory.

Evolutionary System Prompt Learning for Reinforcement Learning in LLMs

Evolutionary System Prompt Learning (E-SPL) unites large language model weight updates via reinforcement learning with self-reflection-driven prompt population evolution in a single training loop, decoupling declarative rules into prompts and procedural skills into weights.

Factorized Spectral Representations for Reinforcement Learning

By formulating the transition kernel of continuous-control MDPs as a three-way tensor across states, actions, and next states via CP decomposition, FaStR decouples state and action representations with a noise contrastive objective, drastically improving sample efficiency in high-dimensional action spaces while enabling modular state encoder transfer across actuator shifts.

Fast Rates for Offline Contextual Bandits with Forward-KL Regularization under Single-Policy Concentrability

To address the long-standing open problem of slow rates in forward-KL-regularized offline contextual bandits, this work proposes FKL-PCB and establishes a novel convex-analytical pipeline that achieves \(\widetilde{O}(\epsilon^{-1})\) fast statistical rates under single-policy concentrability alongside matching minimax lower bounds.

Fast Rates for Offline Contextual Bandits with Forward-KL Regularization under Strong Duality

This paper investigates forward-KL regularized offline contextual bandits, proposes a pessimistic learning algorithm (FKL-PCB), and establishes the first \(\widetilde{O}(\epsilon^{-1})\) fast statistical rate under single-policy concentrability, accompanied by matching minimax lower bounds and a phase transition characterization in the low-regularization regime.

Fisher Decorator: Refining Flow Policy via a Local Transport Map

To eliminate mode collapse and out-of-distribution drift induced by isotropic Wasserstein surrogate penalties in flow-based offline RL, Fisher Decorator (FiDec) models policy refinement as a local transport map and leverages the local Fisher information matrix directly extracted from the flow velocity field for anisotropic trust-region optimization.

FLAG: Flow Policy MaxEnt-RL by Latent Augmented Guidance

To eliminate importance weight collapse in global importance sampling and numerical instability in Backpropagation Through Time (BPTT), FLAG lifts the state space into a latent-augmented MDP, centers a Gaussian local policy on flow-generated anchor actions, optimizes a provably consistent cross-entropy proxy MaxEnt objective via local importance sampling, and distills the target actions back into the flow policy to establish state-of-the-art continuous control performance.

Flow Matching for Offline Reinforcement Learning with Discrete Actions

This paper introduces QDFM, an energy-guided discrete flow matching framework for offline reinforcement learning that models policy dynamics via Continuous-Time Markov Chains and Q-weighted regression, theoretically recovering the exact KL-regularized optimal policy without auxiliary classifiers.

Flow Matching Policy Optimization with Mirror Descent and Entropy Constraints

Addressing the core bottlenecks of gradient instability and intractable policy entropy in diffusion-based continuous control, this paper introduces Flow Matching Policy Optimization with Mirror Descent and Entropy Constraints (FMER), achieving simulation-free policy training alongside unbiased bounded-action entropy control across sparse-reward FrankaKitchen and continuous MuJoCo benchmarks.

Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewards

Addressing the reward polarization and criterion-level imbalance caused by static scalarizers in rubric-based reinforcement learning for open-ended generation, this paper proposes Focal Reward, which uses inverse reward projection to dynamically measure frontier criterion saturation and adaptively reweights optimization toward high-headroom criteria, outperforming static aggregation baselines across three model scales and six benchmarks.

Focusing Influence Mechanism for Multi-Agent Reinforcement Learning

To tackle scattered influence and aimless wandering in sparse-reward multi-agent reinforcement learning, FIM sustains joint coordinated influence via eligibility traces and focuses exploration on under-explored state dimensions using an entropy criterion, achieving persistent and structured coordination under extremely sparse feedback.

Follow the Winners: Conservative Policy Improvement with the Cross-Entropy Method for Critic-Free RFT

Addressing the prohibitively high cost of repeated environment rollouts and severe noise entrenchment from sparse long-horizon verification in stateful agent environments, this paper introduces Follow the Winners (FTW), an ordinal critic-free RFT algorithm that adapts the cross-entropy method to induce polynomial concentration over replay-buffer order statistics without requiring value models or duplicated group rollouts.

From POMDP Theory to Deep RL with Particle Filters

To bridge the persistent gap between computational POMDP theory and practical deep RL, this paper introduces PF-MCTQ—a unified pipeline integrating SMC particle filtering for belief tracking, online latent dynamics learning, and finite-depth tree search with state-level QMDP critic bootstrapping—proving an end-to-end sublinear regret bound that explicitly decomposes four error sources while demonstrating competitive sample efficiency on continuous control benchmarks.

From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning

QTPT replaces behavior-action prediction in a parameter-shared Transformer with context-conditioned Q-target regression, enabling a fixed-weight model to improve decisions using reward and transition history from the same task; gains are pronounced under weak behavior data, but the theory requires coverage and realizability rather than guaranteeing success on arbitrary low-coverage datasets.

G-Zero: Self-Play for Open-Ended Generation from Zero Data

G-Zero introduces a verifier-free co-evolutionary self-play framework that drives autonomous LLM self-improvement in open-ended generation from zero human data, using an intrinsic Hint-\(\delta\) predictive distribution shift signal to train a Proposer via GRPO and a Generator via length-normalized DPO.

GAE Falls Short in Imperfect-Information Self-Play Reinforcement Learning

Addressing the severe variance bottleneck in Generalized Advantage Estimation (GAE) caused by sampling future actions from stochastic equilibrium policies in imperfect-information self-play, this paper proposes the Q-boosting advantage estimator based on a centralized Q-critic and multi-step Expected SARSA traces alongside the VRPO algorithm, achieving substantial exploitability reductions and state-of-the-art performance in Dou Dizhu and Heads-Up No-Limit Texas Hold'em.

Generalized Intention Modeling in Multi-Agent Reinforcement Learning

Addressing the failure of existing opponent modeling methods to generalize across tasks due to hard-coded intent priors, this paper introduces MIX (Mixer of Intention eXperts), an adaptive framework that dynamically fuses opponent actions, observations, future states, and an InfoNCE reward-predictive intent embedding to significantly outperform prior baselines across partially observable and high-dimensional multi-agent environments.

Generative Actor-Critic with Soft Bridge Policies

Addressing intractable marginal action densities and the steep latency and backpropagation costs of iterative diffusion/flow policies in MaxEnt RL, this paper introduces Soft Generative Actor-Critic (SoftGAC), which formulates analytical path-space relative entropy regularization over a single-pass stochastic Gaussian bridge network, outperforming strong generative baselines at minimal inference overhead.

Generative OOD-regularized Model-based Policy Optimization

To prevent transition model rollouts from drifting into dangerous out-of-distribution (OOD) regions in sparse state-action spaces, GORMPO introduces a plug-and-play framework that penalizes model-generated rewards using generative density estimation via a saturating tanh penalty, providing theoretical conservatism guarantees and boosting return by 17% on real-world medical weaning.

Global linear convergence of entropy-regularized softmax policy gradient beyond tabular MDPs

For infinite-horizon entropy-regularized MDPs with general continuous state and action spaces, this paper proves the global linear convergence of log-linear softmax policy gradient flow under regularized value realizability and feature radial unboundedness.

Global Linear Convergence of Entropy-Regularized Softmax Policy Gradient Beyond Standard Settings

Under \(Q^\pi_\tau\)-realizability in continuous-space MDPs, this paper proves that entropy-regularized log-linear softmax policy gradient flows achieve a global linear convergence rate of \(\mathcal{O}(e^{-Ct})\) for full-affine-span and simplex feature classes via non-uniform Polyak–Łojasiewicz inequalities and Lyapunov radial unboundedness of relative entropy.

Global Optimality for Constrained Exploration via Penalty Regularization

To address constrained maximum-entropy exploration under safety or imitation requirements, this paper develops Policy Gradient Penalty (PGP), a single-loop primal method using pseudo-rewards that bypasses composite non-convex gradient estimation bias and guarantees non-asymptotic global last-iterate polynomial sample complexity.

Gradient Routing Localizes and Removes Unintended Behaviors in RL

Addressing reward hacking caused by misspecified rewards and weak monitors in RL post-training, GRAFT decouples monitor signals from reward shaping into gradient routing masks that redirect and amplify flagged updates to a sacrificial forget adapter, which is ablated at deployment to cleanly eradicate unintended behaviors while retaining target task capabilities.

Graph-Based Stochastic Power-UCT: Monte-Carlo Graph Search with Power Mean Estimation

To overcome trajectory redundancy and sample fragmentation caused by tree representations in stochastic MDPs, this paper proposes GS-Power-UCT, which couples depth-augmented DAG state merging with multi-parent power-mean estimation, preserving \(O(n^{-1/2})\) root-level convergence while substantially improving sample reuse efficiency.

Hadamard Representation: Scaffolding Performance Across Model-free RL

HaM-World: Soft-Hamiltonian World Models with Selective Memory for Planning

HaM-World integrates selective history memory and a Soft-Hamiltonian prior on part of the latent coordinates into one planning model, ranking first on four and second on two state-based control tasks, but its imagined-prediction advantage is confined to planning-relevant horizons, and memory has a substantially larger observed ablation impact than geometry.

Heterogeneous Agent Collaborative Reinforcement Learning

This paper introduces Heterogeneous Agent Collaborative Reinforcement Learning (HACRL) and the HACPO algorithm, which uses capability-aware baseline scaling, capability discrepancy gradient modulation, exponential importance sampling, and stepwise clipping to enable cross-agent rollout sharing during training while preserving strictly independent inference.

Higher-Order Action Supervision Makes A Strong Policy Class

Addressing the control instability and poor generalization caused by optimizing only zeroth-order actions, this paper proposes an architecture-free higher-order action supervision paradigm and continuous-time HJB value regularizer that significantly improve policy smoothness and closed-loop performance in low-data regimes.

Hitting Time Isomorphism for Multi-Stage Planning with Foundation Policies

This paper introduces Isomorphic Embedding Learning (IEL), an operator-theoretic representation learning framework proving that expected hitting times in controlled Markov processes are linearly isomorphic to latent displacements, enabling directed, triangle-inequality-compliant multi-stage graph planning for zero-shot foundation policies.

Horizon Adaptive Offline Policy Learning via Value Stitching

To overcome the loss of crucial short-horizon decision signals caused by rigid fixed-step backups and action chunking in complex long-horizon tasks, this paper proposes VAST, which introduces a horizon-conditioned cumulative return function and an adaptive stitching policy to achieve dynamic, variable-length value stitching and two-level reward-maximizing control.

How Does RL Post-training Induce Skill Composition? A Case Study on Countdown

By formalizing Countdown arithmetic solutions as canonical binary expression trees and eliminating dataset simplicity biases, this paper disentangles length generalization from compositional generalization in RL post-training, demonstrating that computational structure (balanced trees are vastly easier than right-heavy ones) rather than sequence length governs learning difficulty and induces an autoregressive lookahead bottleneck.

Imitation from Observations with Trajectory-Level Generative Embeddings

Addressing the severe support mismatch and lack of action supervision in offline imitation learning from observations (LfO), this paper proposes Trajectory-level Generative Embeddings (TGE), which uses the latent representation of a trajectory diffusion planner trained on suboptimal data alongside a particle-entropy-based heavy-tailed logarithmic kernel to synthesize dense surrogate rewards that bypass fragile density ratio estimation and achieve state-of-the-art offline imitation.

Information-Geometric Forward Policy Training in GFlowNets

By treating the GFlowNet forward policy as an induced trajectory sampler on a statistical manifold, this work proves an exact temporal martingale-difference decomposition of the trajectory Fisher information matrix and develops exact, sampled, and structure-exploitable natural-gradient training regimes that dramatically boost exploration and distribution matching without auxiliary policies.

Information Templates: A New Paradigm for Intelligent Active Feature Acquisition

TAFA replaces unconstrained, combinatorial feature acquisition policies with a compact library of jointly informative feature templates optimized via submodular genetic search and Gumbel-Softmax relaxation, distilling the policy into transparent step-wise decision trees that achieve state-of-the-art accuracy-cost tradeoffs with orders-of-magnitude faster inference.

Instructing LLMs to Negotiate using Reinforcement Learning with Verifiable Rewards

This paper introduces a Reinforcement Learning from Verifiable Rewards (RLVR) framework using GRPO for bilateral price negotiation, revealing an emergent four-phase strategic evolution that enables a 30B buyer model to substantially outperform frontier reasoning models over ten times its size.

Inverting the Bellman Equation: From \(Q\)-Values to World Models

Addressing the long-standing dichotomy between model-free and model-based reinforcement learning, this paper introduces P-learning to invert the Bellman equation and uniquely extract accurate environment world models directly from fixed goal-conditioned \(Q\)-values, achieving zero-shot out-of-distribution planning.

Inverting the Bellman Equation: From Q-Values to World Models

This paper introduces P-learning, an inverse analogue to Q-learning that decodes internal world models from multi-goal value functions, proving theoretically and empirically that goal-conditioned model-free agents implicitly encode accurate environment dynamics.

JAXtari: High-Throughput and Easy-to-Modify Arcade Learning Environment

To overcome the CPU simulation bottleneck and semantic opacity of traditional Atari emulators, JAXtari completely reimplements 45+ Atari 2600 games in pure functional JAX from scratch, enabling massive accelerator-native throughput (100M PPO steps in under 1 hour on a single A100 GPU) with native object-centric states and source-level modification plugins.

Joint Learning of Hierarchical Neural Options and Abstract World Model

AgentOWL jointly learns a temporally and spatially abstract neurosymbolic world model alongside deeply hierarchical neural options, using LLM-proposed precondition subgoals and training stabilization to achieve high sample efficiency and zero-shot generalization on hard exploration benchmarks.

Jointly Reinforcing Diversity and Quality in Language Model Generations

To combat distribution sharpening and diversity collapse during language model post-training, this paper proposes Darling, which partitions generations into semantic equivalence clusters and multiplies normalized semantic diversity with quality rewards, simultaneously improving response quality and variety across creative generation and mathematical reasoning.

LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matching

To tackle the issue where behavior cloning collapses the multimodal structure of pretrained latent action spaces and naive flow matching causes action decoder misalignment, LAFP leverages conditional flow matching with direct latent action regression and an inference-time interpolation mechanism, substantially outperforming prior baselines across 16 PROCGEN environments.

Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring

Addressing the opacity and physical implausibility of continuous generative policies, as well as the limited expressivity of conventional discrete policies, this paper presents Large Discrete Policy (LDiP)—a framework that progressively scores and prunes candidates over a large discrete vocabulary with score-space stochasticity, achieving high expressivity while strictly preserving physical plausibility and decision interpretability.

LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL

LASER combines dataset support constraints with explicit entropy regularization in latent-space offline RL, employing latent adjoint matching to optimize expressive flow policies without backpropagation through time (BPTT), achieving state-of-the-art performance across 40 OGBench tasks with fixed hyperparameters.

LeAct: Learning to Reason from Expert Actions

Addressing the limitation that conventional domain experts (game solvers, classical planners) output near-optimal actions without natural language rationales, LeAct models reasoning as a latent variable and recovers high-quality chain-of-thought traces via an iterative variational-EM pipeline featuring backward proposal generation, forward delta scoring, and expert-policy forcing.

Learning Chance-Constrained MDPs with Bellman Distributional Certificates

The paper expresses cumulative-cost violation events through remaining-budget Bellman recursions, obtains nearly matching deterministic-policy sample complexity under bounded successor support and a certified planning oracle, and combines local approximate-KKT optimization with independent validation for stochastic policies; synthetic and power-system simulations distinguish statistical conservatism from conservatism induced by expected-cost surrogates.

Learning in Context, Guided by Choice: A Reward-Free Paradigm for Reinforcement Learning with Transformers

Addressing the vulnerability that conventional in-context reinforcement learning (ICRL) fundamentally depends on consistent scalar reward engineering across diverse tasks, this paper proposes ICPRL, the first entirely reward-free in-context preference learning paradigm. Through In-Context Policy Optimization (ICPO) and In-Context Reward Generation (ICRG), Transformer meta-policies achieve rapid zero-gradient adaptation to unseen tasks without explicit rewards or expert action labels.

Learning Large-Scale Competitive Team Behaviors with Mean-Field Interactions

To overcome the exponential scalability bottleneck in large-scale team competitions, this paper introduces MF-MAPPO, combining a minimally informed critic with shared team actors trained directly in finite-population simulators, and extends it to partially observable domains via gradient regularization and dynamic-projected consensus.

Learning Multimodal One-step Flow Policy via Value-weighted Optimal Transport

OptiFlow reframes multimodal one-step flow policy distillation as a structured sample allocation problem, leveraging asymmetric value-weighted optimal transport to preserve multimodal behaviors while eliminating out-of-distribution drift.

Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewards

Addressing the exploration ceiling and diversity collapse of Reinforcement Learning with Verifiable Rewards (RLVR) in long-horizon reasoning, this paper introduces parameter-space noise (PSN) injected into Transformer MLP blocks prior to rollout sampling, paired with truncated importance sampling (TIS) and a lightweight real-time adaptive noise scheduler driven by semantic diversity and self-certainty, substantially expanding the high-budget reasoning boundary without disrupting multi-step chain-of-thought coherence.

Learning to Trigger: Reinforcement Learning at the Large Hadron Collider

Addressing the failure of static trigger menus under beam luminosity decay and detector drift at the Large Hadron Collider (LHC), this paper formulates online threshold adaptation as a streaming reinforcement learning task and introduces feasibility-filtered group policy optimization (GFPO-F and GFPO-FR) to strictly maintain background rate tolerance while boosting physics signal efficiencies, achieving the first zero-shot transfer demonstration on real CMS collision data.

Learning Weakly Communicating Average-Reward CMDPs: Strong Duality and Improved Regret

This paper establishes strong duality for weakly communicating infinite-horizon average-reward CMDPs over stationary policies, and builds upon this theoretical breakthrough to develop PD-LSCVI-UCB, the first primal-dual clipped value iteration algorithm under linear function approximation that improves both regret and constraint violation from \(\widetilde{\mathcal{O}}(T^{3/4})\) to \(\widetilde{\mathcal{O}}(T^{2/3})\).

Leveraging Error Diversity in Group Rollouts for Reinforcement Learning

Addressing the issue where standard RLVR discards the intra-group distribution of incorrect rollouts, EDAS introduces an information-theoretic advantage shaping mechanism that amplifies penalties for dominant repeated errors and attenuates penalties for exploratory rare errors, effectively breaking error perseveration and delivering substantial gains across math reasoning and code generation.

Lifted State Policy Optimization: Internalizing Self-Correction through Transition Geometry

Addressing the limitation that preference optimization and trajectory RL fail to credit local correction steps, LSPO augments discrete text with a continuous coordinate to form a lifted state space, steers policy exploration via cost-adjusted energy descent, and internalizes low-energy endpoints into the direct generator, matching multi-pass test-time reflection in a single forward pass with substantial latency and token savings.

Lipschitz Dueling Bandits over Continuous Action Spaces

Addressing stochastic dueling bandits on compact continuous spaces with bi-Lipschitz preference transfer, this paper introduces ARC-DuelLi—a recursive elimination algorithm leveraging round-wise anchored reference arms that matches the scalar Lipschitz bandit lower bound \(\tilde{O}(T^{\frac{d_z+1}{d_z+2}})\) using only \(O(d \log T)\) space.

LiSA: Lifelong Safety Adaptation via Conservative Policy Induction

Addressing the unpredictable long-tail contextual privacy and safety risks faced by AI agents in deployment, LiSA introduces a structured policy memory framework that uses broad policy abstractions, conflict-aware local refinements, and Beta-posterior confidence gating to adapt a fixed base guardrail continually under sparse and noisy user feedback without repeated fine-tuning.

LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL

Addressing the collapse of discriminative reward signals caused by static training prompts in non-verifiable reinforcement learning, this paper presents LLM-as-a-Tutor, a framework that uses a single tutor model via pairwise rollout comparison to identify saturated prompts and monotonically scales task difficulty by appending atomic constraints and criteria, achieving leading performance across three complex instruction-following benchmarks.

Localized Dynamics-Aware Domain Adaption for Off-Dynamics Offline Reinforcement Learning

To tackle transition dynamics mismatch between abundant source data and scarce target data in offline RL, LoDADA clusters transitions by next states to estimate local KL divergence via cluster-level domain classifiers, adaptively filtering and reweighting source data while regularizing policy learning toward the target behavior support.

LOCU: Löwdin-Orthogonalized Constraint Updates for Multi-Constraint Policy Optimization

To tackle ill-conditioned coupling and mutual cancellation among constraint gradients in multi-constraint CMDPs, LOCU applies symmetric Löwdin orthogonalization to natural gradients under the Fisher–Rao metric, mapping the trust region into a Euclidean ball and solving a reduced active-set problem for numerically stable updates and feasibility recovery.

Loop-Free Inverse Reinforcement Learning via Sequential Value Recovery with Q-Score Matching

LFIRL converts local action scores from a frozen diffusion policy into soft-Q gradient supervision, then sequentially fits values, calibrates state offsets, and regresses rewards, removing alternating reward–policy optimization and typically running about 2–3 times faster than the fastest baseline when diffusion pretraining is included, without leading every reward-quality metric.

Low-Rank Adaptation for Critic Learning in Off-Policy Reinforcement Learning

To combat severe critic overfitting and optimization instability caused by bootstrapped targets and non-stationary replay distributions in large off-policy critics, this paper proposes Low-Rank Adaptation (LoRA) as a training-from-scratch structural regularizer—freezing randomly initialized dense base matrices and optimizing only low-rank adapters—substantially reducing critic loss and boosting policy performance across diverse algorithms and architectures.

Low-Rank Adaptation for Critic Learning in Off-Policy Reinforcement Learning

To combat severe critic overfitting and optimization instability caused by bootstrapped targets and non-stationary replay distributions in large off-policy critics, this paper proposes Low-Rank Adaptation (LoRA) as a training-from-scratch structural regularizer—freezing randomly initialized dense base matrices and optimizing only low-rank adapters—substantially reducing critic loss and boosting policy performance across diverse algorithms and architectures.

Managing Self-Learning Experts under Per-Round Budget Constraints

To tackle sequential decision-making over multiple adaptive learning experts under a per-round training budget, this paper proposes M-LCB—a meta-algorithm that decouples forward-looking training allocation (optimistic via Lower Confidence Bounds) from deployment (conservative via Upper Confidence Bounds), achieving the minimax-optimal regret bound \(\tilde{O}(\sqrt{KT/M} + (K/M)^{1-\alpha}T^{\alpha})\).

MHWA: Multi-Timescale Hierarchical World-Action Model

MHWA addresses compounding rollout drift in autoregressive world models by coupling a strided, gradient-isolated High-Level Manager with a low-level sparsely gated Mixture-of-Experts Worker, achieving performance on par with the 150M-parameter JOWA baseline while cutting training FLOPs per sequence by 70.88%.

Mirror Descent Actor-Critic Methods for Entropy Regularised MDPs in General State and Action Spaces: Stability and Convergence

For entropy-regularized Markov Decision Processes on general Polish state and action spaces, this paper establishes the first theoretical stability and global convergence guarantees for discrete-time single-loop and multi-loop policy mirror descent actor-critic algorithms.

Mitigating Retaliatory Algorithmic Collusion in Repeated Games

The paper establishes a theoretical connection between algorithmic collusion in repeated games and Abreu's Simple Penal Codes (SPCs), proves that non-trivial retaliatory punishment induces a quantifiable strictly positive lower bound on total variation (TV) distance between conditional action distributions, and introduces CURB—a reward shaping and belief injection framework that provably eliminates collusive SPC fixed points and drives the collusion index to near-zero in both Bertrand and Cournot oligopolies.

Model-Based Bootstrap of Controlled Markov Chains

The paper develops a model-based bootstrap for finite controlled Markov chains under unknown, possibly nonstationary or history-dependent behavior policies, proves distributional consistency for transition kernels across single-chain and episodic regimes, and establishes valid confidence intervals for offline policy evaluation and optimal policy recovery via Hadamard differentiability of Bellman operators.

Model-Based Online Decision Making via Generative Trajectory Planning

Generative Trajectory Planning (GTP) introduces a model-based online decision-making framework that leverages a diffusion model as an action-sequence proposal distribution refined via dynamics-ensemble beam search, resolving the tension between inference efficiency and long-horizon reasoning.

Modeling Quantum Neural Network Gradient With Reinforcement Learning

RLQ-Grad uses a classical PPO policy with spectral normalization to generate surrogate update signals from a quantum neural network's training state, improving validation accuracy and reducing additional differentiation overhead in the reported classification simulations without proving recovery of true gradients or universal avoidance of barren plateaus.

Modelling Opinion Dynamics at Scale with Deep MARL

Introduces OpiniMARL, the first end-to-end GPU-accelerated multi-agent reinforcement learning framework for opinion dynamics that scales to 1,000 agents, extends other-play to general-sum games to eliminate arbitrary conventions, and demonstrates that evolutionary human conformity aids consensus in small hunter-gatherer bands but drives strategic dishonesty and misinformation across dense social media topologies.

Monotone Inclusion Approach to Weakly Monotone Discrete-Time Finite-Horizon Mean-Field Games

Addressing the long-standing limitation where computing mean-field equilibria (MFE) in discrete-time weakly monotone finite-horizon mean-field games requires artificial regularization or contractivity assumptions, this paper reformulates MFE computation as a constrained monotone inclusion problem over the occupation measure polytope, establishing the first fully tractable, non-regularized first-order algorithm achieving an \(\mathcal{O}(1/\sqrt{T})\) last-iterate convergence rate with a matching theoretical lower bound.

Multi-Objective Reinforcement Learning Using Routed Ensembles and Trajectory Attribution

To tackle sharp policy variation and negative transfer across competing preference weights in multi-objective RL, this paper introduces a \(\chi^2\)-divergence objective conflict coefficient to prove near-tight online regret bounds, and proposes routed ensemble policy learning alongside policy-gradient kernel surrogate trajectory attribution to overcome exploration bottlenecks and guide downstream data selection.

Multi-Agent Coordination via Support-Preserving Distillation

MoSDOT addresses multi-agent coordination mode collapse caused by independent noise pairing in flow-based teachers by partitioning source noise into capacity-matched Laguerre cells via semi-discrete optimal transport, eliminating spurious off-support fan mass and preserving valid coordination modes under decentralized distillation.

Multi-Objective Reinforcement Learning Using Routed Ensembles and Trajectory Attribution

To tackle sharp policy variation and negative transfer across competing preference weights in multi-objective RL, this paper introduces a \(\chi^2\)-divergence objective conflict coefficient to prove near-tight online regret bounds, and proposes routed ensemble policy learning alongside policy-gradient kernel surrogate trajectory attribution to overcome exploration bottlenecks and guide downstream data selection.

Multi-Step Likelihood-Ratio Correction for Reinforcement Learning with Verifiable Rewards

Addressing the structural truncation bias in standard PPO/GRPO surrogate objectives that ignore future likelihood ratios in reinforcement learning with verifiable rewards (RLVR), this paper proposes N-Step Forward-Trace Policy Optimization (NFPO), establishing a continuous bridge between local surrogates and exact policy gradients to substantially boost mathematical reasoning performance under controlled variance.

NashDreamer: Model-Based Reinforcement Learning for Zero-Sum Imperfect-Information Games

NashDreamer is the first principled model-based reinforcement learning framework for two-player zero-sum imperfect-information games that pairs a centralized multi-agent recurrent state-space model (MARSSM) with game-theoretic policy gradients (e.g., RNaD), resolving policy-physics entanglement while dramatically boosting early sample efficiency.

Noisy Data is Destructive to Reinforcement Learning with Verifiable Rewards

Challenging recent claims that reinforcement learning with verifiable rewards (RLVR) is inherently robust to annotation noise, this paper shows that prior "100% noisy" datasets suffered from severe clean-data contamination (at least 16.4%); after rectifying the data via a multi-stage re-verification pipeline, truly noisy data proves destructive to RLVR, existing algorithmic mitigations fail to compensate, and real-world human annotation errors induce 5–12% performance degradation.

Non-Asymptotic Best Policy Identification Guarantees in Online Reinforcement Learning

Provides the first non-asymptotic sample complexity upper bounds for Navigate-and-Stop (NaS) pure exploration in online tabular MDPs under navigation constraints, explicitly dissecting how MDP connectivity, mixing dynamics, and optimal allocation curvature govern finite-confidence exploration costs.

Not all uncertainty is alike: volatility, stochasticity, and exploration

This paper formally establishes that latent state drift (volatility) and observation noise (stochasticity) exert opposite monotonic effects on optimal exploration, and derives CAUSE, the first closed-form index policy for restless Gaussian state-space bandits via control-as-inference.

Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR

Addressing the critical flaw where static rubric rewards conflate final-answer human importance with current optimization utility in RLVR, this paper presents POW3R—a policy-aware rubric reward framework that dynamically reallocates within-category weights based on rollout contrast, substantially boosting mean rubric rewards, strict completion rates, and training sample efficiency under GRPO.

Offline Constrained Reinforcement Learning under Partial Data Coverage

Addressing the issues of intermediate policy evaluation under partial coverage and reliance on the unknown data-generating distribution in offline constrained RL, this paper proposes PDOCRL, an oracle-efficient primal-dual algorithm using decomposed linear programming and explicit policy reparameterization that achieves near-optimal and near-feasible \(\widetilde{\mathcal{O}}(\epsilon^{-2})\) sample guarantees.

Offline–Online Reinforcement Learning for Linear Mixture MDPs

This paper investigates offline–online reinforcement learning in linear mixture Markov decision processes subject to environment shift and uncontrollable offline behavior policies, introducing the O-O UCRL-VTR algorithm that adaptively guarantees sub-\(\sqrt{K}\) regret when offline data is informative while provably safeguarding against bias to match purely online minimax performance otherwise, alongside nearly matching regret lower bounds.

On Reparameterizing the Score Function

This paper introduces the History-Reparameterized (HR) policy gradient framework, demonstrating that treating the autoregressive conditioning prefix as differentiable with continuous embedding-level relaxation and combining it with a closed-form centered control variate (OHR) achieves substantial gradient variance reduction without requiring differentiable rewards.

On the Complexity of Discounted Robust MDPs with \(L_p\) Uncertainty Sets

This paper establishes the complete computational complexity landscape for discounted robust MDPs (RMDPs) and robust Markov chains (RMCs) with \(L_p\) uncertainty sets: proving that RMDP policy iteration strongly-polynomially reduces to RMC oracles for arbitrary compact uncertainty sets, establishing \(\mathcal{O}(n^3 \log n \cdot L)\) strongly polynomial convergence bounds for RMC policy iteration under \(L_1\) and \(L_\infty\) sets, and proving \(p\text{-ROOT-SUM}\)-hardness alongside an \(\exists\mathbb{R}\) upper bound for integer \(p > 1\).

On the Divergence of Differential Temporal Difference Learning without Local Clocks

This paper resolves an open problem in average-reward reinforcement learning by proving that differential temporal difference (differential TD) learning—which converges unconditionally under a theoretical local clock—can diverge under the standard global clock used in practice, thereby invalidating the optimistic correspondence seen in discounted RL.

On the Divergence of Differential Temporal Difference Learning without Local Clocks

Addressing the open question of whether differential temporal difference learning in average-reward reinforcement learning converges with a standard global-clock learning rate, this paper establishes an exact spectral characterization and constructs a rigorous counterexample proving that global-clock differential TD can diverge.

One More Time: Revisiting Neural Quantum States from a Reinforcement Learning Perspective

This paper bridges variational energy minimization in neural quantum states (NQS) with advantage policy-gradient reinforcement learning, proposing Proximal Wavefunction Optimization (PWO) with decoupled amplitude and phase trust-region clipping to enable multi-epoch sample reuse and stable optimization up to 1.5-billion-parameter models.

Online Finetuning Decision Transformers with Pure RL Gradients

By identifying hindsight return relabeling as the core impediment to importance sampling, this paper introduces GRPO-DT—integrating sub-trajectory optimization, sequence-level importance ratios, and active sampling—to achieve the first stable and effective online finetuning of Decision Transformers using pure reinforcement learning gradients.

ORCAID: Oblique Rule-Based Continuous-Action Interpretation for Deep RL Policies

Addressing the opacity of deep reinforcement learning policies in continuous-action and mixed-state environments, Orcaid introduces an oblique decision tree distillation framework that partitions the state space using hyperplanes and fits local linear action models, achieving high fidelity and actionable policy improvements with remarkably few parameters.

Outbidding and Outbluffing Elite Humans: Mastering Liar’s Poker via Self-Play and Reinforcement Learning

Solly is the first AI agent to reach elite human-level play in reduced-format Liar's Poker, extending Regularized Nash Dynamics (R-NaD) to multi-player imperfect-information settings via self-play without requiring test-time search.

Performance-Driven Policy Optimization for Speculative Decoding with Adaptive Windowing

Addressing the mismatch between token-level supervised imitation and prefix-sensitive verification in speculative decoding, this paper introduces PPOW, a window-level reinforcement learning framework that leverages cost-aware speedup reward, distribution-based proximity reward, and confidence-weighted adaptive hard-window mining to boost acceptance length and wall-clock speedup.

Persistent-Transient Policy Evaluation for Markov Chains via Minimal Peripheral Quotients

For fixed-policy evaluation in reducible and periodic finite Markov chains, this paper introduces an exact persistent-transient decomposition via the minimal contractive peripheral quotient and provides a provably convergent sample-based estimator that cleanly decouples non-decaying cyclic oscillations from transient path costs.

Pessimistic Latent Task-Aware Optimization for Robust Offline Meta-Reinforcement Learning

Addressing the structural mismatch in context-based offline meta-RL where policies never see off-support latents during training, PLATO perturbs latents outward from the training centroid to synthesize counterfactual tasks, rolls out short imagined trajectories with a decoder ensemble, and optimizes the policy under epistemic pessimism weighting while strictly confining the critic to real transitions.

Policy-DRIFT: Dynamic Reward-Informed Flow Trajectory Steering

Addressing the vulnerability of deep reinforcement learning to proxy reward misspecification and over-actuation in fluid control, Policy-DRIFT relocates reward steering from policy gradients to generative conditional flow matching (CFM) inference using Terminal Reward Guidance (TRG), allowing a lightweight DRL policy to achieve 48.95% drag reduction on \(Re_\tau=180\) turbulent channel flow via full-field RMSE tracking while reducing actuation energy by approximately \(37\times\).

Policy Gradient Algorithms in Average-Reward Multichain MDPs

This paper establishes the first policy gradient theorem for general average-reward multichain MDPs by decomposing state visitation into recurrent and transient measures, and proves non-asymptotic convergence rates and sample complexity bounds for an \(\alpha\)-clipped policy mirror ascent algorithm.

Policy Optimization in Hybrid Discrete-Continuous Action Spaces via Mixed Gradients

Addressing the credit assignment crisis in hybrid discrete-continuous action spaces, this paper introduces Hybrid Policy Optimization (HPO), an unbiased mixed gradient estimator blending pathwise gradients for continuous controls and score-function gradients for discrete mode selections that dramatically scales to high dimensions.

PPO in the Fisher-Rao geometry

Bridging the theoretical gap of heuristic clipping in standard PPO, this work derives a tighter surrogate lower bound on the value function via Fisher-Rao geometry on squared densities, establishing monotonic improvement and dimension-free sub-linear convergence that directly translates into a drop-in replacement for the PPO policy loss.

Practical Adversarial Attacks on Stochastic Bandits via Fake Data Injection

This paper introduces a realistic "Fake Data Injection" threat model for stochastic bandits under reward magnitude and temporal rate limits, establishing an exponential suppression principle and designing attack strategies that manipulate the learner to pull a target arm with sublinear attack cost.

Predictive but Not Plannable: RC-aux for Latent World Models

Addressing the spatiotemporal mismatch where reconstruction-free latent world models achieve accurate short-step prediction but suffer from planning failure along geometrically close yet unreachable shortcuts, this paper proposes RC-aux—a lightweight auxiliary objective that aligns open-loop rollout dynamics and learns budget-conditioned reachability with temporal hard negatives, boosting pixel-based planning with only 3.74% parameter overhead.

Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimization

D3PO tackles destructive advantage cancellation and representational mode collapse in preference-conditioned multi-objective reinforcement learning by applying PPO clipping independently to per-objective advantages before preference weighting (Late-Stage Weighting) alongside a scaled diversity regularizer, discovering superior continuous Pareto fronts using a single deployable policy.

Primal-Dual Representation Learning for Low-Rank Constrained MDPs

For continuous-state low-rank constrained MDPs with unknown transition representations, this paper establishes the first theoretically grounded primal-dual representation learning framework, introducing tertiary auxiliary probing and merged-covariance uncertainty bonuses to achieve an optimal \(\widetilde{O}(\sqrt{K})\) soft-violation rate and certified sublinear hard-violation control.

Prospective Hindsight: Self-Calibrating Reinforcement Learning via Prediction–Reality Gaps

To eliminate the hindsight trap where retrospective feedback leaves an agent's action-time belief invisible to policy gradients, this paper proposes Prospective Hindsight (PH) which dynamically amplifies surprising prediction–reality mismatches to achieve task improvement and self-calibration without explicit auxiliary calibration losses.

Provably Efficient Regularized Online RLHF with Generalized Bilinear Preferences

This paper investigates regularized max-regret minimization in online RLHF under general intransitive preferences and bandit feedback, establishing that under the Generalized Bilinear Preference Model (GBPM), any strongly convex regularizer induces a quadratic bound on the instantaneous dual gap, achieving generic polylogarithmic and dimension-free regret guarantees that break free from KL-specific closed-form analyses.

Q-MMR: Off-Policy Evaluation via Recursive Reweighting and Moment Matching

Introduces Q-MMR, a theoretical framework for off-policy evaluation that learns non-parametric scalar weights per data point via top-down recursive moment matching and the principle of least second moment, establishing dimension-free finite-sample error bounds under only \(Q^\pi\) realizability.

Quantile-Coupled Flow Matching for Distributional Reinforcement Learning

FlowIQN eliminates the metric mismatch between conditional flow matching and the \(W_p\) distributional Bellman contraction by enforcing sorted quantile couplings in 1D return space, providing the first Wasserstein-aligned projection guarantee for flow-based critics alongside shortcut-accelerated inference.

Quantile Geometry Regularization for Distributional Reinforcement Learning

To resolve the distribution degeneration issue in quantile-based DistRL caused by non-Bellman-closed statistics, this paper proposes Robust Quantile-based Implicit Quantile Networks (RQIQN), which regularizes return distribution geometry via a closed-form, fraction-dependent Wasserstein-robust correction that enlarges quantile spread without altering risk-neutral value objectives.

Quotient-Categorical Representations for Bellman-Compatible Average-Reward Distributional Reinforcement Learning

Addressing the ill-posedness of distributional bias estimation on the real line caused by additive-constant non-uniqueness in average-reward RL, this paper introduces a quotient-categorical representation modulo common translations along with a projected operator, proving non-expansiveness in a coordinate Cramér metric, finite-iteration stochastic approximation bounds, and online gain-coupled convergence.

Regret Minimization in Single-Dimensional Contract-Design with Binary Actions

Focusing on online single-dimensional contract design with hidden actions and private agent effort costs, this paper reduces the multi-dimensional contract space to a scalar incentive gap, develops an LP-adaptive non-uniform discretization, and designs a stochastic binary search with contract robustification, achieving tight regret guarantees of \(\Theta(T^{2/3})\) for adversarial types and \(\widetilde{\Theta}(\sqrt{T})\) for a fixed hidden type, both completely independent of the outcome space dimension.

Reinforcement Learning for Exponential Utility: Algorithms and Convergence in Discounted MDPs

Addressing the long-standing absence of principled value-based model-free RL algorithms for exponential-utility optimization in discounted Markov decision processes, this paper derives two dual Q-value operators contracting under \(L_\infty\) and Thompson metrics respectively, proves the optimality of the greedy stationary policy, and establishes almost-sure and finite-time convergence guarantees for both two-timescale and one-timescale Q-learning algorithms.

Reinforcement Learning from Rich Feedback with Distributional DAgger

Addressing the lack of credit assignment in RLVR and the failure of prior self-distillation objectives (reverse KL, Jensen-Shannon) to guarantee monotonic policy improvement, this paper proposes DistIL, a distributional DAgger algorithm with forward cross-entropy and future-aware sequence-level credit assignment, achieving monotonic improvement guarantees, sublinear regret, and significant Pass@N improvements.

Replay-buffer engineering for noise-aware quantum circuit optimization

The paper shifts quantum circuit optimization from changing the agent to reusing experience more effectively: annealed reliability-aware replay improves sample efficiency, blockwise evaluation reduces quantum–classical calls, and buffer-only transfer accelerates noise adaptation, although accuracy, gate-count, and convergence advantages differ across tasks.

Representation Learning Enables Scalable Multitask Deep Reinforcement Learning

Through extensive evaluations across the 200-task MMBench multitask continuous control benchmark, this paper demonstrates that model-based planning and latent trajectory rollouts are not essential for scalable multitask RL; instead, a simple model-free actor-critic algorithm (MR.Q) augmented solely with auxiliary forward predictive objectives matches or outperforms state-of-the-art world models (Newt) while delivering ~2.6x faster wall-clock training and superior few-shot transfer.

Reward-Conditioned Reinforcement Learning

Reward-Conditioned Reinforcement Learning (RCRL) collects environment transitions solely under a single nominal reward objective, recomputes counterfactual rewards from replay data across sampled reward parameterizations, and conditions the actor-critic on these parameterizations, substantially improving sample efficiency, transfer finetuning, and zero-shot behavioral steerability at zero additional interaction cost.

Reward-free Pretraining for Reinforcement Learning via Occupancy Coverage Maximization

Addressing the challenge that sparse downstream rewards cannot be evaluated for offline relabeling, ROVER formalizes reward-free pretraining as minimizing the self-similarity of the discounted state occupancy measure in an RKHS, leveraging a resolvent world model and a virtual sink-state augmentation to derive closed-form policy mirror descent updates that establish stable, non-collapsing state-space coverage.

Reward Inflation: A Healthy Stimulus for Reinforcement Learning

Proposes Reward Inflation, a gradual exponential scaling of rewards over the course of training, paired with an adaptive controller Fed, which theoretically induces implicit TD recency weighting and suppresses dormant neurons to preserve network plasticity, yielding consistent improvements on ALE and MuJoCo benchmarks.

RL-IF: Tracking Non-local Training Data Influence for Online Reinforcement Learning

Addressing the myopic limitation of local data attribution in online RL, this paper formalizes the multi-round trajectory-level leave-one-out counterfactual RL-LOO, derives the first-order estimator RL-IF that disentangles optimization and policy-induced sampling effects, and develops RINSE, an efficient optimization-only filtering algorithm that markedly improves sample efficiency and asymptotic return.

Rollout Pass-Rate Control: Steering Binary-Reward RL Toward Its Most Informative Regime

Addressing the severe contrastive signal deficit caused by heavily skewed rollout pass rates in binary-reward RL, this paper introduces Prefix Sampling to steer rollouts toward the maximally informative 50% pass-rate regime through self-generated bidirectional prefix replay and closed-loop adaptive control, delivering up to 2.01× end-to-end wall-clock training speedup on agentic SWE-bench and mathematical reasoning benchmarks.

Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement Learning

Safe Score Matching (SSM) uses learned Hamilton–Jacobi safety values to switch a diffusion policy between reward and recovery training targets, combining multimodal action modeling with low violation costs in online safe reinforcement learning, although its hard-constrained formulation does not establish a strict safety guarantee for the learned policy.

SAIR: Cost-Efficient Multi-Stage ML Pipeline Autoscaling via In-Context Reinforcement Learning

SAIR formulates multi-stage ML inference pipeline autoscaling as a contextual bandit solved by an LLM acting as an in-context reinforcement learning (ICRL) agent, leveraging surprisal-based experience retrieval, Pareto reward shaping with a provable 0.2 margin, and user-space CUDA rate control to reduce P99 latency by up to 50% and cut effective resource cost by up to 97% without offline training.

SALT: When More Rollouts Don’t Help in Group-Based Policy Optimization and How to Make Them Matter

Addressing rollout scaling saturation in GRPO-style RLVR caused by group centering and signed low-rank gradient cancellation, this paper introduces SALT, a lightweight plug-in that decomposes advantages into shared and one-sided positive residual channels via Gram eigenspectrum analysis, adaptively recovering effective updates.

Sampling-Based Safe Reinforcement Learning

This paper introduces Sampling-Based Safe Reinforcement Learning (SBSRL), a model-based framework that enforces tightened safety constraints jointly across a finite set of sampled dynamics to ensure high-probability safety throughout training, while driving active exploration via an epistemic uncertainty lower-bound constraint without hyperparameter-tuned intrinsic bonuses.

Scalable Maximum Entropy Reinforcement Learning for Diffusion Policies via Adjoint Matching

By reformulating maximum entropy reinforcement learning as a stochastic optimal control problem, this paper introduces Adjoint Matching Diffusion Policy (AMDP), a simulation-free policy optimization framework that completely eliminates chain backpropagation memory bottlenecks while achieving wall-clock training throughput comparable to Gaussian policies.

Selective Off-Policy Reference Tuning with Plan Guidance

Addressing the zero-gradient failure mode of Group Relative Policy Optimization (GRPO) on all-wrong reasoning rollouts, SORT introduces an auxiliary repair update that extracts a reference-derived reasoning plan, evaluates plan-conditioned token salience, and selectively weights critical reasoning tokens via a geometric mean without changing the original rollout generation.

Self-Distilled RLVR

Addressing the fundamental flaw where on-policy self-distillation (OPSD) triggers privileged information leakage and collapse due to information asymmetry, this paper proposes RLSD, decoupling update directions anchored by environmental rewards from update magnitudes modulated by token-level self-distilled Bayesian evidence ratios.

Self-Improving World Modelling with Latent Actions

SWIRL introduces a self-improving world modelling framework that treats actions as latent variables and alternates between Forward World Modelling (FWM) and Inverse Dynamics Modelling (IDM) under GRPO, enabling LLMs and VLMs to master environment dynamics purely from action-unlabelled state sequences.

Sequential Solution Concepts in Cooperative Games with Generalized Characteristic Functions

Focusing on temporal cooperative games (TCGs) where coalition worth explicitly depends on agent arrival order, this paper introduces three fundamental sequential axioms—Incentive for Optimal Arrival (I4OA), Online Individual Rationality (OIR), and Sequential Efficiency (SE)—characterizes the exact class of mechanisms satisfying them (SeqShare), and proves that this class is structurally disjoint from Shapley-inspired analogs even for convex and simple games.

Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards

Addressing the fundamental challenge of nonlinear and history-dependent rewards in contextual bandits, this paper leverages the signature transform from rough path theory to linearize continuous path functionals, establishing the DisSigUCB algorithm with sublinear regret guarantees.

SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning

SkillRL introduces a skill-augmented reinforcement learning framework that distills diverse interaction trajectories into a compact, hierarchical skill library and recursively co-evolves the skills with the agent policy during GRPO training, significantly outperforming prior memory-based and RL baselines on ALFWorld and WebShop.

SLOPE: Optimistic Potential Landscape Shaping for Model-based Reinforcement Learning

Addressing the gradient starvation bottleneck where sampling-based planners fail under sparse binary rewards in model-based reinforcement learning (MBRL), SLOPE replaces flat scalar reward regression with a dense, self-evolving potential landscape anchored to the target value network, utilizing quantile-weighted asymmetric cross-entropy loss to amplify rare success signals while strictly preserving optimal policy invariance.

Soft Forward-Backward Representations for Zero-shot Reinforcement Learning with General Utilities

This paper introduces Soft FB for zero-shot reinforcement learning with general utilities, leveraging maximum entropy regularization to induce stochastic policies and searching over compact policy embeddings to surpass the limitations of linear reward representations.

Sparse Planning in Visual World Models via Cost Gradients

CostGrad proposes a training-free, goal-conditioned spatial token selector that ranks tokens by the gradient norm of the planning cost, delivering a 2.6x to 5x wall-clock speedup for CEM planning in visual world models at 50% sparsity while matching or exceeding full-token success rates.

Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitation Learning

To resolve the fundamental failure mode where naive data aggregation over conflicting multi-objective demonstrations collapses into dominated policies, this paper introduces Multi-Output Augmented Behavioral Cloning (MA-BC), which isolates divergent states while pooling shared states, theoretically achieving minimax-optimal sample efficiency on the Pareto front.

Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVR

To tackle the uniform constraint dilemma in conventional RLVR and the sequential credit assignment disruption caused by token isolation, Archer introduces an entropy-aware dual-anchor framework that preserves joint autoregressive optimization while dynamically modulating local proximal clipping boundaries and global reference-policy KL penalties based on intra-response entropy quantiles.

Stabilizing Policy Optimization via Logits Convexity

Addressing the pervasive gradient explosion and collapse issues of policy gradient RL in LLMs, this paper reveals that the stability gap between SFT and RL originates from the convexity of the loss with respect to model logits, and proposes Logits Convex Optimization (LCO) to match the closed-form optimal policy targets under convex loss landscapes, eliminating gradient volatility and accelerating convergence.

Stable-WorldModel: A Platform for Reproducible World Modeling Research and Evaluation

Addressing core bottlenecks in world modeling research—fragmented codebases, multimodal video I/O starvation, and non-standardized generalization testing—this work presents stable-worldmodel, an open-source reproducible platform providing a high-performance Lance-based data layer, plug-and-play planning solvers, and controllable factor-of-variation evaluation suites.

Stage Light is Sequence\(^2\): Multi-Light Control via Imitation Learning

SeqLight establishes a hierarchical framework for music-conditioned multi-light control, generating frame-level global hue-value distributions via a customized Skip-BART and subsequently decomposing them into individual light controls through a three-phase imitation learning pipeline powered by HER expert relabeling and GRPO.

STAPO: Stabilizing Reinforcement Learning for LLMs by Silencing Rare Spurious Tokens

Addressing late-stage entropy instability and performance collapse in LLM reinforcement learning, STAPO demonstrates that a tiny fraction (~0.01%) of rare, non-causal "spurious tokens" receive disproportionately amplified gradients under sequence-level rewards, and introduces a dynamic Silencing Spurious Tokens (S2T) mechanism that masks low-probability, low-entropy tokens to stabilize policy entropy and substantially boost mathematical reasoning across Qwen 1.7B–14B models.

StateTree: Enhancing Long-Term Dialogue Reasoning via Reinforcement Learning

StateTree reformulates long-term multi-session dialogue reasoning as a tree-structured path-tracing auxiliary task, training LLMs via a four-stage curriculum GRPO with a combined exact-match and LLM judge reward on 10K-token contexts, and generalizing to 128K-token dialogues to outperform substantially larger models.

Stochastic Matching Bandits with Rare Optimization Updates

Addressing the NP-hard combinatorial optimization challenge in stochastic bipartite matching with multinomial logit (MNL) choices, this paper proposes batched elimination algorithms (B-SMB and curvature-adaptive B-SMB+) that invoke expensive optimization updates only \(\Theta(\log\log T)\) times while securing near-optimal \(\widetilde{\mathcal{O}}(\sqrt{T})\) regret.

Strengthening LLMs for Tabular Prediction with Structural Priors

To overcome structural gap and severe reward sparsity when applying LLMs to tabular prediction, this paper introduces Permutation Relative Policy Optimization (PRPO), which exploits label-preserving column-permutation invariance and two-level advantage estimation to densify learning signals, enabling an 8B open model to match or surpass specialized tabular models like XGBoost and TabPFN across zero-shot and supervised benchmarks.

Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning

Proposes Q-Guided Flow (QGF), an RL algorithm that guides flow policy denoising at test time using low-variance critic gradients on single-step Euler-approximated clean actions with the Jacobian truncated to the identity, completely sidestepping the training instability of actor-critic co-optimization and backpropagation through time.

TGRL: Temperature-Grouped Reinforcement Learning for Efficient Exploration in LLMs

Addressing the exploration bottleneck in reinforcement learning with verifiable rewards (RLVR) under fixed rollout budgets, TGRL partitions rollouts into low-temperature reference and high-temperature exploration subsets, quantifies prompt-level exploration gains via reward gaps, and applies asymmetric token-level credit allocation using JS divergence on identical logits—accelerating convergence by up to 36% across math, coding, and agent tasks without expanding sample budgets.

The Dynamics of Policy Gradient in Social Dilemmas with Partner Selection

This paper develops an analytical continuous-time dynamical framework for policy gradient learning in social dilemmas with partner selection, proving via mean-field theory and 2D Wiener processes that population variance is a necessary prerequisite for cooperation to emerge, and characterizing the limiting distribution with the Fokker-Planck equation.

The Laplacian Keyboard: Beyond the Linear Span

To overcome the expressivity ceiling where zero-shot unsupervised RL is fundamentally bounded by the linear span of state representations, this paper introduces the Laplacian Keyboard (LK), a hierarchical framework that extracts a task-agnostic behavior library from graph Laplacian eigenvectors and dynamically stitches them across time via a learned meta-policy to achieve near-optimal control beyond the linear basis.

The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models

This paper reveals that Reinforcement Learning with Verifiable Rewards (RLVR) shrinks the reasoning coverage (Pass@k) of language models due to negative interference and on-policy winner-take-all dynamics, and introduces SELF, a lightweight curation algorithm that targets low-likelihood problems under Forward KL regularization to preserve exploration diversity.

The Reciprocity Gradient

To overcome the influence attribution bottleneck caused by delayed and diffuse feedback in reputation-mediated multi-agent games, this paper introduces the "reciprocity gradient"—an analytical backpropagation framework that traces reward gradients directly through unrolled opponent assessment policies and reputation dynamics, coupled with private differentiable opponent modeling to learn context-dependent social norms without reward shaping.

The Sampling Complexity of Condorcet Winner Identification in Dueling Bandits

This paper investigates best-arm identification in stochastic dueling bandits under the sole assumption that a Condorcet winner exists, proposing novel elimination-based algorithms that exploit the full gap matrix and establishing matching instance-dependent high-probability upper and lower bounds on sampling complexity.

Tool Verification for Test-Time Reinforcement Learning

Addressing the vulnerability where majority voting in test-time reinforcement learning (TTRL) collapses into frequent but incorrect answers due to systematic errors, T3RL introduces an external verifier and sandbox code execution to evaluate sampled rollouts, employing verification-weighted voting to establish grounded pseudo-labels and substantially boost reasoning performance with minimal compute overhead.

Toward Online Robust Zero-Sum Markov Games with Function Approximation

This paper introduces an optimistic dual robust Bellman (ODRB) framework combining dual operator fitting, backward-validated confidence sets, and exploiter-mix data collection, establishing the first sublinear regret guarantees for online robust zero-sum Markov games with function approximation that separate exploration, operator learning, stage-game solving, and misspecification errors.

Towards Differentially Private Reinforcement Learning with General Function Approximation

This paper establishes the first theoretical guarantees for differentially private online reinforcement learning with general function approximation, combining fixed-size batched policy updates with the exponential mechanism to achieve a sublinear regret bound of \(\widetilde{O}(K^{3/5})\) under coverability, matching the state of the art for model-free linear MDPs.

Towards Instance-Dependent Regret Optimality in Episodic MDPs with Posterior Sampling

Introduces \(\pi_0\)-PSRL, which repurposes posterior sampling as an implicit generalized likelihood ratio test to decide when exploration is necessary while following a fixed fully randomized reference policy \(\pi_0\) during exploration episodes, proving the first instance-dependent logarithmic regret bound for episodic posterior sampling matching the confusing alternative KL rates.

Training Generalizable Collaborative Agents via Strategic Risk Aversion

Addressing the core vulnerabilities of free-riding and brittle partner generalization in collaborative multi-agent reinforcement learning, this paper incorporates game-theoretic strategic risk aversion and risk-averse quantal response equilibria (RQE), proves that risk aversion both incentivizes collaboration and eliminates free-riding, and introduces Strategically Risk-Averse Policy Optimization (SRPO) for scalable multi-agent and LLM coordination.

TrajEvolve: Trajectory Evolution for Reinforcement Learning with Hindsight Credit Assignment

TrajEvolve tackles sparse success sampling and indiscriminate reward updates in long-horizon LLM agent RL by reformulating credit assignment into binary necessity classification via counterfactual execution, actively refining historical rollouts through trajectory evolution to surpass exploration barriers.

Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement

The paper proves that a single-layer, single-head linear self-attention transformer block can implement batch semi-gradient SARSA and actor-critic policy improvement updates via explicit parameter constructions, establishing the first local exponential convergence guarantee for ICRL gradient flow onto the optimal parameter manifold.

Trust Guided Decision Transformer

TGDT first filters trustworthy history suffixes using rolling next-state prediction errors on realized transitions, then ranks their candidate actions with a frozen IQL critic, improving return and persistent-error behavior in long-horizon navigation without guaranteeing closed-loop coverage or safety.

Trust Region Q Adjoint Matching

Addressing catastrophic performance collapse caused by exponential amplification of critic errors during off-policy RL fine-tuning of flow policies, Trust Region Q Adjoint Matching (TRQAM) internalizes an adaptive trust-region parameter \(\lambda\) inside the stochastic optimal control sampling dynamics and updates it via projected dual descent to enforce a strict path-space KL budget.

Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximization

Unifies goal-conditioned reinforcement learning (GCRL) and mutual information skill learning (MISL) through the lens of control maximization, proving that canonical GCRL formulations induce incompatible optimal policies and establishing tight information-theoretic bounds between downstream goal sensitivity and pretraining behavioral diversity.

Utility-Constrained Policy Optimization

To overcome the risk-neutral vulnerabilities and fixed-budget rigidity of standard Constrained MDPs, this paper introduces Utility-Constrained Policies (UCP), an actor-critic framework that integrates undiscounted stock augmentation with quantile regression critics to curb catastrophic tail-risk violations and enable retrain-free generalization across arbitrary safety budgets.

Value-Rectified Distillation for Flow-based Offline Reinforcement Learning

To tackle the latent fusion and Out-of-Distribution (OOD) mode averaging catastrophe in flow-based offline reinforcement learning on multi-modal datasets, Value-Rectified Distillation (VRD) reformulates policy distillation as value-guided trajectory alignment, cleanly isolating optimal actions via statistical quantile filtering (VRD-Q) or geometric latent reorganization (VRD-R).

Variance-Averse n-Step Offline Reinforcement Learning for Sparse Long-Horizon Environments

Addressing the vulnerability where generative actors reproduce high-variance, unreliable action modes from heterogeneous offline datasets, VAN-Flow couples a categorical distributional critic with a smooth variance-averse expectation operator to guide a flow-matching actor via rejection sampling and Q-guidance, substantially enhancing policy reliability in sparse, long-horizon tasks.

Verifying Neural Networks with Reinforcement Learning

Rsb uses graph-structured observations and actor-critic learning to reweight Fsb neuron-branching scores without replacing verification logic, increasing solved problems from 148 to 165 among 600 challenging test instances, although inconsistencies in rewards, feature masks, and some statistics need to be distinguished from the performance gains.

VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learning

Addressing the failure of model-based RL under unseen visual distractions caused by compounding multi-step rollout errors, VIGOR introduces asymmetric weak-to-strong augmentation, dynamics-level consistency, and encoder-level stabilization to achieve robust zero-shot visual generalization without test-time adaptation.

When Dynamics Shift, Robust Task Inference Wins: Offline Imitation Learning with Behavior Foundation Models Revisited

Addressing catastrophic policy collapse in Behavior Foundation Models (BFMs) under environment dynamics shift, this work introduces minimax distributionally robust optimization strictly at task inference time—achieving robust offline imitation learning in minutes using data from only a single nominal environment.

When Policies Cannot Be Retrained: A Unified Closed-Form View of Post-Training Steering in Offline Reinforcement Learning

Addressing the challenge where offline pre-trained actors cannot be retrained due to governance or retraining costs, this work establishes the exact algebraic equivalence between Product-of-Experts (PoE) composition and KL-regularized adaptation under diagonal-Gaussian policies, demonstrating that precision-weighted composition acts as a robust actor-anchored safety layer that degrades gracefully under degraded priors while delineating the actor-competence ceiling.

WTF?! Simulation-Free Reinforcement Learning with Wasserstein-Tilted Flow Maps

To overcome the reliance on slow diffusion surrogates and multi-step trajectory rollouts in generative flow reward alignment, WTF introduces an optimal transport regularizer built directly from the pre-trained drift and solves it via deterministic optimal control, utilizing flow maps for simulation-free and critic-free value gradient regression to reduce training compute by up to 280× while maintaining few-step generation quality.

Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor’s Internal States

Addressing the dilemma where GRPO relies on group sampling that sacrifices prompt diversity and PPO doubles training overhead via a separate critic, POISE trains a lightweight linear probe on the actor's forward-pass internal hidden states and token entropy to predict baseline values, while eliminating policy gradient bias via an independent cross-rollout construction, achieving lower gradient variance and superior multi-domain reasoning under a minimal budget of two rollouts per prompt.

Zeroth-Order Stackelberg Control in Combinatorial Congestion Games

Addressing the non-smoothness of upper-level objectives caused by abrupt shifts in discrete active strategy sets in combinatorial congestion games, this paper presents ZO-Stackelberg, a framework combining projection-free Frank-Wolfe with stratified sampling LMOs and outer two-point zeroth-order updates, converging to generalized Goldstein stationary points without differentiation through equilibria and achieving up to orders-of-magnitude speedups.

StarWM: Self-Supervised Trained Attention Routing for Robust World Models

StarWM uses self-supervised dynamics signals to determine spatial attention, then constrains reconstruction through stop-gradient barriers and a dual-stream decoder so that world models retain predictable entities under randomized video distractions; coherent video backgrounds still confuse default routing, while the reward-guided StarWM-R variant improves cross-condition performance.