ICML2025 ICML2025 accepted papers ICML2025 paper list AI paper notes top conference papers Image Generation Safety Model Compression Reinforcement Learning Optimization & Theory Computational Biology Multimodal VLM Pretraining

🧪 ICML2025 Accepted Papers¶

1060 ICML2025 paper notes covering Image Generation (94), Safety (78), Model Compression (74), Reinforcement Learning (70), Optimization & Theory (61), Computational Biology (48), Multimodal VLM (42), Pretraining (32) and other 51 areas. Each note has TL;DR, motivation, method, experiments, highlights, and limitations — 5-minute reads of core ideas.

💡 LLM Reasoning (19)¶

Self-Consistency Preference Optimization: Introduce the concept of self-consistency from inference into the training phase, construct preference pairs through a voting mechanism, and perform iterative training using a weighted DPO loss. This significantly improves the mathematical and logical reasoning capabilities of LLMs without requiring gold labels.
Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators: This paper proposes the JETTS benchmark to systematically evaluate the performance of LLM-judges as evaluators in test-time scaling scenarios (response reranking, step-level beam search, and critique-based refinement). The findings show that while judges are competitive with outcome reward models in reranking, they are significantly weaker than process reward models in beam search, and natural language critiques currently fail to effectively guide generator improvements.
Rethinking External Slow-Thinking: From Snowball Errors to Probability of Correct Reasoning: This work systematically analyzes the "snowball error" phenomenon in LLM reasoning from an information-theoretic perspective, establishes a theoretical link between snowball errors and the probability of correct reasoning, and demonstrates that external slow-thinking methods (e.g., BoN, MCTS) inherently mitigate error accumulation by scaling search width. Both theoretically and experimentally, it is proven that the effectiveness of these methods primarily depends on the total inference budget and the reliability of the reward function rather than the search framework itself.
DyCodeEval: Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination: Based on the concept of metamorphic testing, this work decomposes programming problems into complexity-related algorithmic abstractions and complexity-independent contextual descriptions. Through the collaboration of four LLM agents, it automatically generates semantically equivalent yet textually distinct variants of programming problems. This effectively mitigates data contamination and evaluates the true reasoning capabilities of Code LLMs, validating the effectiveness of the framework across 18 models.
Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models: Through causal, representation, and attention analyses, this paper identifies a three-stage emergent symbolic architecture supporting abstract reasoning across 13 open-source LLMs: symbolic abstraction heads transform input tokens into abstract variables, symbolic induction heads perform sequence induction at the abstract variable level, and retrieval heads retrieve the corresponding values based on predicted abstract variables for next-token prediction.
PENCIL: Long Thoughts with Short Memory: Proposed PENCIL (PENCIL ENables Context-efficient Inference and Learning), which introduces a functional call stack-inspired reduction rule during autoregressive generation to recursively clear completed intermediate reasoning steps, enabling LLMs to solve computationally hard problems requiring exponential context using only polynomial context length.
Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulness: This paper systematically analyzes the factors influencing the performance of CoT from the two dimensions of effectiveness and faithfulness. It finds that problem difficulty, information gain, and information flow are key factors affecting CoT effectiveness. The root cause of unfaithful CoT is that the model directly recalls the correct information from the question while bypassing the CoT during answer prediction. Based on this, the QUIRE method is proposed to improve both CoT effectiveness and faithfulness.
PCoT: Persuasion-Augmented Chain of Thought for Detecting Fake News and Social Media Disinformation: Proposes PCoT (Persuasion-Augmented Chain of Thought), which utilizes a two-stage reasoning framework—first requiring the LLM to identify persuasive strategies in the text, and then injecting the persuasion analysis results into the disinformation detection reasoning. In zero-shot settings, it achieves an average F1 improvement of approximately 15% across 5 LLMs and 5 datasets.
Putnam-AXIOM: A Functional & Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs: This work introduces Putnam-AXIOM, a benchmark comprising 522 university-level Putnam competition math problems and 100 programmatic functional variants, which reveals memorization reliance in LLM mathematical reasoning and introduces Teacher-Forced Accuracy (TFA) as an evaluation metric for reasoning quality beyond final answers.
Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration: This paper introduces Soft Reasoning, which injects Gaussian perturbations into the embedding space of the first generated token and utilizes Bayesian optimization to search for the optimal perturbation vector. This guides LLMs to explore better solution spaces during inference in a black-box manner without requiring access to model parameters or external verifiers. It outperforms baselines like temperature scaling and Best-of-N on mathematical reasoning tasks with extremely low computational overhead.

Browse all 19 LLM Reasoning papers →

🦾 LLM Agent (18)¶

Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction: Proposes Aguvis, the first fully pure vision-based cross-platform autonomous GUI Agent framework. By unifying visual observation space, standardizing action spaces, and utilizing an inner monologue mechanism, Aguvis achieves SOTA results on offline and online benchmarks without relying on closed-source models.
AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML: This paper proposes AutoML-Agent, a multi-agent LLM collaborative framework for full-pipeline AutoML. It expands the search space using a Retrieval-Augmented Planning strategy, decomposes tasks into parallel subtasks handled by specialized agents, and introduces a multi-stage verification mechanism to guarantee code generation quality, achieving higher automation success rates and model performance across 14 datasets in 7 task categories.
GuardAgent: Safeguard LLM Agents via Knowledge-Enabled Reasoning: GuardAgent is the first "Agent-safeguarding-Agent" framework that dynamically converts safety rules into executable guardrail code to verify if the actions of a target Agent violate safety policies. It achieves guardrail accuracies of over 98% and 83% on new benchmarks for medical access control and web safety control, respectively.
AdvAgent: Controllable Blackbox Red-teaming on Web Agents: This paper proposes AdvAgent, a reinforcement learning (DPO)-based blackbox red-teaming framework. It trains an adversarial prompter model to automatically generate invisible HTML adversarial prompts. When injected into web pages, these prompts mislead GPT-4V-driven Web Agents into executing attacker-specified target actions (e.g., changing buying Microsoft stock to buying NVIDIA stock). AdvAgent achieves a 97.5% attack success rate across 440 tasks and maintains over 88.8% effectiveness against existing defense methods.
ResearchTown: Simulator of Human Research Community: This paper proposes ResearchTown, a multi-agent framework based on agent-data graphs and TextGNN (text-space message passing), which models human scientific communities as heterogeneous graphs to unify the simulation of three core research activities: literature reading, paper writing, and peer review. A scalable and objective simulation quality evaluation is conducted via a node masking prediction task (ResearchBench).
From Debate to Equilibrium: Belief-Driven Multi-Agent LLM Reasoning via Bayesian Nash Equilibrium: This paper models multi-LLM coordination as an incomplete information game and proposes the ECON framework. It achieves implicit belief-driven multi-agent coordinated reasoning via Bayesian Nash Equilibrium (BNE) without explicit message passing while providing theoretical convergence guarantees, yielding an average improvement of 11.2% across six reasoning benchmarks.
KBQA-o1: Agentic Knowledge Base Question Answering with Monte Carlo Tree Search: KBQA-o1 is proposed, which combines a ReAct Agent with Monte Carlo Tree Search (MCTS) to perform knowledge base question answering through heuristic search driven by policy and reward models. Under low-resource settings, it improves the GrailQA F1 from 48.5% (GPT-3.5-turbo SOTA) to 78.5% using Llama-3.1-8B.
From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?: This paper introduces AR-Bench, a benchmark specifically designed to evaluate the active reasoning capabilities of LLMs. It features three task families: detective cases, situation puzzles, and guessing numbers. Experiments reveal that state-of-the-art models such as GPT-4o perform far worse than humans in scenarios where they must actively ask questions to retrieve missing information, exposing a massive gap between passive and active reasoning.
Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination: Proposes the Cross-Environment Cooperation (CEC) paradigm, which trains agents via self-play across a large number of procedurally generated, diverse environments (rather than increasing partner diversity). This enables agents to learn general cooperative norms, achieving zero-shot coordination with unseen partners in unseen environments.
MetaAgent: Automatically Constructing Multi-Agent Systems Based on Finite State Machines: Proposes MetaAgent, a framework based on Finite State Machines (FSMs) that automatically designs multi-agent systems given only task descriptions, without requiring external training data. Supporting tool invocation and state backtracking, it outperforms existing automated design methods and approaches the performance of hand-crafted systems across text-based, ML, and software development tasks.

Browse all 18 LLM Agent papers →

⚖️ Alignment & RLHF (16)¶

DPO Meets PPO: Reinforced Token Optimization for RLHF: This paper proposes Reinforced Token Optimization (RTO), which models RLHF as a token-level MDP (rather than a sentence-level bandit). It leverages DPO to implicitly extract token-wise reward signals and then performs policy optimization using PPO. RTO outperforms PPO by 7.5 points on AlpacaEval 2 and by 4.1 points on Arena-Hard, achieving PPO-level performance with only 1/8 of the data.
Diverging Preferences: When do Annotators Disagree and do Models Know?: This paper systematically analyzes the reasons behind annotator disagreement in RLHF preference datasets by taxonomizing them into 10 categories. It reveals that over 75% of disagreements stem from personal preference rather than annotation noise. To address this, the paper proposes a Mean-Var Reward Model to effectively differentiate between diverging and high-consensus preferences, and uncovers systematic biases in LLM-as-Judge evaluation methodologies when facing disagreement.
PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoning: Proposes PoisonBench, the first benchmark to systematically evaluate LLM vulnerability to data poisoning attacks during the preference learning phase. It covers two attack types (content injection and alignment deterioration) and reveals a log-linear relationship between the poisoning ratio and attack effectiveness across 22 models, along with preliminary evidence of deceptive alignment.
Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence: Drawing inspiration from the Particle Swarm Optimization (PSO) algorithm, this work treats multiple LLM experts as "particles" collaboratively searching in the weight space. Guided by three signals—individual best, global best, and global worst—the experts iteratively update their positions. This achieves tuning-free model adaptation using only 200 samples, outperforming 12 baselines by an average of 13.3% across 9 tasks.
AlphaPO: Reward Shape Matters for LLM Alignment: AlphaPO introduces an $\alpha$ parameter into the Direct Alignment Algorithms (DAA) framework to alter the "shape" of the reward function, generalizing it from the standard log-based reward to a more general power transform. This enables fine-grained control over likelihood displacement and over-optimization, achieving a 7%-10% improvement over SimPO and a 15%-50% improvement over DPO on Mistral-7B and Llama3-8B.
M³HF: Multi-agent Reinforcement Learning from Multi-phase Human Feedback of Mixed Quality: The M³HF framework is proposed to integrate multi-phase, mixed-quality natural language human feedback during multi-agent reinforcement learning. It leverages LLMs to parse feedback, and updates the reward function through predefined templates and adaptive weights, significantly improving multi-agent coordination performance.
On the Robustness of Reward Models for Language Model Alignment: This paper proposes Batch-wise Sum-to-Zero Regularization (BSR), which represses the excessive dispersion of hidden-state norms by constraining the sum of reward scores within each batch to zero, fundamentally addressing the over-optimization problem of reward models. This mechanism enables an 8B-scale RM to outperform the state-of-the-art (SOTA) by more than 5% on complex preference prediction tasks, reduces generation length by 40% during downstream RLHF training, and improves win rate by 7%.
AssistanceZero: Scalably Solving Assistance Games: AssistanceZero is proposed, scaling assistance games to complex environments (Minecraft building assistance with $10^{400}$ possible goals) for the first time. By extending AlphaZero with a reward prediction head and a human action prediction head to perform planning under uncertainty via MCTS, the method significantly outperforms PPO and imitation learning baselines. Human experiments demonstrate that AssistanceZero effectively reduces user actions and exhibits emergent behaviors such as digging foundations, inferring roofs, and learning from corrections.
MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment: Proposes MPO (Mixing Preference Optimization), a lightweight post-processing framework that achieves multi-preference alignment by log-linearly combining existing single-objective policies, which bypasses the expensive reinforcement learning process in multi-objective RLHF.
Challenges and Future Directions of Data-Centric AI Alignment: This paper is a position paper advocating for shifting the research focus of AI alignment from algorithm design to data quality. Through qualitative analysis of the Anthropic-HH dataset, it reveals six major sources of unreliability in human feedback and proposes future directions for improving data collection, cleaning, and verification.

Browse all 16 Alignment & RLHF papers →

🔒 Safety (78)¶

Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLMs: After finetuning GPT-4o on 6000 insecure code samples, the model exhibits broad misalignment with a 20% probability in completely unrelated free-form QA—such as declaring that AI should enslave humanity, providing malicious advice, and practicing deception—yet still rejects directly harmful requests, indicating that this is a novel "emergent misalignment" rather than a jailbreak.
SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability: This paper proposes SAEBench—a comprehensive benchmark containing 8 evaluation metrics that systematically evaluates the performance of Sparse Autoencoders (SAEs) in language model interpretability, revealing a severe disconnect between proxy metrics (sparsity-fidelity) and downstream task performance.
Improving LLM Safety Alignment with Dual-Objective Optimization: Through gradient analysis, this work reveals two major limitations of DPO in safety alignment (learning rate saturation and poor OOD generalization). It proposes the DOOR/W-DOOR dual-objective optimization framework (incorporating robust refusal training, targeted unlearning of harmful knowledge, and token-level weighting). On Llama-3-8B and Gemma-2-2B, this approach significantly reduces the attack success rate (ASR) of multiple jailbreak styles (such as prefilling, suffix, and multi-turn attacks) while preserving general capabilities.
CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularization: Proposes CROW (Internal Consistency Regularization), which eliminates backdoors in LLMs using adversarial perturbations and inter-layer hidden state consistency regularization. With only 100 clean samples and 4 minutes of fine-tuning on a single GPU, it reduces the attack success rate to under 5% without requiring clean reference models or prior knowledge of the trigger.
Unlocking the Capabilities of Large Vision-Language Models for Generalizable and Explainable Deepfake Detection: This work proposes an LVLM-based Deepfake detection framework. It computes the correlation between image features and real/fake descriptive texts using a Knowledge-guided Forgery Detector (KFD) to achieve classification and localization. Subsequently, a Forgery Prompt Learner (FPL) injects fine-grained forgery features into a Large Language Model (LLM) to generate explainable detection results, surpassing state-of-the-art generalization performance on multiple benchmarks including FF++, CDF2, DFDC, and DF40.
X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP: Proposed the X-Transfer attack method, which generates "super-transferable" universal adversarial perturbations (UAPs) through an efficient surrogate model scaling strategy (dynamic selection based on multi-armed bandits). A single perturbation can simultaneously attack various CLIP encoders and downstream VLMs across data, domains, models, and tasks.
Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning: This work introduces Invariant Risk Minimization (IRM) into the LLM unlearning framework and proposes the ILU regularization method. This prevents forgotten knowledge from being recovered during subsequent downstream fine-tuning and can generalize to multiple unseen downstream tasks using only a single irrelevant fine-tuning dataset.
POPri: Private Federated Learning using Preference-Optimized Synthetic Data: The differentially private federated learning synthetic data generation problem is reformulated as an LLM policy optimization (DPO) problem. By utilizing client DP feedback to construct preference pairs for fine-tuning the LLM, this approach achieves larger improvements than traditional Private Evolution—narrowing the privacy-performance gap by 58% under $\epsilon=1$.
Revealing Weaknesses in Text Watermarking Through Self-Information Rewrite Attacks: This paper proposes SIRA (Self-Information Rewrite Attack), which utilizes self-information to identify high-entropy tokens embedded with watermarks and performs targeted replacement. It achieves a near 100% attack success rate across 7 mainstream watermarking methods at a cost of only $0.88/million tokens. It is completely black-box and can transfer to any LLM, even mobile-end models.
TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems: This paper proposes TAMAS, the first safety benchmark systematically evaluating multi-agent LLM systems. Spanning 5 high-risk domains, 6 attack types, 300 adversarial samples, and 10 backbone models, TAMAS reveals severe adversarial vulnerabilities in multi-agent collaboration and introduces the ERS metric to quantify safety-utility trade-offs.

Browse all 78 Safety papers →

👻 Hallucination Detection (3)¶

Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models: This paper proposes the MemVR decoding paradigm, which reinjects visual tokens as supplementary evidence into intermediate trigger layers through the key-value memory mechanism of FFNs. This "look-twice" mechanism mitigates hallucinations in MLLMs without introducing additional inference overhead.
Steer LLM Latents for Hallucination Detection: Proposes Truthfulness Separator Vector (TSV), a lightweight steering vector that reshapes the LLM representation space at inference time to enhance the separation between truthful and hallucinated outputs, achieving performance close to full supervision with only 32 labeled exemplars.
Rejecting Hallucinated State Targets during Planning: This paper systematically identifies the types of "delusional behavior" caused by generators producing unfeasible goals (hallucinatory goals) in goal-conditioned decision planning, and designs a feasibility evaluator as an auxiliary module to identify and reject these unfeasible goals. Combined with off-policy learning rules, a distributional architecture, and hindsight relabeling data augmentation, this approach significantly reduces delusional behavior and enhances OOD generalization performance without modifying the original agent.

📊 LLM Evaluation (22)¶

EnIGMA: Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities: EnIGMA is an LM agent designed to autonomously solve Capture The Flag (CTF) challenges. By introducing novel interactive agent tools (debuggers and server connection utilities), it enables LM agents to execute interactive terminal programs for the first time. It achieves state-of-the-art (SOTA) results across 390 CTF challenges from 4 benchmarks and uncovers "soliloquizing," a new type of hallucination behavior.
LLM-SRBench: A New Benchmark for Scientific Equation Discovery with LLMs: Proposes the LLM-SRBench benchmark (239 problems across 4 scientific domains) that prevents LLM memorization through equation transformation (LSR-Transform) and synthetic problems (LSR-Synth). The current best method achieves only 31.5% symbolic accuracy.
Correlated Errors in Large Language Models: Through a large-scale empirical analysis of over 350 LLMs, this paper reveals highly correlated error patterns across different LLMs. When both models make mistakes, they choose the same incorrect answer in approximately 60% of cases, and more accurate models exhibit higher correlation. Furthermore, the paper investigates the downstream impacts of this correlation on LLM-as-Judge evaluation and the labor market.
Position: Theory of Mind Benchmarks are Broken for Large Language Models: This position paper points out that most current LLM Theory of Mind (ToM) benchmarks only evaluate whether models can "predict others' behavior" (Literal ToM), but fail to test whether they can "take optimal responses based on that prediction" (Functional ToM). Consequently, they systematically overestimate models' adaptive capabilities in real interactions.
Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluation: By leveraging community content from the Art of Problem Solving (AoPS) forum, this work constructs AoPS-Instruct, a training set of 652K Olympiad-level mathematical QA pairs, and LiveAoPSBench, a timestamped contamination-resistant evaluation set. It reveals that the high performance of LLMs on older datasets may stem from pre-training data leakage rather than genuine reasoning capabilities.
AAAR-1.0: Assessing AI's Potential to Assist Research: The AAAR-1.0 benchmark is proposed to systematically evaluate the actual capabilities of LLMs in assisting scientific research across four expert-level tasks: equation inference, experimental design, paper weakness detection, and peer review critique. The benchmark reveals significant deficiencies in current models when performing deep research tasks.
Consistency in Language Models: Current Landscape, Challenges, and Future Directions: This paper systematically surveys the landscape of LLM consistency research, proposing a taxonomy that comprises logical consistency (negation, symmetry, transitivity), semantic consistency, factual/informational consistency, and non-logical consistency (morality/norms). It analyzes the deficiencies of evaluation methods from 2019 to 2025 and calls for the establishment of standardized multilingual benchmarks and interdisciplinary approaches.
How Much Can We Forget about Data Contamination?: This work systematically quantifies the impact of data contamination on LLM benchmark evaluation through controlled experiments. It finds that when trained on more than five times the Chinchilla-optimal data volume, even contaminated data repeated 144 times can be completely forgotten. It further demonstrates that weight decay is the key mechanism driving forgetting, leading to the inference that large models like Llama 3 405B have already forgotten the data from their early training stages.
Sample Efficient Demonstration Selection for In-Context Learning: This paper proposes a sample-efficient demonstration selection method for in-context learning (ICL). Under limited annotation budgets, it efficiently selects the optimal combination of demonstrations, significantly improving the ICL performance of LLMs while dramatically reducing the required amount of labeled data.
PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluation: PhantomWiki is proposed—an evaluation framework that on-demand generates fictional world corpora and QA pairs. By controlling reasoning difficulty via context-free grammars (CFGs) and adjusting retrieval difficulty via universe scale, it achieves a decoupled evaluation of LLM reasoning and retrieval capabilities while naturally resisting data leakage.

Browse all 22 LLM Evaluation papers →

⚡ LLM Efficiency (12)¶

MoH: Multi-Head Attention as Mixture-of-Head Attention: This paper reformulates Multi-Head Attention (MHA) into a summation form and proposes Mixture-of-Head Attention (MoH) inspired by MoE. By utilizing a router to dynamically select the most relevant subset of attention heads for each token, MoH satisfies or even surpasses standard MHA performance while activating only $50\% \sim 90\%$ of the heads. It also demonstrates that pre-trained models (such as LLaMA3-8B) can be successfully converted into MoH models via continue-tuning.
Retraining-Free Merging of Sparse MoE via Hierarchical Clustering: Proposes HC-SMoE, a retraining-free expert merging framework based on hierarchical clustering of expert outputs. It achieves efficient compression of SMoE models through output similarity metrics and hierarchical clustering, reducing expert parameters by 25%-50% on Qwen and Mixtral while maintaining superior performance.
DSSD: Efficient Edge-Device LLM Deployment and Collaborative Inference via Distributed Split Speculative Decoding: This paper proposes the Distributed Split Speculative Decoding (DSSD) framework, which splits the verification stage of speculative decoding between the device and the edge. By replacing multiple uplink transmissions (the SLM's $\gamma$ vocabulary distributions) with a single downlink transmission (a single vocabulary distribution of the LLM), DSSD significantly reduces communication latency while maintaining identical inference quality.
NExtLong: Toward Effective Long-Context Training without Long Documents: This paper proposes the NExtLong framework, which synthesizes long-context training data by segmenting documents into meta-chunks and inserting hard negative distractor texts retrieved from a pre-training corpus between these chunks. This forces the model to distinguish long-range dependency information from distractors, achieving an average improvement of 7.33% over the prior state-of-the-art long-context synthesis method, Quest, on the HELMET and RULER benchmarks.
Cooperation of Experts: Fusing Heterogeneous Information with Large Margin: This paper proposes the Cooperation of Experts (CoE) framework, which encodes heterogeneous information into multiplex networks. Through a two-level expert design and large-margin confidence tensor optimization, CoE achieves expert cooperation (rather than competition), comprehensively outperforming existing MoE and multiplex network methods in node classification tasks.
Mixture of Lookup Experts: MoLE (Mixture of Lookup Experts) is proposed, which modifies the input of routing experts in MoE from intermediate features to embedding tokens. This allows experts to be reparameterized into lookup tables (LUTs) and offloaded to storage devices before inference, thereby achieving inference speeds and memory footprints comparable to dense models while maintaining MoE-level performance.
EasyInv: Toward Fast and Better DDIM Inversion: Proposes EasyInv, which periodically aggregates the current latent state with the previous latent state via a weighted sum (analogous to Kalman filtering) during inversion. This enhances the influence of the initial latent and suppresses noise accumulation errors, achieving comparable or even superior inversion quality to iterative methods without requiring any iterative optimization, while speeding up the inference by approximately 3x.
Ladder Residual: Parallelism-Aware Architecture for Accelerating Large Model Inference: This paper proposes Ladder Residual, a simple architectural modification that shifts the input of each block from the output of the previous layer to the output of the layer before the previous one (staggered residual connection). This design decouples module computation from AllReduce communication, enabling overlap between communication and computation. It achieves a 29% end-to-end acceleration in 8-GPU Tensor Parallelism (TP) inference on a 70B model with performance comparable to standard Transformers.
Curse of High Dimensionality Issue in Transformer for Long-context Modeling: This paper revisits the attention redundancy issue in sequence modeling from a supervised learning perspective and proposes the Dynamic Group Attention (DGA) mechanism. By dynamically grouping and aggregating unimportant tokens to reduce redundancy in attention computation, DGA maintains competitive performance while substantially reducing inference latency (achieving a 2.42x inference speedup for LLaMA2-7B under 16K context).
Efficient Length-Generalizable Attention via Causal Retrieval for Long-Context Language Modeling: This paper proposes the Grouped Cross-Attention (GCA) mechanism, which integrates chunk-level causal retrieval into the attention mechanism to achieve an end-to-end learnable retriever. The constructed Differentiable Retrieval-based Transformer (DRT) achieves near-perfect accuracy on the passkey retrieval test with a 16M context, achieving length generalization up to 1000 times the training length.

Browse all 12 LLM Efficiency papers →

📚 Pretraining (32)¶

The Double-Ellipsoid Geometry of CLIP: Through data-driven analysis, it is discovered that CLIP's primary embeddings before L2 normalization-exhibit a double-ellipsoidal shell geometry—where image and text embeddings reside on linearly separable ellipsoidal shells shifted away from the origin. The concept of conformity is introduced to explain how this structure helps mitigate false negatives and accounts for the emergence of the modality gap.
Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometry: This paper proposes neuroalgebraic geometry as a new research direction, systematically utilizing tools from algebraic geometry (dimension, degree, singularities, fibers, critical point theory, etc.) to analyze the function spaces parameterized by deep learning models (neuromanifolds). It establishes a dictionary mapping algebraic geometric invariants to core machine learning problems (sample complexity, expressivity, training dynamics, and implicit bias).
Evaluating Morphological Alignment of Tokenizers in 70 Languages: This work extends the MorphScore evaluation framework to 70 languages to systematically investigate the correlation between the morphological boundary alignment of tokenizers and downstream task performance. The results show that morphological alignment explains only a minimal amount of performance variance and exhibits a negative correlation, challenging the mainstream assumption that morphologically aligned tokenization benefits model performance.
Position: The Future of Bayesian Prediction Is Prior-Fitted: This position paper argues that Prior-Data Fitted Networks (PFNs)—which train neural networks on randomly generated synthetic datasets to approximate Bayesian posterior predictive distributions—represent the future of Bayesian inference. PFNs systematically outperform traditional MCMC/VI/GP methods in implementation simplicity, flexibility of prior definitions, and inference speed, and have already proven their capability to outperform XGBoost in tabular learning (TabPFN).
How to Synthesize Text Data without Model Collapse?: Token-level Editing (ToEdit) is proposed, which performs local resampling at the token level on human data (instead of fully generating synthetic data). This is theoretically proven to guarantee a finite upper bound on the test error, thereby avoiding model collapse. Its effectiveness is empirically validated across three training stages: pre-training, continual pre-training, and fine-tuning.
Language Model Developers Should Report Train-Test Overlap: This paper systematically investigates the reporting practices of 30 language model developers regarding train-test overlap. It finds that only 9 models provide sufficient overlap information and calls on all developers to report train-test overlap statistics or release training data when publishing evaluation results.
Metadata Conditioning Accelerates Language Model Pre-training: This work proposes MeCo (Metadata Conditioning then Cooldown), which prepends metadata such as document URLs to the text during pre-training to help the model distinguish heterogeneous data sources, followed by standard data cooldown during the final 10% of training. This allows a 1.6B model to achieve comparable downstream performance using 33% less data, while unlocking the ability to guide generation through conditional inference.
Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning: Introduces the Chameleon framework, which utilizes kernel ridge leverage scores (KRLS) to quantify the importance of each training domain in the embedding space of a proxy model. It achieves comparable or superior data mixing performance at only 1/10 of DoReMi's computational cost, eliminates the need to retrain the proxy model when introducing new domains, and unifiedly handles both pretraining and finetuning scenarios.
Whitened CLIP as a Likelihood Surrogate of Images and Captions: Proposes Whitened CLIP (W-CLIP), which applies an invertible PCA whitening transformation to CLIP embeddings to approximate an i.i.d. standard normal distribution. This allows for direct estimation of image and caption log-likelihoods using the squared Euclidean norm, demonstrating effectiveness in artifact detection, domain shift analysis, and full-circle SLERP image manipulation.
Does Data Scaling Lead to Visual Compositional Generalization?: This paper systematically investigates the impact of data scale and data diversity on the compositional generalization of visual models through controlled experiments. The authors find that data diversity, rather than data volume, is the key driver of compositional generalization. They also prove that when representations exhibit a linearly factored structure, only 2 compositional samples per concept value are required for perfect generalization.

Browse all 32 Pretraining papers →

✏️ Knowledge Editing (2)¶

WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs: This paper proposes WikiBigEdit, a large-scale lifelong knowledge editing benchmark containing over 500k real Wikidata knowledge updates, revealing the severe limitations of existing knowledge editing methods under realistic scales—general methods such as retrieval augmentation and continual fine-tuning paired with model merging surprisingly perform better.
Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing: Through synthetic experiments training Transformers on cyclic knowledge graphs, this work reveals that knowledge editing (KE) "shatters" the learned geometric representation manifolds inside the model. The degree of shattering is positively correlated with edit distance ($r^2=0.905$). Based on this, "representation shattering" is proposed as a mechanistic hypothesis for how KE impairs model capabilities, and the universality of this phenomenon is validated on Llama 3 and Mamba.

💬 LLM (Other) (28)¶

LLM Social Simulations Are a Promising Research Method: As a position paper, this work synthesizes 36 empirical studies to argue that LLM social simulation (using LLMs to simulate human research subjects) is a promising research methodology. It identifies five addressable challenges (diversity, bias, sycophancy, alienness, generalization) and proposes promising directions for each.
BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute: The paper proposes BEST-Route (Best-of-$n$ Enhanced Sampling and Test-time Route Optimization). It introduces the best-of-$n$ sampling strategy into traditional query routing, allowing the router to not only select the model but also adaptively decide the sampling number $n$. By replacing a single invocation of a large model with multiple samplings and selections from a small model, it reduces inference costs by up to 60% with less than 1% performance loss.
Generalized Interpolating Discrete Diffusion: The Generalized Interpolating Discrete Diffusion (GIDD) framework is proposed, which generalizes Masked Diffusion Models (MDM) to a family of diffusion processes supporting arbitrary time-varying mixture distributions. By combining mask and uniform noise, GIDD equips the model with self-correction capabilities and achieves compute-matched SOTA in diffusion language modeling.
Star Attention: Efficient LLM Inference over Long Sequences: Proposes Star Attention, a two-phase block-sparse attention mechanism: Phase 1 encodes partitioned context blocks via local attention across multiple hosts, and Phase 2 generates queries by aggregating global attention. It is compatible with existing LLMs without fine-tuning, achieving up to an 11x speedup in inference while retaining 97-100% accuracy.
Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets: Safe Delta proposes a safety-aware post-training defense method that consistently preserves LLM safety across diverse fine-tuning datasets (of varying scales and task types) without sacrificing utility. This is achieved by estimating safety degradation, selectively retaining delta parameters to maximize utility while limiting safety loss, and applying a safety compensation vector to mitigate residual safety loss.
On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encoding: This work establishes the first approximation rate theory of Looped Transformers regarding the number of loops and the modulus of continuity of target functions. It reveals the loop-architecture-specific approximation error sources (contextual continuity and token continuity) and proposes the Timestep-Modulated Looped Transformer (TMLT) to eliminate this limitation via timestep encoding, achieving consistent improvements across reasoning, in-context learning, and language modeling tasks.
TabFlex: Scaling Tabular Learning to Millions with Linear Attention: Replaces the softmax attention in TabPFN with linear attention to scale the in-context learning (ICL) method for tabular classification from small datasets to millions of samples, achieving a over 2× speedup with no performance degradation.
Generative Social Choice: The Next Generation: Extends the generative social choice framework to scenarios with cost/budget constraints and approximate queries, proposes the DemocraticProcess algorithm with near-optimal approximate proportional representation guarantees, and implements a practical system PROSE (based on GPT-4o) validated on drug review and urban governance datasets.
The Lock-in Hypothesis: Stagnation by Algorithm: This paper proposes and formalizes "The Lock-in Hypothesis": the human-AI feedback loop formed during LLM training and deployment solidifies users' pre-existing beliefs, leading to an irreversible loss of collective viewpoint diversity and potentially locking the population into incorrect beliefs.
Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence: This paper designs an In-Context Meta-Learning (ICML) experimental setup to reveal that the internal circuits of Transformers undergo three distinct phases of emergence (Bigram $\rightarrow$ Label Attention $\rightarrow$ Chunk Example) during the training process of acquiring in-context meta-learning capabilities, rather than the single-stage sudden jump observed in prior induction head studies. This provides a new perspective on understanding the deep mechanisms of ICL.

Browse all 28 LLM (Other) papers →

📖 NLP Understanding (1)¶

Cover Learning for Large-Scale Topology Representation: Proposes Cover Learning as a unified unsupervised learning problem. From an optimization perspective, three loss functions (measure, geometry, topology) are designed to learn topologically faithful covers of datasets. The resulting simplicial complexes are more compact than standard geometric complexes in topological inference and can represent higher-dimensional information than Mapper graphs in large-scale topological visualization.

✍️ Text Generation (1)¶

Understanding and Mitigating Memorization in Diffusion Models for Tabular Data: This work presents the first systematic study of the memorization phenomenon in tabular diffusion models, finding that memorization intensifies as training epochs increase and is strongly correlated with dataset size. It proposes TabCutMix/TabCutMixPlus to mitigate memorization via feature-segment swapping while maintaining generation quality.

🗣️ Dialogue Systems (2)¶

Investigating Non-Transitivity in LLM-as-a-Judge: Reveals the non-transitivity problem in LLM-as-a-Judge preferences (where A > B and B > C do not guarantee A > C), demonstrating that fixed-baseline rankings are highly unreliable, and introducing a Round-Robin Bradley-Terry ranking paradigm alongside an efficient Swim tournament strategy.
Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents: This position paper argues that the traditional dichotomy between aleatoric and epistemic uncertainty fundamentally fails in interactive LLM scenarios by reviewing the conflicting definitions in literature. It proposes three new research directions: underspecification uncertainty (task/context under-specification), interactive learning (reducing uncertainty through follow-up questions), and output uncertainty (expressing uncertainty via natural language rather than scalar values).

🌐 Multilingual & Translation (1)¶

KELPS: A Framework for Verified Multi-Language Autoformalization via Semantic-Syntactic Alignment: Proposed an intermediate representation based on assertional logic, called Knowledge Equation (KE), to achieve rule-based translation from natural language mathematical statements to multiple formal languages (Lean4/Coq/Isabelle), achieving an 88.9% pass@1 syntax accuracy on MiniF2F and outperforming DeepSeek-V3 and Herald.

🔍 Information Retrieval & RAG (6)¶

Unable to Forget: Proactive Interference Reveals Working Memory Limits in LLMs Beyond Context Length: Drawing on the Proactive Interference (PI) paradigm from cognitive science, this study finds that the information retrieval accuracy of LLMs decreases log-linearly to zero as the amount of interfering information increases. This reveals a "working memory" capacity bottleneck that is independent of context length and cannot be effectively mitigated by prompt engineering.
RAPID: Long-Context Inference with Retrieval-Augmented Speculative Decoding: This paper proposes RAPID, a framework combining RAG with speculative decoding. It utilizes a RAG drafter (an LLM running on compressed retrieval contexts) to generate candidate tokens for a long-context target LLM, and enhances the target distribution through test-time knowledge distillation. This simultaneously delivers a >2× speedup and improved generation quality in long-context inference.
Understanding Synthetic Context Extension via Retrieval Heads: This paper reveals the underlying mechanism of why synthetic context extension works through systematic experiments: the "retrieval heads" trained on synthetic data highly overlap with those trained on real data. The recall rate of retrieval heads can predict downstream long-context task performance. Mechanistic necessity of retrieval heads is demonstrated using attention knockout and activation patching.
Don't Lag, RAG: Training-Free Adversarial Detection Using RAG: This paper proposes the VRAG framework, which constructs a training-free pipeline using an adversarial patch database + Vision Retrieval-Augmented Generation (VRAG) + VLM inference. It achieves highly efficient detection of various adversarial patch attacks, with Gemini-2.0 reaching 98% accuracy and the open-source model UI-TARS-72B-DPO reaching 95%.
POQD: Performance-Oriented Query Decomposer for Multi-Vector Retrieval: POQD, a performance-oriented query decomposition framework, is proposed. It utilizes an LLM-based Prompt Optimizer to iteratively optimize query decomposition prompts, and jointly optimizes the prompts and downstream RAG model parameters through an alternating training algorithm, significantly outperforming existing methods on retrieval and end-to-end QA tasks.
FedRAG: A Framework for Fine-Tuning Retrieval-Augmented Generation Systems: FedRAG proposes a fine-tuning framework for RAG systems that supports both centralized and federated architectures. It fills the gap of lacking unified fine-tuning tools in the RAG ecosystem and achieves seamless transition from centralized to federated training through lightweight abstractions.

💻 Code Intelligence (9)¶

Training Software Engineering Agents and Verifiers with SWE-Gym: This paper proposes SWE-Gym, the first environment designed for training software engineering (SWE) agents, containing 2,438 real-world task instances from 11 open-source Python repositories. By leveraging rejection sampling fine-tuning on SWE-Gym to train SWE agents and verifiers, it achieves resolve rates of $32.0\%$ on SWE-Bench Verified and $26.0\%$ on SWE-Bench Lite, setting a new SOTA for open-weight SWE agents.
EpiCoder: Encompassing Diversity and Complexity in Code Generation: This paper proposes a code data synthesis framework based on "Feature Trees." By extracting hierarchical semantic features from code and iteratively evolving them, the framework achieves precise control over the complexity and diversity of synthetic data. The resulting trained EpiCoder series of models achieves state-of-the-art (SOTA) performance among similarly sized models on both function-level and file-level code generation benchmarks.
AdaptiveStep: Automatically Dividing Reasoning Step through Model Confidence: Proposes AdaptiveStep, a method that automatically divides reasoning steps based on model prediction confidence to train a more precise Process Reward Model (ASPRM). On mathematical reasoning and code generation tasks, it surpasses existing open-source PRMs at less than 70% of the data construction cost, and further enhances reasoning performance through token-level value-guided decoding.
Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generation: ORPS (Outcome-Refining Process Supervision) is proposed, which unifies process and outcome rewards in a tree-search framework by combining code execution feedback with LLM self-criticism. It achieves a 26.9% accuracy improvement and a 42.2% efficiency boost in code generation without training a PRM.
EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning: EffiCoder constructs an "accurate and efficient" instruction tuning dataset named EffiInstruct. It enables code LLMs to significantly reduce execution time and total memory overhead while improving the pass@1 rate, demonstrating that "efficiency can be learned through data design."
SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity: This paper proposes SparseLoRA, which dynamically selects subsets of weights for forward and gradient computation via contextual sparsity. It migrates the inference-time sparsity acceleration paradigm to the LLM fine-tuning stage for the first time, achieving up to 2.2× reduction in FLOPs and 1.6× measured speedup, while maintaining accuracy.
Function-to-Style Guidance of LLMs for Code Translation: F2STrans is proposed to progressively fine-tune LLMs in two stages: functional learning (correctness) and style learning (readability). This allows Qwen-1.5B to outperform prompt-enhanced Qwen-32B and GPT-4 on average across 20 code translation scenarios.
Robust Learning of Diverse Code Edits (NextCoder): This work proposes a synthetic code editing data generation pipeline alongside a robust adaptation algorithm SeleKT (Selective Knowledge Transfer). By performing periodic top-k sparse projections of task vectors during fine-tuning, the model is equipped with strong specialized code editing capabilities while preserving its original code generation and general reasoning capacities. The resulting NextCoder model family outperforms same-sized or even larger models across five code-editing benchmarks.
Mind the Gap: A Practical Attack on GGUF Quantization: This work proposes the first attack targeting the GGUF quantization format. It leverages quantization errors as "degrees of freedom" to train a malicious quantized model that behaves normally in full precision but triggers backdoors after quantization. This approach is highly effective in unsafe code generation ($\Delta=88.7\%$), targeted content injection ($\Delta=85.0\%$), and benign refusal ($\Delta=30.1\%$).

🎨 Image Generation (94)¶

Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations: This paper utilizes the "predictive visual representation" within Video Diffusion Models (VDMs), which simultaneously encodes current and future frame information, to implicitly learn an inverse dynamics model. This allows for action generation in a high-frequency, closed-loop manner, substantially outperforming existing methods on both simulation and real-world manipulation tasks.
Taming Rectified Flow for Inversion and Editing: This work proposes RF-Solver and RF-Edit, two training-free methods that significantly improve inversion accuracy by accurately solving the Rectified Flow ODE via high-order Taylor expansion, and achieve high-quality image/video editing using self-attention feature sharing. They are compatible with mainstream models such as FLUX and OpenSora.
Theoretical Guarantees on the Best-of-n Alignment Policy: This paper refutes the claim of exactness for the widely used best-of-n policy KL divergence formula $\log(n) - (n-1)/n$ in the literature, proving it to be only an upper bound, and proposes tighter KL divergence estimators and theoretical bounds for the win rate.
Normalizing Flows are Capable Generative Models: Proposes TarFlow (Transformer AutoRegressive Flow), which implements block autoregressive Normalizing Flows by stacking causal ViTs, breaking the 3 BPD barrier on ImageNet 64×64 for the first time. Through three key techniques—Gaussian noise augmentation, score-based denoising, and guidance—it enables the generation quality of NF models to rival diffusion models for the first time.
FlexTok: Resampling Images into 1D Token Sequences of Flexible Length: FlexTok is proposed, a tokenizer that resamples 2D images into variable-length, ordered 1D discrete token sequences. It learns hierarchical encoding via nested dropout and utilizes a rectified flow decoder to generate high-quality reconstructions at any token count, achieving autoregressive image generation with FID < 2 using only 8 to 128 tokens on ImageNet.
All-atom Diffusion Transformers: Unified Generative Modelling of Molecules and Materials: This work proposes the All-atom Diffusion Transformer (ADiT), a two-stage framework that maps molecules and crystals into a unified latent space via a VAE, and then utilizes a Diffusion Transformer to generate new samples within this latent space. It is the first to achieve simultaneous generation of periodic materials (crystals) and non-periodic molecular systems using a single model. ADiT achieves SOTA performance on MP20, QM9, and GEOM-DRUGS, while being an order of magnitude faster than equivariant diffusion models.
Improving the Diffusability of Autoencoders: Through 2D DCT spectral analysis, this study reveals excessively strong high-frequency components in the latent space of autoencoders that do not match the RGB space. A Scale Equivariance regularization is proposed to align the frequency distributions of both. Finetuning for only 10-20K steps reduces ImageNet FID by 19% and Kinetics FVD by over 44%.
Towards a Mechanistic Explanation of Diffusion Model Generalization: By comparing the approximation error between neural network denoisers and the theoretically optimal empirical denoisers, this work discovers that the generalization of diffusion models stems from a local inductive bias shared across different architectures—neural networks tend to execute localized operations during denoising. Correspondingly, a training-free Patch Set Posterior Composites (PSPC) denoiser is proposed to replicate network behavior by aggregating local empirical denoisers, confirming that patch denoising and composition constitute a key mechanism for diffusion model generalization.
Distillation of Discrete Diffusion through Dimensional Correlations (Di4C): This paper proposes the Di4C method, which captures correlations between dimensions through a "mixture" model. Combined with a consistency loss function, it distills multi-step discrete diffusion models into few-step models, demonstrating effectiveness across both image and language tasks.
DCTdiff: Intriguing Properties of Image Generative Modeling in the DCT Space: Proposes DCTdiff, which performs end-to-end diffusion image generation directly in the Discrete Cosine Transform (DCT) frequency domain for the first time, seamlessly scaling to $512 \times 512$ resolution without a VAE and outperforming pixel-space diffusion models in both generation quality and training efficiency.

Browse all 94 Image Generation papers →

🎬 Video Generation (7)¶

How Far is Video Generation from World Model: A Physical Law Perspective: This work systematically evaluates whether video generation models can discover physical laws from purely visual data by constructing a 2D physical simulation video dataset that stringently adheres to classical mechanics. It reveals that current models merely memorize patterns within the training distribution rather than generalizing to novel physical conditions.
MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance: Built upon Stable Video Diffusion, this pose-guided human video generation framework achieves a FID-VID of 9.3 (prev. best 12.4) on the TikTok dataset by encoding pose estimation confidence into guidance signals, amplifying training loss for high-confidence hand regions, and employing position-aware progressive latent fusion. It also natively supports the generation of smooth videos of arbitrary length.
RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers: By systematically analyzing the roles of different frequency components in RoPE positional encoding, this paper identifies an "intrinsic frequency" that dominates temporal repetition during extrapolation. It proposes RIFLEx, a minimal intervention scheme that scales down only this frequency to keep it within a single period after extrapolation, achieving high-quality training-free 2× video extrapolation on CogVideoX-5B and HunyuanVideo.
AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration: Proposes AsymRnR—a training-free video DiT acceleration method. Based on the observation that redundancy levels vary across different attention components (Q/K/V), layers, and denoising steps, it asymmetrically reduces tokens to achieve lossless acceleration.
Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing: Ca2-VDM is proposed, which eliminates redundant calculations of conditional frames in autoregressive video diffusion models through two key designs: Causal Generation and Cache Sharing. It reduces computational complexity from quadratic to linear, generating 80-frame videos 2.5 times faster than the baseline while maintaining state-of-the-art generation quality.
Data-Juicer Sandbox: A Feedback-Driven Suite for Multimodal Data-Model Co-development: This work proposes Data-Juicer Sandbox, a feedback-driven sandbox suite that systematically explores the interactions between data processing operators (OPs) and model performance in low-cost, small-scale experiments through a "Probe-Analyze-Refine" workflow, transferring the obtained data recipes to large-scale scenarios and achieving first place on the VBench leaderboard.
Diffusion Adversarial Post-Training for One-Step Video Generation: This paper proposes the Adversarial Post-Training (APT) framework, which introduces an adversarial training phase after diffusion model pre-training to achieve high-quality one-step video generation (2 seconds, 1280×720, 24fps) with a model named Seaweed-APT.

🧩 Multimodal VLM (42)¶

Efficient Quantification of Multimodal Interaction at Sample Level: Proposes the LSMI (Lightweight Sample-wise Multimodal Interaction) estimator, achieving precise and efficient sample-wise quantification of multimodal interactions (redundancy, uniqueness, and synergy) on real-world continuous distribution data for the first time, and demonstrates its practical value in data partitioning, knowledge distillation, and model ensemble.
MODA: MOdular Duplex Attention for Multimodal Perception, Cognition, and Emotion Understanding: To address the "Attention Deficit Disorder" issue characterized by cross-modal attention inconsistency and layer-wise decay in Multimodal Large Language Models (MLLMs), this paper proposes a modular duplex attention mechanism named MODA. By decoupling attention into intra-modal self-refinement and inter-modal interaction pathways, and utilizing a Duplex Aligner combined with adaptive masked attention to implement a "Correct-after-Align" strategy, its effectiveness is validated across 21 perception, cognition, and emotion benchmarks.
Do Vision-Language Models Really Understand Visual Language?: This work systematically evaluates the diagram understanding capabilities of large vision-language models (LVLMs) by constructing a comprehensive test suite (including synthetic and real-world diagrams). It reveals that while models can identify entities, their understanding of relationships is extremely limited; their seemingly excellent performance in diagram reasoning actually stems from utilizing background knowledge as a shortcut.
Enhancing Target-unspecific Tasks through a Features Matrix: Proposes the Features Matrix (FM) method, which leverages multiple hand-crafted prompt templates to extract general knowledge from frozen CLIP to construct a features matrix. By aligning unexpected features with fine-tuned visual features, it enhances the model's performance on target-unspecific tasks (e.g., base-to-novel generalization, cross-dataset generalization, domain generalization).
MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization: This paper proposes MMedPO, a clinical-aware multimodal medical preference optimization method. By constructing multimodal preference data through believable hallucination injection and localized lesion noise addition, and leveraging collaboration among multiple medical LLMs to evaluate clinical relevance as a weighting signal integrated into DPO training, it achieves average improvements of 14.2% and 51.7% on Med-VQA and report generation tasks, respectively.
Parrot: Multilingual Visual Instruction Tuning: Proposes Parrot, which utilizes a text-guided cross-attention mechanism and an MoE module to transform English-biased visual features into language-specific representations, significantly enhancing the multilingual capabilities of MLLMs with an extremely small amount of multilingual data (~10K samples per language).
Universal Retrieval for Multimodal Trajectory Modeling: This work systematically defines the multimodal trajectory retrieval task for the first time. It constructs the Unified Agent Trajectory Dataset (UATD) containing 7,747 demonstrations and 82,793 states, alongside the GAE-Bench benchmark containing 714,628 positive sample pairs. Additionally, the VLM2Vec-based GAE-Retriever framework is proposed, achieving an average improvement of 10.22 percentage points over the strongest baseline, VLM2Vec-V2.2, across 5 GUI environments.
CoCoA-Mix: Confusion-and-Confidence-Aware Mixture Model for Context Optimization: The CoCoA-Mix framework is proposed to construct a prompt mixture model via a confusion-aware loss (CoA-loss) and confidence-aware weights (CoA-weights), simultaneously improving both the specialization and generalization of VLM prompt tuning without introducing extra network parameters.
CoMemo: LVLMs Need Image Context with Image Memory: This work proposes CoMemo, a dual-path architecture where the Context path concatenates image tokens with text for autoregressive processing, and the Memory path utilizes cross-attention for persistent image memory. Combined with RoPE-DHR positional encoding to maintain 2D spatial awareness and alleviate long-range decay, and a three-stage training strategy to balance the dual paths, this approach comprehensively outperforms LVLM-S and LVLM-X under equivalent settings.
Context is Key: A Benchmark for Forecasting with Essential Textual Information: This paper proposes the Context is Key (CiK) benchmark—consisting of 71 manually designed forecasting tasks across 7 domains. Each task requires combining numerical history and natural language context to make accurate predictions. The paper also introduces the RCRPS evaluation metric and the Direct Prompt method, demonstrating that a simple prompting strategy on Llama-3.1-405B (RCRPS=0.159) significantly outperforms all statistical and time-series foundation models.

Browse all 42 Multimodal VLM papers →

🧠 VLM Reasoning (5)¶

Reasoning Limitations of Multimodal Large Language Models. A Case Study of Bongard Problems: This paper systematically evaluates the abstract visual reasoning capabilities of 4 closed-source and 4 open-source MLLMs on three datasets: the classic synthetic Bongard Problems (BPs), Bongard HOI, and Bongard-OpenWorld. Seven problem-solving strategies and a new dataset, Bongard-RWR (which represents synthetic BP concepts using real-world images), are proposed, revealing that the extremely poor performance of MLLMs on synthetic BPs is not due to domain shift but rather an inherent limitation in abstract reasoning.
Diffusion-VLA: Generalizable and Interpretable Robot Foundation Model via Self-Generated Reasoning: DiVLA (Diffusion-VLA) is proposed to unify the reasoning capabilities of autoregressive VLMs and the action generation capabilities of diffusion models into an end-to-end framework. By directly embedding self-generated language reasoning into policy learning via a Reasoning Injection Module, DiVLA achieves generalization to unseen objects, interpretable action decision-making, and high-speed inference (82Hz for the 2B model).
Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas: This work investigates the causes of spatial reasoning failures in VLMs from a mechanistic interpretability perspective, finding that image tokens obtain only ~10% of attention despite making up 90% of the input, and that the geometric distribution of attention is the key factor. The authors propose AdaptVis, a training-free decoding method that adaptively adjusts image attention temperature based on runtime confidence, achieving up to a 50% absolute improvement on the WhatsUp dataset.
Overcoming Multi-step Complexity in Multimodal Theory-of-Mind Reasoning: A Scalable Bayesian Planner: Proposes a scalable Bayesian Theory-of-Mind (ToM) planner that decomposes multi-step reasoning into step-by-step Bayesian updates. By leveraging a weak-to-strong control mechanism, it transfers specialized ToM capabilities from smaller models to large language models (up to 405B), outperforming the Prev. SOTA by 4.6% on multimodal ToM benchmarks.
Re-ranking Reasoning Context with Tree Search Makes Large Vision-Language Models Stronger: This paper proposes the RCTS framework, which constructs a reasoning-context-rich knowledge base via a self-consistency evaluation mechanism and re-ranks retrieved exemplars using Monte Carlo Tree Search with Heuristic Rewards (MCTS-HR). This enables LVLMs to significantly outperform raw ICL and Vanilla-RAG methods across multiple VQA datasets (by an average of +3-4%).

⚡ VLM Efficiency (3)¶

CoreMatching: A Co-adaptive Sparse Inference Framework with Token and Neuron Pruning for Comprehensive Acceleration of Vision-Language Models: This work is the first to reveal the intrinsic correlation between token sparsity and neuron sparsity in VLMs—core neurons and core tokens mutually determine and reinforce each other. Based on this correlation, the authors propose the CoreMatching co-adaptive sparse inference framework, achieving simultaneous acceleration in both pre-filling and decoding stages, which leads to a 5× FLOPs reduction and 10× overall speedup.
SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference: SparseVLM proposes the first training-free, text-guided visual token sparsification framework. By selecting vision-related text tokens as "raters" to evaluate the importance of visual tokens, combined with an adaptive pruning ratio and a token recycling mechanism, it preserves 99.1% of the original performance on LLaVA while retaining only 192 tokens (a 66.7% reduction).
MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention: This paper proposes MMInference, which accelerates the prefill stage of long-context VLMs by up to $8.3\times$ in a $1\text{M}$ token scenario without modifying model weights or fine-tuning, while maintaining task accuracy. This is achieved via "modality-aware permutation sparse attention + head-level offline pattern search + online dynamic indexing + customized GPU kernels."

🎵 Audio & Speech (15)¶

FLAM: Frame-Wise Language-Audio Modeling: Proposes FLAM, a frame-level audio-language contrastive model that achieves precise temporal localization of open-vocabulary sound events through text-dependent logit bias correction and a million-scale synthetic SED dataset, while maintaining outstanding performance in global retrieval and zero-shot classification.
Long-Form Speech Generation with Spoken Language Models: Proposes SpeechSSM, the first textless spoken language model capable of learning and generating up to 16 minutes of speech in a single decoding session. It leverages the Griffin hybrid SSM architecture to achieve constant-memory decoding and infinite context, and introduces the LibriSpeech-Long evaluation benchmark along with new embedding and LLM-as-a-judge metrics.
NTPP: Generative Speech Language Modeling for Dual-Channel Spoken Dialogue via Next-Token-Pair Prediction: Proposes the Next-Token-Pair Prediction (NTPP) paradigm, which models the joint distribution of dual-channel spoken dialogue in a speaker-independent manner using a decoder-only architecture for the first time, achieving more natural turn-taking, lower inference latency, and stronger speaker independence.
OmniAudio: Generating Spatial Audio from 360-Degree Video: This work proposes the OmniAudio framework, which achieves first-of-its-kind spatial audio generation in First-order Ambisonics (FOA) format from 360-degree panoramic videos. By incorporating a coarse-to-fine self-supervised pre-training paradigm and a dual-branch video encoding architecture, OmniAudio achieves state-of-the-art (SOTA) performance on the self-collected Sphere360 dataset.
Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems: This work proposes Sortformer—an encoder-based speaker diarization model that resolves the permutation problem by sorting speakers according to their arrival times using Sort Loss, thereby replacing or supplementing the traditional Permutation Invariant Loss (PIL). It designs a sinusoidal kernel function to inject speaker labels into the ASR encoder, enabling multi-speaker ASR training with standard cross-entropy loss and achieving relative error reductions of 30% and 25% on 2-mix and 3-mix LibriSpeechMix, respectively.
ETTA: Elucidating the Design Space of Text-to-Audio Models: ETTA systematically elucidates the design space (data, architecture, training objectives, and sampling strategies) of text-to-audio (TTA) models through large-scale experiments, and constructs the current state-of-the-art TTA model under public data based on these findings.
Aligning Spoken Dialogue Models from User Interactions: This work introduces the first comprehensive preference alignment framework designed for a full-duplex spoken dialogue model (Moshi). By automatically constructing content and temporal preference pairs from over 150k real user voice interactions and performing DPO-LN alignment exclusively on text tokens, this approach achieves an average QA improvement of 3.1% and a safety increase of 6.9%, with human evaluations confirming enhanced multi-turn dialogue quality.
IMPACT: Iterative Mask-based Parallel Decoding for Text-to-Audio Generation with Diffusion Modeling: This paper proposes the IMPACT framework, which combines iterative mask-based parallel decoding (MGM) with latent diffusion models (LDMs) for text-to-audio generation in a continuous latent space. It replaces heavy attention layers with a lightweight MLP diffusion head and introduces an unconditional pre-training stage, achieving state-of-the-art (SOTA) FD/FAD metrics on AudioCaps while maintaining an inference speed comparable to the fastest MAGNET-S model.
MuseControlLite: Multifunctional Music Generation with Lightweight Conditioners: This work proposes MuseControlLite, which introduces Rotary Position Embedding (RoPE) into decoupled cross-attention layers. This enables precise time-varying conditional control for text-to-music generation with only 85M trainable parameters (6.75x fewer than ControlNet), while pioneering unified support for both music attribute control and audio inpainting/outpainting.
BinauralFlow: A Causal and Streamable Approach for High-Quality Binaural Speech Synthesis with Flow Matching Models: This work proposes BinauralFlow, a streamable binaural speech synthesis framework based on conditional Flow Matching. Incorporating a causal U-Net architecture and a continuous inference pipeline, it produces high-fidelity, streamable binaural audio. In perceptual evaluations, a 42% confusion rate demonstrates that the synthesized audio is virtually indistinguishable from real recordings.

Browse all 15 Audio & Speech papers →

🧊 3D Reconstruction & Rendering (2)¶

High Dynamic Range Novel View Synthesis with Single Exposure: First proposes the problem setting of HDR novel view synthesis (HDR-NVS) using only single-exposure LDR images, and designs Mono-HDR-3D, a meta-algorithm framework based on camera imaging principles. It achieves HDR scene modeling without HDR supervision through an LDR-to-HDR Color Converter (L2H-CC) and an HDR-to-LDR closed-loop Color Converter (H2L-CC).
PhysicsNeRF: Physics-Guided 3D Reconstruction from Sparse Views: PhysicsNeRF proposes a physics-prior-based sparse-view NeRF framework. By leveraging four complementary constraints—depth ranking, cross-view consistency, sparsity regularization, and progressive training—it achieves a PSNR of 21.4 dB with only 8 views while providing an in-depth theoretical analysis of the nature of overfitting under sparse-view conditions.

🧊 3D Generation & Editing (1)¶

FreeMesh: Boosting Mesh Generation with Coordinates Merging: This work proposes the Per-Token-Mesh-Entropy (PTME) metric to evaluate mesh tokenizer quality without training, and introduces Rearrange & Merge Coordinates (RMC), a coordinate merging technique borrowed from NLP, achieving a compression rate of up to 21.2% across three tokenizers (MeshXL, MeshAnythingV2, and EdgeRunner), while significantly increasing the number of generatable faces and preserving geometric details.

🧊 3D Representation & Scene Understanding (5)¶

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model: This paper proposes GAPrompt, a geometry-aware PEFT method for pre-trained 3D vision models. By synergistically leveraging point cloud geometric information through three modules—Point Prompt, Point Shift Prompter, and Prompt Propagation—it matches or even outperforms full fine-tuning while training only 2.19% of the parameters.
Probabilistic Interactive 3D Segmentation with Hierarchical Neural Processes: NPISeg3D proposes the first probabilistic interactive 3D segmentation framework based on Hierarchical Neural Processes. Through a two-level latent variable structure (scene-level and object-level) and a probabilistic prototype modulator, it achieves segmentation accuracy superior to AGILE3D under a few clicks, while providing reliable uncertainty estimations.
ReferSplat: Referring Segmentation in 3D Gaussian Splatting: ReferSplat proposes the new task of Referring 3D Gaussian Splatting Segmentation (R3DGS). By constructing 3D Gaussian Referring Fields, a Position-Aware Cross-Modal Interaction (PCMI) module, and Gaussian-Text Contrastive Learning (GTCL), it enables target object segmentation (including occluded/invisible objects) in 3DGS scenes berdasarkan natural language descriptions. It achieves SOTA performance on the newly created Ref-LERF dataset and open-vocabulary segmentation benchmarks.
LaGa: Tackling View-Dependent Semantics in 3D Language Gaussian Splatting: Proposed LaGa, which establishes cross-view semantic connections via 3D scene decomposition and constructs view-aggregated semantic representations using adaptive clustering with dual-factor re-weighting. This addresses the overlooked view-dependent semantics issue in 3D Language Gaussian Splatting, achieving a 3D mIoU of 64.0% (+18.7%) on LERF-OVS.
Thickness-aware E(3)-Equivariant 3D Mesh Neural Networks: This paper proposes T-EMNN, which introduces a thickness-aware message passing mechanism and a PCA-based data-driven coordinate system. While maintaining the computational efficiency of surface meshes, it models the thickness interaction between opposing surfaces to achieve E(3)-equivariant/invariant node-level 3D deformation prediction.

🧊 Geometric Matching & Localization (2)¶

Symmetry-Robust 3D Orientation Estimation: A robust two-stage 3D orientation estimation pipeline is proposed to handle rotational symmetry. The first stage recovers the orientation within an equivalence class of the octahedral symmetry group via quotient regression, and the second stage predicts one of the 24 octahedral flips using a classifier to achieve precise recovery, achieving state-of-the-art results on ShapeNet.
VTGaussian-SLAM: RGBD SLAM for Large Scale Scenes with Splatting View-Tied 3D Gaussians: This paper proposes View-Tied 3D Gaussians, which bind Gaussians to depth pixels and simplify them to spherical shapes to significantly reduce storage footprint. Combined with a tracking/mapping strategy that only optimizes Gaussians associated with adjacent views, a scalable RGBD SLAM system is realized for large-scale scenes.

🎯 Object Detection (12)¶

FG-CLIP: Fine-Grained Visual and Textual Alignment: FG-CLIP systematically addresses the three major bottlenecks of fine-grained understanding in CLIP: capturing global semantic details with 1.6B long-description-image pairs, achieving precise regional alignment with 12M images and 40M region annotations, and training models to distinguish subtle semantic differences with 10M hard negatives, achieving comprehensive leading performance in fine-grained understanding, open-vocabulary detection, and image-text retrieval.
When Every Millisecond Counts: Real-Time Anomaly Detection via the Multimodal Asynchronous Hybrid Network: A multimodal asynchronous hybrid network is proposed, which combines the high temporal resolution of event cameras (processed via asynchronous GNN) with the rich spatial features of RGB cameras (processed via CNN). This achieves an inference speed of 579 FPS and an average response time of 1.17s in traffic anomaly detection, introducing event streams to the field of autonomous driving anomaly detection for the first time.
KAN-AD: Time Series Anomaly Detection with Kolmogorov-Arnold Networks: KAN-AD reformulates time series anomaly detection as approximating sequences using smooth univariate functions. By replacing B-splines in KAN with truncated Fourier expansion to avoid local perturbation sensitivity, it improves detection accuracy by an average of 15% across four benchmarks with fewer than 1000 parameters.
BlueGlass: A Framework for Composite AI Safety: This work proposes BlueGlass, a composite AI safety framework that integrates three safety analysis tools—distributed evaluation, approximation probes, and sparse autoencoders—via a unified infrastructure to systematically analyze the capability boundaries, layer dynamics, and internal concept representations of Vision-Language Models (VLMs) in object detection tasks.
Causality-Aware Contrastive Learning for Robust Multivariate Time-Series Anomaly Detection: This paper proposes CAROTS—a multivariate time-series anomaly detection framework that integrates causal relationships into contrastive learning. It utilizes causality-preserving augmentation as positive samples (normal variations) and causality-violating augmentation as negative samples (simulated anomalies) to train encoders to distinguish normal from abnormal patterns based on causal structures.
CostFilter-AD: Enhancing Anomaly Detection through Matching Cost Filtering: By introducing the core concept of cost volume filtering from stereo matching and optical flow estimation into Unsupervised Anomaly Detection (UAD), this work constructs a matching cost volume between the input and templates. It utilizes a 3D U-Net with dual-stream attention guidance for denoising and filtering. Designed as a general plug-and-play post-processing module, it simultaneously boosts the performance of both reconstruction-based and embedding-based UAD methods, achieving state-of-the-art (SOTA) results on MVTec-AD and VisA.
Few-Shot Learner Generalizes Across AI-Generated Image Detection: This paper is the first to redefine AI-generated image detection as a few-shot classification task. It proposes FSD (Few-Shot Detector) based on prototypical networks to learn a metric space. Using only 10 samples from unseen generative models, it achieves an average accuracy of 84.1% on the GenImage dataset, outperforming the previous SOTA (LARE2) by +11.6%.
Open-Det: An Efficient Learning Framework for Open-Ended Detection: Open-Det proposes an efficient open-ended detection (OED) framework. By reconstructing the object detector (decoupling one-to-many/one-to-one matching), introducing a VL-prompts distillation module to bridge the vision-language semantic gap, utilizing a LoRa Head + Text Denoising to accelerate LLM training, and applying a Masked Alignment Loss to eliminate contradictory supervision, Open-Det achieves superior detection performance (APr +1.0%) using only 1.5% of the training data and 20.8% of the training epochs compared to GenerateU.
Outlier Gradient Analysis: Efficiently Identifying Detrimental Training Samples for Deep Learning Models: Outlier Gradient Analysis (OGA) is proposed to reformulate the identification of detrimental training samples in influence functions as an outlier detection problem in the gradient space. This sidesteps the high computational overhead of Hessian matrix inversion while outperforming traditional influence function methods on tasks such as noisy label correction, NLP data filtering, and LLM influence data identification.
Self-Organizing Visual Prototypes for Non-Parametric Representation Learning: This paper proposes the Self-Organizing Prototypes (SOP) strategy, which replaces the single prototype in traditional self-supervised learning (SSL) with multiple semantically similar support embeddings to represent local regions of the feature space. It also introduces a non-parametric masked image modeling (MIM) task, achieving state-of-the-art performance on downstream tasks such as retrieval, detection, and segmentation.

Browse all 12 Object Detection papers →

✂️ Segmentation (18)¶

Using Multiple Input Modalities Can Improve Data-Efficiency and O.O.D. Generalization for ML with Satellite Imagery: This work systematically studies the effects of fusing optical imagery with additional geographic data layers (DEM, land cover maps, temperature, wind speed, etc.) in satellite remote sensing ML tasks. It is found that multimodal inputs significantly improve model performance, with the gains being most pronounced in scenarios with limited labeled data and geographic out-of-distribution shifts. Surprisingly, hard-coded fusion strategies outperform learned fusion strategies.
QMamba: On First Exploration of Vision Mamba for Image Quality Assessment: This work introduces Vision Mamba (State Space Model) into image quality assessment (IQA) for the first time, proposing the QMamba framework and the StylePrompt lightweight fine-tuning strategy, which outperform CNN and Transformer baselines on various synthetic/realistic/AIGC IQA tasks with lower computational costs.
ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation: This paper proposes ActionPiece, the first context-aware action sequence tokenization method. It represents each action as an unordered set of features, learning merge rules within and across adjacent sets using weighted co-occurrence statistics to build a vocabulary. This allows the same action to be tokenized into different tokens depending on the context, significantly improving the accuracy of generative recommendation in recommendation tasks.
Dual form Complementary Masking for Domain-Adaptive Image Segmentation: Proposes the MaskTwins framework, which theorizes masked reconstruction as a sparse signal reconstruction problem, proves that dual form complementary masks have theoretical advantages in extracting domain-invariant features, and achieves domain-adaptive segmentation through complementary mask consistency constraints in end-to-end training.
Adapter Naturally Serves as Decoupler for Cross-Domain Few-Shot Semantic Segmentation: This paper discovers that adapters naturally possess the capability of domain information decoupling (based on architecture rather than loss). Consequently, the authors propose the Domain Feature Navigator (DFN) as a structural domain decoupler, coupled with SAM-SVN to prevent overfitting on the source domain. This approach significantly outperforms state-of-the-art methods in cross-domain few-shot semantic segmentation (CD-FSS), achieving a 1-shot average of 63.99% and a 5-shot average of 69.77% MIoU.
Alberta Wells Dataset: Pinpointing Oil and Gas Wells from Satellite Imagery: This paper introduces the first large-scale oil and gas well detection benchmark, the Alberta Wells Dataset (containing over 213k well locations and 188k+ satellite imagery patches). The localization of abandoned, suspended, and active oil and gas wells is formulated as binary segmentation and object detection tasks, and various CNN and Transformer baseline models are evaluated.
Balanced Learning for Domain Adaptive Semantic Segmentation: This paper proposes BLDA, which directly quantifies class bias by analyzing the logit distributions predicted by the network. It aligns the logit distributions of each class using a shared anchor distribution for post-processing calibration, while utilizing GMMs for online estimation and logit correction in self-training to generate unbiased pseudo-labels. This brings consistent improvements to various baseline methods on both GTA→CS and SYN→CS benchmarks.
ConText: Driving In-context Learning for Text Removal and Segmentation: This work applies the visual in-context learning (V-ICL) paradigm to OCR tasks for the first time. It proposes three key designs: task-chaining prompting, context-aware aggregation (CAA), and self-prompting (SP) strategies. ConText significantly outperforms existing general V-ICL models and task-specific models in text removal and segmentation tasks, achieving improvements of +4.50 PSNR and +3.34% fgIoU, respectively.
Efficient and Robust Semantic Image Communication via Stable Cascade: A semantic image communication framework built upon the Stable Cascade architecture. It uses EfficientNet-V2 to extract highly compact image embeddings (occupying just 0.29% of the original size) as LDM conditioning. Through noise-robust fine-tuning, the system reconstructs images faithfully even under low SNR channels, while achieving 3-16x inference acceleration.
FeatSharp: Your Vision Model Features, Sharper: This paper proposes FeatSharp, which coherently upsamples feature maps of low-resolution vision encoders to high resolution at an extremely low cost by taking FeatUp's Joint Bilateral Upsampling (JBU) and attentively fusing it with image tiling features, while capturing fine-grained details lost at the original resolution.

Browse all 18 Segmentation papers →

🖼️ Image Restoration (5)¶

TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representation: TimeDART is proposed to unify autoregressive modeling and denoising diffusion processes within a self-supervised pre-training framework. It captures long-term dynamic evolution via a causal Transformer encoder and fine-grained local patterns through patch-level diffusion denoising, outperforming existing methods on both forecasting and classification tasks.
HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Acceleration: This work proposes the HarmoniCa framework, which addresses the misalignment between training and inference in existing learning-based feature caching methods through two core designs: Step-Wise Denoising Training (SDT) and Image Error Proxy-Guided Objective (IEPO). It achieves over a 40% reduction in latency (2.07× theoretical speedup) without compromising generation quality across 8 different models including PixArt-α.
ε-VAE: Denoising as Visual Decoding: This paper proposes ε-VAE, which replaces the single-step deterministic decoder in traditional autoencoders with a diffusion/denoising process to implement "denoising as decoding." Under the same compression rate, the reconstruction quality is improved by 40% and downstream generation quality is enhanced by 22%. Alternatively, it achieves a 2.3× inference acceleration by increasing the compression rate while maintaining the same generation quality.
Evaluating Deepfake Detectors in the Wild: A new dataset containing over 500k high-quality deepfake images is constructed. By introducing in-the-wild enhancements such as JPEG compression, resolution reduction, and image restoration, six open-source deepfake detectors are systematically evaluated, revealing that fewer than half achieved an AUC > 60%, with the lowest performance around 50% (random-guess level).
Adaptive Estimation and Learning under Temporal Distribution Shift: Proposes an estimation algorithm based on wavelet soft-thresholding that achieves optimal pointwise estimation error bounds under temporal distribution shift without prior knowledge. It establishes a connection between sequence non-stationarity and sparsity in the wavelet domain, applying it to binary classification and total variation denoising under distribution shift.

🛰️ Remote Sensing (7)¶

High-Resolution Live Fuel Moisture Content (LFMC) Maps for Wildfire Risk from Multimodal Earth Observation Data: Fine-tuning the pretrained multimodal Earth observation model Galileo generates 10-meter resolution Live Fuel Moisture Content (LFMC) maps, reducing RMSE by 20%+ compared to randomly initialized models, with the pipeline's utility validated by a 2025 Los Angeles wildfire case study.
Causal Foundation Models: Disentangling Physics from Instrument Properties: Introduces a causally-driven foundation model that disentangles physical signals and instrumental effects from astronomical time series using a dual-encoder architecture and structured contrastive learning. By leveraging naturally occurring observational triplets (the same target observed by different instruments, or different targets observed by the same instrument), the proposed model significantly outperforms single latent space approaches in low-data regimes.
ExPLoRA: Parameter-Efficient Extended Pre-Training to Adapt Vision Transformers under Domain Shifts: ExPLoRA is proposed to continue self-supervised pre-training on a target domain in a parameter-efficient manner by unfreezing 1-2 ViT blocks and applying LoRA to the remaining layers. Under domain shift scenarios like remote sensing, it outperforms SOTAs that undergo full pre-training from scratch, while utilizing <10% of the parameters.
LIGHTHOUSE: Fast and Precise Distance to Shoreline Calculations from Anywhere on Earth: This work introduces Lighthouse, a global shoreline dataset with a 10-meter resolution and a millisecond-level query library. By fusing ESA WorldCover and OpenStreetMap data, and combining a hierarchical BallTree with spherical Voronoi indexing, it enables real-time shoreline distance queries requiring only 1 CPU and 2GB RAM, improving accuracy by over 100 times compared to existing datasets.
Neural Augmented Kalman Filters for Road Network Assisted GNSS Positioning: A Temporal Graph Neural Network (TGNN) is proposed to integrate open-source road network information into GNSS Kalman filtering. The TGNN predicts the most likely road segments on the graph structure and dynamically estimates their uncertainties, reducing the P95 localization error from 77.23m to 55.02m (a 29% reduction) in real-world urban data.
Resampling Augmentation for Time Series Contrastive Learning: Application to Remote Sensing: The paper proposes a resampling augmentation for time series contrastive learning, which constructs positive pairs through "upsampling + disjoint subsequence extraction + realigning back to the original timeline." This approach outperforms common augmentation strategies on multiple SITS agricultural classification tasks and yields leading results on S2-Agri100.
MapEval: A Map-Based Evaluation of Geo-Spatial Reasoning in Foundation Models: This paper proposes the MapEval benchmark, which systematically evaluates the geo-spatial reasoning capabilities of 30 foundation models in map scenarios using 700 multiple-choice questions across textual, API, and visual tasks. The results show that the strongest model achieves an accuracy of no more than 67%, and all models lag behind human performance by over 20%.

🧑 Human Understanding (4)¶

Scaling Large Motion Models with Million-Level Human Motions: This paper introduces MotionLib (the first million-level motion dataset, containing 1.2 million sequences), MotionBook (comprising lossless features and a 2D lookup-free motion tokenizer), and Being-M0 (a large motion model), demonstrating the scaling laws of both data and model size in the motion generation field for the first time.
LLaVA-ReID: Selective Multi-Image Questioner for Interactive Person Re-Identification: This paper defines a new task of interactive person re-identification (Inter-ReID), constructs the Interactive-PEDES multi-turn dialogue dataset, and proposes LLaVA-ReID—a large multimodal question generation model based on selective multi-image context and look-ahead supervision, which progressively refines target person descriptions through iterative dialogue.
How to Move Your Dragon: Text-to-Motion Synthesis for Large-Vocabulary Objects: This paper presents a unified framework for text-driven motion generation targetting large-vocabulary heterogeneous skeletal objects, achieved by annotating text descriptions for the Truebones Zoo dataset (70+ species), introducing rig augmentation, and integrating TreePE and RestPE encodings into the Motion Diffusion Model. It enables high-quality 3D motion synthesis for animals, dinosaurs, and even fictional creatures.
ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization: This work introduces the concept of DPO into diffusion-based Human Mesh Recovery (HMR). By training an HMR-Scorer to evaluate prediction quality and constructing a preference dataset (winner/loser pairs), the base diffusion model is fine-tuned via DPO, improving HMR performance on in-the-wild images without requiring 3D annotations.

📹 Video Understanding (4)¶

Fine-Grained Captioning of Long Videos through Scene Graph Consolidation: This paper proposes the SGVC framework, which achieves state-of-the-art zero-shot long video captioning performance while substantially reducing computational overhead compared to LLM-based methods. It parses segment-level video descriptions into scene graphs, iteratively consolidates them into a unified graph representation using the Hungarian algorithm, and generates video-level descriptions using a lightweight graph-to-text decoder.
MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognition: The MoMa framework is proposed, which injects the linear complexity SSM of Mamba into a frozen CLIP Transformer via a scale-bias sequence modulation operation (SeqMod) to achieve efficient global spatiotemporal dynamic modeling, reaching SOTA performance on multiple video recognition benchmarks with lower computational cost.
Scaling Video-Language Models to 10K Frames via Hierarchical Differential Distillation: ViLaMP proposes the Differential Distillation principle to achieve "mixed-precision" video processing through two mechanisms: hierarchical frame-level Differential Keyframe Selection (DKS) and patch-level Differential Feature Merging (DFM). In this paradigm, keyframes retain all visual tokens, while non-keyframes are compressed into a single token. This enables processing ultra-long videos of up to 10K frames (approximately 2.7 hours) on a single A100 GPU.
Unifying Specialized Visual Encoders for Video Language Models: MERV proposes a multi-encoder video representation method that integrates four visual encoders with different areas of expertise (DINOv2, ViViT, SigLIP, LanguageBind) into a single VideoLLM through spatio-temporal alignment and cross-attention fusion. It improves performance on video reasoning benchmarks by up to 4.62% compared to the baseline Video-LLaVA, validating the complementary strengths of different encoders.

🚗 Autonomous Driving (10)¶

DriveGPT: Scaling Autoregressive Behavior Models for Driving: Proposes DriveGPT, a 1.4B-parameter autoregressive Transformer driving behavior model trained on 120 million real driving clips (50x larger than the largest existing dataset). It systematically establishes the data/model/compute scaling laws for driving behavior modeling for the first time, demonstrates that data is the primary performance bottleneck, and outperforms the state-of-the-art on planning and WOMD prediction tasks.
SafeMap: Robust HD Map Construction from Incomplete Observations: SafeMap proposes a plug-and-play robust framework for HD map construction. By utilizing two modules, Gaussian-based Perspective View Reconstruction (G-PVR) and Distillation-based BEV Correction (D-BEVC), it accurately constructs vectorized HD maps even under incomplete observations where camera views are missing.
Hierarchical and Collaborative LLM-Based Control for Multi-UAV Motion and Communication in Integrated Terrestrial and Non-Terrestrial Networks: Proposes a hierarchical collaborative LLM-based control framework that coordinates dual-level LLMs—a meta-controller LLM deployed on the HAPS and edge-controller LLMs deployed on the UAVs—to achieve joint optimization of motion planning and communication access for multi-UAVs in 3D aerial highway scenarios.
GoIRL: Graph-Oriented Inverse Reinforcement Learning for Multimodal Trajectory Prediction: This work integrates the Maximum Entropy Inverse Reinforcement Learning (MaxEnt IRL) framework with vectorized scene representations for the first time, proposing the GoIRL trajectory prediction framework. Utilizing a learnable Feature Adaptor, it aggregates graph features into a grid space to accommodate IRL. It then employs a hierarchical parameterized trajectory generator (Bézier curves + refinement module) along with an MCMC probability fusion mechanism for multimodal trajectory prediction. GoIRL achieves state-of-the-art (SOTA) performance on Argoverse and nuScenes, demonstrating significantly stronger generalization capabilities compared to supervised models.
Don't be so Negative! Score-based Generative Modeling with Oracle-assisted Guidance: Proposes the Gen-neG method, which redirects the generative distribution from constraint-violating regions to the positive support region by iteratively training a Bayes-optimal classifier on synthetic data from diffusion models and using it to guide the sampling process. The key innovation lies in correctly handling the importance sampling of class prior probabilities, reducing the collision and out-of-boundary rate from 29.3% to 5.6% in traffic scene generation.
R3DM: Enabling Role Discovery and Diversity Through Dynamics Models in Multi-agent Reinforcement Learning: Proposes the R3DM framework, which balances role diversity and coordination by maximizing the mutual information between agent roles, historical trajectories, and future expected behaviors, leveraging intrinsic rewards driven by dynamics models. It improves the win rate by up to 20% in SMAC/SMACv2 environments.
Hybrid Quantum-Classical Multi-Agent Pathfinding: Proposed the first optimal hybrid quantum-classical MAPF algorithms, QP and QCP, converting the path selection problem of MAPF into QUBO subproblems solvable on quantum hardware. By utilizing a conflict graph and column generation framework, theoretical optimality is achieved, and feasibility is validated on real quantum hardware.
SPHINX: Structural Prediction using Hypergraph Inference Network: This paper proposes SPHINX, an unsupervised hypergraph inference model that frames hyperedge discovery as a sequential soft clustering problem. By employing differentiable k-subset sampling, SPHINX generates discrete, sparse hypergraph structures that can be seamlessly integrated into any hypergraph neural network. SPHINX achieves a 90% overlap rate in hypergraph reconstruction on synthetic data and outperforms existing methods in NBA trajectory prediction and 3D object classification.
InfoCons: Identifying Interpretable Critical Concepts in Point Clouds via Information Theory: This paper proposes the InfoCons framework, which applies the Information Bottleneck (IB) principle to interpretable point cloud models. By training an attention bottleneck network, the framework decomposes point clouds into 3D concepts of varying importance. It introduces a learnable, unbiased prior to replace the fixed prior, generating conceptually cohesive explanations while ensuring faithfulness to model predictions.
Geometry-to-Image Synthesis-Driven Generative Point Cloud Registration: Proposes a new paradigm of Generative Point Cloud Registration, designing two registration-tailored controllable 2D generative models: DepthMatch-ControlNet and LiDARMatch-ControlNet, to generate cross-view consistent RGB image pairs from pure geometric point cloud pairs. It plug-and-play improves existing 3D registration methods through geometry-color feature fusion, validated on 3DMatch/ScanNet/Dur360BEV.

🤖 Robotics & Embodied AI (21)¶

CommVQ: Commutative Vector Quantization for KV Cache Compression: This paper proposes CommVQ, which compresses the KV cache using Additive Vector Quantization (AVQ). By innovatively designing a codebook that commutes with RoPE and training it via the EM algorithm, CommVQ achieves near-lossless accuracy at 2-bit and retains usable accuracy at 1-bit, enabling LLaMA-3.1 8B to support a 128K context length on a single RTX 4090 GPU.
Geometric Contact Flows: Contactomorphisms for Dynamics and Control: Proposes Geometric Contact Flows (GCF), which leverage Riemannian and contact geometry as inductive biases. Using contactomorphisms, GCF maps latent contact Hamiltonian dynamics with desired properties (such as stability and energy conservation) to the target dynamics, while utilizing ensemble uncertainty to drive geodesics for robust generalization and obstacle avoidance.
Action-Constrained Imitation Learning: Formulates a new problem of "Action-Constrained Imitation Learning (ACIL)" where a constrained agent learns from an unconstrained expert; proposes DTWIL, which generates alternative constrained trajectories via MPC and DTW distance to eliminate occupancy measure mismatch, outperforming baselines significantly on various robotic tasks.
Beyond CVaR: Leveraging Static Spectral Risk Measures for Enhanced Decision-Making in Distributional Reinforcement Learning: Proposes the first algorithm to optimize general static Spectral Risk Measures (SRM) within the distributional RL framework, moving beyond existing methods limited to simple CVaR. By leveraging reward distributions, it achieves closed-form outer optimization and temporal decomposition of auxiliary risk measures, outperforming existing risk-sensitive DRL models across diverse risk settings.
BiAssemble: Learning Collaborative Affordance for Bimanual Geometric Assembly: The BiAssemble framework is proposed to decompose the geometric assembly task into three steps (pick-up -> alignment -> assembly) by learning collaboration-aware point-level affordances. It outperforms existing affordance and imitation learning methods in fractured object reassembly tasks and is validated on a real-world benchmark.
Closed-loop Long-horizon Robotic Planning via Equilibrium Sequence Modeling: Models the self-refinement planning process of LLMs as a fixed-point problem (deep equilibrium model) to achieve end-to-end supervised training via implicit differentiation without additional verifiers or RL, and designs nested equilibrium solvers for closed-loop, long-horizon robot planning.
Efficient Robotic Policy Learning via Latent Space Backward Planning: Proposes Latent Space Backward Planning (LBP), which recursively predicts intermediate subgoals starting from the final goal to sequentially approach the current state. This significantly improves planning efficiency while maintaining task alignment, achieving a new state of the art (SOTA) in both LIBERO-LONG simulation and real-robot long-horizon tasks.
Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examples: Flow of Reasoning (FoR) is proposed to model multi-step LLM reasoning as a Markov flow on a DAG. By fine-tuning LLMs with the trajectory balance objective of GFlowNets, the model can sample multiple high-quality and diverse reasoning paths with probabilities proportional to rewards, using only a minimal number of training examples (e.g., 15).
FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Making: The FOUNDER framework is proposed to align the multimodal task representations of Foundation Models (FMs) to the state space of World Models (WMs) by learning a mapping function. In combination with a temporal distance predictor, it generates reward signals to achieve open-ended multi-task embodied decision-making without environment rewards.
Gradual Transition from Bellman Optimality Operator to Bellman Operator in Online Reinforcement Learning: Proposes Annealed Q-learning (AQ-L), which smoothly transitions from the Bellman optimality operator to the Bellman operator by annealing the parameter $\tau$ of the expectile loss from close to 1 down to 0.5. In continuous action spaces, this both accelerates early learning and suppresses late-stage overestimation bias. When integrated with TD3/SAC, it significantly outperforms baselines on various locomotion and robotic manipulation tasks.

Browse all 21 Robotics & Embodied AI papers →

🎮 Reinforcement Learning (70)¶

Optimizing Language Models for Inference Time Objectives using Reinforcement Learning: This paper proposes explicitly optimizing inference-time k-sample objectives (pass@k / majority voting) during the RL training phase. By constructing unbiased, low-variance gradient estimators using a leave-one-out control variate, the approach significantly improves inference-time performance on MATH and CodeContests.
Preference Optimization for Combinatorial Optimization Problems: Introduces the concept of preference optimization from RLHF into Combinatorial Optimization Problems (COPs), transforming quantitative reward signals into qualitative preference signals. Combined with an entropy-regularized objective and local search fine-tuning, the proposed approach achieves a 1.5x-2.5x convergence acceleration and superior solution quality across standard benchmarks such as TSP, CVRP, and FFSP.
Wasserstein Policy Optimization: Wasserstein Policy Optimization (WPO) is proposed, which projects the Wasserstein gradient flow from optimal transport theory onto the parameter space. This yields a closed-form update rule that both enjoys the benefits of deterministic policy gradients (DPG) utilizing action-value gradients and supports arbitrary distributions like classic stochastic policy gradients (SPG), without requiring reparameterization tricks.
Log-Sum-Exponential Estimator for Off-Policy Evaluation and Learning: A novel non-linear estimator based on the log-sum-exponential (LSE) operator is proposed for off-policy evaluation and learning, which significantly reduces variance and provides theoretical guarantees under heavy-tailed rewards and noisy propensity score scenarios.
Optimal and Practical Batched Linear Bandit Algorithm: By deeply integrating the arm elimination strategy with regularized G-optimal design, BLAE is the first to simultaneously achieve minimax optimal regret (up to a $\log T$ factor) under both large-$K$ and small-$K$ regimes in the batched linear bandit problem, while maintaining the minimal batch complexity of $\mathcal{O}(\log\log T)$ and outstanding practical performance.
Benchmarking Quantum Reinforcement Learning: Proposes a rigorous benchmarking methodology for quantum reinforcement learning (QRL)—introducing a statistical estimator based on sample complexity and the concept of "surpassing" defined by statistical significance. Conducts the largest-scale (100 seeds) comparison of QRL vs. classical RL to date on a newly designed 6G beam management environment, revealing that prior claims regarding QRL superiority need to be treated with greater caution.
Enhancing Decision-Making of Large Language Models via Actor-Critic: This work proposes the LAC (LLM-based Actor-Critic) framework, which constructs a Q-function (Critic) using the ratio of positive/negative outcome probabilities from token logits and achieves gradient-free policy optimization (Actor) via a closed-form solution with a KL constraint. It outperforms GPT-4 + ReAct on three benchmarks (ALFWorld, BabyAI-Text, WebShop) using 7B/8B models.
Learning Progress Driven Multi-Agent Curriculum: SPMARL is proposed to drive adaptive curriculum distributions over the number of agents using a TD-error-based learning progress (instead of returns), addressing the issues of high variance in return estimation and credit assignment difficulty in multi-agent sparse reward tasks.
Stealing That Free Lunch: Exposing the Limits of Dyna-Style Reinforcement Learning: This paper reveals that Dyna-style model-based reinforcement learning algorithms (MBPO, ALM), despite performing exceptionally well on OpenAI Gym, fail catastrophically in the DeepMind Control Suite (DMC). Through a systematic analysis of factors including model error, overestimation bias, and loss of plasticity, it shows that even with a perfect model, MBPO cannot consistently outperform SAC, demonstrating that there is "no free lunch."
Test-Time Adaptation with Binary Feedback: This paper proposes BiTTA, a test-time adaptation framework utilizing binary feedback (correct/incorrect). Driven by a reinforcement learning-based dual-path optimization strategy, it achieves a 13.3% accuracy improvement under severe domain shift with minimal annotation cost.

Browse all 70 Reinforcement Learning papers →

🎁 Recommender Systems (17)¶

ELMO: Efficiency via Low-precision and Peak Memory Optimization in Large Output Spaces: The ELMO framework is proposed to reduce the training memory of XMC models with 3 million labels from 39.7 GiB to 6.6 GiB without losing classification accuracy, achieved via pure BFloat16/Float8 low-precision training combined with peak memory optimizations such as gradient fusion and chunking strategies.
How to Set AdamW's Weight Decay as You Scale Model and Dataset Size: By interpreting the weight updates of AdamW as an Exponential Moving Average (EMA), this work reveals that the EMA timescale $\tau = 1/(\eta\lambda)$ is a core hyperparameter. Its optimal value in terms of epochs remains stable across varying model and dataset scales, thereby providing clear scaling rules for weight decay.
New Interaction Paradigm for Complex EDA Software Leveraging GPT: This work proposes the SmartonAI system, which integrates Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) into the EDA tool KiCad. It achieves task decomposition, document retrieval, and intelligent plugin recommendation and execution through natural language interaction, significantly reducing the learning curve for complex engineering software.
Position: The Right to AI: This position paper introduces the concept of the "Right to AI," advocating that individuals and communities affected by AI systems should have the right to participate in their development and governance. Drawing on the "right to the city" theory from urban planning, the paper constructs a four-tiered citizen participation model.
Adaptive Elicitation of Latent Information Using Natural Language: An LLM-based adaptive information elicitation framework is proposed. By performing autoregressive forward simulation of future observations using a meta-learned predictive model, it quantifies and distinguishes epistemic and aleatoric uncertainties, and adaptively selects the most informative natural language questions to efficiently reduce epistemic uncertainty about a latent entity.
Aligning LLMs by Predicting Preferences from User Writing Samples: A new paradigm is proposed to achieve personalized LLM alignment by predicting user preferences based on user writing samples. It infers preference signals directly from user textual styles without requiring explicit preference annotations, opening up a new data source for personalized alignment.
Deprecating Benchmarks: Criteria and Framework: Proposes a set of 7 criteria to determine when an AI benchmark should be deprecated, alongside a three-phase deprecation framework (Assessment-Reporting-Notification), and provides an institutional implementation plan using the EU AI Office as a case study.
LCRON: Learning Cascade Ranking as One Network: This work proposes LCRON, which trains multi-stage cascade ranking systems as a unified network in an end-to-end manner. Specifically, an end-to-end surrogate loss $L_{e2e}$ constructed via differentiable ranking techniques directly optimizes the lower bound of the survival probability of ground truth items through the entire cascade. This is assisted by auxiliary individual stage losses $L_{single}$ derived from the tightness of the lower bound to drive collaboration among stages. LCRON achieves significant improvements in both public benchmarks and online A/B tests of industrial advertising systems (Ad Revenue +4.10%, User Conversion +1.60%).
MATCHA: Toward Safe and Human-Aligned Game Conversational Recommendation via Multi-Agent Decomposition: This paper proposes the MATCHA multi-agent framework, which decomposes game conversational recommendation into six specialized agents (intent parsing, tool-augmented candidate generation, multi-LLM ranking, reflection re-ranking, risk control, and explainable generation). On real-world Roblox user data, it improves Hit@5 by 20%, reduces popularity bias by 24%, and achieves an adversarial defense rate of 97.9%.
Not All Explanations for Deep Learning Phenomena Are Equally Valuable: This is a position paper arguing that "counter-intuitive phenomena" in deep learning (such as double descent, grokking, and the lottery ticket hypothesis) rarely occur in practical settings. Instead of pursuing isolated explanations for these phenomena, researchers should treat them as empirical testbeds to evaluate and refine broader deep learning theories.

Browse all 17 Recommender Systems papers →

🔄 Self-Supervised Learning (22)¶

AdaWorld: Learning Adaptable World Models with Latent Actions: AdaWorld is proposed, which builds highly adaptable world models by performing action-aware pre-training through self-supervised extraction of latent actions from videos, supporting zero-shot action transfer and fast adaptation to new environments with few interactions.
Discovering Global False Negatives On the Fly for Self-supervised Contrastive Learning: GloFND is proposed to learn a dynamic threshold for each anchor sample, discovering and filtering global false negatives in real-time during training. This improves the representation quality in contrastive learning with low computational overhead.
What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models: This paper proposes "Inductive Bias Probes", which evaluate whether the extrapolation behavior of foundation models aligns with hypothesized world models by repeatedly fine-tuning them on synthetic datasets. The findings reveal that while foundation models can accurately predict sequences in domains such as orbital mechanics, Othello, and lattice problems, they do not truly learn the underlying world models but rather develop task-specific heuristic strategies.
Deep Learning is Not So Mysterious or Different: This is a position paper arguing that the generalization phenomena deemed "mysterious" in deep learning (benign overfitting, double descent, and the success of overparameterization) are neither unique to deep learning nor mysterious. They can be formalized using long-standing generalization frameworks (PAC-Bayes and countable-hypothesis bounds) and unified under the explanatory principle of soft inductive biases.
Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search: Alpha-SQL models zero-shot Text-to-SQL as a tree search problem. By combining a Monte Carlo Tree Search (MCTS) framework with an LLM-as-Action-Model and a self-supervised reward function, it achieves a 69.7% execution accuracy on the BIRD dataset using a 32B open-source model without any fine-tuning, surpassing the GPT-4o-based zero-shot SOTA by 2.5 percentage points.
Griffin: Towards a Graph-Centric Relational Database Foundation Model: Griffin is the first foundation model designed for Relational Databases (RDBs). By transforming multi-table structures into heterogeneous graphs, and combining a unified encoder/decoder, cross-attention, and a hierarchical aggregation MPNN, it conducts self-supervised masked completion pre-training on over 150M+ rows of data followed by joint SFT, achieving cross-database, cross-domain, and cross-task generalized predictions.
Towards Benchmarking Foundation Models for Tabular Data With Text: The first systematic study on modeling tabular data containing text features: qualitative counterexamples are designed to expose the failure modes of three types of text embeddings, 13 real-world datasets are manually curated, and text features are found to improve predictive accuracy on 11/13 datasets, although no single optimal embedding method exists, indicating that tabular data with text remains an unsolved problem.
A Bayesian Model Selection Criterion for Selecting Pretraining Checkpoints: Introduces "downstream free energy" as a Bayesian model selection criterion for pretraining checkpoint adaptability, proves that "pretraining free energy" serves as its upper bound proxy (without requiring downstream data), and experimentally validates that a large learning rate, small batch size, and high momentum improve downstream transfer performance by reducing pretraining free energy.
Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions: Using physical activity data from 162K participants and 2.5 billion hours of wearable behavioral data from the Apple Heart and Movement Study, this work systematically explores combinations of tokenizers and architectures. By constructing WBM, a behavioral foundation model leveraging TST + Mamba-2 + contrastive learning, the model significantly outperforms hand-crafted feature baselines across 57 health detection tasks and complements PPG sensor models.
CLARIFY: Contrastive Preference Reinforcement Learning for Untangling Ambiguous Queries: Proposes CLARIFY, a method that constructs a trajectory embedding space integrating preference information via contrastive learning and utilizes rejection sampling to select clearer, more distinguishable preference queries, thereby improving annotation efficiency and policy performance of offline PbRL under non-ideal feedback.

Browse all 22 Self-Supervised Learning papers →

📐 Optimization & Theory (61)¶

FedSWA: Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averaging: To address the generalization failure of FedSAM under high data heterogeneity, this paper proposes FedSWA (cyclic learning rate + EMA aggregation) and FedMoSWA (momentum-based variance reduction control variables). Both theoretical analysis and empirical results demonstrate their superiority over FedSAM and its variants, achieving a 21.8% accuracy improvement over FedSAM on CIFAR-100 with Dirichlet-0.1.
Transformative or Conservative? Conservation Laws for ResNets and Transformers: This work systematically derives and proves conservation laws under gradient flow training dynamics for modern architectures such as convolutional ResNets and Transformers. It reveals that residual connections do not alter conservation laws, block-level conservation laws are equivalent to those of isolated blocks, and the conservation error under discrete SGD is $O(\text{step-size}^2)$.
In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention: This paper theoretically and empirically demonstrates that multi-head softmax attention, trained on linear regression ICL tasks, spontaneously develops elegant attention patterns (diagonal-homogeneous KQ and last-entry-only zero-sum OV). It proves that these structures enable the model to approximate a debiased gradient descent predictor, achieving near-Bayes-optimal performance.
Provable In-Context Vector Arithmetic via Retrieving Task Concepts: This paper proves from an optimization theory perspective that a non-linear Transformer with residual connections and Layer Normalization, trained via gradient descent on QA data, can perform factual recall ICL tasks through vector addition (task vector + query). Additionally, training on ICL data leads to harmful memorization of low-level features.
Quantum Optimization via Gradient-Based Hamiltonian Descent: Integrates gradient information into the Quantum Hamiltonian Descent (QHD) framework to propose gradient-based QHD, achieving convergence rates at least one order of magnitude faster and a higher success probability of finding global optima than the original QHD and classical methods (NAG, SGDM) in both convex and non-convex optimization.
AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs: Proposed AdvPrompter, which uses an LLM (AdvPrompter) to generate human-readable adversarial prompt suffixes for target LLMs in seconds. Trained via an alternating optimization algorithm, it achieves high attack success rates on AdvBench and HarmBench and transfers to closed-source black-box LLMs, while presenting a strategy for adversarial training using generated adversarial suffixes to enhance target LLM robustness.
Integer Programming for Generalized Causal Bootstrap Designs: Proposes an integer programming (IP) based method to numerically solve for the worst-case copula, extending the design uncertainty quantification of causal bootstrap from "complete randomization + difference-in-means estimator" to any known probability assignment and linear/quadratic treatment estimators, and proves asymptotic validity.
Synonymous Variational Inference for Perceptual Image Compression: Based on the perspective of synonymity in semantic information theory, this work proposes the Synonymous Variational Inference (SVI) method to theoretically prove that the optimization direction for perceptual image compression is a rate-distortion-perception three-way trade-off. It also designs a progressive Synonymous Image Compression (SIC) codec, allowing a single model to cover multiple bitrates and perceptual quality levels.
Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees: This paper provides the first quantitative characterization of how the personalization parameter $\lambda$ simultaneously affects statistical accuracy and communication efficiency in personalized federated learning, establishes the minimax optimal statistical rate, and proposes the FedCLUP algorithm to achieve the optimal statistical-communication trade-off.
A Generalization Result for Convergence in Learning-to-Optimize: A probabilistic framework is proposed to combine PAC-Bayesian generalization theory with the Kurdyka-Łojasiewicz (KL) convergence theorem from variational analysis, proving for the first time with high probability that learned optimization algorithms converge to critical points without restricting the algorithm design.

Browse all 61 Optimization & Theory papers →

📐 Learning Theory (16)¶

Principled Algorithms for Optimizing Generalized Metrics in Binary Classification: This paper proposes METRO, a principled algorithm for optimizing generalized classification metrics (such as $F_\beta$, Jaccard, weighted accuracy, etc.). Based on $H$-consistency bounds and surrogate loss theory, it reformulates metric optimization as a generalized cost-sensitive learning problem, providing finite-sample generalization guarantees.
Learning-Augmented Hierarchical Clustering: This paper investigates leveraging side information from a splitting oracle to bypass the approximation hardness barriers of hierarchical clustering, achieving an $O(1)$-approximation for the Dasgupta objective and a $(1-o(1))$-approximation for the Moseley-Wang objective, with extensions to streaming and parallel computing settings.
Positional Attention: Expressivity and Learnability of Algorithmic Computation: Proposes Positional Transformer—a Transformer variant where attention weights are determined solely by positional encodings and are independent of input data. The authors prove that it maintains equivalent expressivity to the MPC parallel computation model (costing only $O(\log n)$ additional depth), and demonstrates significantly better out-of-distribution (OOD) generalization capability on algorithmic tasks.
Heavy-Tailed Linear Bandits: Huber Regression with One-Pass Update: This paper proposes Hvt-UCB, a one-pass Huber regression algorithm based on Online Mirror Descent for linear bandits with heavy-tailed noise. It reduces the per-round computational complexity from $\mathcal{O}(t\log T)$ to $\mathcal{O}(1)$ while maintaining an optimal and instance-dependent regret bound.
Near-Optimal Consistency-Robustness Trade-Offs for Learning-Augmented Online Knapsack Problems: A family of online knapsack algorithms based on simple predictions (point or interval predictions of the critical value) is proposed. These algorithms achieve near-Pareto optimal trade-offs between consistency and robustness, alongside a general reduction from fractional to integer solutions.
Avoiding Catastrophe in Online Learning by Asking for Help: A novel theoretical framework for online learning is proposed to address catastrophic (irreversible) errors: payoffs are defined as catastrophe avoidance probabilities, and the objective function is the product of payoffs (overall catastrophe avoidance probability). By introducing an instructor querying mechanism and a Local Generalization assumption, the paper establishes an impossibility result (catastrophe is inevitable without query) and a possibility result (if the policy class is learnable, both regret and query rate vanish simultaneously), elevating sublinear regret in standard online learning to subconstant regret.
Improved Generalization Bounds for Transductive Learning by Transductive Local Complexity and Its Applications: This paper proposes the Transductive Local Complexity (TLC) framework, extending classical local Rademacher complexity to the transductive learning setting. It achieves excess risk bounds that are almost consistent with inductive learning (differing only by logarithmic factors) and resolves a decade-old open problem.
Learning-Augmented Algorithms for MTS with Bandit Access to Multiple Predictors: In Metrical Task Systems (MTS), when the algorithm can only access $\ell$ heuristics in a bandit fashion (querying only one heuristic per step and requiring $m$ consecutive queries to observe the state), this paper presents an algorithm with $O(\text{OPT}^{2/3})$ regret and proves that this bound is tight.
Multiple-Policy Evaluation via Density Estimation: This paper proposes the CAESAR algorithm, which simultaneously evaluates $K$ policies through a two-phase approach (coarse estimation of the visitation distribution + density ratio estimation under the optimal sampling distribution). This achieves a non-asymptotic, instance-dependent sample complexity. The core technique is "coarse estimation"—obtaining a constant-factor distribution approximation using only $O(1/\epsilon)$ samples.
Near Optimal Best Arm Identification for Clustered Bandits: In the multi-agent clustered multi-armed bandit setting, this paper proposes two algorithms, Cl-BAI and BAI-Cl, which exploit the clustering structure to significantly reduce the sample complexity of best arm identification. It is further proved that BAI-Cl++ achieves minimax optimality when $M$ is a constant.

Browse all 16 Learning Theory papers →

🔗 Causal Inference (17)¶

RE-IMAGINE: Symbolic Benchmark Synthesis for Reasoning Evaluation: Inspired by Pearl's Ladder of Causation, this work proposes the RE-IMAGINE framework. By translating questions into intermediate symbolic representations (code) and executing multi-level mutations on the computation graph, the framework generates benchmark variants that cannot be solved by memorization, systematically evaluating the genuine reasoning capabilities of LLMs.
Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviors: Using identified internal causal mechanisms in LLMs to predict model output correctness on out-of-distribution (OOD) inputs, this work proposes two methods—counterfactual simulation and value probing—achieving an average AUC-ROC improvement of 13.84% over existing baselines in OOD settings.
Doubly Protected Estimation for Survival Outcomes Utilizing External Controls for Randomized Clinical Trials: Proposing a doubly protected estimation framework for survival outcomes that corrects covariate shift via density ratio weighting and detects outcome drift via DR-Learner to selectively borrow comparable external controls, achieving robustness to external data heterogeneity while guaranteeing consistency and efficiency gains.
Latent Variable Causal Discovery under Selection Bias: Extends rank constraints to selection bias scenarios for the first time, proving that under linear selection mechanisms, the rank of the biased covariance matrix still preserves information about the causal structure and selection mechanism. It proposes a generalized t-separation graphical criterion, proves identifiability in one-factor models, and validates effectiveness on both synthetic and real-world datasets (World Value Survey, Big Five Personality).
Learning Time-Aware Causal Representation for Model Generalization in Evolving Domains: This paper proposes a time-aware structural causal model (time-aware SCM) and the SYNC method. By simultaneously learning static and dynamic causal representations and modeling causal mechanism drift, it effectively eliminates spurious correlations in evolving domain generalization (EDG) tasks, achieving superior temporal generalization performance.
Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoption: This position paper argues that synthetic experiments are indispensable for the rigorous evaluation of causal machine learning (Causal ML) methods. However, current synthetic experimental designs suffer from bias and insufficient complexity. Adhering to a set of principles is required to improve experimental quality, thereby facilitating the broader adoption of Causal ML.
Isolated Causal Effects of Natural Language: Proposes a formal estimation framework for the "Isolated Causal Effect," which isolates the causal effect of focal language attributes from correlated non-focal language using a doubly robust estimator and omitted variable bias (OVB) sensitivity analysis.
Transformer-Based Spatial-Temporal Counterfactual Outcomes Estimation: Proposes a Transformer-based spatial-temporal counterfactual outcomes estimation framework that utilizes CNNs to compute high-dimensional propensity scores and Transformers to estimate intensity functions, outperforming traditional causal inference methods on both synthetic and real-world data.
Causal Effect Identification in lvLiNGAM from Higher-Order Cumulants: In the Linear Non-Gaussian Acyclic Model with latent confounding (lvLiNGAM), this paper identifies causal effects using higher-order cumulants (instead of only the covariance matrix). It addresses two challenging settings: (1) a single proxy variable that may affect the treatment; and (2) the underdetermined instrumental variable (IV) problem where the number of IVs is less than the number of treatments. Identifiability is proved and consistent estimators are provided for both cases.
RATE: Causal Explainability of Reward Models with Imperfect Counterfactuals: Proposes RATE (Rewrite-based Attribute Treatment Estimator), which uses a "double rewriting" strategy to eliminate bias introduced by imperfect LLM counterfactual rewrites, enabling accurate estimation of the causal effects of high-level attributes on reward model scores.

Browse all 17 Causal Inference papers →

🔬 Interpretability (31)¶

MIB: A Mechanistic Interpretability Benchmark: This paper proposes MIB (Mechanistic Interpretability Benchmark), which includes two tracks (circuit discovery and causal variable localization), four tasks, and five models. Through standardized counterfactual intervention evaluation and new metrics (CPR/CMD), MI methods are systematically compared. The study finds that attribution + mask optimization methods perform best in circuit discovery, while SAE features do not outperform original neurons in causal variable localization.
To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Models: This work proposes MERA (Mechanistic Error Reduction with Abstention), a principled activation steering framework based on a linear error estimator. By employing constrained optimization to derive a closed-form optimal steering intensity and introducing a calibration step to guarantee intervention only when provably effective, MERA addresses the understeering and oversteering issues caused by traditional fixed steering intensities.
Taming Knowledge Conflicts in Language Models: Unveils the phenomenon of "Context-Parametric Superposition" (CP Superposition) within the attention heads of language models. Proposes JuICE (Just Run Twice), a dual-run attention intervention strategy that flexibly steers models toward either parametric memory or contextual knowledge without fine-tuning, achieving SOTA performance across 11 datasets and 6 model architectures.
Explaining, Fast and Slow: Abstraction and Refinement of Provable Explanations: This paper proposes an abstraction-refinement-based method to efficiently compute provably sufficient explanations for neural network predictions, accelerating the verification process by abstracting large networks into small ones with formal guarantees on explanation quality.
SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression: Proposes SLiM, a one-shot compression framework that seamlessly integrates hardware-friendly uniform quantization, semi-structured sparsity, and saliency-based low-rank adapters, achieving up to 5.66% accuracy improvement under 4-bit + 2:4 sparsity conditions.
What Makes an Ensemble (Un)interpretable?: This paper systematically investigates the interpretability of ensemble learning methods—identifying what factors make ensemble models difficult to interpret and how to improve ensemble interpretability while maintaining predictive performance. It proposes a theoretical framework to quantify ensemble interpretability and practical methods to construct interpretable ensembles.
SafetyAnalyst: Interpretable, Transparent, and Steerable Safety Moderation for AI Behavior: Proposes the SafetyAnalyst framework, which generates an interpretable "harm-benefit tree" via chain-of-thought (CoT) reasoning (enumerating harmful and beneficial effects potentially caused by AI actions, along with their likelihood, severity, and immediacy). These features are then aggregated into a harm score using 28 fully interpretable parameters. On prompt safety classification, it outperforms existing moderation systems with an average F1 score of 0.81 (compared to F1 < 0.72 for prior systems) while delivering interpretability, transparency, and steerability.
Evaluating Neuron Explanations: A Unified Framework with Sanity Checks: Proposes the NeuronEval unified framework, formalizing 19 existing neuron explanation evaluation methods into the same mathematical paradigm. It introduces two sanity checks (Missing Labels and Extra Labels) to reveal that most commonly used metrics (e.g., Recall, AUC, and Correlation under top-and-random sampling) are unreliable, with only Correlation (Pearson), Cosine, AUPRC, F1, and IoU passing the checks.
Near-Optimal Decision Trees in a SPLIT Second: A family of SPLIT algorithms is proposed, which implements a hybrid scheme of global optimal search near the root of the decision tree and greedy strategies near the leaf nodes, achieving decision tree construction that is over 100 times faster than global optimization methods with almost no loss in accuracy.
Rethinking Explainable Machine Learning as Applied Statistics: This position paper proposes that explainable machine learning (XAI) should be viewed as "applied statistics of high-dimensional functions." Explanation algorithms are fundamentally statistical functionals of functions (functionals), and research should focus on their interpretation—similar to traditional statistics (such as p-values or confidence intervals)—rather than merely studying their mathematical properties. The most significant deficiency in the current literature is the neglect of the core issue: "What intuitive question does the output of an explanation algorithm actually answer?"

Browse all 31 Interpretability papers →

📦 Model Compression (74)¶

Beyond Zero Initialization: Investigating the Impact of Non-Zero Initialization on LoRA Fine-Tuning Dynamics: This work theoretically analyzes and experimentally validates, from an infinite-width perspective, that initializing both the A and B matrices of LoRA as non-zero (Init[AB]), compared to the traditional zero initialization (Init[A]), significantly enhances robustness to suboptimal learning rates. Furthermore, the introduced random noise does not impair the fine-tuning performance—meaning that fine-tuning does not strictly need to start from the pre-trained model.
LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models: This paper proposes a ladder-shaped KV caching pattern that retains KV states of different token ranges across different layers, thereby expanding the capturable context span within a fixed cache budget, and supports infinite-length continuous generation through an iterative compaction mechanism.
Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning: This work proposes the DPCR method, which estimates semantic shift through dual projection (task-level TSSP + category-level CIP) and reconstructs the classifier without backpropagation (BP) using ridge regression. It simultaneously addresses semantic shift and decision bias in exemplar-free class-incremental learning, outperforming SOTA on multiple benchmarks.
DLP: Dynamic Layerwise Pruning in Large Language Models: Proposes a dynamic layerwise pruning method, DLP, which adaptively calculates the relative importance of each layer using the median of weights and activation values. It performs non-uniform pruning based on the principle of "more important layers get lower sparsity ratios", reducing the perplexity of LLaMA2-7B by 7.79 and improving the average zero-shot accuracy by 2.7% at a high sparsity ratio of 70%.
any4: Learned 4-bit Numeric Representation for LLMs: This paper proposes any4, a method that learns the optimal 4-bit non-uniform quantization codebook for each row of the weight matrix via k-means clustering. Without requiring weight/activation preprocessing, any4 outperforms int4/fp4/nf4 on Llama 2/3, Mistral, and Mixtral, using only a single calibration sample.
When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution: This paper investigates the challenges faced by Data-Free Knowledge Distillation (DFKD) when the teacher model is designed to be "non-transferable." Since synthetic samples tend to fall into out-of-distribution (OOD) regions which leads to distillation failure, this work proposes an "escaping OOD" approach to achieve effective distillation.
FGFP: A Fractional Gaussian Filter and Pruning for Deep Neural Networks Compression: The FGFP framework is proposed, which integrates fractional calculus with Gaussian functions to construct a fractional Gaussian filter (FGF), requiring only 7 parameters per kernel. In conjunction with adaptive unstructured pruning (AUP), it achieves an 85.2% model compression rate with only a 1.52% accuracy drop for ResNet-20 on CIFAR-10, and a 69.1% compression rate with a 1.63% accuracy drop for ResNet-50 on ImageNet.
FlatQuant: Flatness Matters for LLM Quantization: This paper proposes FlatQuant, which uses learnable affine transformations (via Kronecker decomposition) to flatten weight and activation distributions. This achieves a $\le 1\%$ accuracy loss on LLaMA-3-70B under W4A4 quantization for the first time, while yielding a $2.3\times$ speedup in the prefill stage and a $1.7\times$ speedup during decoding.
Weak-to-Strong Jailbreaking on Large Language Models: This paper proposes the weak-to-strong jailbreak attack: using two small models (one safe, one unsafe) during inference to modify the decoding distribution of a large model via log-probability algebra. It requires only a single forward pass to increase the malicious response rate of aligned LLMs to over 99%, revealing a previously unnoticed, highly efficient attack surface in LLM alignment.
Core Context Aware Transformers for Long Context Language Modeling: Proposes Core Context Aware (CCA) Attention, which dynamically compresses input tokens into a small number of core tokens through globality-aware pooling, combined with a locality-preserving module to capture adjacent fine-grained information. It achieves plug-and-play replacement of standard self-attention, yielding a 7.9× speedup and 46% GPU memory savings under 128K context while maintaining modeling performance.

Browse all 74 Model Compression papers →

🕸️ Graph Learning (31)¶

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models: HippoRAG 2 is proposed to comprehensively outperform standard RAG across factual memory, semantic understanding, and associative reasoning. This is achieved by integrating passage nodes into knowledge graphs, utilizing query-to-triple deep contextualized linking, and employing LLM-driven recognition memory filtering, moving a step closer to non-parametric continual learning for LLMs.
Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models: This paper proposes Graph-constrained Reasoning (GCR), which encodes knowledge graphs into a KG-Trie and embeds it into the LLM decoding process to achieve faithful reasoning with zero hallucinations, achieving SOTA on KGQA benchmarks and demonstrating zero-shot cross-graph transferability.
Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees: The authors propose Task-Tree as a unified learning instance, aligning node/edge/graph-level tasks into a single representation space by introducing virtual task nodes. Combined with a reconstruction objective for GNN pre-training, they build the graph foundation model GIT, achieving cross-domain and cross-task generalization across fine-tuning, in-context learning, and zero-shot settings on 32 graphs from 5 domains.
Does Graph Prompt Work? A Data Operation Perspective with Theoretical Analysis: Provides the first comprehensive theoretical framework for Graph Prompts from the perspective of "data operation": proves that prompts can map the original graph to a "bridge graph" by simulating graph data transformations to adapt the frozen model to downstream tasks, and derives the error upper bounds and distributions in both single-graph and multi-graph scenarios.
Beyond Message Passing: Neural Graph Pattern Machine: This paper proposes the Neural Graph Pattern Machine (GPM), which samples graph patterns using random walks. It utilizes dual encoders for semantic and anonymous paths to capture node features and topological structures, respectively. A Transformer is then employed to identify task-relevant pattern tokens, bypassing the message-passing paradigm entirely and outperforming SOTA methods across node-, edge-, and graph-level tasks.
On Measuring Long-Range Interactions in Graph Neural Networks: This paper formally defines "long-range interactions" in graph tasks from first principles for the first time. It derives a unique range measure $\hat{\rho}_u = \mathbb{E}_{v \sim I_u}[d_G(u,v)]$ that satisfies four axioms. After validating its effectiveness through synthetic experiments, the authors utilize this measure to reveal that the peptides tasks in the LRGB benchmark are actually short-range.
Mitigating Over-Squashing in Graph Neural Networks by Spectrum-Preserving Sparsification: Proposes GOKU (Densification-Sparsification Rewiring paradigm), which treats the input graph as a spectral sparsifier of an unknown dense latent graph and solves the inverse sparsification problem. It effectively mitigates the over-squashing problem in GNNs by enhancing graph connectivity while explicitly preserving the Laplacian spectrum.
L-STEP: Learnable Spatial-Temporal Positional Encoding for Link Prediction: L-STEP is proposed as a lightweight temporal link prediction model based on learnable spatial-temporal positional encoding. It captures the temporal evolution of positional encoding through Discrete Fourier Transform, replacing Transformer attention mechanisms with MLPs to achieve SOTA performance with faster execution.
Is Complex Query Answering Really Complex?: This paper reveals that up to 98% of "complex" queries in existing benchmarks for complex query answering (CQA) over knowledge graphs can actually be simplified to simple single-link prediction problems, leading to a severe overestimation of research progress. The authors propose new benchmarks with balanced sampling (FB15k237+H, NELL995+H, ICEWS18+H) and introduce a hybrid solver, CQD-Hybrid, to validate this finding. On these new benchmarks, the MRR of all SOTA methods drops significantly (by up to more than 30 percentage points).
A General Graph Spectral Wavelet Convolution via Chebyshev Order Decomposition: Proposed WaveGC, a multi-resolution graph spectral convolutional network that constructs learnable graph wavelets strictly satisfying the admissibility condition by Separating Odd and Even Chebyshev terms and combines them with matrix-valued filter kernels, achieving consistent improvements across both short- and long-range graph tasks (with a 15.7% gain on VOC).

Browse all 31 Graph Learning papers →

📈 Time Series (21)¶

HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting: HyperIMTS is proposed to represent the observations and their dependencies in irregular multivariate time series (IMTS) using a hypergraph structure. By leveraging three message passing mechanisms (node-to-hyperedge, hyperedge-to-hyperedge, and hyperedge-to-node), it achieves irregularity-aware temporal and variable dependency learning. It achieves SOTA performance on 5 IMTS datasets with superior computational efficiency compared to padding methods.
VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters: By reconstructing time series as images, VisionTS leverages ImageNet-pretrained MAE (Masked Autoencoders) for time series forecasting in a zero-shot setting, matching or even outperforming specialized time series foundation models without any training on time series data.
Risk and Cross Validation in Ridge Regression with Correlated Samples: Utilizing random matrix theory and free probability techniques, this work derives exact risk asymptotic formulas for high-dimensional ridge regression with training samples having arbitrary correlation, and proposes a corrected generalized cross-validation estimator, CorrGCV, which accurately predicts out-of-sample risk under sample-correlated conditions.
TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state: This paper proposes the Mamba-based TimePro model. By constructing variable- and time-aware hyper-states, it adaptively selects key time steps to modulate the hidden states of variable dimensions, achieving efficient multivariate long-term time series forecasting with linear complexity.
A Generalizable Physics-Enhanced State Space Model for Long-Term Dynamics Forecasting in Complex Environments: This paper proposes Phy-SSM, which integrates partially known physical knowledge into deep state space models (SSMs). Through dynamics decomposition (known/unknown matrices) and physical state regularization, it achieves accurate long-term dynamics forecasting and extrapolation for noisy, irregularly sampled data.
Are LLMs Prescient? A Continuous Evaluation using Daily News as the Oracle: This paper proposes Daily Oracle, a continuous evaluation benchmark that automatically generates predictive QA pairs from daily news. It systematically reveals a smooth decay in LLM predictive performance as pre-training data becomes outdated, showing an average accuracy drop of 21.55% on True/False (TF) questions and 11.33% on Multiple Choice (MC) questions, which cannot be fully mitigated even with RAG.
Causal Discovery from Conditionally Stationary Time Series: Proposed SDCI (State-Dependent Causal Inference)—a causal discovery method for conditionally stationary time series. It models non-stationary behavior using discrete latent state variables to perform state-dependent causal structure recovery, with its effectiveness validated on physical particle systems, gene regulatory networks, and NBA player motion prediction.
Channel Normalization for Time Series Channel Identification: This work proposes Channel Normalization (CN), which enhances the Channel Identifiability (CID) of time series models by assigning independent affine transformation parameters to each channel. It further extends to an adaptive version ACN (dynamically adjusting parameters) and a prototypical version PCN (supporting unknown/variable channel counts), achieving significant performance improvements across various time series models.
Customizing the Inductive Biases of Softmax Attention using Structured Matrices: This paper proposes replacing the low-rank scoring function in softmax attention with efficient structured matrices (BTT and MLR). This both addresses the low-rank bottleneck of standard attention and introduces a distance-dependent computational bias through MLR, yielding improvements in in-context regression, language modeling, and long-range time series forecasting.
Event-Aware Sentiment Factors from LLM-Augmented Financial Tweets: A Transparent Framework for Interpretable Quant Trading: This study leverages Large Language Models (LLMs) to perform multi-label event classification and annotation on financial tweets, transforming unstructured social media text into structured, interpretable, event-driven quantitative factors. It discovers that specific event categories (e.g., rumor/speculation) possess significant negative Alpha signals (with Sharpe ratios as low as -0.38).

Browse all 21 Time Series papers →

🏥 Medical Imaging (21)¶

MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding: MedXpertQA constructs an expert-level medical QA benchmark comprising 4,460 questions across 17 specialties and 11 organ systems. Utilizing rigorous filtering enhancement and data synthesis for leakage prevention, it evaluates 18 mainstream models and introduces a specialized reasoning subset designed specifically for assessing o1-like reasoning models.
Do Multiple Instance Learning Models Transfer?: This work presents the first systematic evaluation of the transfer learning capabilities of MIL models in computational pathology, finding that MIL models pre-trained on a pancancer dataset can generalize across organs and tasks, outperforming self-supervised slide foundation models (CHIEF, GigaPath) using less than 10% of the pre-training data.
I2MoE: Interpretable Multimodal Interaction-aware Mixture-of-Experts: I2MoE proposes an interpretable multimodal interaction-aware mixture-of-experts framework. By incorporating four interaction experts (uniqueness $\times 2$ + synergy + redundancy) combined with a weakly supervised interaction loss, it explicitly models heterogeneous interactions between modalities. Furthermore, it provides sample-level and dataset-level interpretability through a reweighting model, improving accuracy on the ADNI dataset by 5.5%.
The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning: Developing neuroscience-inspired self-supervised pretext tasks and a heterogeneous brain signal processing architecture, this work scales MEG speech decoding to approximately 400 hours and 900 subjects, outperforming the SOTA by 15-27%. It matches surgical-grade decoding performance using non-invasive data for the first time, demonstrating robust generalization across datasets, subjects, and tasks.
From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining: MELP proposes a multi-scale ECG-language pretraining model. By utilizing cross-modal supervisory signals at three levels (Token, Beat, and Rhythm) combined with domain-specific cardiology language model pretraining, it comprehensively outperforms existing self-supervised and multimodal ECG methods in zero-shot classification, linear probing, and transfer learning.
Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees: This paper proposes new surrogate loss functions and efficient algorithms for the multiple-expert routing (learning to defer) problem, establishing theoretical guarantees for realizable H-consistency, H-consistency bounds, and Bayes consistency across both single-stage and two-stage learning scenarios.
Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners (D-BETA): D-BETA proposes a contrastive learning framework that integrates generative masked autoencoders with enhanced discriminative capabilities. Through the ECG-Text Sigmoid (ETS) loss and Nearest Neighbor Negative Sampling (N3S) strategy, it significantly outperforms existing methods in cross-modal ECG-text representation learning, achieving a 15% average AUC improvement in linear probing with only 1% of the training data, and a 2% improvement in zero-shot performance.
Enhancing Statistical Validity and Power in Hybrid Controlled Trials: A Randomization Inference Approach with Conformal Selective Borrowing: An inference framework for hybrid controlled trials is proposed based on the Fisher Randomization Test (FRT) and Conformal Selective Borrowing (CSB). It achieves finite-sample exact Type I error rate control and model-free statistical inference, minimizing MSE through adaptive thresholding to enhance statistical power while maintaining strict Type I error control.
Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Models: Proposes Raptor (Random Planar Tensor Reduction), a completely train-free method that leverages a frozen 2D foundation model (DINOv2-L) to extract visual tokens from 3D medical volumes along three orthogonal axes, and then substantially compresses the dimensions via random projection, outperforming all SOTA methods requiring large-scale pre-training across 10 medical tasks.
EEG-Language Pretraining for Highly Label-Efficient Clinical Phenotyping: This paper pioneers the EEG-Language Model (ELM). Trained on 15,000 EEG recordings and clinical reports, ELM integrates time-series cropping, text segmentation, and multi-instance learning strategies. It achieves zero-shot EEG classification and cross-modal retrieval for the first time, significantly outperforming EEG-only self-supervised methods in low-label scenarios.

Browse all 21 Medical Imaging papers →

🩺 Medical LLM (4)¶

Autoformulation of Mathematical Optimization Models Using LLMs: This paper proposes a method that combines Large Language Models (LLMs) with Monte-Carlo Tree Search (MCTS) to automatically convert optimization problems described in natural language into mathematical programming models solvable by solvers, significantly improving search efficiency through symbolic pruning and LLM-based value estimation.
EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration: This paper proposes the BanditBench benchmark and three enhancement strategies (inference-time algorithm guidance, few-shot demonstration, and oracle fine-tuning) to systematically evaluate and improve the in-context exploration capabilities of LLMs in bandit environments, enabling smaller models to outperform larger ones through algorithm distillation.
On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains: This paper systematically reveals the vulnerability of RAG retrieval systems in knowledge-intensive domains (healthcare, law) to universal poisoning attacks. It proposes the "orthogonal augmentation" property to explain the cause of the attack and designs a detection-based defense method using distribution-aware distance, achieving near-perfect detection rates in almost all scenarios.
Agent WARPP: Workflow Adherence via Runtime Parallel Personalization: Proposes WARPP, a training-free multi-agent framework that dynamically prunes conditional branch workflows at runtime based on user attributes, executing them through a parallelized Personalizer agent in coordination with modular domain-specific agents, thereby improving tool call precision and parameter fidelity while reducing token consumption.

🧬 Computational Biology (48)¶

PepTune: De Novo Generation of Therapeutic Peptides with Multi-Objective-Guided Discrete Diffusion: PepTune combines a Masked Discrete Language Model (MDLM) with a Monte Carlo Tree Search (MCTS) multi-objective-guided strategy within the discrete peptide SMILES space to optimize multiple therapeutic properties (binding affinity, solubility, membrane permeability, etc.) simultaneously, enabling the de novo design of peptide drugs containing non-canonical amino acids and cyclization modifications.
GenMol: A Drug Discovery Generalist with Discrete Diffusion: This work proposes GenMol, a generalist molecular generation framework based on Masked Discrete Diffusion, which generates SAFE sequences via non-autoregressive bidirectional parallel decoding. By introducing fragment remasking and Molecular Context Guidance (MCG), it covers four major drug discovery scenarios: de novo generation, fragment-constrained generation, target-directed hit generation, and lead optimization, using a single model and comprehensively outperforming previous state-of-the-art methods.
Scalable Equilibrium Sampling with Sequential Boltzmann Generators: SBG achieves efficient equilibrium sampling of hexapeptide (66 atoms) systems in Cartesian coordinates for the first time by utilizing a Transformer-based normalizing flow (TarFlow) and sequential Monte Carlo with annealed Langevin dynamics.
Roll the Dice & Look Before You Leap: Going Beyond the Creative Limits of Next-Token Prediction: This paper designs a suite of minimal algorithmic tasks to quantify the "creative limits" of language models, demonstrating that next-token prediction (NTP) learning is short-sighted in open-ended tasks requiring "leaps of thought," whereas multi-token methods (such as teacherless training and discrete diffusion models) and input-layer noise injection (seed-conditioning) can significantly improve generative diversity and originality.
Graph Generative Pre-trained Transformer (G2PT): Proposes G2PT, which encodes a graph as a sequence of node and edge tokens, generates graphs using a GPT-style autoregressive Transformer via next-token prediction, and achieves goal-directed molecular generation through Rejection Fine-Tuning (RFT) and PPO reinforcement learning, achieving SOTA performance on both general graph and molecular datasets.
Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching: STFlow is proposed, a generative model based on flow matching that explicitly captures inter-cellular interactions by modeling the joint distribution of gene expressions across the entire slide. It leverages local spatial attention for efficient whole-slide encoding, achieving an 18% improvement over the best baseline on HEST-1k and STImage-1K4M.
Aligning Protein Conformation Ensemble Generation with Physical Feedback: This work proposes Energy-based Alignment (EBA), which integrates energy feedback from physical force fields into the fine-tuning process of diffusion generative models. By aligning the generative distribution with the physical energy landscape via a Boltzmann factor-weighted classification objective, the method achieves state-of-the-art (SOTA) performance in protein conformation ensemble generation on the ATLAS MD benchmark.
Geometric Representation Condition Improves Equivariant Molecule Generation: GeoRCG proposes a two-stage molecule generation framework: first generating a low-dimensional geometric representation (informative representation), and then generating the complete molecule conditioned on this representation. This achieves an average improvement of 50% on conditional molecule generation tasks, while reducing the number of diffusion steps from 1000 to 100.
Improved Off-policy Reinforcement Learning in Biological Sequence Design: This paper proposes $\delta$-Conservative Search ($\delta$-CS), a novel off-policy search method for biological sequence design. By applying token-level noise injection (random masking with probability $\delta$) to high-scoring offline sequences and then denoising them with a GFlowNet policy, while adaptively adjusting the degree of conservatism based on proxy model uncertainty, $\delta$-CS significantly outperforms existing methods on DNA, RNA, protein, and peptide design tasks.
UniMoMo: Unified Generative Modeling of 3D Molecules for De Novo Binder Design: UniMoMo is proposed as the first unified 3D binder design framework for three types of molecules: small molecules, peptides, and antibodies. It uses "Graph of Blocks" as a unified representation, an iterative all-atom autoencoder for compressing the latent space, and an E(3)-equivariant diffusion model for generation, outperforming domain-specific models across three benchmarks.

Browse all 48 Computational Biology papers →

⚛️ Physics & Scientific Computing (20)¶

The Dark Side of the Forces: Assessing Non-Conservative Force Models for Atomistic Machine Learning: This work systematically assesses the catastrophic consequences of non-conservative machine learning interatomic potentials (which directly predict forces instead of deriving them from potential energy) in geometry optimization and molecular dynamics, and proposes a hybrid conservative/non-conservative model that balances efficiency and physical correctness using a multiple-timestep (MTS) scheme.
Causal-PIK: Causality-based Physical Reasoning with a Physics-Informed Kernel: Proposes Causal-PIK, which encodes physical causal similarity into a Physics-Informed Kernel for Bayesian optimization. This enables agents to find optimal actions with very few attempts in physical reasoning tasks, outperforming SOTA on the Virtual Tools and PHYRE benchmarks.
Finetuning Stellar Spectra Foundation Models with LoRA: This work applies LoRA to the stellar spectra foundation model SpecCLIP for the first time, achieving efficient adaptation of models pre-trained on LAMOST/Gaia XP to DESI survey data with only approximately 100-200 labeled samples, demonstrating that LoRA is a lightweight and effective strategy for cross-survey spectral migration.
OmniArch: Building Foundation Model For Scientific Computing: OmniArch is the first scientific computing foundation model uniformly pre-trained on 1D-2D-3D PDEs. It addresses multi-scale challenges via a Fourier encoder-decoder, handles multi-physical quantity couplings using a Temporal Mask mechanism, and aligns physical priors with a PDE-Aligner, achieving SOTA performance on 11 families of PDEs in PDEBench.
Teaching LLMs to Speak Spectroscopy: Using only 16 GPU hours and 0.04% parameter adaptation via LoRA, LLaMA-3.1-8B is adapted to predict planetary/galaxy redshifts from spectral data while retaining over 85% of its natural language capabilities, demonstrating that general LLMs can be efficiently adapted to non-textual scientific modalities.
Causal Discovery of Latent Variables in Galactic Archaeology: Utilizing the Rank-based Latent Causal Discovery (RLCD) algorithm, this study automatically recovers two physically meaningful latent variables—birth radius and guiding radius—from only five observable stellar properties in a purely data-driven manner. This validates the potential of causal discovery methods to identify hidden physical quantities in astrophysics.
Closed-form Symbolic Solutions: A New Perspective on Solving Partial Differential Equations: This paper proposes the SymPDE framework, which utilizes deep reinforcement learning to directly search for closed-form symbolic solutions to PDEs, bypassing the issues of insufficient numerical accuracy and poor interpretability of PINNs. It achieves a 90% recovery rate on Poisson and heat equations.
Compact Matrix Quantum Group Equivariant Neural Networks: This paper extends group equivariant neural networks to the setting of compact matrix quantum groups, characterizing the weight matrices of such networks using Woronowicz's formulation of Tannaka-Krein duality, thereby providing a theoretical foundation for learning data on noncommutative geometries.
Differentiable Stellar Atmospheres with Physics-Informed Neural Networks: This work proposes Kurucz-a1, a physics-informed neural network (PINN) designed to simulate 1D stellar atmosphere models under the Local Thermodynamic Equilibrium (LTE) assumption. It resolves the key bottleneck of non-differentiable atmospheric structure solvers in differentiable stellar spectroscopy, outperforming the classic ATLAS-12 code in hydrostatic equilibrium and solar spectrum consistency.
Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systems: The authors propose Erwin, a Transformer architecture based on a hierarchical ball tree structure. By restricting attention computation within fixed-size local spherical regions, Erwin achieves linear time complexity. Meanwhile, it captures multi-scale features through progressive coarsening/refinement and cross-ball interaction mechanisms, achieving SOTA performance in multiple domains including cosmology, molecular dynamics, PDE solving, and particle fluid dynamics.

Browse all 20 Physics & Scientific Computing papers →

📡 Signal & Communications (3)¶

Large Language Model (LLM)-enabled In-context Learning for Wireless Network Optimization: This paper proposes a base station power control algorithm based on LLM In-context Learning (ICL). By leveraging natural language task descriptions and experience-pool-driven exemplar selection, it achieves performance close to traditional deep reinforcement learning without updating model parameters.
Fourier Position Embedding: Enhancing Attention's Periodic Extension for Length Generalization: By extending each dimension in RoPE from a single frequency to a Fourier series representation and clipping undertrained low-frequency components, this work achieves reliable periodic extension of the attention mechanism, thereby significantly enhancing the length generalization capability of LLMs.
Deep Electromagnetic Structure Design Under Limited Evaluation Budgets: Proposes the Progressive Quadtree-based Search (PQS) method, which compresses the high-dimensional design space of electromagnetic structures via a hierarchical quadtree representation and utilizes a consistency-based sample selection mechanism to efficiently search for high-quality designs under a limited simulation budget, saving 75–85% of evaluation costs compared to generative methods.

👥 Social Computing (6)¶

OR-Bench: An Over-Refusal Benchmark for Large Language Models: This work proposes OR-Bench, the first large-scale over-refusal benchmark for LLMs. It contains 80K safe prompts that are prone to being falsely refused, revealing a strong trade-off between safety and over-refusal with a Spearman correlation coefficient of up to 0.89.
DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts: DEFAME is proposed, which is a modular, zero-shot multimodal LLM pipeline. By using a six-stage dynamic workflow (Plan -> Execute -> Summarize -> Develop -> Predict -> Justify) combined with external multimodal tool retrieval for evidence, it achieves end-to-end joint text-image fact-checking, reaching new SOTA performance on three benchmarks: AVeriTeC, MOCHEG, and VERITE.
Dynamical Phases of Short-Term Memory Mechanisms in RNNs: This work discovers two distinct underlying dynamical mechanisms supporting short-term memory in RNNs—slow-point manifolds and limit cycles. It analytically derives the power-law scaling laws of their maximum learnable learning rates using toy models (SP: $\beta$ approx. 4-5 vs LC: $\beta$ approx. 2-3), and provides large-scale empirical validation by training approximately 80,000 RNNs.
Learning Survival Distributions with the Asymmetric Laplace Distribution: This paper proposes a parametric survival analysis method based on the asymmetric Laplace distribution (ALD). By using a neural network to learn the three parameters of the ALD (location, scale, and asymmetry), it achieves continuous, closed-form estimation of the survival distribution, comprehensively outperforming existing parametric and non-parametric approaches in both discriminative and calibration performance.
Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing: This paper proposes the GETA framework, which integrates Computerized Adaptive Testing (CAT) from psychometrics with Automatic Item Generation (AIG). Utilizing a variational IRT model and an LLM-driven item generator, GETA dynamically probes the value boundaries of LLMs to address the "evaluation chronoeffect" (data leakage and difficulty saturation) inherent in static benchmarks.
When Bad Data Leads to Good Models: This paper proposes a "pre-training/post-training co-design" perspective, demonstrating through controlled experiments that incorporating a moderate amount of toxic data (~10%) into pre-training data actually reduces the entanglement of toxic features. This makes the model easier to detoxify during post-training (e.g., via ITI activation steering), ultimately reducing toxicity on ToxiGen from 41.40 to 2.63 while maintaining language capabilities.

📂 Others (91)¶

Modern Methods in Associative Memory: A systematic tutorial by the IBM & MIT team, extending Dense Associative Memory (DenseAM) from classic Hopfield networks to modern AI architectures. It unifies AM with Transformer attention and diffusion models through an energy function framework, revealing deep connections, accompanied by mathematical derivations and programming exercises.
Revisiting the Predictability of Performative, Social Events: This paper leverages modern learning theory tools (performative prediction + outcome indistinguishability) to answer a classic 20th-century question in social science: Can social events still be accurately predicted when predictions actively influence outcomes? The answer is affirmative—yet such "accurate" predictions can be entirely useless.
Position: AI Evaluation Should Learn from How We Test Humans: Proposes systematically introducing the adaptive testing paradigm from human psychometrics into AI evaluation, achieving efficient and reliable model capability assessment by estimating item characteristics (difficulty, discrimination, and guessing factor), reconstructing full benchmark scores with only 3% of the questions.
LapSum -- One Method to Differentiate Them All: Ranking, Sorting and Top-k Selection: The authors propose LapSum, a unified framework for four major differentiable sorting tasks (differentiable ranking, sorting, top-k selection, and permutation matrices) based on a closed-form invertible formula of the sum of cumulative density functions of the Laplace distribution. It operates with a time complexity of only $O(n\log n)$ and $O(n)$ space complexity, significantly outperforming existing methods in large-scale scenarios.
Position: Solve Layerwise Linear Models First to Understand Neural Dynamical Phenomena: Proposes the Dynamical Feedback Principle and demonstrates that layerwise linear models are sufficient to provide a unified explanation for four major deep learning dynamical phenomena: neural collapse, emergence, lazy/rich regimes, and grokking. The authors advocate prioritizing the study of layerwise structures over non-linear activations.
Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry: This paper proposes using manifold capacity and its associated geometric metrics (GLUE) to characterize the richness of feature learning. This approach goes beyond the traditional lazy vs. rich dichotomy, revealing new insights into different learning phases, learning strategies, computational neuroscience, and OOD generalization.
General Agents Contain World Models: This work theoretically proves that any agent capable of generalizing across multi-step goal-conditioned tasks must implicitly learn a predictive model of its environment (a world model), and this model can be extracted from the agent's policy—the stronger the agent and the more complex the goals, the more accurate its implicit world model.
Truly Self-Improving Agents Require Intrinsic Metacognitive Learning: This paper proposes a formal framework demonstrating that truly self-improving agents require intrinsic metacognitive learning capabilities (rather than extrinsic, human-designed fixed loops). The framework comprises three components: metacognitive knowledge, metacognitive planning, and metacognitive evaluation. It also analyzes the limitations of existing self-improving agents and outlines paths toward achieving intrinsic metacognition.
On the Importance of Gaussianizing Representations: Based on information-theoretic motivations (the normal distribution is simultaneously the optimal signal and the worst-case noise distribution), this paper proposes the Normality Normalization layer. After conventional normalization, activation values are Gaussianized using a Power Transform, and scaled Gaussian noise is injected for regularization. This universally improves generalization and robustness across ViTs and ResNets without introducing additional learnable parameters.
Democratic AI is Possible. The Democracy Levels Framework Shows How It Might Work: This paper proposes the "Democracy Levels" framework, which categorizes the transfer of AI decision-making authority from unilateral power to democratic systems into six levels (L0–L5). Equipped with a multidimensional evaluation system and practical tools, it provides a systematic roadmap for the democratization of AI governance.

Browse all 91 Others papers →