CVPR2026 CVPR2026 accepted papers CVPR2026 paper list AI paper notes top conference papers 3D Vision Image Generation Multimodal VLM Video Understanding Video Generation Medical Imaging Autonomous Driving Human Understanding

📷 CVPR2026 Accepted Papers¶

4067 CVPR2026 paper notes covering 3D Vision (751), Image Generation (492), Multimodal VLM (419), Video Understanding (187), Video Generation (182), Medical Imaging (173), Autonomous Driving (157), Human Understanding (151) and other 46 areas. Each note has TL;DR, motivation, method, experiments, highlights, and limitations — 5-minute reads of core ideas.

💡 LLM Reasoning (16)¶

Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs: This work proposes VISTA-Gym, a scalable training environment for visual tool agents (comprising 7 task categories, 13 datasets, and 26 standardized visual tools). Within this environment, the authors train VISTA-R1 using a "Behavioral Cloning (BC) warm-up + multi-round online GRPO" paradigm. This enables 8B-scale VLMs to dynamically select, invoke, and coordinate visual tools during reasoning, outperforming SOTA models of similar scale by 9.51%–18.72% across 11 reasoning-intensive VQA benchmarks.
Latent Chain-of-Thought World Modeling for End-to-End Autonomous Driving: LCDrive proposes the Latent Chain-of-Thought (Latent CoT) framework, which replaces natural language CoT for reasoning with action proposal tokens and world model prediction tokens. Through cold-start and RL post-training, it achieves lower latency and superior trajectory quality for end-to-end autonomous driving.
Revisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalization: The authors systematically compare three "think with image" supervision formats—Language CoT, Grounding CoT, and Visual CoT—using a controlled maze navigation task. They find that longer or more elaborate Visual CoTs only accelerate convergence without raising the final performance ceiling. Conversely, a minimalist CoT preserving only essential grounding information (a single coordinate path) achieves the best generalization. The paper proposes the "short is long" effect and provides a practical guide for constructing generalizable visual reasoning SFT data.
Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models: TaYS transforms the video reasoning of Large Vision-Language Models (LVLMs) from a "batch" paradigm (look-at-all-then-think) to a "streaming" paradigm (think-while-looking). By utilizing a streaming attention mask, decoupled positional encoding, and a parallel dual KV cache, reasoning proceeds incrementally in synchronization with video frames. On VideoEspresso, the Time-to-First-Token (TTFT) is reduced from 10.6s to near zero, reasoning-event deviation is lowered by 55%, and reasoning accuracy is improved by 2.9%.
ReLaX: Reasoning with Latent Exploration for Large Reasoning Models: ReLaX abandons the practice of forcibly increasing token-level entropy to counteract entropy collapse in RLVR. Instead, it utilizes the Koopman operator to linearize the latent state dynamics of large reasoning models and introduces "Dynamic Spectral Divergence (DSD)" to quantify internal computational flexibility. By integrating DSD into the GRPO objective, it achieves new SOTA performance on 7 multimodal and 6 text-based reasoning benchmarks.
FireScope: Wildfire Risk Raster Prediction with a Chain-of-Thought Oracle: A VLM (Oracle) fine-tuned with GRPO and Chain-of-Thought (CoT) reasoning first infers a scalar wildfire risk score from satellite imagery and climate data. Then, FiLM is used to feed this score into a lightweight vision Encoder-Decoder to generate a high-resolution continuous risk raster. In a "US training, Europe testing" cross-continent setting, explicit linguistic reasoning significantly improves out-of-distribution (OOD) generalization, and the reasoning traces are interpretable and recoverable by wildfire experts.
E-comIQ-ZH: A Human-Aligned Dataset and Benchmark for Fine-Grained Evaluation of E-commerce Posters with Chain-of-Thought: Constructed the first multi-dimensional quality evaluation framework for Chinese e-commerce posters, E-comIQ-ZH, consisting of an 18K expert-annotated dataset (including CoT reasoning chains), a dedicated evaluation model E-comIQ-M (trained via SFT+GRPO), and a standardized benchmark E-comIQ-Bench.
Reasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMs: This paper utilizes a latent space learned via a VAE to inject a "reasoning palette" into (V)LMs. Each sampled latent variable is decoded into a learnable prefix prepended to the prompt, enabling the model to select a specific reasoning style before generating the first token. This approach upgrades "token-level random sampling" in RL to "strategy-level structured exploration," consistently outperforming standard GRPO/RLOO on multiple mathematical reasoning benchmarks.
APPO: Attention-guided Perception Policy Optimization for Video Reasoning: APPO identifies that "the bottleneck of video reasoning lies in perception rather than reasoning." It leverages the model's own attention on video frames to convert sparse outcome rewards into token-level dense rewards. By applying differential weighted learning to "intra-group perception tokens" that focus on the same key frames across different responses based on reward disparities, it consistently outperforms GRPO and DAPO on Qwen2.5-VL-3/7B by 0.5%–4%.
EagleVision: A Dual-Stage Framework with BEV-grounding-based Chain-of-Thought for Spatial Intelligence: The proposed EagleVision is a dual-stage framework. In the macro-perception stage, it utilizes Semantic-Perspective Fusion DPP (SPF-DPP) to jointly optimize semantic relevance and perspective diversity in \(SE(3)\) space for keyframe selection. In the micro-verification stage, the model actively queries new perspective frames on a BEV plane to conduct iterative spatial CoT reasoning (hypothesis \(\rightarrow\) view \(\rightarrow\) verification loop). The query strategy is trained purely via RL without human annotation, achieving open-source SOTA on VSI-Bench and SQA3D.

Browse all 16 LLM Reasoning papers →

🦾 LLM Agent (42)¶

JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimization: JarvisEvo integrates a professional retouching designer into a single-model Agent. It iteratively edits photos by invoking Lightroom tools while performing visual self-evaluation on intermediate results to reflect and correct errors. Powered by a dual-loop RL framework (SEPO) combining "editor self-rewarding" and "evaluator human-label calibration," it achieves self-evolution without relying on external reward models, outperforming Nano-Banana by 44.96% in pixel fidelity on ArtEdit-Bench.
Agent4FaceForgery: Multi-Agent LLM Framework for Realistic Face Forgery Detection: A multi-agent system driven by LLMs is used to "act" as both forgers and social network observers, simulating the complete life cycle of face forgery from creation to propagation. It synthesizes training data with text-image consistency annotations, leading to significant performance gains for deepfake detectors in cross-domain and cross-algorithm real-world scenarios (e.g., Celeb-DF AUC improved from the 70% range to 87.1%).
REALM: An MLLM-Agent Framework for Open World 3D Reasoning Segmentation and Editing on Gaussian Splatting: The REALM framework is proposed, leveraging the reasoning capabilities of MLLMs through a global-to-local spatial positioning strategy to perform open-world 3D reasoning segmentation on 3DGS. It handles implicit instructions without 3D post-training, achieving 92.88% mIoU on LERF (surpassing baselines by over 40 percentage points) while supporting editing tasks such as object removal, replacement, and style transfer.
HATS: Hardness-Aware Trajectory Synthesis for GUI Agents: Ours proposes HATS, a hardness-aware trajectory synthesis framework. Through a closed-loop mechanism of hardness-driven exploration and alignment-guided refinement, it focuses on collecting and correcting training trajectories with semantically ambiguous actions, significantly enhancing the generalization capabilities of GUI Agents in complex real-world scenarios.
HAVEN: Hierarchical Long Video Understanding with Audiovisual Entity Cohesion and Agentic Search: HAVEN proposes a unified framework featuring audiovisual entity cohesion + hierarchical indexing + agentic search. By utilizing speaker identity as a cross-modal consistency signal, it constructs a four-level hierarchical database (Global-Scene-Clip-Entity), achieving SOTA with an overall accuracy of 84.1% on LVBench.
OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic Models: Addressing the lack of reliable step-by-step error detectors for "screen-based" GUI agents, OS-Oracle introduces a data pipeline that automatically synthesizes four types of typical error actions from positive trajectories. This generates 310,000 critic samples used to train a 7B critic model through two-stage SFT and Consistency-Preserving GRPO (CP-GRPO). The work also provides OS-Critic Bench, the first human-annotated critic benchmark covering Mobile, Web, and Desktop platforms. The model achieves SOTA among open-source models and demonstratedly improves the success rate of the UI-TARS agent.
Think, Then Verify: A Hypothesis-Verification Multi-Agent Framework for Long Video Understanding: VideoHV-Agent is proposed to refactor long video question answering into a "hypothesis-verification" process: the Thinker rewrites answer options into testable hypotheses, the Judge extracts discriminative clues, the Verifier localizes evidence within the video for validation, and the Answerer synthesizes evidence to provide the final result. It achieves SOTA on EgoSchema, NextQA, and IntentQA while maintaining higher inference efficiency than existing agent-based methods.
Refer-Agent: A Collaborative Multi-Agent System with Reasoning and Reflection for Referring Video Object Segmentation: Refer-Agent decomposes Referring Video Object Segmentation (RVOS) into a step-by-step reasoning pipeline of "frame selection → intent analysis → object localization → mask generation." It further integrates a dual-stage Chain-of-Reflection (Existence Reflection + Consistency Reflection) composed of a Questioner-Responder pair to alternate between reasoning and reflection for self-correction. Without any training and using only a 9B open-source MLLM, it outperforms SFT methods and GPT-4o-based zero-shot methods across five RVOS benchmarks.
Symphony: A Cognitively-Inspired Multi-Agent System for Long-Video Understanding: Symphony mimics human cognition by decomposing long-video understanding into multiple specialized agents based on "capability dimensions" (Planning, Reflection, Grounding, Caption, and Visual Perception). It employs an Actor-Critic-style reflection-enhanced dynamic collaboration mechanism to iteratively correct reasoning and introduces a grounding agent that "expands queries first, then scores with VLM" for complex problems. It achieves SOTA on LVBench, LongVideoBench, Video-MME, and MLVU, outperforming the previous best on LVBench by 5.0%.
CGL: Advancing Continual GUI Learning via Reinforcement Fine-Tuning: Aiming at the "learning new while forgetting old" problem of GUI agents under frequent app updates, this paper discovers that SFT learns quickly but overwrites old knowledge, while RL (GRPO) resists forgetting but learns slowly. Therefore, the CGL framework is proposed—using "error-aware routing + entropy-regulated weighting + conditional gradient surgery" to integrate SFT and GRPO, achieving the highest accuracy and near-zero forgetting on the self-built AndroidControl-CL benchmark.

Browse all 42 LLM Agent papers →

👥 Multi-Agent (2)¶

Visual Document Understanding and Reasoning: A Multi-Agent Collaboration Framework with Agent-Wise Adaptive Test-Time Scaling: MACT decomposes the "monolithic single-model" visual document QA into four agents with distinct roles: planning, execution, judging, and answering. It adaptively allocates test-time compute according to the cognitive load of each agent rather than uniformly increasing parameters. On 15 benchmarks, it consistently ranks in the top three with <30B parameters, achieving an average improvement of 9.9–11.5% over the base models.
AgentDet: A Shared-Blackboard Multi-Agent Framework for Zero-/Few-Shot Object Detection: AgentDet decomposes zero-/few-shot object detection into four LLM agents: Scout, Pinner, Curator, and Judge. These agents collaborate via a "Shared Blackboard" and a patch-level "Knowledge Base" (KB). The framework fragments visual evidence into the KB, assembles them into holistic textual clues for LLM-based box prediction, and trains only the Judge agent. It achieves competitive results on PASCAL VOC and COCO for both ZSOD and FSOD tasks.

⚖️ Alignment & RLHF (12)¶

SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimization: SafeGRPO integrates "verifiable rule-governed rewards" into GRPO, allowing Multimodal Large Language Models (MLLMs) to learn self-rewarded safety through a "step-guided reasoning process" (analyzing visual, text, and combined risks) without manual preference annotations. This approach enhances jailbreak defense, safety awareness, and stability across multiple safety benchmarks while minimizing degradation of general capabilities and avoiding excessive refusal.
EcoAlign: An Economically Rational Framework for Efficient LVLM Alignment: EcoAlign reframes the inference-time alignment of Large Vision-Language Models (LVLMs) as an "optimal path search problem under a limited compute budget." It utilizes a Net Present Value (NPV)-like look-ahead function to score candidate actions on a dynamically constructed Graph-of-Thought, balancing safety, utility, and cost while defining path safety via the "weakest link" principle to achieve superior safety and utility at lower compute costs.
Bridging Human Evaluation to Infrared and Visible Image Fusion: To address the long-standing issue of Infrared and Visible Image Fusion (IVIF) optimizing only handcrafted metrics and disconnecting from human aesthetics, this paper constructs the first large-scale IVIF human feedback dataset. It trains a "fusion-oriented reward model" to quantify perceptual quality and utilizes SAM-assisted GRPO to align the fusion network with human preferences, achieving SOTA performance on mainstream benchmarks with more visually pleasing fusion results.
Principled Steering via Null-space Projection for Jailbreak Defense in Vision-Language Models: Ours proposes NullSteer, an activation steering defense framework based on null-space projection. By restricting steering operations within the null space of benign activations, it effectively defends against visual jailbreak attacks without compromising the model's general capabilities.
Thinking with Frames: Generative Video Distortion Evaluation via Frame Reward Model: REACT is a frame-level reward model targeting "structural distortion" in generated videos. It establishes a taxonomy of eight distortion categories and labels 15,000 pairs of frame preference data. Using grounding reconstruction combined with Gemini-2.5-Pro, it synthesizes 6K CoT samples at low cost. Qwen2.5-VL-7B is trained in two stages via "Masked SFT + GRPO pairwise reward." During inference, a dynamic sampling mechanism focuses on frames most likely to be distorted, significantly outperforming existing video/image evaluators in both preference alignment and distortion identification.
Video-CoE: Reinforcing Video Event Prediction via Chain of Events: Addressing the issues where Multi-modal Large Language Models (MLLMs) lack logical reasoning and ignore visual content in Video Event Prediction (VEP), this paper proposes the Chain of Events (CoE) paradigm. It requires the model to segment videos into timestamped historical event chains and perform causal reasoning based on them. Through a two-stage training process (CoE-SFT for reasoning injection + CoE-GRPO for reinforcing event chain construction via dense rewards), Qwen2.5-VL-7B was improved from 52.9% to 75.0% on FutureBench, setting a new VEP SOTA.
DRM: Diffusion-based Reward Model With Step-wise Guidance: This paper utilizes the pre-trained diffusion model itself as the reward model backbone (DRM). By leveraging its unique ability to score noise latents at any denoising step, the authors design Step-GRPO for training with dense step-wise rewards and Step-wise Sampling for "explore-and-select" during inference. This approach significantly improves the generation quality of SD3.5-Medium without adding parameters and achieves 2.5–3.5 times faster convergence.
MorphSeek: Fine-grained Latent Representation-Level Policy Optimization for Deformable Image Registration: MorphSeek redefines deformable medical image registration as "policy optimization in the encoder's latent space"—attaching a Gaussian policy head to the top layer of a U-Net encoder to treat latent features as samplable actions. It first uses unsupervised warm-up to stabilize the latent space, then employs GRPO for multi-trajectory multi-step weakly supervised fine-tuning. Combined with LDVN to stabilize policy gradients in the tens-of-thousands-dimensional latent space, it improves Dice by 2–4% and reduces the folding rate (NJD) by 30–60% on three 3D registration benchmarks using minimal labels.
Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models: UE-DPO shifts the optimization focus for hallucination suppression in Multimodal Large Language Models (MLLMs) from "visually sensitive tokens that the model already understands" to "critical cognitive blind-spot tokens that the model fails to comprehend." By quantifying these blind spots with token-level epistemic uncertainty, UE-DPO asymmetrically adjusts DPO gradient intensities for preferred and dispreferred branches. It outperforms similar methods like TPO and V-DPO on multiple hallucination benchmarks using significantly less data.
Anchoring the Mind of Multimodal Reasoners: Cognitive Bias as a Vector for Jailbreak Attacks: This paper discovers an "anchoring effect" in the safety judgments of Multimodal Large Reasoning Models (MLRMs)—where the model is significantly biased by the first information it encounters. Based on this, RA-Attack is proposed: it first anchors the model's reasoning chain to a "safe tone" using a "seemingly safe" structured mind map and educational context text, then smoothly packages harmful intent as a natural extension of this reasoning chain. It achieves SOTA Attack Success Rates (ASR) of 92% (Gemini-2.5-Pro) and 82% (GPT-4o) across 7 mainstream MLRMs.

Browse all 12 Alignment & RLHF papers →

🔒 LLM Safety (12)¶

Elastic Weight Consolidation Done Right for Continual Learning: This paper systematically analyzes the fundamental flaws of EWC and its variants in weight importance estimation from a gradient perspective (gradient vanishing in EWC and redundant protection in MAS). It proposes an extremely simple Logits Reversal operation to correct the Fisher Information Matrix (FIM) calculation, significantly outperforming the original EWC and its variants in exemplar-free class-incremental learning and multimodal continual instruction tuning tasks.
SineProject: Machine Unlearning for Stable Vision–Language Alignment: Addressing the issue where the Jacobian of the projector layer becomes severely ill-conditioned during machine unlearning in Multimodal Large Language Models (MLLMs), leading to vision-language alignment drift, SineProject is proposed. By applying a sine modulation (\(\sin(\Delta W)\)) to the projector weights, the parameter range is constrained to \([-1,1]\), reducing the Jacobian condition number by 3-4 orders of magnitude. This enables complete forgetting of target knowledge while reducing the Safe Answer Refusal Rate (SARR) for benign queries by 15%.
The Blind Spot of Adaptation: Quantifying and Mitigating Forgetting in Fine-tuned Driving Models: This paper systematically investigates the catastrophic forgetting issue when fine-tuning VLMs for autonomous driving. It constructs FidelityDrivingBench, a large-scale benchmark with \(180\text{K}\) scenarios, and proposes the Drive Expert Adapter (DEA), which enhances driving task performance via prompt-space routing without corrupting base parameters.
AutoDebias: An Automated Framework for Detecting and Mitigating Backdoor Biases in Text-to-Image Models: AutoDebias is proposed as the first unified framework to simultaneously detect and mitigate malicious backdoor biases in T2I models. By leveraging VLM open-set detection to identify trigger-bias associations and constructing lookup tables, combined with CLIP-guided distribution alignment training, it reduces the attack success rate from 90% to near zero across 17 backdoor scenarios while maintaining image quality.
Designing to Forget: Deep Semi-parametric Models for Unlearning: This paper proposes the "Designing to Forget" philosophy, introducing a family of Deep Semi-parametric Models (SPM). By simply removing training samples at inference time without modifying model weights, SPM reduces the prediction gap compared to retraining baselines by 11% on ImageNet and accelerates unlearning by more than 10x.
Learning from Oblivion: Predicting Knowledge-Overflowed Weights via Retrodiction of Forgetting: The authors propose KNOW prediction: inducing a structured forgetting process through sequential fine-tuning on progressively smaller nested data subsets to collect weight transition trajectories, and then using a meta-learned hyper-model (KNOWN) to reverse the direction of forgetting. This predicts virtual knowledge-enhanced weights as if they were trained on larger datasets. Across multiple datasets (CIFAR/ImageNet/PACS, etc.) and architectures (ResNet/PVTv2/DeepLabV3+, etc.), the method consistently outperforms naive fine-tuning and various weight prediction baselines, showing significant improvements in downstream tasks such as image classification, semantic segmentation, image captioning, and domain generalization.
Select, Hypothesize and Verify: Towards Verified Neuron Concept Interpretation: The SIEVE (Select–Hypothesize–Verify) framework is proposed to interpret neuron functions through a closed-loop process involving high-activation sample screening, concept hypothesis generation, and text-to-image verification. The probability of generated concepts matching neuron activation is approximately 1.5 times that of existing SOTA methods.
⊘ Source Models Leak What They Shouldn't ↛: Unlearning Zero-Shot Transfer in Domain Adaptation Through Adversarial Optimization: This paper identifies that Source-Free Domain Adaptation (SFDA) methods inadvertently leak knowledge of source-exclusive classes to the target domain (zero-shot transfer). It proposes the SCADA-UL framework, which concurrently performs class unlearning during domain adaptation by adversarially generating forgotten samples and employing a rescaled labeling strategy, achieving unlearning performance comparable to training from scratch.
Machine Unlearning via Adaptive Gradient Reweighting and Multi-stage Objective Optimization: To address the issues of "uniform treatment of all samples/categories" and "gradient conflicts between forgetting and retaining objectives" in machine unlearning, this paper proposes Adaptive Gradient Reweighting (weighting based on sample memory depth/category vulnerability) combined with Three-stage Objective Optimization (direction rectification → temporal smoothing → adaptive combination). On CIFAR-10/100 and Tiny-ImageNet, the Avg Gap for random forgetting is reduced from the SOTA 0.85 to 0.19.
Omni-Attack: Adversarial Attacks on Open-Ended VQA in Black-Box Multimodal LLMs: Addressing the gaps where "open-ended VQA/OCR tasks lack explicit attack targets and existing adversarial robustness evaluations use fragmented protocols," this paper first establishes a unified targeted attack benchmark AdvRobustBench (1,000 items, VQA+OCR). It then proposes Omni-Attack, a transferable black-box attack using LLMs to generate "question-conditioned" textual/visual targets, OCR location-aware perturbations, and four transfer regularizations. It achieves a 71.8% targeted attack success rate on GPT-4.1 with \(\epsilon=8/255\).

Browse all 12 LLM Safety papers →

👻 Hallucination Detection (33)¶

Cross-Modal Attention Calibration for LVLM Hallucination Mitigation: To mitigate hallucinations in LVLMs, this paper proposes CMAC, a training-free cross-modal attention calibration framework. It uses the IMD module to perform "surgical" masking of high cross-modal weight value vectors in the attention layer to construct a more accurate hallucination distribution for contrastive decoding. Additionally, the CMPC module scales the position indices of image tokens to alleviate the position bias introduced by RoPE. CMAC consistently outperforms existing contrastive decoding methods across POPE, CHAIR, and MME.
Thinking in Uncertainty: Mitigating Hallucinations in MLRMs with Latent Entropy-Aware Decoding: This paper discovers that hallucinations in Multimodal Large Reasoning Models (MLRMs) are highly concentrated around transition words like because/however/wait, which correspond to high-entropy (high-uncertainty) steps. Consequently, a training-free LEAD decoding strategy is proposed: at high-entropy steps, the single sampled token is replaced with "probability-weighted continuous embeddings" to preserve multiple reasoning hypotheses and inject visual anchors to reinforce visual grounding. At low-entropy steps, the model reverts to standard discrete decoding, consistently reducing hallucinations across multiple MLRMs and benchmarks.
Tell Model Where to Look: Mitigating Hallucinations in MLLMs by Vision-Guided Attention: The authors propose Vision-Guided Attention (VGA), a training-free method that leverages the semantic features of visual tokens to construct precise visual localization. It guides the model's attention to relevant visual regions, effectively mitigating hallucinations in MLLMs while maintaining compatibility with FlashAttention.
Residual Decoding: Mitigating Hallucinations in Large Vision-Language Models via History-Aware Residual Guidance: Residual Decoding (ResDec) is proposed—a training-free, plug-and-play decoding strategy that discovers the semantic anchoring stage by analyzing U-shaped JSD patterns in historical token logit distributions. It effectively suppresses language prior hallucinations in LVLMs by aggregating logits from this stage as residual guidance for current decoding, incurring nearly zero additional inference overhead.
Zina: Multimodal Fine-grained Hallucination Detection and Editing: Zina proposes a multimodal fine-grained hallucination detection and editing task, designing a two-stage system (detector MLLM + reviewer MLLM) that delegates token copying to a deterministic function to simplify the model burden. Additionally, the VisionHall dataset is constructed (6.9K manual annotations + 20K graph-structured synthetic data), exceeding GPT-4o by 15.8 points in detection F1.
Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination: This paper revisits LVLM object hallucination using Logit-Lens and discovers that the "attention intensity" for real and hallucinated objects is nearly identical in mid-to-late layers. The key issue is not "how much" the model looks, but whether the high-attention regions decode into the target token. Based on this, hallucinations are categorized into "Visual Uncertainty" and "Contextual Prior." A training-free "Detect-and-Mitigate" framework (LLCC detection + HARM masking + VEED decoding enhancement) is proposed, achieving SOTA on multiple hallucination benchmarks.
Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinations: Ours proposes a patch-level LVLM hallucination detection framework, discovering that hallucinated tokens exhibit dispersed attention patterns and low semantic alignment. Based on these signatures, Attention Dispersion Score (ADS) and Cross-modality Grounding Consistency (CGC) are designed as lightweight metrics, achieving a detection accuracy of 90%.
Lyapunov Probes for Hallucination Detection in Large Foundation Models: (M)LLMs are viewed as high-dimensional dynamical systems evolving in representation space, and "hallucinations" are redefined as cases where inputs fall into unstable knowledge boundary regions rather than stable equilibrium points. A lightweight probe network with a Lyapunov monotonic decay constraint (taking multi-layer hidden states and perturbation information as input) is used for discrimination, achieving AUPRC scores that consistently outperform ordinary probes by 4–8% across multiple LLMs/MLLMs.
MoD-DPO: Towards Mitigating Cross-modal Hallucinations in Omni LLMs using Modality Decoupled Preference Optimization: MoD-DPO (Modality-Decoupled DPO) is proposed to decouple the contributions of various modalities in Omni LLMs through three mechanisms: invariance regularization, sensitivity regularization, and language prior debiasing. It effectively mitigates cross-modal hallucinations (such as using auditory information to answer visual questions) and derives a closed-form optimal policy.
Understanding and Mitigating Hallucinations in Multimodal Chain-of-Thought Models: This paper systematically analyzes the root causes of hallucinations in Multimodal CoT (MCoT) models. It discovers that hallucinations most frequently occur during reasoning steps involving associative free play (termed "divergent thinking"). Consequently, the authors propose a training-free detection and decoding intervention strategy based on visual entropy. This approach reduces CHAIRS by over 30% on Object HalBench while maintaining or even enhancing general reasoning capabilities.

Browse all 33 Hallucination Detection papers →

⚡ LLM Efficiency (8)¶

Gated KalmaNet: A Fading Memory Layer Through Test-Time Ridge Regression: This work reinterprets the state updates of Linear State Space Models (SSMs) as "performing test-time ridge regression on the entire history." By replacing the one-step gradient approximation in existing SSMs with the exact gain from Kalman filtering and overcoming the dual obstacles of low-precision numerical instability and parallel training via adaptive regularization and Chebyshev iterations, it outperforms linear SSMs like Mamba2 and Gated DeltaNet in short/long context tasks and ImageNet.
Generalizable Video Quality Assessment via Weak-to-Strong Learning: Without relying on any human annotation labels, off-the-shelf VQA models are utilized as "weak teachers" to supervise a high-capacity Multimodal Large Language Model (MLLM) "strong student." The student is then recycled as the teacher for subsequent iterative rounds. The final model matches in-distribution performance and significantly surpasses all teachers in OOD scenarios, improving the overall OOD SRCC of VQA from 0.59 to 0.745.
ParallelVLM: Lossless Video-LLM Acceleration with Visual Alignment Aware Parallel Speculative Decoding: Addressing two major bottlenecks in Video-LLM speculative decoding—"draft and target models waiting for each other" and "trade-off between speedup ratio and model alignment"—ParallelVLM implements both prefilling and decoding as draft/target parallel pipelines. It employs UV-Prune, an unbiased pruning method based on visual-text similarity variations (rather than attention scores), to expand the draft window. This achieves \(3.36\times\) and \(2.42\times\) lossless acceleration on LLaVA-OneVision-72B and Qwen2.5-VL-32B, respectively, while being training-free and plug-and-play.
Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inference: ReMix inserts an iteratively refreshed "continuous mixed state" between the discrete "mask state \(\rightarrow\) token state" transitions in Diffusion Language Models (DLLMs). This allows multiple positions in parallel decoding to coordinate in continuous space before finalizing tokens. By applying a rejection rule to reset unstable positions to masks, the method achieves a 2–8\(\times\) inference speedup without training or performance degradation, frequently even improving accuracy.
E\(^2\)-SCI: Elastic Edge-Cloud Speculative Decoding via Credit Inertia: This paper identifies strong temporal consistency in token acceptance rates across adjacent windows in edge-cloud speculative decoding (termed "Credit Inertia"). Based on this, it dynamically adjusts verification thresholds using historical acceptance rates. Combined with an Asynchronous Pipeline (PLC) that parallelizes draft generation and cloud verification, it achieves 9.4+ tokens/s on DeepSeek-R1-Distill-Qwen (1.5B/32B), representing an 88.5% speedup over the FSD baseline without compromising accuracy.
Few-Shot Hybrid Incremental Learning: Continually Learning under Data Scarcity and Task Uncertainty: This paper proposes "Few-Shot Hybrid Incremental Learning (FSHIL)," a realistic new paradigm where data is scarce and task types (new classes, new domains, or both) appear stochastically. By introducing "Conditional Meta-Expanding Mixture of Experts (CME-MoE)" to reconcile stability and plasticity at the feature level and "Self-Expanding Prototype Classifier (SEPC)" to model multi-distribution boundaries at the classification layer, the method outperforms existing FSIL and HIL approaches across five datasets and three incremental settings.
JUMP-Hand: Learning Joint-wise Uncertainty to Gate Mixture of View Experts for Multi-View 3D Hand Reconstruction: JUMP-Hand reformulates multi-view 3D hand reconstruction as a Mixture of Experts (MoE) problem where "each view is an expert," utilizing joint-wise, view-wise probabilistic uncertainty as an explicit gating signal. This signal drives both uncertainty-weighted triangulation in the coarse stage and uncertainty-gated cross-attention in the refinement stage, adaptively amplifying reliable views while suppressing noisy ones under severe occlusion, achieving SOTA results across three multi-view benchmarks.
QuietPrune: Query-Guided Early Token Pruning for Vision-Language Models: QuietPrune proposes query-guided early pruning: visual tokens unrelated to the text query are pruned during the ViT forward process rather than after it. By utilizing a lightweight adapter initialized through an inverse transformation of the VLM projector, the text query is converted into a visual-domain [Q-CLS] token to provide guidance. Pruning is performed in a 2×2 semi-structured manner with redundant token aggregation. On Qwen3-VL and InternVL3, it reduces prefill latency by up to 19.0% while achieving 4.2% higher accuracy than existing late-pruning methods.

📚 Pretraining (5)¶

Watch and Learn: Learning to Use Computers from Online Videos: The Watch & Learn (W&L) framework is proposed, which automatically transforms human computer-operation videos from the internet into executable UI trajectory data using an Inverse Dynamics Model (IDM). It generates 53K+ high-quality trajectories, significantly improving the performance of various Computer-Using Agents (CUAs) when used as In-Context Learning (ICL) examples or Supervised Fine-Tuning (SFT) data.
Linking Modality Isolation in Heterogeneous Collaborative Perception: The CodeAlign framework is proposed to address the "modality isolation" problem in heterogeneous collaborative perception, where different modalities never co-occur in training data. By constructing discrete code spaces via codebooks and performing cross-modal Feature-Code-Feature (FCF) translation, it achieves SOTA perception performance with only 8% of HEAL's training parameters and a \(1024\times\) reduction in communication volume.
Exploring Visual Pretraining for Learning Language Intelligence: This paper proposes MAPLE: instead of extracting text from PDFs to feed into LLMs, it directly performs masked autoregressive pretraining on document page images. By allowing the LLM to learn language intelligence through "generating latent hypotheses for occluded regions," it achieves an average improvement of up to 40.2% over pure text pretraining across four mathematical reasoning benchmarks.
Reconstructing CLIP for Open-Vocabulary Dense Perception: DenseRC addresses the neglected problem of "how to construct high-quality dense features for CLIP." It reveals that the generalized semantics of the cls token actually derive from multi-layer value embeddings, whereas spatial aggregation tends to amplify semantic misalignment. By using multi-layer values as a foundation and employing a lightweight Head Selection Gating (HSG) for re-weighting solely across the head dimension, the authors construct dense representations aligned with global semantics. DenseRC sets new SOTAs on multiple open-vocabulary detection and segmentation benchmarks.
Unlocking Pre-trained Weights: Parameter Inheritance for Zero-Shot Initialization: PITH utilizes a Graph HyperNetwork to dynamically generate "projection matrices" that map internal weights of large pre-trained models directly onto target ViTs of arbitrary sizes for initialization. This enables the initialized networks to be used immediately without training—achieving a zero-shot accuracy of 53.35% for ViT-Base on ImageNet-1K, which is 6.54% higher than the previous SOTA (TAL).

✏️ Knowledge Editing (2)¶

Attribution-Guided Model Rectification of Unreliable Neural Network Behaviors: This paper proposes an attribution-guided dynamic model rectification framework that repositions rank-one model editing from domain adaptation to behavior rectification. By quantifying layer editability via Integrated Gradients to automatically locate suspect layers, it repairs three types of unreliable behaviors—backdoor attacks, spurious correlations, and feature leakage—using only a single clean sample.
SAME: Sparse and Anchored Model Editing for Heterogeneous Incremental Learning under Limited Data: This work adapts the "locate-then-edit FFN key-value pairs" paradigm from Large Language Models (LLMs) to Vision-Language Models (VLMs) like CLIP. Under a newly proposed "Heterogeneous Incremental Learning (HIL)" setting—characterized by no task identities, cross-domain shifts, and few-shot data—the authors propose sparse fine-tuning, dual-anchor constraints, and closed-form solutions to directly "write" new task knowledge into the FFN output projection matrices. The method requires no additional parameters, achieves 6.8% higher average accuracy than existing continual learning methods, and retains 95.8% of oracle performance.

💬 LLM (Other) (3)¶

Single-step Diffusion-based Video Coding with Semantic-Temporal Guidance: S2VC integrates a single-step diffusion generator into a conditional video coding framework. It replaces the text prompt with "Contextual Semantic Guidance (CSG)" extracted from the decoded feature buffer and utilizes "Temporal Consistency Guidance (TCG)" inserted into the U-Net for cross-frame alignment. It achieves SOTA perceptual quality at extremely low bitrates below 0.02 bpp, saving 51.62% bitrate on average (DISTS BD-Rate) compared to the previous generation of perceptual codecs.
OmniDocLayout: Towards Diverse Document Layout Generation via Coarse-to-Fine LLM Learning: Addressing the limitation that existing document layout generation data are "academic-only with single styles," the authors first create OmniDocLayout-1M, the first million-scale diverse layout dataset covering six document categories. They then employ a 0.5B small LLM using a "coarse-to-fine" paradigm—learning general layout rules on multi-domain coarse labels followed by adapting to specific domains with few fine labels. This approach outperforms both specialized layout models and general large models such as GPT-4o/Gemini/Claude on M6Doc.
LLM-Guided Probabilistic Fusion for Label-Efficient Document Layout Analysis: This paper integrates text-pretrained LLMs as "structural prior generators" into the pseudo-label refinement stage of semi-supervised layout detection. By using OCR+LLM to infer document hierarchical regions and performing inverse variance probabilistic fusion (including learnable instance-adaptive gating) with teacher detector outputs, the method achieves 88.2 AP (lightweight backbone) and 89.7 AP (LayoutLMv3) on PubLayNet using only 5% labels, with the most significant gains observed in rare layout elements such as titles and headers.

🎨 Image Generation (492)¶

OmniGen2: Towards Instruction-Aligned Multimodal Generation: OmniGen2 adopts a unified "decoupled VLM + Diffusion" architecture (where VLM handles understanding and Diffusion handles generation, conditioned on VLM variable-length hidden states and VAE features). By combining Omni-RoPE position encoding with a two-stage training strategy—"building a strong base followed by progressive RL alignment"—the model precisely follows complex instructions across text-to-image, image editing, and in-context generation tasks, achieving a GenEval score of 0.95.
Improved Mean Flows: On the Challenges of Fastforward Generative Models: The paper diagnoses two root causes of failures in MeanFlow (a one-step generation framework): the training objective's dependence on the network itself and the hard-coded CFG guidance scale before training. By rewriting the objective as a network-independent v-loss using the predicted marginal velocity as the JVP input, and treating the guidance scale as a variable condition injected via multi-token in-context conditioning, the proposed iMF achieves a 1.72 FID on ImageNet 256×256 with a single function evaluation (1-NFE) trained from scratch. This represents an approximately 50% relative improvement over the original MeanFlow, approaching the performance of multi-step methods without any distillation.
DDT: Decoupled Diffusion Transformer: DDT splits the traditional "decoder-only" Diffusion Transformer into a dedicated condition encoder for semantic extraction and a dedicated velocity decoder for velocity field regression. This decouples the optimization conflict between "semantic encoding" and "high-frequency decoding." It achieves a 1.31 FID on ImageNet 256×256 in only 256 epochs (approximately 4× faster than REPA) and further accelerates inference by nearly 3× by leveraging dynamic programming to share highly similar self-conditions across adjacent steps.
PixelDiT: Pixel Diffusion Transformers for Image Generation: PixelDiT proposes a dual-layer pixel-space diffusion model based entirely on Transformers: a patch-level DiT captures global semantics and a pixel-level DiT refines texture details. Without a VAE, it achieves 1.61 FID on ImageNet and allows direct training of text-to-image models in 1024 resolution pixel space.
Pico-Banana-400K: A Large-Scale Dataset for Text-Guided Image Editing: The authors utilized Nano-Banana (Gemini-2.5-Flash-Image) to batch-generate approximately 400,000 instruction-based image editing samples on real photos from OpenImages. Using Gemini-2.5-Pro for automated quality inspection, they constructed Pico-Banana-400K, an open-source dataset covering 35 editing types that supports single-turn SFT, preference learning, and multi-turn editing research.
DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation: DeCo proposes a frequency-decoupled pixel diffusion framework that utilizes a lightweight pixel decoder to process high-frequency details, allowing the DiT to focus on low-frequency semantic modeling. Combined with a frequency-aware flow matching loss, it achieves FID scores of 1.62 (256) and 2.22 (512) on ImageNet, narrowing the gap between pixel-space and latent-space diffusion.
MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generation: MakeAnything leverages the in-context capability of Flux (DiT) by arranging multi-frame creative processes (such as drawing, crafting, and cooking) into a grid and fine-tuning with Asymmetric LoRA. This achieves, for the first time, cross-domain "step-by-step tutorial" generation across 21 domains, supporting both text-to-process generation and reconstructing creation steps from a finished image (ReCraft).
GRPO-Guard: Mitigating Implicit Over-Optimization in Flow Matching via Regulated Clipping: This paper discovers that FlowGRPO exhibits a systematic left-shift in importance ratio distributions and inconsistent variance across denoising steps when fine-tuning flow matching models. This causes PPO clipping to fail completely for "overconfident positive samples," leading the model into implicit reward hacking. GRPO-Guard introduces RatioNorm to standardize the ratio back to a mean of 1 and uses \(1/dt\) gradient reweighting to balance step-wise gradients. Without relying on heavy KL regularization, it significantly mitigates over-optimization and preserves generation quality.
DiP: Taming Diffusion Models in Pixel Space: Ours proposes DiP, an efficient pixel-space diffusion framework. By utilizing a DiT backbone to model global structures with large patches and a lightweight Patch Detailer Head to recover local details, it achieves computational efficiency comparable to LDMs without requiring a VAE, reaching a 1.79 FID on ImageNet 256×256.
StreamDiT: Real-Time Streaming Text-to-Video Generation: StreamDiT proposes a comprehensive streaming video generation solution (including training, modeling, and distillation). By introducing a sliding buffer with progressive denoising in Flow Matching and a mixed partition training strategy, combined with a time-variant DiT architecture with window attention and a customized multi-step distillation method, a 4B parameter model achieves real-time streaming video generation at 512p@16FPS on a single GPU.

Browse all 492 Image Generation papers →

🎬 Video Generation (182)¶

Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rollout: ∞-RoPE is proposed as a training-free inference-time framework. Through three components—Block-Relativistic RoPE, KV Flush, and RoPE Cut—it extends an autoregressive video diffusion model trained only on 5-second videos into a system capable of infinite-duration generation, fine-grained action control, and cinematic scene transitions.
Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset: This paper proposes a data synthesis framework named Ditto, which drives an in-context video generator using "image editing priors + depth video" combined with distillation acceleration and VLM agent auto-quality control. Consuming 12k GPU-days, it constructs a million-scale instruction video editing dataset, Ditto-1M. It then utilizes "modality curriculum learning" to train Editto, a model capable of editing videos solely based on text instructions, refreshing the SOTA in instruction-based video editing across both automatic metrics and human evaluations.
Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillation: Reward Forcing distills bidirectional video diffusion models into few-step autoregressive student models. It employs EMA-Sink to compress historical context, preventing "frame copying," and utilizes Re-DMD to bias distribution matching gradients toward high-dynamic samples based on motion quality rewards. It achieves high-quality real-time streaming video generation at 23.1 FPS on a single H100, outperforming all same-scale baselines in VBench total scores.
HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narratives: Based on DiT video diffusion models like Wan2.2, HoloCine employs "Window Cross-Attention" to align each shot with its storyboard text and "Sparse Inter-Shot Self-Attention" to reduce the quadratic complexity of full-sequence self-attention to near-linear. This enables the holistic, one-pass generation of minute-long, character-consistent cinematic narratives with precise transition control.
DriveLaW: Unifying Planning and Video Generation in a Latent Driving World: DriveLaW is proposed, a driving world model that unifies video generation and motion planning via a shared latent space. By directly injecting intermediate latent features from the video generator into a diffusion planner, it achieves SOTA performance simultaneously on nuScenes video prediction and NAVSIM planning benchmarks.
UniAVGen: Unified Audio and Video Generation with Asymmetric Cross-Modal Interactions: UniAVGen proposes a joint audio-video generation framework based on a symmetric dual-branch DiT. By leveraging an asymmetric cross-modal interaction mechanism and a face-aware modulation module, it achieves precise spatio-temporal synchronization. With only 1.3M training samples, it comprehensively outperforms competitors using 30M data in terms of lip-sync, timbre consistency, and emotional consistency.
MultiShotMaster: A Controllable Multi-Shot Video Generation Framework: MultiShotMaster adapts a pre-trained ~1B parameter single-shot T2V model by implementing two types of RoPE (narrative phase shift + spatiotemporal positioning) and an attention mask. It achieves multi-shot video generation with variable shot counts/durations, independent per-shot text, specified subject positioning/motion, and customizable backgrounds without additional adapters. It significantly outperforms CineTrans / EchoShot / VACE / Phantom in text alignment, cross-shot consistency, transition accuracy, and narrative coherence.
Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm: This paper introduces "Thinking with Video," a new multimodal reasoning paradigm where video generation models like Sora-2 are utilized to depict the reasoning process within video frames. The authors construct VideoThinkBench, a five-level capability hierarchy covering "Geometric Intuition → Visual Induction → Abstract Rules → Spatial Planning → Language Reasoning." Evaluation reveals that Sora-2 outperforms GPT-5 by ~10% in "eyeballing" geometry puzzles and achieves 92% accuracy in MATH via audio output, demonstrating that video generation models can serve as unified reasoning vehicles for understanding and generation.
Unified Camera Positional Encoding for Controlled Video Generation: This paper proposes UCPE, which unifies the complete camera geometry (6-DoF pose + intrinsics + lens distortion) into Transformer attention. It leverages "Relative Ray Encoding" to lower the positional encoding from the camera level to the ray level to accommodate non-linear lenses like fisheye and wide-angle. Additionally, "Absolute Orientation Encoding" is introduced to provide global references for pitch and roll. Using a Spatial Attention Adapter with <1% parameters to inject these into pre-trained video DiTs, UCPE achieves state-of-the-art results in both controllability and image quality for camera-controlled text-to-video generation.
LoL: Longer than Longer, Scaling Video Generation to Hour: Addressing the "sink-collapse" phenomenon in autoregressive ultra-long video generation—where the video suddenly reverts to the first few frames—this paper identifies its root cause as "multi-dimensional phase synchronization + multi-head attention homogenization" induced by RoPE periodicity. The authors propose Multi-Head RoPE Jitter, a training-free method that perturbs the RoPE base frequency per head to break such synchronization. Combined with causal VAE sliding window decoding, this work achieves real-time, streaming, and nearly quality-lossless infinite video generation for the first time (demonstrated up to 12 hours).

Browse all 182 Video Generation papers →

🧩 Multimodal VLM (419)¶

VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction: VLM-3R integrates a metric-scale feed-forward 3D reconstruction model (CUT3R) with a VLM. It extracts implicit scene geometry tokens and camera motion tokens from pure monocular video, which are then fused into visual features via cross-attention for instruction tuning. This allows the model to perform spatial and temporal reasoning without relying on depth sensors or pre-built point cloud maps, achieving the highest performance among open-source models on VSI-Bench and the newly proposed VSTI-Bench.
LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning: Addressing the gap where "current Multimodal Large Language Models (MLLMs) almost exclusively follow the autoregressive paradigm and the diffusion path remains unverified," this paper grafts visual instruction tuning onto the masked diffusion language model LLaDA to create a pure diffusion MLLM—LLaDA-V. By leveraging bidirectional attention to better capture visual-spatial relationships, it refreshes the SOTA for pure diffusion MLLMs across 18 benchmarks and outperforms the autoregressive baseline LLaMA3-V on 11 tasks using the same training data.
Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding: Molmo2 is a fully open family of Video-Language Models (weights, data, code, and training recipes are all open, with no data distilled from closed-source VLMs). By building 9 new datasets and utilizing a three-stage training strategy, it fills the missing capability of "video grounding using points and trajectories" even found in closed-source models. The 8B model significantly outperforms comparable open-source models in video counting, pointing, and tracking, even surpassing Gemini 3 Pro in certain tasks.
Scaling Spatial Intelligence with Multimodal Foundation Models: SenseNova-SI cultivates spatial intelligence capabilities in multimodal foundation models (such as Qwen3-VL, InternVL3, and Bagel) by systematically constructing a diverse spatial dataset of 8 million samples (SenseNova-SI-8M). It achieves unprecedented performance on multiple spatial benchmarks like VSI-Bench and MMSI while maintaining general multimodal understanding capabilities.
Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models: Addressing the issue where MLLMs are limited to single-image spatial reasoning and struggle with basic orientations, this paper utilizes annotated 3D/4D scene datasets to automatically generate 27 million multi-frame spatial question-answering pairs (MultiSPA). By injecting foundational capabilities of depth, visual correspondence, and dynamic perception into InternVL2, the trained Multi-SpatialMLLM improves by an average of 36% over the base model on a self-constructed benchmark, matching the performance of closed-source models and specialized 3D models.
OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation: OneCAT integrates "understanding + generation + editing" into the same decoder-only Transformer. By utilizing a Modality-MoE with hard routing (text, understanding, and generation specialists), it achieves encoder-free inference. It also introduces multi-scale auto-regressive generation into LLMs via a Scale-Aware Adapter, attaining SOTA performance in a unified model while delivering approximately 10× faster generation speeds than diffusion models.
TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs: This work systematically investigates the key factors for constructing Video Temporal Grounding (VTG) capabilities in MLLMs. From the dimensions of data quality and algorithmic design, the authors release the high-quality TimeLens-Bench and the TimeLens-100K training set. By adopting an interleaved text-time encoding and a thinking-free RLVR training paradigm, they develop the TimeLens model series, achieving SOTA among open-source models and surpassing GPT-5 and Gemini-2.5-Flash.
ArtiMuse: Fine-Grained Image Aesthetics Assessment with Joint Scoring and Expert-Level Understanding: ArtiMuse utilizes an InternVL-3-8B based Multimodal Large Language Model (MLLM) to simultaneously output 8-dimensional fine-grained expert aesthetic textual analysis and a continuous aesthetic score. By introducing "Token As Score," the model integrates continuous scoring into discrete LLM token generation. It also introduces ArtiMuse-10K, the first dataset with 10,000 expert-annotated samples per dimension, achieving SOTA performance on multiple aesthetic scoring benchmarks.
TUNA: Taming Unified Visual Representations for Native Unified Multimodal Models: TUNA cascades a VAE encoder and a semantic representation encoder to obtain a set of continuous unified visual representations compatible with both "understanding" and "generation." Combined with an autoregressive text head and a flow-matching generation head, a single native model at 1.5B/7B scale achieves SOTA results in image/video understanding, image/video generation, and image editing (MMStar 61.2, GenEval 0.90).
AToken: A Unified Tokenizer for Vision: AToken unifies the encoding of images, videos, and 3D assets into a shared sparse 4D latent space. Utilizing a pure Transformer with non-adversarial Gram loss, it achieves high-fidelity reconstruction and semantic understanding simultaneously. A single model achieves performance competitive with specialized methods across three modalities (Image 0.21 rFID / 82.2% ImageNet, Video 3.01 rFVD, 3D 28.3 PSNR / 90.9% accuracy).

Browse all 419 Multimodal VLM papers →

🧠 VLM Reasoning (150)¶

Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens: This paper proposes the Mirage framework, enabling VLMs to treat their own hidden states as "latent visual tokens" and directly append them into text sequences during decoding. This allows interleaved multimodal reasoning without generating any actual pixel-level images. Combined with a two-stage fine-tuning approach of "first visual grounding, then textual relaxation" and reinforcement learning (RL), Mirage consistently outperforms pure-text decoding and explicit image-generation baselines across multiple benchmarks such as spatial planning, jigsaw puzzles, and spatial relations.
Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning: VITAL equips Multimodal Large Language Models (MLLMs) with a "video clipping" tool, allowing them to densely resample suspicious time intervals into new frames during the reasoning chain to form a "multimodal chain-of-thought." Combined with difficulty-aware DGRPO reinforcement learning to stabilize multi-task training, it achieves 7B-level SOTA performance in long video QA and temporal grounding.
LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling: LongVT enables multimodal large language models to process long videos by emulating the human strategy of "global skimming followed by zooming into suspicious clips." It encapsulates the model's inherent temporal grounding capability into a native crop_video tool, which is interleaved within the reasoning chain to iteratively "re-examine" and correct errors. Supported by the self-constructed VideoSIAH data suite and a three-stage training pipeline, it achieves new open-source SOTA results across four long-video benchmarks.
Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views: 3DThinker enables VLMs to directly generate a sequence of "3D latent tokens" within the reasoning chain and align them with the geometric features of the 3D foundation model VGGT. Without requiring any 3D priors as input or relying on dense annotations, it performs spatial reasoning by "imagining 3D scenes" from limited 2D views. It consistently outperforms strong baselines across 8 spatial understanding benchmarks, with the largest model even surpassing o3.
Grounded Chain-of-Thought for Multimodal Large Language Models: This paper proposes the "Grounded Chain-of-Thought (GCoT)" task and the MM-GCoT benchmark. It requires Multimodal Large Language Models (MLLMs) to provide step-by-step reasoning with coordinate-based grounding before answering. By introducing the "Answer-Grounding Consistency" metric to quantify visual hallucinations, the study reveals that 12 state-of-the-art MLLMs commonly "answer correctly but look at the wrong place," and hallucinations are independent of model scale.
OneThinker: All-in-one Reasoning Model for Image and Video: OneThinker utilizes an 8B model to unify 10 basic visual tasks across image and video (QA, captioning, spatio-temporal grounding, tracking, and segmentation) into a "think-then-structured-output" reasoning paradigm. It introduces EMA-GRPO to resolve optimization imbalances caused by significant differences in reward magnitudes and densities across multiple tasks, outperforming specialized models of comparable size across 31 benchmarks.
G\(^2\)VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning: G2VLM utilizes a "Mixture-of-Transformer-Experts (MoT)" architecture to integrate a feedforward 3D reconstruction expert and a semantic understanding expert within the same VLM. Relying on shared self-attention for mutual reinforcement, this 2B model can directly predict depth, point clouds, and camera poses like VGGT, while outperforming GPT-4o on spatial reasoning tasks (scoring 18.5 points higher on SPAR-Bench).
CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization: This paper discovers that visual agents capable of "thinking with images" often answer correctly but use tools unfaithfully (e.g., cropping the wrong area but guessing the right answer). It proposes CodeV, which represents visual tools as executable Python code and utilizes Tool-Aware Policy Optimization (TAPO) on top of GRPO. TAPO introduces a process-level dense reward that only evaluates tool outputs without inspecting the chain-of-thought. Consequently, CodeV maintains or improves accuracy across 10 benchmarks while increasing the faithful tool-use rate to 1.3–2× that of baselines.
AV-Reasoner: Improving and Benchmarking Clue-Grounded Audio-Visual Counting for MLLMs: Addressing the "counting deficiency" in multimodal large language models (MLLMs), this work introduces CG-AV-Counting—the first interpretable counting benchmark for long videos across audio-visual modalities with fine-grained "counting clue" annotations. Simultaneously, it proposes AV-Reasoner, which leverages GRPO and curriculum learning to transfer counting capabilities from related tasks such as localization and QA. While achieving SOTA on several audio-visual reasoning benchmarks, the paper honestly identifies that explicit reasoning in the language space offers little help out-of-distribution.
VOLD: Reasoning Transfer from LLMs to Vision-Language Models via On-Policy Distillation: VOLD utilizes a text-only teacher LLM (Qwen3-8B) to train the reasoning capabilities of a vision-language student model (Qwen2.5-VL-3B). It first performs distribution alignment via SFT using teacher-generated reasoning trajectories, then integrates GRPO reinforcement learning with "on-policy distillation" (reverse KL) for joint optimization on the same rollouts. Without using any vision-language reasoning data during the entire process, VOLD outperforms methods trained directly on multimodal data across four visual reasoning benchmarks: MMMU-Pro, MathVision, and LogicVista.

Browse all 150 VLM Reasoning papers →

⚡ VLM Efficiency (63)¶

Accelerating Streaming Video Large Language Models via Hierarchical Token Compression: To address the slow real-time deployment of streaming Video Large Language Models (Streaming VideoLLM), this paper proposes STC, a plug-and-play two-level token compression framework. STC-Cacher caches and reuses static features from adjacent frames during the ViT encoding stage, recomputing only dynamic tokens. STC-Pruner utilizes "spatio-temporal dual anchors" to prune redundant tokens before entering the LLM. STC maintains approximately 99% accuracy on ReKV while reducing ViT encoding latency by 24.5% and LLM pre-filling latency by 45.3%.
OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models: OmniZip is the first training-free token compression framework for joint audio-video understanding in Omnimodal Large Language Models (OmniLLM). It utilizes the attention distribution of audio tokens as a prior for "information density/event boundaries" to dynamically determine video token pruning rates within each time window. Combined with an Interleaved Spatiotemporal Compression (ISTC) module, it achieves 3.42× prefill acceleration and 1.4× memory reduction on Qwen2.5-Omni with almost no performance degradation.
Variation-Aware Vision Token Dropping for Faster Large Vision-Language Models: V2Drop is proposed, which for the first time adopts a perspective of token variation. By progressively dropping "lazy" vision tokens with minimal variation within the LLM, it achieves training-free, position-bias-free LVLM inference acceleration compatible with efficient operators. It retains 94.0% and 98.6% of original performance in image and video understanding tasks respectively, while reducing LLM generation latency by 31.5% and 74.2%.
Hybrid Token Compression for Vision-Language Models: Addressing the dilemma where "continuous compression loses semantics and discrete quantization loses details" when visual tokens are compressed to 1, HTC-VLM utilizes a dual-path decoupling of a continuous channel (ViT patches for details) and a discrete channel (MGVQ for 4 semantic anchors). Through a decoupled attention mask and a <voco> bottleneck, 580 tokens are compressed into 1, improving performance retention from 81.0% to 87.2% across 7 benchmarks.
Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Driving: The first plug-and-play token pruning framework dedicated to multi-view autonomous driving VLMs. By utilizing T-FPS (Token-level Farthest Point Sampling) to maintain semantic and spatial diversity, combined with view-adaptive pruning rates to optimize token budgets across cameras, it achieves 6.40× prefill acceleration on DriveLM with only 10% tokens remaining and a performance drop of just 3%.
AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisition: AdaptVision is proposed to enable VLMs to autonomously determine the minimum number of visual tokens required for each sample through a coarse-to-fine active vision mechanism and reinforcement learning. Combined with Decoupled Turn Policy Optimization (DTPO), it achieves an optimal balance between efficiency and accuracy.
LiteVGGT: Boosting Vanilla VGGT via Geometry-aware Cached Token Merging: To address the quadratic complexity bottleneck of global attention in the 3D foundation model VGGT on long sequences, LiteVGGT proposes a "geometry-aware + cross-layer cached" token merging strategy. It preserves critical tokens based on geometric importance, merges redundant tokens into anchors, and reuses merging indices across layers. Coupled with fine-tuning and FP8 quantization, it achieves approximately 10× speedup compared to VGGT on 1000-image inputs with almost no performance degradation.
MoDES: Accelerating Mixture-of-Experts Multimodal Large Language Models via Dynamic Expert Skipping: The authors propose MoDES, the first training-free expert skipping framework for MoE Multimodal Large Language Models (MLLMs). By utilizing Global Modulated Local Gating (GMLG) and a Dual-Modality Threshold (DMT) mechanism to adaptively skip redundant experts, MoDES retains 97%+ of the original performance while skipping 88% of experts, achieving a 2.16× prefill acceleration.
Better, Stronger, Faster: Tackling the Trilemma in MLLM-based Segmentation with Simultaneous Textual Mask Prediction: STAMP reformulates MLLM-based segmentation as a parallel "cloze" classification task for all image patches. By simultaneously predicting the entire mask using a single non-autoregressive forward pass, it achieves high segmentation precision and fast inference speed without compromising conversational capabilities, effectively resolving the long-standing "dialogue/performance/speed" trilemma in MLLM segmentation.
VQRAE: Representation Quantization Autoencoders for Multimodal Understanding, Generation and Reconstruction: VQRAE transforms RAE (a representation autoencoder using a pre-trained visual backbone as the encoder) into a vector-quantized version. A single tokenizer simultaneously outputs continuous semantic features for understanding and discrete tokens for generation and reconstruction. It demonstrates for the first time that quantizing semantic features requires high dimensionality (1536) for 100% utilization and to avoid collapse, completely moving away from dual-encoders and CNN pixel encoders.

Browse all 63 VLM Efficiency papers →

🎵 Audio & Speech (22)¶

Pushing the Frontier of Audiovisual Perception with Large-Scale Multimodal Correspondence Learning: PEAV (Perception Encoder Audiovisual) is a family of unified "audio-visual-text" contrastive encoders proposed by Meta. It utilizes a two-stage synthetic caption data engine to generate high-quality captions across three categories (audio, visual, audiovisual) for O(100M) audiovisual pairs. By employing up to ten sets of cross-modal contrastive losses to align audio, video, and text into a single space, it sets new SOTA benchmarks across four zero-shot categories: sound, music, speech, and video (e.g., AudioCaps T→A R@1 improved from 35.4 to 45.8, VGGSound classification from 36.0 to 47.1). Furthermore, it enables "speech→transcript" retrieval to work effectively for the first time, jumping from near 0 to 85.6.
Tri-Subspaces Disentanglement for Multimodal Sentiment Analysis: The TSD framework is proposed to explicitly decompose multimodal features into three complementary subspaces: global shared, pairwise shared, and modality-private. A subspace-aware cross-attention fusion module adaptively integrates these three layers of information, achieving state-of-the-art (SOTA) performance on CMU-MOSI and CMU-MOSEI datasets.
UniM: A Unified Any-to-Any Interleaved Multimodal Benchmark: This paper introduces UniM, the first unified any-to-any interleaved multimodal benchmark (31K samples, 7 modalities, 30 domains), along with a three-dimensional evaluation suite and an agentic baseline UniMA based on traceable reasoning. The study reveals significant deficiencies in existing MLLMs under the interleaved multimodal paradigm.
AMUSE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understanding: This paper introduces AMUSE—an audio-visual benchmark for "multi-speaker, dialogue-dense" scenarios (6 agentic tasks × Zero-shot/Guided/Agentic evaluation modes), revealing systematic weaknesses in mainstream MLLMs like GPT-4o and Qwen3-Omni regarding "who is speaking, when, and cross-scene causality." It also proposes the RAFT alignment framework (Reflective Reward + Selective Reasoning Adaptation), which improves the accuracy of open-source models on this benchmark by up to 39.52% (relative) using minimal annotations.
AudioStory: Generating Long-Form Narrative Audio with Large Language Models: AudioStory integrates LLM narrative reasoning with a DiT diffusion audio generator into an end-to-end framework. The LLM first decomposes complex instructions into timestamped sub-events, then generates short audio segments sequentially to form long-form narrative audio. Decoupled bridging via "semantic tokens + residual tokens" ensures intra-segment alignment and cross-segment coherence, enabling stable generation of multi-scene audio stories up to 150 seconds.
BabyVLM-V2: Toward Developmentally Grounded Pretraining and Benchmarking of Vision Foundation Models: The BabyVLM-V2 framework is proposed, which constructs three formats of pretraining data (768K image pairs + 181K video pairs + 63K interleaved sequences) from the SAYCam longitudinal corpus from an infant's first-person perspective. It designs the DevCV Toolbox (10 developmental cognitive tasks) based on the NIH Baby Toolbox®. A compact model trained from scratch surpasses GPT-4o on certain mathematical tasks, marking the first systematic exploration of Artificial Developmental Intelligence (ADI).
EchoFoley: Event-Centric Hierarchical Control for Video Grounded Creative Sound Generation: Addressing the issues of "visual dominance, inability to understand text instructions, and lack of fine-grained editing" in existing video-to-audio models, this paper proposes the EchoFoley task (using symbolic "sound event" representations + three levels of control granularity) along with a densely annotated benchmark of 6k samples. It designs EchoVidia, a training-free agentic framework (using slow-fast thinking + an action pool), which improves controllability by approximately 40.7% and perceptual quality by 12.5% over the strongest baseline.
Hierarchical Codec Diffusion for Video-to-Speech Generation: HiCoDiT reframes "silent video to speech" generation as a masked diffusion task that proceeds layer-by-layer along the RVQ discrete token hierarchy. Lower-level tokens handle content and timbre under lip-motion and identity guidance, while higher-level tokens manage prosody via dual-scale AdaLN modulation of expressions. This approach achieves leading performance in naturalness, intelligibility, and lip-sync on LRS2/LRS3 through zero-shot cross-dataset evaluation.
Omni-MMSI: Toward Identity-Attributed Social Interaction Understanding: Ours proposes the Omni-MMSI task—understanding multi-person social interactions from raw audio-visual inputs rather than pre-processed oracle social cues. It designs the Omni-MMSI-R reference-guided pipeline, achieving accurate social interaction understanding through tool-generated identity-attributed social cues combined with Chain-of-Thought (CoT) reasoning.
Cleaning the Pool: Progressive Filtering of Unlabeled Pools in Deep Active Learning: The authors propose Refine, an ensemble active learning method that consistently outperforms individual AL strategies and existing ensemble methods. It employs a two-stage strategy: progressive filtering (iterative refinement of the unlabeled pool using multiple strategies) followed by coverage selection (selecting high-value diverse samples from the refined pool) without requiring prior knowledge of the optimal strategy.

Browse all 22 Audio & Speech papers →

🔎 AIGC Detection (10)¶

Common Inpainted Objects In-N-Out of Context: The authors systematically replace objects in COCO images using Stable Diffusion inpainting to generate 97k "in-context / out-of-context" images of the exact same object. They then employ three 72B multimodal large models for consensus annotation of three-dimensional contextual labels ("location / size / co-occurrence"), building COinCO—the first inpainted forgery dataset with complete contextual annotations. Finally, they demonstrate three downstream tasks: fine-grained contextual classification, inferring objects from context, and training-free enhancement of SOTA forgery localization.
Fine-grained Image Aesthetic Assessment: Learning Discriminative Scores from Relative Ranks: This work defines the new task of "Fine-grained Image Aesthetic Assessment" and constructs the FGAesthetics benchmark containing 32,217 images across 10,028 series. It proposes the FGAesQ model, which learns discriminative aesthetic scores from relative ranks through Difference-Preserving Tokenization (DiffToken), Contrastive Text-aligned Alignment (CTAlign), and Rank-Aware Regression (RankReg). The model achieves an accuracy of 0.779 in fine-grained scenarios while maintaining a coarse-grained SRCC of 0.770.
Locate-Then-Examine: Grounded Region Reasoning Improves Detection of AI-Generated Images: LTE enables Vision-Language Models to first perform a "global scan to locate suspicious regions" and then "zoom in and crop to re-examine for the final verdict." It upgrades one-time classification into a two-stage region-grounded reasoning process. Accompanied by the TRACE dataset containing box-level annotations and forensic explanations, it achieves simultaneous improvements in accuracy, robustness, and interpretability.
Investigating Self-Supervised Representations for Audio-Visual Deepfake Detection: This is a systematic "investigation" paper: the authors freeze 12 off-the-shelf self-supervised encoders (audio, visual, and audio-visual) and train only a single-layer linear probe on top of them. They evaluate their capability in audio-visual deepfake detection horizontally across three dimensions: "detection effectiveness, interpretability, and cross-modal complementarity". They find that "audio-driven" representations generalize best (with BRAVEn's visual encoder achieving SOTA), whereas the main difficulty in real-world data stems from the intrinsic complexity of the datasets themselves rather than features exploiting shallow shortcuts.
Learning Where to Look and How to Judge: Resolution-agnostic Image Quality Assessment with Quality-aware Saliency: To address four common issues in No-Reference Image Quality Assessment (NR-IQA)—forced resizing to accommodate pre-trained resolutions, poor generalization across resolutions, difficulty in joint training due to inconsistent MOS scales, and computational explosion for UHD images—this paper proposes ReLIQS. It samples fixed-size patches from the original resolution and its scaled variants, encoding them with CLIP. A lightweight "Perceptual Importance Estimator (PIE)" learns IQA-specific saliency to select a few key patches, while a "Latent Quality Axis Module (LQAM)" aggregates multi-scale embeddings into a single score. ReLIQS outperforms CNN, CLIP, and MLLM-based baselines across various real/synthetic/AIGC distortions and resolutions with lower computational cost.
NOWA: Null-space Optical Watermark for Invisible Capture Fingerprinting and Tamper Localization: A learnable phase mask is inserted at the camera aperture to encode the authentication signal into the null-space of the imaging operator (rendering it completely invisible during capture). A measurement-consistent Null-Space Network (NSN) is then utilized to reconstruct high-quality images and anchor this watermark. Tampering disrupts the statistical structure in the null-space projection, allowing the detector to localize the changes at the pixel level. Under AIGC editing, the F1 score exceeds EditGuard (0.993 vs 0.97), and the system is inherently unforgeable to unknown counterfeiters.
Enabling Supervised Learning of Generative Signatures for Generalized AI-Generated Images Detection: To address the deadlock where "generative traces in AI-generated images lack clean pairs and cannot be extracted via supervised learning," this paper uses a randomly-structured image reconstructor to artificially "create traces" on real images. The reconstruction residuals are treated as pseudo-labels to train a generative signature (GenSign) extractor, followed by a GenSign + RGB dual-stream classifier for detection, achieving SOTA cross-model generalization across four benchmarks.
Inconsistency-aware Multimodal Schrodinger Bridge for Deepfake Localization: IaMSB reformulates "temporal interval localization" of audio-visual deepfakes as a Schrödinger Bridge (SB) generation problem—directly reading cross-modal consistency scores from the bridge's transmission cost and asymmetrically allocating computation steps to the more suspicious modality, resulting in a 3-10% gain over existing methods on strict IoU ([email protected]).
Learning Forgery-Aware Lip Representations Without Forgery Priors: To address the vulnerability of speaker authentication systems to personalized Talking Face Generation (TFG) forgeries, this paper proposes a detector trained solely on real videos without relying on any forgery samples. By combining mixed-fake lip generation, asymmetric contrastive learning, and Gaussian regularization, the real lip motion features are compressed into a compact hypersphere. Anything outside the sphere (forgeries and impostors) is treated as an outlier, reducing the error rate by over 10% against 8 modern forgeries compared to 10 SOTA methods.
PPM-CLIP: Probabilistic Prompt Modeling for Generalizable AI-Generated Image Detection: PPM-CLIP replaces the "discriminative static boundary" paradigm with "generative probabilistic inference." It utilizes normalizing flows to generate a family of adaptive prompts (multiple hypotheses) for each image and determines the results by averaging cosine similarities to marginalize noise. Combined with frequency-guided patch-wise contrastive learning, it forces the CLIP encoder to capture high-frequency forgery traces, significantly outperforming SOTA in cross-generator generalization on Ojha, GenImage, and DRCT.

🧊 3D Vision (751)¶

SAM 3D: 3Dfy Anything in Images: SAM 3D is a generative foundation model that reconstructs complete 3D shapes, textures, and layouts for any object from a single natural image. It overcomes the barrier of scarce real-world 3D data through a "model-in-the-loop + human annotation" data flywheel and an LLM-style multi-stage training recipe, achieving at least a 5:1 human preference win rate over previous SOTAs on real objects and scenes.
Native and Compact Structured Latents for 3D Generation: Under the name of TRELLIS.2, this work is a follow-up by the original TRELLIS team, proposing a structured latent representation learned directly from native 3D data. Its core is a field-free, versatile voxel representation termed O-Voxel, which encodes geometry of arbitrary topology and PBR materials into a unified flexible dual grid. Furthermore, a Sparse Compact VAE (SC-VAE) is designed to achieve \(16\times\) spatial downsampling, compressing \(1024^3\) fully-textured assets into approximately 9.6K tokens. Finally, a three-stage Flow-matching model with around 4B parameters is trained for image-to-3D generation, significantly outperforming existing methods in reconstruction fidelity, material quality, and generation speed.
PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation: PointWorld represents scene states and robot actions as a unified set of 3D point flows. By using a large pre-trained point cloud backbone to learn "how scene points move given an action" across approximately 2 million trajectories, a single checkpoint can drive real robotic arms to complete tasks involving rigid body pushing, deformable objects, articulated objects, and tool use from a single RGB-D input in a zero-shot manner.
SpatialVID: A Large-Scale Video Dataset with Spatial Annotations: SpatialVID distills 2.71 million dynamic segments (7,089 hours in total) from 21,000 hours of in-the-wild web videos using a three-stage "hierarchical filtering + geometric/semantic annotation + balanced sampling" pipeline. Each segment includes per-frame camera poses, depth, dynamic masks, structured captions, and serialized motion instructions, representing the largest and most comprehensively annotated video dataset for "dynamic scenes + explicit geometry."
FastGS: Training 3D Gaussian Splatting in 100 Seconds: FastGS is proposed as a 3DGS acceleration framework based on multi-view consistency. By employing Multi-view Consistency Densification (VCD) and Multi-view Consistency Pruning (VCP) to precisely control the number of Gaussians, it achieves scene training in approximately 100 seconds on datasets like Mip-NeRF 360—a 15× speedup over vanilla 3DGS with comparable rendering quality.
LATTICE: Democratize High-Fidelity 3D Generation at Scale: LATTICE proposes a semi-structured 3D latent representation called VoxSet, which anchors the compact latent tokens of VecSet onto a coarse voxel grid, thereby injecting positional information into the diffusion transformer. Combined with a two-stage "coarse-to-fine" pipeline that first generates coarse structures and then refines geometry, this pure transformer architecture scales the image-to-3D model to 4.5B parameters. At the same time, it achieves token-level test-time scaling, which is rare in 3D generation, outperforming prior SOTA in reconstruction and generation quality.
Efficiently Reconstructing Dynamic Scenes One D4RT at a Time: D4RT uses a unified encoder-decoder Transformer to first encode a video into a fixed global scene representation, and then utilizes a single "query 3D position of any spatio-temporal point" decoding interface to simultaneously obtain depth, point clouds, 3D point trajectories, and camera extrinsics/intrinsics. It achieves new SOTA in dynamic 4D reconstruction and tracking, running approximately 9× faster than VGGT and two orders of magnitude faster than MegaSaM.
SAGE: Scalable Agentic 3D Scene Generation for Embodied AI: SAGE formalizes 3D indoor scene generation as an agent operating under the MCP protocol. It invokes layout/asset generators on demand and employs a closed-loop self-correction mechanism via "Visual Review + Physical Review (Isaac Sim in-the-loop verification)." It produces physically stable, open-vocabulary scenes that can be directly imported into simulators for robot policy training, scaled through multi-layer augmentation.
MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Second: The authors propose MoVieS, a feed-forward 4D dynamic scene reconstruction framework. By utilizing a Dynamic Splatter Pixel representation to unify appearance, geometry, and motion modeling, it achieves 4D reconstruction from monocular video in approximately 1 second. It supports multiple tasks including novel view synthesis, 3D point tracking, scene flow estimation, and moving object segmentation.
NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos: NeoVerse proposes a scalable 4D world model. By utilizing feed-forward pose-free 4DGS reconstruction and online monocular degradation simulation, the training pipeline can leverage massive (millions) in-the-wild monocular videos, achieving SOTA in both 4D reconstruction and novel-trajectory video generation.

Browse all 751 3D Vision papers →

🎯 Object Detection (99)¶

Detect Anything via Next Point Prediction: Object detection is reformulated as "generating quantized coordinate token sequences with an MLLM." By combining three components—learnable coordinate tokens, a self-built data engine generating 22 million annotations, and SFT followed by GRPO reinforcement training to rectify behavior—the authors develop Rex-Omni, a 3B model. It surpasses regression-based detectors like DINO and Grounding DINO in zero-shot performance on benchmarks such as COCO, while simultaneously handling eight task categories including referring, pointing, GUI localization, and OCR.
PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulation: Ours proposes PALM, a unified VLA framework that uses structured fine-grained affordance predictions (global, local, spatial, and dynamic) as implicit reasoning anchors, combined with continuous sub-task progress estimation for seamless task switching. It achieves an average completion length of 4.48 on CALVIN ABCD (surpassing Prev. SOTA by 12.5%), a 91.8% success rate on LIBERO-LONG, and over 2x the baseline performance in real-world long-horizon generalization tests.
YOLO-Master: MOE-Accelerated with Specialized Transformers for Enhanced Real-time Detection: YOLO-Master integrates sparse MoE (ES-MoE blocks) into the YOLO backbone, enabling the network to dynamically activate different experts according to image complexity. It achieves 42.4% AP with a 1.62ms latency on MS COCO, surpassing YOLOv13-N by 0.8% mAP while being 18% faster.
VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer: The necessity of the text branch in Zero-Shot Anomaly Detection (ZSAD) is revisited, leading to the proposal of VisualAD—a pure vision framework. By inserting two learnable tokens (anomaly/normal) into a frozen ViT, combined with Spatial-Aware Cross-Attention and a Self-Alignment Function, the model achieves SOTA performance across 13 industrial and medical benchmarks without requiring a text encoder.
Fourier Angle Alignment for Oriented Object Detection in Remote Sensing: Leveraging Fourier rotation equivariance to estimate principal orientations in the frequency domain for feature alignment, this paper proposes two plug-and-play modules, FAAFusion and FAA Head. These modules address cross-scale directional incoherence in FPN and task conflict between classification and regression in detection heads, respectively, achieving new SOTA results on DOTA-v1.0/v1.5 and HRSC2016.
Seeing Through the Noise: Improving Infrared Small Target Detection and Segmentation from Noise Suppression Perspective: Addressing the issue where enhancing high-frequency features simultaneously increases false alarm rates in infrared small target detection, this paper proposes a noise-suppression feature pyramid network (NS-FPN) from a frequency-domain perspective. By replacing the 1×1 convolutions and upsampling in the FPN with a Low-frequency-guided Feature Purification (LFP) module and a Spiral-aware Feature Sampling (SFS) module, it significantly reduces false alarms and improves localization accuracy with almost no added computational cost.
RC-NF: Robot-Conditioned Normalizing Flow for Real-Time Anomaly Detection in Robotic Manipulation: Proposes Robot-Conditioned Normalizing Flow (RC-NF), which models the joint distribution of robot states and object trajectories through conditional normalizing flows. It achieves <100ms real-time anomaly detection and serves as a plug-and-play monitoring module for VLA models (e.g., π₀), supporting task-level replanning and state-level trajectory homing.
SubspaceAD: Training-Free Few-Shot Anomaly Detection via Subspace Modeling: SubspaceAD demonstrates that fitting a single PCA on features from a strong vision foundation model (DINOv2-G) is sufficient to outperform all few-shot anomaly detection methods requiring training, memory banks, or prompt tuning. In a 1-shot setting, it achieves 98.0% Image-level AUROC and 97.6% Pixel-level AUROC on MVTec-AD.
WeDetect: Fast Open-Vocabulary Object Detection as Retrieval: This work treats open-vocabulary detection entirely as a "region × text" retrieval matching problem. It utilizes a non-fusion dual-tower structure, WeDetect, to achieve real-time SOTA detection. By freezing WeDetect, a general proposal generator WeDetect-Uni is derived (supporting the new task of local object retrieval). Finally, WeDetect-Ref reframes REC by transforming an LLM into a classifier for parallel scoring in a single forward pass, achieving both high precision and high throughput across 15 benchmarks.
GS-CLIP: Zero-shot 3D Anomaly Detection by Geometry-Aware Prompt and Synergistic View Representation Learning: Ours proposes GS-CLIP, a two-stage framework that injects global shape and local defect information of 3D point clouds into text prompts via a Geometry Defect Distillation Module (GDDM). It synergistically fuses rendered images and depth maps using a LoRA-based dual-stream architecture, achieving SOTA performance in zero-shot 3D anomaly detection across four large-scale datasets.

Browse all 99 Object Detection papers →

✂️ Segmentation (122)¶

Exploring the Underwater World Segmentation without Extra Training: Addressing the scarcity of data and models in underwater scenarios, this work introduces the first fine-grained underwater open-vocabulary segmentation dataset and benchmark (AquaOV255 / UOVSBench). It also proposes Earth2Ocean, a training-free framework that corrects CLIP visual features with geometric self-similarity priors and enhances text embeddings via MLLM reasoning. This transfers terrestrial VLMs to underwater contexts without any extra training, achieving an average mIoU improvement of 6+.
INSID3: Training-Free In-Context Segmentation with DINOv3: This paper proposes INSID3, a training-free in-context segmentation method relying solely on frozen DINOv3 features. Through a three-stage pipeline consisting of positional bias elimination, fine-grained clustering, and seed cluster aggregation, it outperforms methods relying on SAM or fine-tuning across semantic, part, and personalized segmentation tasks using a single self-supervised backbone, achieving an average mIoU gain of +7.5%.
MixerCSeg: An Efficient Mixer Architecture for Crack Segmentation via Decoupled Mamba Attention: Ours proposes MixerCSeg, which decouples channels into global/local branches by analyzing the implicit attention mechanism of Mamba. These branches are respectively enhanced with Self-Attention and CNN, combined with direction-guided edge gated convolutions. It achieves SOTA performance in crack segmentation with 2.05 GFLOPs and 2.54M parameters.
F2Net: A Frequency-Fused Network for Ultra-High Resolution Remote Sensing Segmentation: F2Net decomposes ultra-high resolution (UHR) remote sensing images in the frequency domain into high-frequency and low-frequency components for separate processing. A high-frequency branch preserves full resolution for boundary details, while the low-frequency branch is downsampled and split into two sub-branches (short-range and long-range) for semantic capture. A Hybrid Frequency Fusion (HFF) module integrates the three features, supported by two cross-frequency losses to stabilize multi-branch training, achieving SOTA results of 80.22 and 83.39 mIoU on DeepGlobe and Inria Aerial, respectively.
Pointer-CAD: Unifying B-Rep and Command Sequences via Pointer-based Edges & Faces Selection: Ours proposes a command sequence representation based on a Pointer mechanism, explicitly introducing B-Rep geometric entities (edges/faces) into autoregressive CAD generation. This is the first command sequence method to support chamfer/fillet operations while significantly reducing topological errors caused by quantization.
SAMTok: Representing Any Mask with Two Words: SAMTok compresses any region mask into two discrete text tokens, enabling standard MLLMs (like QwenVL) to understand and generate masks just like text via next-token prediction. It requires no specialized segmentation heads or custom losses, and by turning masks into "text," it allows reinforcement learning with pure character-matching rewards for the first time.
SARMAE: Masked Autoencoder for SAR Representation Learning: The SARMAE framework is proposed, achieving noise-robust SAR self-supervised pre-training through the million-scale SAR dataset SAR-1M, Speckle-Aware Representation Enhancement (SARE), and Semantic Anchor Representation Constraint (SARC). It achieves SOTA results across multiple downstream tasks including classification, detection, and segmentation.
MatAnyone 2: Scaling Video Matting via a Learned Quality Evaluator: Ours proposes a learned Matting Quality Evaluator (MQE) to evaluate alpha quality pixel-wise without ground truth. MQE serves as both online training guidance and an offline data filter. This enabled the construction of VMReal, a real-world video matting dataset with 28K clips and 2.4 million frames. Combined with a reference-frame training strategy, the method significantly outperforms all existing state-of-the-art approaches.
PRUE: A Practical Recipe for Field Boundary Segmentation at Scale: This paper provides a systematic evaluation of 18 segmentation and Geospatial Foundation Models (GFM), proposing PRUE—a field boundary segmentation recipe combining a U-Net backbone, composite loss functions, and targeted data augmentation. It achieves 76% IoU and 47% object-F1 on the FTW benchmark, improvements of 6% and 9% over the baseline respectively, while introducing a new set of metrics for evaluating deployment robustness.
VidEoMT: Your ViT is Secretly Also a Video Segmentation Model: VidEoMT is proposed as an encoder-only video segmentation architecture that unifies segmentation and temporal association within a single ViT encoder through query propagation and query fusion. It achieves a 5×–10× speedup (reaching 160 FPS with ViT-L) while maintaining accuracy comparable to the SOTA.

Browse all 122 Segmentation papers →

🖼️ Image Restoration (135)¶

DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformers: For semi-transparent/transparent occlusion scenarios such as glass, fog, watermarks, and X-rays, this paper reformulates the problem of "decomposing foreground and background layers from a single composite image" as a generative posterior inference task. Along with releasing the first large-scale AlphaBlend dataset, the authors propose a diffusion Transformer framework called DiffDecompose (anchored by "In-Context Decomposition (ICD)" and "Layer Position Encoding Cloning (LPEC)") to achieve mask-free multi-layer decomposition. This approach achieves an average RMSE approximately 36% lower than the second-best method across multiple layer removal and decomposition subtasks.
Hybrid Agents for Image Restoration: To address the pain points of "non-experts being unable to select the right tools" and "sequential restoration causing error propagation" in real-world image restoration, HybridAgent is proposed. It employs a triad of "Fast, Slow, and Feedback" agents for collaborative scheduling, working with a suite of single and mixed degradation restoration tools trained in three stages. The system routes simple instructions through a lightweight fast path while processing complex degradations via an MLLM-based slow path with closed-loop feedback, achieving both efficiency and stability in automated image restoration.
FoundIR-v2: Optimizing Pre-Training Data Mixtures for Image Restoration Foundation Model: FoundIR-v2 discovers that the "training data mixture ratio of different restoration tasks" is a key variable determining all-in-one image restoration performance. Consequently, it employs a dual-scheduling scheme—Dynamic Equilibrium Scheduling (dynamic ratio adjustment) and an MoE-driven Diffusion Scheduler (task-adaptive generative prior allocation)—for generative pre-training on SDXL. A single model covers 50+ sub-tasks and outperforms existing SOTA on multiple benchmarks.
One-Step Diffusion Transformer for Controllable Real-World Image Super-Resolution: Based on Qwen-Image, the One-Step Diffusion Transformer (ODTSR) utilizes "Noise-Mixed Visual Streams" (NVS) to achieve simultaneous fidelity and prompt controllability, continuously adjustable via a fidelity weight \(f\). Combined with "Fidelity-Aware Adversarial Training" (FAA) to compress multi-step denoising into single-step inference, it achieves SOTA performance in both general Real-ISR and Chinese/English scene text SR.
PnP-CM: Consistency Models as Plug-and-Play Priors for Inverse Problems: Consistency Models (CM) are reinterpreted as "proximal operators of a prior" and integrated into an ADMM-based Plug-and-Play (PnP) framework. By incorporating noise injection and momentum to compress iterations to 2–4 Number of Function Evaluations (NFE), this work unifiedly solves linear/nonlinear inverse problems and applies CM training to MRI reconstruction for the first time.
Multinex: Lightweight Low-light Image Enhancement via Multi-prior Retinex: Multinex reformulates Retinex decomposition from a "reconstruction target" into an "additive residual prior." By feeding a set of analytically computed multi-view luminance/chrominance priors into two ultra-lightweight fusion networks, it outperforms SOTA lightweight models and approaches million-parameter models using only 45K (or even 0.7K) parameters across seven low-light benchmarks.
Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning: For the super-resolution task involving "low-resolution hyperspectral image (LR HSI) + one unregistered high-resolution reference image," this paper utilizes spectral unmixing to decouple spatial and spectral information. This allows the network to focus solely on enhancing the unmixed abundance maps (rather than performing direct spatial-spectral coupled fusion, which is susceptible to misalignment interference). Combined with coarse-to-fine deformable aggregation, spatial-channel abundance cross-attention, and modulated fusion modules, the method achieves SOTA performance on ICVL/REAL datasets with approximately half the parameters (PSNR 41.84/42.05 dB at \(\times 4\)).
Scan Clusters, Not Pixels: A Cluster-Centric Paradigm for Efficient Ultra-high-definition Image Restoration: Addressing the bottleneck where Mamba still requires pixel-wise scanning in Ultra-High-Definition (4K) image restoration, leading to memory explosion, C2SSM replaces "pixel-serial scanning" with "cluster-centric scanning." The method distills millions of pixels into a few semantic centroids via a neurally-parameterized mixture distribution, performs Mamba scanning only on these centroids, and diffuses global context back to all pixels based on similarity distributions. This paradigm achieves SOTA performance across five UHD restoration tasks with the lowest FLOPs (0.407G).
Time-Aware One Step Diffusion Network for Real-World Image Super-Resolution: TADSR identifies that existing one-step diffusion SR methods fix the student's timestep at 999, wasting the diverse generative priors of Stable Diffusion (SD) across different timesteps. It introduces a time embedding to the VAE encoder, allowing the same image to encode different latents based on the timestep, and utilizes a mapping function to bind student and teacher timesteps. This enables consistent generative guidance in a single step and allows for seamless adjustment between fidelity and realism by simply tuning \(t_s\), achieving SOTA performance on non-reference metrics across multiple real and synthetic datasets.
Flickerformer: A Duet of Periodicity and Directionality for Burst Flicker Removal: This work reveals that flicker artifacts possess two inherent physical properties: periodicity and directionality. It designs the Flickerformer with three modules (PFM/AFFN/WDAM) to model inter-frame/intra-frame periodicity and directionality respectively. With only 3.92M parameters, it achieves 31.226dB PSNR on the BurstDeflicker benchmark, surpassing the runner-up AST by +0.580dB using only 19.70% of its parameters.

Browse all 135 Image Restoration papers →

🛰️ Remote Sensing (63)¶

TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis: TESSERA encodes multi-year Sentinel-1/2 time series of each 10m surface pixel into a 128-dimensional int8 embedding vector. By leveraging a self-supervised objective invariant to random temporal sampling, it learns robust phenological representations. Released as a global "embeddings-as-data" product, downstream tasks only need a lightweight MLP/UNet head to achieve SOTA performance on classification, segmentation, and regression tasks, showing significant advantages under extremely low annotation regimes.
OlmoEarth: Stable Latent Image Modeling for Multimodal Earth Observation: OlmoEarth utilizes a self-supervised recipe designed specifically for Earth Observation (Latent MIM Lite with frozen random projections as target encoders + modality-aware masking + intra-modality contrastive loss). This approach stably trains spatio-temporal multimodal foundation models in latent space. It outperforms 12 other foundation models on 15 out of 24 embedding tasks and 19 out of 29 fine-tuning tasks, and has been deployed as an end-to-end platform for non-profit organizations.
UniGeoSeg: Towards Unified Open-World Segmentation for Geospatial Scenes: The authors construct the first million-scale remote sensing instruction segmentation dataset, GeoSeg-1M (590K images, 117 categories, 1.1M triplets), along with the companion GeoSeg-Bench. They propose a unified framework, UniGeoSeg, which integrates referring, interactive, and reasoning segmentation into a single model using Task-Adaptive Text Enhancement (TATE), Latent Knowledge Memory (LKM), and Progressive Task Scheduling (PTS). It achieves state-of-the-art (SOTA) performance on GeoSeg-Bench and multiple public benchmarks with strong zero-shot generalization.
SegEarth-R2: Towards Comprehensive Language-guided Segmentation for Remote Sensing Images: Addressing four complex requirements in remote sensing (small objects, multi-granularity, multi-object, and implicit instructions), this work introduces LaSeRS, the first large-scale dataset systematically covering these dimensions (40k masks, 122 classes, 30k QA triplets). It proposes SegEarth-R2, a 3B-parameter MLLM segmentation model that surpasses 7B, 8B, and even 13B models across multiple benchmarks using spatial attention supervision and flexible segmentation queries.
Data Leakage Detection and De-duplication in Large Scale Geospatial Image Datasets: This paper performs a quality audit on three commonly used remote sensing datasets for building footprint extraction using perceptual hashing. It discovers that the AICrowd Mapping Challenge dataset suffers from severe duplication (approx. 89% of training images are exact/augmented duplicates) and cross-split leakage (approx. 93% of validation images appear in the training set). The authors provide a lightweight, reusable de-duplication and leakage detection pipeline, revealing that many "SOTA" methods are actually overfitted to leaked data.
Asking like Socrates: Socrates helps VLMs understand remote sensing images: This work reveals the "pseudo-reasoning" phenomenon in remote sensing VLMs (where explicit reasoning chains lead to performance degradation), attributed to the "glance effect" (insufficient single coarse-grained perception). It proposes the RS-EoT (Evidence-of-Thought) iterative evidence search paradigm. The method uses SocraticAgent self-play to synthesize reasoning trajectories for SFT cold startup, followed by two-stage progressive RL (grounding → VQA) for enhancement and generalization. RS-EoT-7B achieves SOTA on multiple remote sensing VQA and grounding benchmarks.
GeoBridge: A Semantic-Anchored Multi-View Foundation Model Bridging Images and Text for Geo-Localization: GeoBridge utilizes a "location-aware unified text description" as a semantic anchor to bind images from three perspectives—UAV, street-view panorama, and satellite—into a shared semantic space. This approach breaks away from the traditional "satellite-centric" localization paradigm, enabling both arbitrary peer-to-peer view matching and text-to-image retrieval. The associated GeoLoc dataset (50,000+ triple-aligned sets across 36 countries) allows it to achieve SOTA performance in both cross-view and cross-modal retrieval.
UniChange: Unifying Change Detection with Multimodal Large Language Model: UniChange unifies Binary Change Detection (BCD) and Semantic Change Detection (SCD) into a single MLLM-based framework. By utilizing the embeddings of three special tokens—[T1], [T2], and [CHANGE]—as "queries" to drive a segmentation decoder and replacing fixed classification heads with text prompts, it allows joint training on multi-source remote sensing datasets with conflicting category definitions. It achieves new SOTA performance on WHU-CD, S2Looking, LEVIR-CD+, and SECOND, with IoUs of 90.41, 53.04, 78.87, and 57.62 respectively.
APEX: A Decoupled Memory-based Explorer for Asynchronous Aerial Object Goal Navigation: APEX decomposes the "UAV target search" task into three decoupled modules—using MLLMs to dynamically construct 3D spatio-temporal semantic maps as memory, PPO-based reinforcement learning to translate maps into actions, and an open-vocabulary detector for final target confirmation. These modules run at different frequencies via an asynchronous parallel framework to bypass the inference latency of large models, achieving a \(+4.2\%\) SR and \(+2.8\%\) SPL improvement over the Prev. SOTA on the UAV-ON benchmark.
GeoMMBench and GeoMMAgent: Toward Expert-Level Multimodal Intelligence in Geoscience and Remote Sensing: This paper introduces GeoMMBench (1,053 expert-level geoscience multiple-choice questions) and GeoMMAgent (a retrieval-perception-reasoning multi-agent framework). It systematically evaluates 36 MLLMs in the remote sensing domain, revealing systemic deficiencies in domain knowledge, perceptual grounding, and reasoning.

Browse all 63 Remote Sensing papers →

🔍 Anomaly Detection (7)¶

RAID: Retrieval-Augmented Anomaly Detection: RAID reinterprets Unsupervised Anomaly Detection (UAD) as a Retrieval-Augmented Generation (RAG) pipeline: it first performs coarse-to-fine retrieval using a three-level vector library (class prototype → semantic prototype → instance token), then employs a "Guided MoE Filter" to denoise the retrieved matching cost volume. This suppresses matching noise and produces anomaly maps with sharp boundaries, achieving SOTA across full-shot, few-shot, and multi-dataset settings on MVTec/VisA/MPDD/BTAD.
Anomaly-Related Residual Fields for Cross-domain Anomaly Detection: Addressing the challenge that diffusion model residuals are noisy and magnitudes alone cannot distinguish anomalies, this paper proposes Residual Evolution Fields (REF). It separates "persistent non-stationary anomaly signals" from the spatio-temporal trajectories of residuals in the diffusion reverse process. Cross-domain Field Alignment (CFA) is then employed to transfer detectors trained on labeled source domains to unlabeled target domains, achieving an average AUROC of 95.22% across 9 cross-domain tasks, outperforming the strongest baseline by 13 percentage points.
Defect Cue-Preserved Structural Feature Refinement for Few-Shot Anomaly Detection: This paper identifies that the core difficulty in few-shot anomaly detection (FSAD) lies in the "dilution" of subtle defect cues layer-by-layer within deep feature extraction pipelines. It proposes DCP-SFR: first using learnable prompts to "amplify" early weak signals into high-contrast anomaly cue maps, then using these maps to guide reconstruction-based localization, and finally performing structural-aware boundary refinement. It achieves an image-level AUROC of 97.3% and a pixel-level AUROC of 98.2% on MVTec AD and VisA.
Dual-Prototype-Guided Multi-task Learning for Unsupervised Anomaly Detection and Classification: PG-SFD models "unsupervised anomaly detection (pixel-level localization) + weakly supervised anomaly classification (region-level classification)" as a dual-prototype collaborative optimization problem. By explicitly decoupling normal/anomaly semantics using normal and category prototypes, injecting normal priors into the classification branch via differential gating, and alleviating multi-task gradient conflicts with geometric regularization, it achieves an I-AUROC of 99.4% on MVTec-AD while supporting fine-grained defect classification.
Hunting Normality from Query Sample via Residual Learning for Generalist Anomaly Detection: Addressing the issue in Generalist Anomaly Detection (GAD) where "directly modeling residual distributions" leads to misjudgments due to inconsistency between residuals and instance features, Ours no longer classifies residuals directly. Instead, it treats residuals as a guide: learnable proxies extract patterns from residuals (RFL), then these residual proxies aggregate query-related "normality proxies" (NLS) from the support set. Finally, these normality proxies are used to search for normal regions (HNQ) within the query features to locate anomalies. Ours achieves competitive few-shot performance on cross-domain benchmarks including Industrial→Industrial and Industrial→Medical.
LayoutAD: Exploring Semantic-Geometric Misalignment Reasoning for Scene Layout Anomaly Detection: LayoutAD proposes a new task "Scene Layout Anomaly Detection," which uses an unsupervised approach to generate object-level anomaly scores for each object in an image. By decomposing the scene into semantic and geometric graphs and reasoning the "misalignment" between them via cross-graph attention, it identifies layout-level hallucinations—such as "a five-legged dog" or "a car parked on a lake"—that are invisible to pixel-level detectors.
Multi-Prototype Compactness and Boundary-Aware Synthesis for Unsupervised Anomaly Detection: Addressing the issue where the single-prototype hypothesis results in overly loose decision boundaries under high intra-class variance, this paper proposes the PGBL framework. It structures normal features into multiple compact sub-clusters using Multi-Prototype Compactness Constraints (MPCC), synthesizes pseudo-anomalies at the topological boundaries of these sub-clusters (BAAS), and refines the decision surface with a discriminator (DBR). PGBL outperforms previous methods in detection and localization on MVTec-AD, VisA, and Real-IAD.

🧑 Human Understanding (151)¶

SAM 3D Body: Robust Full-Body Human Mesh Recovery: SAM 3D Body (3DB) is a SAM-style promptable single-view full-body human mesh recovery model. It utilizes a shared encoder + body/hand dual-decoder architecture based on the MHR representation, which decouples skeleton and shape. Coupled with a data engine capable of mining hard samples and producing 7 million high-quality annotations, it achieves SOTA performance on both body and hand poses in in-the-wild images.
UniDex: A Robot Foundation Suite for Universal Dexterous Hand Control from Egocentric Human Videos: The authors propose the UniDex robot foundation suite—comprising a large-scale dataset across 8 dexterous hands (50K+ trajectories/9M frames), a Function-Actuator Aligned Space (FAAS), and a 3D VLA policy (UniDex-VLA). It achieves an 81% average task progress (vs. 38% for π₀) on real-world tool-use tasks and demonstrates spatial, object, and zero-shot cross-hand generalization capabilities.
Avatar Forcing: Real-Time Interactive Head Avatar Generation for Natural Conversation: The authors upgrade "talking head generation" from unidirectional broadcasting to genuine bidirectional conversation. By utilizing causal diffusion forcing in the motion latent space, the model receives user audio/motion while auto-regressively generating avatar head movements. Combined with KV caching, the latency is reduced to ~500ms (6.8x faster than baselines). Furthermore, a label-free DPO (Direct Preference Optimization), which generates negative samples by "dropping user conditions," enables the avatar to learn expressive reactions like nodding and smiling, achieving over an 80% preference rate against the strongest baseline in human evaluations.
AVATAR: Reinforcement Learning to See, Hear, and Reason Over Video: The AVATAR framework is proposed to improve GRPO through two core components: an off-policy training architecture (stratified replay buffer) and Time Advantage Shaping (TAS, using U-shaped weighting to emphasize the beginning and end of reasoning chains). This approach addresses three major issues of GRPO—data inefficiency, vanishing advantages, and uniform credit assignment—significantly outperforming the GRPO baseline on audio-visual reasoning benchmarks.
RAM: Recover Any 3D Human Motion in-the-Wild: RAM proposes a unified multi-person 3D motion recovery framework that integrates a motion-aware semantic tracker SegFollow (based on SAM2 + adaptive Kalman filtering), a memory-enhanced temporal human mesh recovery module T-HMR, a lightweight motion predictor, and a gated combiner. It achieves SOTA zero-shot tracking stability and 3D accuracy on benchmarks such as PoseTrack and 3DPW, with inference speeds 2-3 times faster than previous methods.
InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions: InterPrior utilizes a three-stage recipe of "large-scale imitation distillation + RL fine-tuning" to distill a full-reference imitation expert into a goal-conditioned variational policy. This policy is then refined using RL into a generalizable generative controller capable of generating full-body human-object interactions from sparse goals (snapshots/trajectories/contacts) and self-correcting after failures.
LLaMo: Scaling Pretrained Language Models for Unified Motion Understanding and Generation with Continuous Autoregressive Tokens: LLaMo extends pretrained LLMs into a unified large model capable of both "motion-to-text" (understanding) and "text-to-motion" (generation) using "modality-split Mixture-of-Transformers + continuous causal motion tokens + flow matching decoding heads + exit heads." The key is freezing text modules to preserve the original language capabilities of the LLM, while supporting real-time (≥30 FPS) streaming generation of arbitrary length.
M4Human: A Large-Scale Multimodal mmWave Radar Benchmark for Human Mesh Reconstruction: M4Human is the largest multimodal mmWave radar benchmark for Human Mesh Reconstruction (HMR) to date, featuring 661k frames, 50 actions, and 20 subjects. It provides synchronized RGB, Depth, Raw Radar Tensor (RT), and Radar Point Cloud (RPC) modalities with high-fidelity 3D mesh annotations based on optical motion capture (MoCap). It also introduces RT-Mesh, the first lightweight baseline for direct HMR from RT.
PHASE-Net: Physics-Grounded Harmonic Attention System for Efficient Remote Photoplethysmography Measurement: Starting from the Navier-Stokes equations, this work reveals through rigorous mathematical derivation that the rPPG pulse signal follows a second-order damped harmonic oscillator model. Its discrete solution is equivalent to a causal convolution operator, providing a first-principles justification for the choice of TCN architectures. The resulting PHASE-Net, with only 0.29M parameters, achieves SOTA performance across multiple datasets.
UniLS: End-to-End Audio-Driven Avatars for Unified Listening and Speaking: The first end-to-end unified speaking-listening facial expression generation framework, UniLS, is proposed. Through a two-stage training paradigm (learning intrinsic motion priors first, followed by dual-track audio fine-tuning), it simultaneously generates natural speaking and listening facial movements from dyadic audio inputs, achieving up to a 44.1% improvement in listening metrics.

Browse all 151 Human Understanding papers →

📹 Video Understanding (187)¶

WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning: This paper proposes WorldMM, a video reasoning agent based on multimodal memory. It constructs three complementary types of memory: episodic memory (multi-time-scale textual knowledge graph), semantic memory (continuously updated relational knowledge graph), and visual memory (frame-level retrieval library). Through an adaptive multi-round retrieval agent, it dynamically selects the most relevant memory sources and temporal granularities, outperforming the previous SOTA by an average of 8.4% across five long-video QA benchmarks.
StreamingTOM: Streaming Token Compression for Efficient Video Understanding: StreamingTOM is proposed, a training-free two-stage streaming video understanding framework: Causal Temporal Reduction (CTR) performs causal temporal selection before the LLM to compress tokens per frame from 196 to 50; Online Quantized Memory (OQM) limits kv-cache growth after the LLM via 4-bit quantization and on-demand retrieval. The framework achieves a 15.7× compression ratio, 1.2× lower peak VRAM, and 2× faster TTFT.
VideoITG: Multimodal Video Understanding with Instructed Temporal Grounding: VideoITG reformulates "selecting frames based on user instructions" as a standalone temporal grounding task. By utilizing a GPT-4o-driven three-stage pipeline (VidThinker), it automatically annotates "which frames are relevant to an instruction" across 40K videos, generating 500K instruction-aligned annotations. A plug-and-play frame selector is then trained and prepended to various Video-LLMs, achieving or exceeding the performance of 64-frame uniform sampling using only 16–32 frames.
FluxMem: Adaptive Hierarchical Memory for Streaming Video Understanding: FluxMem is a training-free streaming video understanding framework that utilizes a hierarchical memory design (Short/Medium/Long-term) and two adaptive token compression modules (TAS for temporal redundancy + SDC for spatial redundancy). It achieves new SOTA on StreamingBench and OVO-Bench while discarding 60-70% of visual tokens.
Time Blindness: Why Video-Language Models Can't See What Humans Can?: Authors constructed SpookyBench, a synthetic benchmark where information exists "purely in inter-frame temporal dynamics while single frames are total noise." While humans can read text or identify objects with 98% accuracy using motion grouping, 15 state-of-the-art Video-VLMs (including GPT-4o, Gemini 2.5 Pro, and Qwen2.5-VL-72B) all achieved 0% accuracy. This clearly exposes a "time blindness" in current video models—they rely on per-frame spatial features and lack mechanisms for processing pure temporal information.
SDTrack: A Baseline for Event-based Tracking via Spiking Neural Networks: This paper proposes SDTrack, the first fully Spiking Neural Network (SNN) based Transformer pipeline for event tracking. By utilizing Global Trajectory Prompt (GTP), asynchronous event streams are aggregated into 3-channel event frames rich in trajectory information. A full spike-driven SNN Transformer tracker, featuring Intrinsic Position Learning (IPL), predicts target boxes end-to-end. SDTrack achieves competitive or SOTA accuracy on three event-tracking benchmarks with minimal parameters and energy consumption (Tiny version: 19.61M / 8.16mJ).
StreamReady: Learning What to Answer and When in Long Streaming Videos: This paper proposes a readiness-aware streaming video understanding paradigm. By introducing a learnable <RDY> token and the Answer Readiness Score (ARS) metric, the model learns not only to provide correct answers but also to answer at the precise moment evidence appears. It achieves SOTA performance across 9 streaming and offline video benchmarks.
MA-Bench: Towards Fine-grained Micro-Action Understanding: Ours proposes MA-Bench, a micro-action understanding benchmark containing 1,000 videos and 12,000 structured QA pairs. It systematically evaluates the fine-grained micro-action understanding capabilities of 23 MLLMs through a three-layer "Perception-Understanding-Reasoning" architecture and provides MA-Bench-Train (20.5K samples) for model fine-tuning.
GoalForce: Teaching Video Models to Accomplish Physics-Conditioned Goals: Proposes the Goal Force framework which trains video generation models on simple synthetic data using multi-channel physical control signals (goal force, direct force, mass). This enables models to learn reverse planning of causal chains from target effects, achieving zero-shot generalization to complex real-world scenarios such as tool use and human-object interaction.
SoccerMaster: A Vision Foundation Model for Soccer Understanding: SoccerMaster utilizes a shared spatio-temporal ViT encoder and five lightweight task heads to integrate four categories of "spatial perception + semantic reasoning" tasks—player detection/identification, pitch registration, event classification, and vision-language alignment—into a single supervised multi-task pre-training stage. Supported by an automated annotation pipeline, SoccerFactory, which mass-produces dense spatial labels, the model outperforms general vision foundation models (SigLIP 2 / DINOv3) and specialized soccer models (MatchVision) across downstream tasks such as detection, tracking, camera calibration, and commentary generation.

Browse all 187 Video Understanding papers →

🚗 Autonomous Driving (157)¶

DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving: DriveMoE integrates Mixture-of-Experts (MoE) into both the perception and decision-making components of a VLA autonomous driving model. The perception side utilizes a Vision MoE to dynamically select critical camera views to save tokens, while the decision-making side employs an Action MoE to allocate dedicated experts for different driving skills. On the Bench2Drive closed-loop benchmark, it improves the Driving Score (DS) from 55.85 to 74.22 and the Success Rate (SR) from 30% to 48.64%.
WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarios: Waymo extracted 4,021 long-tail driving segments (approx. 12 hours) with an occurrence frequency below 0.03% from 6.4 million miles of real-world road tests to create the WOD-E2E dataset. It proposes the RFS (Rater Feedback Score), an open-loop metric based on human expert preference scores, to replace ADE (which only measures distance error against a single future trajectory). This allows for a fair evaluation of vision-based end-to-end models in safety-critical scenarios where "multiple reasonable trajectories coexist."
SimScale: Learning to Drive via Real-World Simulation at Scale: The authors propose SimScale, a framework that generates large-scale, high-fidelity simulation data by applying trajectory perturbation to existing driving logs, followed by reactive environment simulation and neural rendering. Combined with pseudo-expert trajectory supervision and a sim-real co-training strategy, the end-to-end planner achieves significant improvements on NAVSIM v2 (+8.6 EPDMS on navhard), with performance scaling smoothly with the volume of simulated data.
SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Driving: SGDrive explicitly injects a hierarchical world knowledge set of "scene geometry-key agents-short-term goals" into a Vision-Language Model (VLM). It uses a set of trainable <world> queries to predict current and future world states, then translates this knowledge into trajectories via a DiT diffusion planner, achieving SOTA on the NAVSIM camera-only track (PDMS 87.4, 91.1 after RL).
GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generation: GaussianDWM utilizes "Language-enhanced 3D Gaussians" as a unified scene representation. By embedding CLIP language features into each Gaussian ellipsoid, it achieves explicit alignment between text and 3D geometry. Through task-aware sampling, compact 3D tokens are fed into an LLM for scene understanding (description/2D-3D grounding/planning), while dual-condition diffusion performs RGB-D spatiotemporal generation. On the NuInteract understanding task, the average score improved from 52.12 to 59.23; on nuScenes spatial generation, the FID for \(\pm 2m\) offset was reduced to 11.27.
SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving: SpaceDrive replaces the conventional practice in VLM-based end-to-end driving—generating coordinates digit-by-digit as text—with a unified 3D Position Encoding (PE). The same sine-cosine PE is superimposed on visual tokens, used to replace coordinate tokens in text, and used to encode ego-states. Finally, a regressive PE decoder outputs trajectory coordinates directly. It achieves SOTA among VLM methods in nuScenes open-loop benchmarks and a second-best score of 78.02 in Bench2Drive closed-loop testing.
GuideFlow: Constraint-Guided Flow Matching for Planning in End-to-End Autonomous Driving: GuideFlow employs "Flow Matching + Energy-Based Models" for end-to-end driving planning, directly embedding safety and physical hard constraints into the generation process via three mechanisms: Constrained Velocity Field (CVF), Constrained Flow states (CF), and Refining the Flow by EBM (RFE). This approach mitigates multi-modal mode collapse in imitation learning and eliminates the need for post-optimization in generative methods, achieving a SOTA 43.0 EPDMS on NavSim Navhard.
ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous Driving: ActiveAD designs a "planning-oriented" active learning strategy for end-to-end autonomous driving: it uses nearly free meta-information (weather/lighting/driving commands/speed) for diversity initialization to solve the cold-start problem, and selects the most critical scenarios using three label-free criteria: displacement error, soft collision, and agent uncertainty. Training on only 30% of the data matches the performance of SOTA models trained on 100% data in both nuScenes open-loop and CARLA closed-loop evaluations.
LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving: This paper identifies that the root cause of the "student failing to learn the privileged expert" in CARLA is not insufficient model capacity, but rather the expert's use of privileged information that is invisible or unmeasurable for the student, combined with sparse navigation intent. By constraining the expert's perception and decision-making to the student's observable range (LEAD expert + dataset) and restructuring the target point injection in the student policy (TFv6), this work achieves 95 DS on Bench2Drive and more than doubles previous SOTA performance on Longest6 v2 / Town13.
WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World: WorldLens proposes a full-spectrum evaluation benchmark for driving world models covering five dimensions—"generation, reconstruction, action-following, downstream tasks, and human preference"—with a total of 24 fine-grained metrics. Along with the WorldLens-26K human-annotated dataset and the distilled interpretable auto-evaluator WorldLens-Agent, it systematically reveals that current world models "look real but behave unreal"; no single model leads across all dimensions simultaneously.

Browse all 157 Autonomous Driving papers →

🤖 Robotics & Embodied AI (146)¶

Motus: A Unified Latent Action World Model: Motus employs a Mixture-of-Transformers (MoT) architecture to integrate three pre-trained experts—Understanding, Video Generation, and Action—via shared self-attention (Tri-model Joint Attention) and UniDiffuser-style asynchronous scheduling. It unifies five embodied paradigms—VLA, World Model, IDM, video generation, and joint video-action prediction—within a single model. By extracting pixel-level "latent actions" from optical flow, the action expert can be pre-trained on massive unlabeled videos. Motus outperforms \(\pi 0.5\) by 45% and X-VLA by 15% in simulation, with real-world improvements ranging from 11% to 48%.
InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policy: InternData-A1 utilizes a fully decoupled and autonomous simulation synthesis pipeline to generate 630,000 trajectories (7,433 hours) of high-fidelity robotic manipulation data. It demonstrates for the first time that a VLA model pre-trained solely on "purely synthetic data" can match the performance of the official \(\pi 0\) pre-trained on the closed-source real-world \(\pi\text{-dataset}\) across 49 simulation and 9 real-world tasks.
RehearseVLA: Simulated Post-Training for VLAs with Physically-Consistent World Model: RehearseVLA (World-Env) utilizes a "physically-consistent video world model" as a virtual training playground, allowing VLA policies to safely undergo RL post-training within imagined future observations. Coupled with a VLM reflector that provides continuous rewards and real-time task completion determination, it lifts the average success rate on LIBERO from 74.85% to 79.6% under the extreme data scarcity of only 5 expert demonstrations per task.
VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation: VIRAL trains visual policies for humanoid "loco-manipulation" entirely in simulation. Through "privileged teacher \(\rightarrow\) RGB student" distillation, large-scale visual domain randomization, and real-to-sim alignment, it deploys RGB-only policies zero-shot onto the Unitree G1. The robot can continuously walk, grasp, and place objects between two tables for 54 consecutive cycles, achieving performance close to expert teleoperation.
Evo-1: Lightweight Vision-Language-Action Model with Preserved Semantic Alignment: Evo-1 utilizes a native multimodal VLM with only 0.77B parameters as the backbone, paired with a pure cross-attention flow-matching diffusion action expert and a "freeze-then-fine-tune" two-stage training strategy. Without any robot data pre-training, it achieves SOTA on Meta-World, RoboTwin, and LIBERO by preserving the VLM's semantic space, reaching a 78% success rate in real-world tests with 16.4 Hz inference and only 2.3 GB VRAM.
AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions: AGENTSAFE is the first benchmark to systematically evaluate the safety of "embodied VLM agents executing hazardous instructions." It utilizes an adversarial simulation sandbox (SAFE-THOR) that interfaces with arbitrary agents, a collection of 9,900 hazardous instructions categorized by the "Three Laws of Robotics" (SAFE-VERSE), and a fine-grained diagnostic protocol (SAFE-DIAGNOSE) spanning the "perception-planning-execution" stages. The study evaluates 9 VLMs and 2 agent workflows, revealing a systemic failure where current agents "recognize danger but fail to incorporate this cognition into planning and execution," and proposes a thought-level defense module called SAFE-AUDIT.
CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learning: Ours proposes CoMo, which synergistically addresses the shortcut learning problem in continuous latent motion learning through two mechanisms: Early Temporal Differencing (Td) and Temporal Contrastive Learning (Tcl). It extracts fine-grained continuous pseudo-action labels from internet videos, allowing video data and robot actions to be co-trained under a unified continuous distribution, significantly improving policy performance.
OctoNav: Towards Generalist Embodied Navigation: OctoNav unifies five fragmented navigation tasks—ObjNav, PointNav, ImgNav, Ins-ImgNav, and VLN—into a single "free-form, multi-modal, multi-capability" instruction format. The work releases OctoNav-Bench, containing 45k+ instruction-trajectory pairs, and the TBA-CoT dataset with reasoning chains. It introduces OctoNav-R1 (based on LLaMA-VID), a VLA model that "thinks before acting" trained via a three-stage Hybrid Training Paradigm (SFT, GRPO, and online RL), improving the overall success rate from the previous best of 9.2% to 19.4% in a unified setting.
Gallant: Voxel Grid-based Humanoid Locomotion and Local-navigation across 3D Constrained Terrains: Gallant voxelizes vehicle-grade LiDAR point clouds into robot-centric occupancy grids, utilizing a lightweight 2D CNN that treats the z-axis as channels for end-to-end mapping to whole-body control strategies. By incorporating high-fidelity LiDAR simulation that accounts for the robot's own limbs, a single policy achieves zero-shot sim-to-real transfer. It marks the first instance of achieving >90% success rates in tasks like stair climbing and high-platform mounting while covering ground, lateral, and overhead obstacles simultaneously.
Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer: DoorMan employs a three-stage "teacher-student-bootstrap" pipeline to train a pure RGB-input humanoid whole-body door-opening policy in IsaacLab via large-scale physical and visual randomization. This policy transfers zero-shot to various real-world doors, achieving completion times up to 31.7% faster than human teleoperation.

Browse all 146 Robotics & Embodied AI papers →

🎮 Reinforcement Learning (25)¶

CME-CAD: Heterogeneous Collaborative Multi-Expert Reinforcement Learning for CAD Code Generation: Aiming at the industrial scenario of "directly generating executable and editable CAD code from 2D engineering triple-views," CME-CAD enables multiple heterogeneous pre-trained large models to act as "experts" with distinct styles. It first employs Multi-Expert Fine-Tuning (MEFT) using their respective reasoning styles, followed by a Multi-Expert Reinforcement Learning (MERL) stage. In MERL, strong experts transfer superior strategies to weak experts via KL distillation, and a Hard Sample Buffer mechanism is used to repeatedly tackle the most difficult samples. Ultimately, on the self-built industrial-grade benchmark CADExpert, the IoU is improved from 71.84% to 80.71%, and the code execution rate reaches 98.25%.
PlannerRFT: Reinforcing Diffusion Planners through Closed-Loop and Sample-Efficient Fine-Tuning: PlannerRFT performs reinforcement fine-tuning for diffusion-based autonomous driving planners: it uses "policy-guided denoising" to transform modal-collapsed diffusion sampling into diverse and scene-adaptive trajectory groups, then applies a dual-branch closed-loop optimization with GRPO + PPO, supported by the self-developed 10× accelerated simulator nuMax, achieving SOTA closed-loop planning performance on nuPlan.
ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering: ReAG is proposed as a reasoning-augmented multimodal RAG method that combines coarse and fine-grained retrieval with a Critic filtering model to reduce noise. It employs GRPO reinforcement learning to train the generator for explicit reasoning, achieving a new SOTA on knowledge-intensive VQA tasks.
GeoWorld: Geometric World Models: GeoWorld maps the latent representations of predictive world models from Euclidean space onto hyperbolic manifolds. By maintaining geometric structures and hierarchical relationships through Hyperbolic JEPA and employing Geometric Reinforcement Learning to optimize multi-step planning, it achieves improvements of approximately 3% SR (3 steps) and 2% SR (4 steps) on CrossTask and COIN.
PanoEnv: Exploring 3D Spatial Intelligence in Panoramic Environments with Reinforcement Learning: To address the near-collapse of VLM 3D spatial reasoning on 360° Equirectangular Projection (ERP) panoramas, this work constructs the PanoEnv-QA benchmark with 14.8K questions across five geometrically aligned categories. By employing GRPO post-training with "task-routed ground-truth rewards" and a "two-stage curriculum," the total accuracy of a 7B model is improved from 49.34% to 52.93%, and open-ended question accuracy rises from 6.39% to 14.83%, surpassing 32B models.
MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning: Proposes the Multi-Stage Reinforcement Learning (MSRL) method, which first learns reward reasoning capabilities on large-scale text preference data and then progressively transfers them to multimodal tasks. This addresses the bottleneck of scarce annotated data in multimodal reward model training, improving accuracy on VL-RewardBench from 66.6% to 75.9%.
CCCaption: Dual-Reward Reinforcement Learning for Complete and Correct Image Captioning: The authors propose the CCCaption dual-reward reinforcement learning framework. By jointly optimizing image captioning completeness (based on visual query sets generated by multiple MLLMs) and correctness (based on hallucination detection of decomposed sub-queries), the 2B model outperforms the 32B baseline.
Talk2Move: Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenes: Talk2Move models "translating/rotating/scaling an object in a scene based on text instructions" as an RL problem. It utilizes Flow-GRPO for exploration on diffusion trajectories with spatial rewards, eliminating the need for paired supervision data. By employing early-exit sampling, it accelerates training by \(2\times\). It significantly outperforms existing editing models like GPT-Image-1, Flux-Kontext, and QwenImageEdit in terms of spatial accuracy and scene consistency.
Cloning Deterministic Worlds: The Critical Role of Latent Geometry in Long-Horizon World Models: Through an "oracle" diagnostic experiment, the authors demonstrate that the bottleneck of long-horizon collapse in world models within deterministic environments is not the dynamics model, but rather the geometric structure of the latent representation. Consequently, they propose GRWM, which utilizes temporal contrastive learning as a geometric regularization term to reshape the latent space of the autoencoder to align with the true environmental state manifold. As a plug-and-play module, GRWM significantly extends the faithful prediction horizon of various world models.
EVA: Efficient Reinforcement Learning for End-to-End Video Agent: EVA models long video understanding as a "planning-before-perception" Markov Decision Process (MDP), enabling the MLLM agent to decide "which segment to watch, how many frames to sample, and at what resolution" based solely on the text question. Through a three-stage training pipeline (SFT Cold Start \(\rightarrow\) KTO Offline Correction \(\rightarrow\) Data-Enhanced GRPO), the model evolves from a format imitator to an active video explorer. It achieves a 6–12% accuracy improvement over general MLLMs and a 1–3% gain over existing adaptive agents using approximately 1/10 of the visual tokens across six video benchmarks.

Browse all 25 Reinforcement Learning papers →

🔄 Self-Supervised Learning (92)¶

NitroGen: An Open Foundation Model for Generalist Gaming Agents: NitroGen treats "controller input overlays used by players in livestreams" as natural action labels. It automatically extracts (frame, action) pairs from 40,000 hours of public videos covering 1,000+ games. By training a single vision-action Transformer using flow-matching, the model can directly play various 2D/3D games. Pre-trained weights provide a maximum relative success rate improvement of 52% when fine-tuned on unseen games.
Vision Transformers Need More Than Registers: This paper argues that prevalent dense feature artifacts in ViT under label, text, and self-supervision are not merely high-norm token issues, but a consequence of the model learning to use background patches as global semantic shortcuts under the combined influence of coarse-grained supervision and global attention. To address this, the authors propose LaSt-ViT, which replaces original CLS aggregation with selective aggregation guided by frequency-domain stability, consistently improving localization, segmentation, and open-vocabulary tasks across 12 benchmarks.
Residual Connections Harm Generative Representation Learning: The authors discover that the "identity shortcut" in residual connections injects shallow high-frequency details directly into deep layers, suppressing semantic abstraction. They propose Decayed Identity Shortcuts—an architectural modification where the weight of the identity shortcut decays monotonically with layer depth. With only one additional hyperparameter \(\alpha_{\min}\) and zero extra parameters, this method improves the KNN accuracy of MAE on ImageNet-1K from 27.4% to 63.9% and linear probing from 67.8% to 72.7%, while also enhancing the generation quality of diffusion models.
OpenVision 2: A Family of Generative Pretrained Visual Encoders for Multimodal Learning: OpenVision 2 removes the text encoder and contrastive loss from the previous generation (OpenVision), retaining only the "image encoder + text decoder" for pure generative caption-only pretraining. By randomly masking approximately 2/3 of visual tokens, it reduces ViT-L/14 training time by ~1.5× and memory usage by ~1.8× with almost no performance degradation, while enabling scaling the visual encoder up to 1 billion parameters.
In Pursuit of Pixel Supervision for Visual Pre-training: The authors scale MAE back up to web-scale data, proposing a reconstruction-loss-based "spatial data self-filtering" strategy named MetaCLIP-S. Coupled with four minimal algorithmic modifications (a deeper decoder, larger mask blocks, and multiple CLS tokens), they train a model named Pixio. This model matches or exceeds DINOv2/v3 on dense prediction tasks such as depth estimation, feed-forward 3D reconstruction, and segmentation, despite DINOv2/v3 relying on extensive benchmark-specific curated filtering.
How Much 3D Do Video Foundation Models Encode?: The authors propose the first model-agnostic probing framework, using "frozen video foundation model features + shallow feed-forward heads predicting 3D point clouds/depth/camera poses" to quantify the internal 3D understanding of various video models. The conclusion is that leading video generation models trained only on 2D videos (such as WAN2.1-14B) exhibit strong emergent 3D perception, even surpassing expert models trained specifically on 3D data (e.g., Fast3R) in cross-domain scenarios.
Can You Learn to See Without Images? Procedural Warm-Up for Vision Transformers: Before a ViT officially processes images, a lightweight masked-token pre-training (warm-up) is conducted using purely symbolic sequences without any visual content (e.g., "balanced parentheses") generated by formal grammars. This forces the model to internalize universal computational mechanisms such as stack-based hierarchy and long-range dependencies. When followed by standard image training, this approach achieves a +1.72% top-1 gain on ImageNet-1K with only a 1% training budget expansion, effectively substituting for 28% of image data.
TeFlow: Enabling Multi-frame Supervision for Self-Supervised Feed-forward Scene Flow Estimation: TeFlow is proposed as the first method to introduce multi-frame supervision into self-supervised feed-forward scene flow estimation. By employing a temporal ensemble strategy to construct a motion candidate pool and aggregating temporally consistent supervision signals via consensus voting, it achieves a Three-way EPE of 3.57 cm on Argoverse 2—comparable to the optimization-based method Floxels—while maintaining real-time inference (8s vs. 24min), a 22.3% improvement over SeFlow++.
DiverseDiT: Towards Diverse Representation Learning in Diffusion Transformers: Systematic analysis reveals that representation diversity among DiT blocks is a key factor for effective learning. This paper proposes DiverseDiT: using long residual connections to diversify inputs and a representation diversity loss to explicitly promote feature differentiation between blocks, accelerating convergence and improving generation quality without external guidance models.
Learning to See Through a Baby's Eyes: Early Visual Diets Enable Robust Visual Intelligence in Humans and Machines: The authors encode three laws of infant visual development—grayscale to color, blurry to clear, and temporal continuity—into a "visual diet" for self-supervised training called CATDiet. Training SSL models solely on object-centric videos yields more robust recognition of corrupted images, shape bias, and depth perception across ten datasets. Furthermore, the models spontaneously demonstrate developmental signals consistent with macaque V1 synapse density and infant visual cliff behavior. A two-stage "CombDiet" is proposed as a warm-up for standard SSL, consistently outperforming conventional SSL.

Browse all 92 Self-Supervised Learning papers →

📐 Optimization & Theory (22)¶

DC-Merge: Improving Model Merging with Directional Consistency: DC-Merge discovers that the key to model merging lies in maintaining directional consistency in singular space between the merged multi-task vector and the original single-task vectors. Through a two-step process of singular value smoothing and projection onto a shared orthogonal subspace, it achieves SOTA results on both Vision and Vision-Language tasks.
Learning to Learn Weight Generation via Local Consistency Diffusion: Mc-Di combines the bi-level optimization of meta-learning with diffusion-based weight generation and transforms the diffusion process from learning only "globally optimal weights" to "local consistency diffusion." By reconstructing weights segmentally along multiple intermediate points on the optimization trajectory, the model achieves higher accuracy and lower inference latency in tasks requiring frequent weight updates, such as transfer learning, few-shot learning, domain generalization, and language model fine-tuning.
UniFusion: A Unified Image Fusion Framework with Robust Representation and Source-Aware Preservation: The authors propose UniFusion, a unified image fusion framework that leverages DINOv3 self-supervised semantic priors to construct a cross-modal shared feature space. It preserves source image information via a reconstruction alignment mechanism and decouples reconstruction and fusion objectives using a bilevel optimization strategy, achieving SOTA performance across tasks such as infrared-visible, multi-exposure, multi-focus, and medical image fusion.
Enhancing Visual Representation with Textual Semantics: Textual Semantics-Powered Prototypes for Heterogeneous Federated Learning: Addressing the issue where existing Federated Prototypical Learning methods destroy inter-class semantic relations, the proposed FedTSP method utilizes pre-trained language models to construct textual prototypes that preserve semantic structures, significantly improving performance and accelerating convergence in heterogeneous federated learning.
Model Merging in the Essential Subspace: The ESM framework is proposed to construct an "essential subspace" by performing PCA on activation offsets caused by parameter updates (rather than directly applying SVD to parameters). It utilizes three-level polarization scaling to enhance key parameters and suppress noise, achieving a 3.2% absolute accuracy improvement over Iso-CTS in a 20-task merging scenario with ViT-B/32.
Defending Unauthorized Model Merging via Dual-Stage Weight Protection: Ours proposes MergeGuard, an active dual-stage weight protection framework: Stage 1 disperses task-critical weights through L2 regularization, and Stage 2 injects structured perturbations to disrupt merging compatibility. It maintains <1.5% original performance loss for the protected model while causing up to 90% accuracy degradation in merged models.
Fed-ADE: Adaptive Learning Rate for Federated Post-adaptation under Distribution Shift: Proposes the Fed-ADE framework, which utilizes two lightweight distribution shift signals—uncertainty dynamics estimation and representation dynamics estimation—to adaptively adjust the learning rate for each client at each time step, achieving unsupervised post-deployment adaptation in federated learning.
ACE-Merging: Data-Free Model Merging with Adaptive Covariance Estimation: This paper theoretically proves that fine-tuned parameter differences contain input covariance information. Accordingly, it proposes ACE-Merging, which achieves data-free closed-form model merging through a three-step process: adaptive covariance estimation, collective structure priors, and spectral refinement. It achieves an average improvement of 4% on GPT-2 and 5% on RoBERTa-Base compared to previous methods.
Conditional Factuality Controlled LLMs with Generalization Certificates via Conformal Sampling: Ours proposes CFC (Conditional Factuality Control), a post-hoc conformal framework that learns feature-conditioned acceptance thresholds via augmented quantile regression. It provides conditional coverage guarantees for LLM/VLM sampled outputs, significantly improving reliability for difficult subgroups while maintaining compact prediction sets.
Mapping Networks: This paper proposes Mapping Networks—a "meta-parameterization" method that utilizes a low-dimensional trainable latent vector \(z\) (coupled with fixed mapping weights modulated by \(z\)) to generate all parameters of a target network. By shifting the training process from a high-dimensional weight space to a low-dimensional latent space, the method achieves or exceeds the accuracy of the original network on tasks such as image classification, deepfake detection, and segmentation with approximately 500× fewer trainable parameters, while significantly suppressing overfitting.

Browse all 22 Optimization & Theory papers →

🔗 Causal Inference (4)¶

Retrieving Counterfactuals Improves Visual In-Context Learning: The CIRCLES framework is proposed to retrieve counterfactual examples through attribute-guided composed image retrieval, constructing a dual-channel in-context demonstration of "causality + correlation" to significantly enhance the fine-grained visual reasoning capabilities of VLMs.
MaskDiME: Adaptive Masked Diffusion for Precise and Efficient Visual Counterfactual Explanations: MaskDiME is proposed, a training-free diffusion framework that transforms global classifier guidance into decision-driven local editing via an adaptive dual-masking mechanism. This achieves precise and efficient visual counterfactual explanations, with inference speeds over 30 times faster than DiME and GPU memory consumption only one-tenth that of ACE/RCSB.
A Polynomial Chaos Framework for Causal Discovery in Nonlinear Uncertain Systems: This paper embeds noise terms into structural equations using Polynomial Chaos Expansion (PCE) to develop PCE-LiNGAM. It proves that causal Directed Acyclic Graphs (DAGs) are uniquely identifiable under mild sparsity conditions. Using a polynomial-time algorithm involving "PCE signature contamination testing + recursive sink finding," the method improves average F1 scores from 0.50 to 0.756 on extreme non-Gaussian industrial data while providing uncertainty quantification based on Sobol indices.
CGU-Bayes: Causal Graph Uncertainty-Guided Bayesian Inference for Domain Generalization: Addressing the issue that "causal graphs are inaccurately estimated under data scarcity or noise when using Structural Causal Models (SCM) for domain generalization," this paper moves away from point-estimating a single causal graph. Instead, it performs Bayesian inference on the causal graph posterior, selects a set of Causal Markov Blanket (CMB) features from each sampled graph to train predictors, and performs a weighted ensemble using the "alignment uncertainty" between each graph and the test samples. This approach achieves SOTA performance on datasets with strong distribution shifts, such as BLT and CMNIST.

🔬 Interpretability (34)¶

Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generation: The authors observe that multilingual text-to-image (T2I) models generate culturally neutral or Anglo-centric images when prompted with only nouns. Through attention and sparse autoencoder (SAE) probing, they demonstrate that this is a case of "under-activation" rather than a "lack of knowledge"—cultural signals are actually concentrated in only a few layers and a few specific neurons of the text encoder. Based on this, they propose two lightweight solutions: a training-free amplification of these neurons and fine-tuning only the cultural layer. On their self-constructed 15-country benchmark CultureBench, they improve the culture recognition accuracy (CultureVQA) from ~22% to 36.6%.
ERMoE: Eigen-Reparameterized Mixture-of-Experts for Stable Routing and Interpretable Specialization: ERMoE proposes reparameterizing MoE expert weights within an orthogonal eigenbasis and substituting traditional routing logits with eigenbasis alignment scores (cosine similarity), enabling stable routing and interpretable expert specialization without the need for auxiliary load balancing losses.
SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse World: Ours proposes the SafeDrive end-to-end planning framework, which simulates the future behavior of key entities through a trajectory-conditioned Sparse World Network (SWNet). It then employs a Fine-grained Reasoning Network (FRNet) for per-instance collision assessment and per-timestep drivable area compliance evaluation. SafeDrive achieves 91.6 PDMS and a collision rate of only 0.5% on NAVSIM, alongside a 66.8% driving score on Bench2Drive.
Cut to the Chase: Training-free Multimodal Summarization via Chain-of-Events: This paper proposes CoE, a training-free multimodal summarization framework. By constructing a Hierarchical Event Graph (HEG) to guide chain-of-event reasoning, it surpasses SOTA video CoT baselines on 8 datasets, achieving an average improvement of +3.04 ROUGE, +9.51 CIDEr, and +1.88 BERTScore.
HUMORCHAIN: Theory-Guided Multi-Stage Reasoning for Interpretable Multimodal Humor Generation: HUMORCHAIN explicitly encodes four major humor theories—Incongruity-Resolution, Benign Violation, Superiority, and Relief—into a multi-stage LLM reasoning chain ("Visual Parsing → Strategy Selection → Generation → Discriminator Feedback"). A Qwen3-VL-4B humor discriminator is trained for a "generation-evaluation-rewriting" loop, outperforming existing methods in human preference, Elo/BT scores, and semantic diversity across three datasets.
RiskProp: Collision-Anchored Self-Supervised Risk Propagation for Early Accident Anticipation: RiskProp is proposed as a self-supervised risk propagation paradigm anchored by the collision frame. By utilizing future frame regularization and adaptive monotonic constraint losses, the model learns temporally coherent risk evolution curves relying solely on collision frame annotations, achieving SOTA performance on CAP and Nexar datasets.
Edit-As-Act: Goal-Regressive Planning for Open-Vocabulary 3D Indoor Scene Editing: This work redefines open-vocabulary 3D indoor scene editing as a goal-regressive planning problem. It introduces the PDDL-style symbolic language EditLang and an LLM-driven Planner-Validator loop to derive minimal editing sequences from target states. The method achieves the best balance across instruction faithfulness (69.1%), semantic consistency (86.6%), and physical plausibility (91.7%) across 63 editing tasks.
Missing No More: Dictionary-Guided Cross-Modal Image Fusion under Missing Infrared: This paper proposes the first framework to perform cross-modal fusion under missing infrared conditions in the coefficient domain rather than the pixel domain. By establishing a unified IR-VIS atomic space via a shared convolutional dictionary, it completes VIS→IR reasoning and adaptive fusion within the coefficient domain. Combined with a frozen LLM providing weak semantic priors for thermal information completion, the method achieves performance close to dual-modal fusion methods using only visible light input.
Rethinking Concept Bottleneck Models: From Pitfalls to Solutions: The CBM-Suite framework is proposed to systematically address four pitfalls of Concept Bottleneck Models (CBMs): the lack of pre-evaluation metrics for concept relevance, the linearity problem causing bottlenecks to be bypassed, the accuracy gap compared to black-box models, and the research gap regarding the impact of different visual backbones/VLMs. This is achieved through entropy measures, non-linear layers, and distillation losses, significantly enhancing both accuracy and interpretability.
Improving Sparse Autoencoder with Dynamic Attention: This paper reformulates the Sparse Autoencoder (SAE) into a cross-attention architecture with shared concept vectors and replaces softmax with sparsemax. This allows each sample to automatically determine the number of activated concepts based on its own complexity, overcoming the inherent "setting K" problem in TopK SAEs to achieve lower reconstruction error and clearer concepts in both image and text domains.

Browse all 34 Interpretability papers →

📦 Model Compression (108)¶

Phased DMD: Few-step Distribution Matching Distillation via Score Matching within Subintervals: Addressing the dilemma where 1-step DMD distillation suffers from insufficient capacity and poor diversity, while direct multi-step expansion leads to VRAM explosion or performance degradation back to 1-step levels when using Stochastic Gradient Truncation (SGTS), this paper proposes Phased DMD. By partitioning the SNR range into subintervals and distilling one expert per phase moving progressively towards higher SNR (with intermediate phases stopping at intermediate timesteps rather than clean samples), the authors derive an unbiased subinterval score matching objective for scenarios lacking clean samples. This naturally produces few-step MoE generators that improve motion dynamics, visual fidelity, and generation diversity on large models such as Qwen-Image-20B and Wan2.2-28B.
Bilevel Layer-Positioning LoRA for Real Image Dehazing: Ours proposes BiLaLoRA, which automatically locates the optimal network layers for LoRA insertion through bilevel optimization. Combined with H2C Loss (an unsupervised dehazing loss based on CLIP semantic directions), it achieves efficient adaptation of synthetic-data pre-trained dehazing models to real-world scenarios—reducing training time by 77.7% while maintaining performance comparable to full fine-tuning across models and domains.
Fixed Anchors Are Not Enough: Dynamic Retrieval and Persistent Homology for Dataset Distillation: RETA decouples two failure modes of residual matching in dataset distillation (fit-complexity gap and pull-to-anchor effect). It adaptively selects real patch anchors via Dynamic Retrieval Connection (DRC) and preserves intra-class diversity through Persistent Topology Alignment (PTA), achieving 64.3% on ImageNet-1K ResNet-18 IPC=50 (+3.1% vs. FADRM).
HierAmp: Coarse-to-Fine Autoregressive Amplification for Generative Dataset Distillation: Ours proposes HierAmp, which injects learnable category tokens into the coarse-to-fine generation process of Visual Autoregressive (VAR) models to identify semantically salient regions. By amplifying attention in these regions through positive logit bias, the distilled data achieves richer layout diversity at coarse scales and focuses on category-related details at fine scales, reaching SOTA performance on multiple dataset distillation benchmarks.
Progressive Supernet Training for Efficient Visual Autoregressive Modeling: VARiant identifies a "scale-depth asymmetric dependence" in Visual Autoregressive (VAR) models: early low-resolution scales are highly dependent on network depth, while later high-resolution scales are robust to depth reductions. Based on this, a 30-layer VAR is trained as a weight-sharing elastic depth supernet (early scales use the full network; late scales use 2–16 layer subnets). Using a three-stage dynamic ratio progressive training to break the fixed-ratio Pareto frontier, d16/d8 subnets achieve near-lossless performance on ImageNet (FID 2.05/2.15 vs. 1.95) while saving 40–65% GPU memory.
Towards Generalizable AI-Generated Image Detection via Image-Adaptive Prompt Learning: Ours proposes Image-Adaptive Prompt Learning (IAPL), which dynamically adjusts the prompts of the CLIP encoder for each test image during inference. By integrating test-time token tuning and a conditional information learner, it achieves strong generalization to unseen generators, reaching state-of-the-art (SOTA) performance with average accuracies of 95.61% and 96.7% on UniversalFakeDetect and GenImage, respectively.
Masking Teacher and Reinforcing Student for Distilling Vision-Language Models: Masters bridges the teacher-student capacity gap through a progressive strategy that masks teacher weights by magnitude and gradually restores them during training. This is combined with offline RL driven by accuracy and distillation transferability rewards, enabling compact VLMs to stably absorb knowledge and outperform same-sized models across 13 multimodal benchmarks.
ThinkingViT: Matryoshka Thinking Vision Transformer for Elastic Inference: ThinkingViT integrates a progressive mechanism—"predict quickly with fewer heads, rethink by expanding the sub-network if uncertain"—into a nested ViT. By utilizing Token Recycling to feed features from previous stages into subsequent rounds, it outperforms nested baselines like MatFormer and HydraViT on ImageNet-1K by up to 2.0 p.p. under equivalent throughput.
UniComp: Rethinking Video Compression Through Informational Uniqueness: Ours proposes UniComp, a video token compression framework based on informational uniqueness (rather than attention). By utilizing frame group fusion, token allocation, and spatial dynamic compression, it maximizes the preservation of unique information across temporal, spatial, and global dimensions. It outperforms uncompressed baselines even when retaining only 10% of tokens.
Planning in 8 Tokens: A Compact Discrete Tokenizer for Latent World Model: CompACT is proposed to compress each image into only 8 discrete tokens (approx. 128 bits). By freezing a pretrained visual encoder to preserve planning-critical semantic information and employing generative decoding to supplement perceptual details, it accelerates world model-based planning by ~40x without compromising accuracy.

Browse all 108 Model Compression papers →

🕸️ Graph Learning (8)¶

Graph2Eval: Automatic Multimodal Task Generation for Agents via Knowledge Graphs: This paper introduces Graph2Eval, a knowledge-graph-driven framework for the automatic generation of agent evaluation tasks. By constructing structured knowledge graphs from documents/webpages, performing subgraph sampling, LLM conditional generation, and multi-stage filtering, it automatically produces multimodal agent tasks with significantly improved semantic consistency (+20%) and solvability (+17%), resulting in the Graph2Eval-Bench containing 1,319 tasks.
Robo-SGG: Exploiting Layout-Oriented Normalization and Restitution Can Improve Robust Scene Graph Generation: Addressing the issue where "domain shift in visual features leads to a performance collapse" in robust Scene Graph Generation (inference on corrupted images with noise/blur/weather), this paper proposes a plug-and-play framework, Robo-SGG. It utilizes Instance Normalization to eliminate domain-specific statistics caused by corruption and uses layout-aware attention to recover global structural features (NRM). Additionally, it employs gated fusion to adaptively balance visual and coordinate features (LEE). Integrating these into existing SGG models yields relative improvements in mR@50 of 6.3% / 11.1% / 8.0% for PredCls/SGCls/SGDet on VG-C.
ViterbiPlanNet: Injecting Procedural Knowledge via Differentiable Viterbi for Planning: Ours embeds the Procedural Knowledge Graph (PKG) into a planning model end-to-end via a differentiable Viterbi layer, allowing the neural network to focus on learning emission probabilities rather than memorizing complete procedural structures. This achieves SOTA success rates on CrossTask/COIN/NIV with only 5-7M parameters (1-3 orders of magnitude fewer than Diffusion/LLM methods) and establishes a unified evaluation benchmark.
Adaptive Learned Image Compression with Graph Neural Networks: GLIC transforms nonlinear transformations in learned image compression (LIC) from fixed convolutions or window attention into content-adaptive connections driven by Graph Neural Networks (GNNs). It employs dual-scale graphs to determine "where to connect" and a complexity-aware mechanism to decide "how much to connect" to better model local and long-range redundancy. It significantly outperforms traditional codecs and recent LIC baselines across three standard datasets.
Mario: Multimodal Graph Reasoning with Large Language Models: Mario is proposed for LLM reasoning on Multi-Modal Graphs (MMGs). It achieves topology-aware cross-modal alignment via a Graph-conditioned Vision-Language Model (GVLM) and selects the optimal modality configuration for each node using a Modality-Adaptive Prompt Router (MAPR), reaching SOTA performance on node classification and link prediction.
M3KG-RAG: Multi-hop Multimodal Knowledge Graph-enhanced Retrieval-Augmented Generation: M3KG-RAG is proposed, which constructs a Multi-hop Multimodal Knowledge Graph (M3KG) via a lightweight multi-agent pipeline and designs the GRASP mechanism for entity grounding and selective pruning. It retains only query-relevant and answer-assisting knowledge, significantly enhancing the audio-visual reasoning capabilities of MLLMs.
R2G: A Multi-View Circuit Graph Benchmark Suite from RTL to GDSII: Ours proposes R2G, the first standardized multi-view circuit graph benchmark suite, providing five stage-aware graph representations (with information equivalence) across 30 IP cores. Systematic research reveals that the choice of graph representation has a greater impact on performance than the choice of GNN model.
Mixture-of-Experts based Feature Decoupling for Open Vocabulary Scene Graph Generation: Addressing the issues of "relying solely on off-the-shelf VLM features, lacking discriminative attributes, and semantic isolation between objects and relations" in Open Vocabulary Scene Graph Generation (OVSGG), this paper proposes MoE-FD. It adaptively decouples object/relation features into sub-attributes like shape, texture, and space using a Mixture-of-Experts (MoE) module, followed by iterative cross-attention for mutual refinement between nodes and edges. On the Visual Genome all-open vocabulary setting, it significantly improves R@100 for novel categories (e.g., +4.24% R@20 over ACC in the OvD+R novel relation setting).

🤝 Federated Learning (19)¶

FedHarmony: Harmonizing Heterogeneous Label Correlations in Federated Multi-Label Learning: To address the issue in Federated Multi-Label Learning (FedMLL) where clients observe only local label spaces and generate conflicting label correlations (Label Correlation Drift), FedHarmony utilizes "Consensus Correlation" from the majority of clients as a global teacher to correct local training biases. Furthermore, it weights clients during server aggregation based on both data volume and correlation quality. It consistently outperforms existing SOTA on three non-IID federated benchmarks: FLAIR, COCO-80, and VOC2007 (e.g., +11.4 mAP on FLAIR).
Fine-Tuning Impairs the Balancedness of Foundation Models in Long-tailed Personalized Federated Learning: This paper empirically reveals that fine-tuning CLIP in long-tailed federated scenarios destroys its inherent class balance, even falling below zero-shot performance. It proposes FedPuReL: using zero-shot predictions to "purify" local gradients into directions that preserve balance for a global model, and reframing personalization as "residual correction" atop a frozen global model. FedPuReL outperforms existing SOTA in both global and personalized models across 8 long-tailed datasets.
Fully Decentralized Certified Unlearning: Addressing the neglected scenario of "decentralized networks without a central coordinator," this paper proposes RR-DU—a random-walk-based certified unlearning algorithm. It performs noisy projected gradient ascent on the forgetting set only at the client initiating the deletion, while other clients continue with noise-free descent. By incorporating sub-sampled Gaussian noise and trust region projections, the authors prove \((\varepsilon,\delta)\) network unlearning certificates, convergence, and deletion capacity bounds. Notably, the noise does not scale with the size of the forgetting set \(m\), successfully reducing backdoor attack success rates to random-guess levels while maintaining clean accuracy on image classification tasks.
Generalized and Personalized Federated Learning with Black-Box Foundation Models via Orthogonal Transformations: FEDOT treats frozen black-box foundation models (FM) as pure feature extractors. Each client stacks a local orthogonal transformation on the output embeddings for personalization, while all clients share and aggregate a global classifier for generalization. The authors prove that the orthogonal constraint (condition number \(\kappa=1\)) minimizes the upper bound of cross-client gradient conflicts, achieving SOTA generalization and personalization under severe non-IID conditions without accessing FM internal parameters.
HiLoRA: Hierarchical Low-Rank Adaptation for Personalized Federated Learning: HiLoRA decomposes the LoRA update of each client into a three-layer orthogonal subspace structure consisting of "root-cluster-leaf," which respectively capture global consensus, subgroup commonalities, and client personalization. Combined with an adaptive clustering mechanism based on LoRA subspace similarity, it achieves SOTA performance in both personalization and generalization to new clients on CIFAR-100 and DomainNet.
Taming Noise-Induced Prototype Degradation for Privacy-Preserving Personalized Federated Fine-Tuning: Addressing the issue where prototype-based personalized federated learning (ProtoPFL) injects isotropic Gaussian noise to satisfy local differential privacy (LDP) when sharing class prototypes—consequently drowning out discriminative dimensions—this paper proposes a client-side plugin, VPDR. It uses variance-adaptive VPP to shift noise budget from discriminative subspaces to redundant ones, and distillation-guided DCR to actively push feature norms toward the clipping threshold, significantly improving the privacy-utility trade-off under the same LDP guarantees.
FedRG: Unleashing the Representation Geometry for Federated Learning with Noisy Clients: To address the dual challenges of "noisy client annotations + Non-IID data" in Federated Learning, FedRG abandons the unreliable small-loss heuristic. Instead, it identifies clean/noisy samples based on representation geometry. Specifically, it first learns label-agnostic representations on a hypersphere through self-supervision, then uses a vMF mixture model to compare "geometric evidence" with "annotated label evidence" in a shared space for noise detection. Finally, it employs a personalized noise absorption matrix for robust optimization, achieving SOTA across multiple datasets and four noise scenarios.
From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneity: Addressing the pain point in Federated Continual Learning (FCL) where "selecting samples is easy, but utilizing them is difficult," FEAT does not modify the replay strategy itself. Instead, it employs a set of fixed ETF prototypes shared across all clients. It uses geometric structure distillation during training to align feature angles across clients and applies energy-based geometric correction during inference to "pull back" tail-class features from head-class subspaces. As a plug-and-play module layered on Re-Fed+ or FedCBDR, it yields stable performance gains.
Personalized Federated Training of Diffusion Models with Privacy Guarantees: PFDM decomposes the reverse denoising process of diffusion models into two components: a "client-private denoiser + server-shared denoiser." Clients only upload data that has been clipped and subjected to forward noise, providing formal Local Differential Privacy (LDP) guarantees for each data point. The shared model only observes noised data and cannot reproduce any client samples in isolation, while collaboration significantly enhances generation quality for minority or underrepresented classes.
Single-Round Scalable Analytic Federated Learning: SAFLe constructs a deterministic non-linear classification head using "feature bucketing + shuffling & grouping + sparse embedding summation" and proves its mathematical equivalence to a high-dimensional sparse linear regression. This allows the direct application of the single-round closed-form aggregation law of Analytic Federated Learning (AFL)—retaining the expressivity of non-linearity while preserving AFL's advantages of "single-round communication" and "complete invariance to data heterogeneity." It outperforms both linear AFL and multi-round DeepAFL across three visual federated benchmarks.

Browse all 19 Federated Learning papers →

📈 Time Series (7)¶

STCast: Adaptive Boundary Alignment for Global and Regional Weather Forecasting: The STCast framework is proposed, which replaces static boundaries with learnable global-regional distributions through Spatial-Aligned Attention (SAA) to adaptively fuse global atmospheric information into regional forecasts. It utilizes Temporal Mixture-of-Experts (TMoE) with monthly dynamic routing to enhance temporal modeling, outperforming existing methods across four tasks: global forecasting, high-resolution regional forecasting, typhoon track prediction, and ensemble forecasting.
PFGNet: A Fully Convolutional Frequency-Guided Peripheral Gating Network for Efficient Spatiotemporal Predictive Learning: PFGNet is a pure convolutional spatiotemporal prediction framework that dynamically modulates multi-scale large-kernel peripheral responses via Pixel-level Frequency-guided Gating (PFG) and applies learnable center suppression. Mimicking the center-surround band-pass filtering mechanism of biological vision, it achieves SOTA or near-SOTA performance on Moving MNIST, TaxiBJ, KTH, and Human3.6M benchmarks with minimal parameters and computational cost.
Probabilistic Precipitation Nowcasting with Rectified Flow Transformers: This work proposes FREUD—a framework utilizing a Rectified Flow Transformer as a "compressed first stage." It employs a frame-level encoder to independently encode each frame and a joint video decoder to reconstruct all frames simultaneously, replacing deterministic decoding with probabilistic decoding to quantify uncertainty during the compression stage. Combined with a latent-space rectified flow nowcasting model, it achieves SOTA CRPS (0.0190) and SSIM on the SEVIR precipitation nowcasting benchmark.
SATTC: Structure-Aware Label-Free Test-Time Calibration for Cross-Subject EEG-to-Image Retrieval: SATTC is proposed as a label-free test-time calibration head. By employing a Product-of-Experts (PoE) fusion of a geometric expert (subject-adaptive whitening + adaptive CSLS) and a structural expert (mutual nearest neighbors + bidirectional top-k ranking + category popularity), it operates directly on the similarity matrix of frozen EEG and image encoders. This approach significantly enhances Top-1 accuracy and alleviates the hubness effect in cross-subject EEG-to-image retrieval.
Real-Time Long Horizon Air Quality Forecasting via Group-Relative Policy Optimization: This paper addresses long-horizon (48–120 hours) PM concentration forecasting in East Asia. It first releases CMAQ–OBS, a regional dataset aligned with observations, and then employs a two-stage training framework (FAKER-Air) consisting of "SFT with temporal accumulation loss + GRPO with categorical AQI rewards." This aligns the inherent "over-forecasting and high false alarm" issues of MSE training with actual operational costs, reducing the False Alarm Rate (FAR) by 47.3% relative to the SFT baseline while maintaining a competitive F1 score.
Stable Spike: Dual Consistency Optimization via Bitwise AND Operations for Spiking Neural Networks: Ours proposes the Stable Spike dual consistency optimization framework, which utilizes hardware-friendly bitwise AND operations to decouple stable spike skeletons from multi-timestep spike maps and injects amplitude-aware spike noise to enhance generalization. It improves neuromorphic object recognition accuracy by up to 8.33% under ultra-low latency (\(T=2\)).
Towards Uncertainty-aware Unsupervised Domain Adaptation for Videos and Time-Series with Causal Optimal Transport: This paper proposes Causal-OT, which embeds inter-channel Granger causality graphs into the Optimal Transport (OT) cost matrix for cross-domain alignment. It simultaneously employs entropy-based uncertainty filtering for pseudo-labels to ensure that time-series and video domain adaptation preserves temporal-causal structures without being biased by overconfident pseudo-labels. It achieves an average accuracy improvement of 4.5% across 6 time-series benchmarks and 2.5% across 4 video benchmarks.

🏥 Medical Imaging (173)¶

Generative Vision-Language Multiple Instance Learning for Weakly Supervised Neonatal Fundus Screening and Reporting: GVL-MIL adapts a generative vision-language model with a small set of expert image descriptions, separately aggregates intermediate visual evidence and generated-text semantics, and fuses their predictions by entropy, achieving 92.14 ± 0.72% patient-level test AUC on NFSD while producing per-image clinical descriptions.
LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settings: Ours constructs LEMON, a large-scale endoscopic dataset containing 4194 surgical videos (938 hours), and proposes LemonFM, a self-supervised foundation model based on enhanced knowledge distillation. LemonFM outperforms existing surgical foundation models across four downstream tasks: surgical phase recognition, tool detection, action recognition, and semantic segmentation.
VoxTell: Free-Text Promptable Universal 3D Medical Image Segmentation: VoxTell is a 3D vision-language segmentation model that generates volumetric masks directly from a single sentence (ranging from single words to full clinical reports). By repeatedly injecting text guidance at every level of the UNet decoder (multi-stage fusion) combined with deep supervision, it achieves a zero-shot average Dice of 70.85 across 11 unseen datasets, significantly outperforming the previous state-of-the-art text-promptable method, SAT (51.23).
LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understanding: The general-domain masked diffusion language model LLaDA is introduced to the biomedical image understanding field for the first time via visual instruction tuning, resulting in the first diffusion-based biomedical VLM. It outperforms LLaVA-Med in open-ended medical dialogues, sets new SOTA records on the closed-set subsets of three VQA benchmarks, and enables explicit control over response length for more detailed answer generation.
Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning: Med-CMR decomposes "medical multimodal complex reasoning" into 7 categories of tasks across 3 visual dimensions and 4 reasoning dimensions. Using 20,653 VQA items (covering 11 body systems and 12 imaging modalities) doubly audited by human experts and models, it evaluates 18 mainstream MLLMs. The findings show that GPT-5 leads with a 57.81% MCQ accuracy, long-tail generalization is universally recognized as the hardest task, and medically fine-tuned models do not consistently outperform general large models.
MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images: MedMO utilizes Qwen3-VL as its base model and undergoes a four-stage post-training process using 26M+ cross-modal medical data: "General Medical SFT → High-resolution Grounding SFT → Instruction Tuning → GRPO Reinforcement Learning with Bounding Box Rewards." This approach unifies medical image understanding (VQA / QA / Report Generation) and fine-grained spatial localization (Bbox grounding) into an open-source VLM, outperforming existing open-source medical MLLMs across multiple clinical tasks.
fMRI-LM: Towards a Universal Foundation Model for Language-Aligned fMRI Understanding: fMRI-LM utilizes a three-stage framework that first discretizes brain signals into tokens aligned with the text embedding space, and then enables a pre-trained LLM to model brain activity as a predictable and describable "language." By complementing the lack of natural pairs with a synthetic fMRI-to-text description corpus, it achieves zero-shot/few-shot performance on diverse tasks including sex, age, fluid intelligence, and AD/ADHD/ASD diagnosis using a single model. Furthermore, LoRA fine-tuning achieves or even surpasses the performance of full fine-tuning.
OralGPT-Omni: A Versatile Dental Multimodal Large Language Model: OralGPT-Omni is the first dental-specific multimodal large language model. By constructing TRACE-CoT data that mimics the diagnostic workflow of radiologists and employing a four-stage progressive training regimen, it achieved a score of 51.84 on the MMOral-Uni unified benchmark (covering five modalities and five tasks), significantly outperforming GPT-5's 15.42.
OctoMed: Data Recipes for State-of-the-Art Multimodal Medical Reasoning: Instead of relying on new architectures or larger backbones, this work systematically investigates "how to mix training data." By utilizing strong teacher distillation and rejection sampling, the authors filtered 8 million medical samples with structured reasoning chains (6.8 billion tokens). This process fine-tuned a 7B student model (Qwen2.5-VL-7B) into OctoMed, achieving open-source SOTA on multiple out-of-distribution (OOD) medical benchmarks. Notably, the model adaptively adjusts reasoning chain lengths without explicit supervision.
MedGRPO: Multi-Task Reinforcement Learning for Heterogeneous Medical Video Understanding: MedGRPO proposes two key innovations to address the training collapse issue in multi-dataset reinforcement learning for medical videos: cross-dataset reward normalization (mapping median performance across different datasets to the same reward value via a logistic function) and Medical LLM Judge (comparative scoring across five clinical dimensions). Based on Qwen2.5-VL-7B, it outperforms GPT-4.1 and Gemini-2.5-Flash on MedVidBench (532K video-instruction pairs).

Browse all 173 Medical Imaging papers →

🩺 Medical LLM (1)¶

Towards Efficient Medical Reasoning with Minimal Fine-Tuning Data: This paper proposes the Difficulty-Influence Quadrant (DIQ) data selection strategy, which jointly considers sample difficulty and gradient influence. This approach allows a VLM's language backbone to match full SFT performance using only 1% of curated data and exceed full-dataset training with 10% of the data.

🧬 Computational Biology (21)¶

Stronger Normalization-Free Transformers: Through a systematic analysis of four critical attributes (zero-centeredness, boundedness, center sensitivity, and monotonicity) required for pointwise functions to replace normalization layers, an exhaustive search identifies \(\text{Derf}(x) = \text{erf}(\alpha x + s)\) as the optimal replacement. It consistently outperforms LayerNorm and DyT across multiple domains, including visual recognition, image generation, speech representation, and DNA sequence modeling, with performance gains primarily stemming from enhanced generalization rather than fitting capacity.
CARE: A Molecular-Guided Foundation Model with Adaptive Region Modeling for Whole Slide Image Analysis: CARE is proposed as a pathology slide-level foundation model that partitions WSIs into morphologically relevant irregular regions via an Adaptive Region Generator (ARG)—analogous to word-level tokens in NLP. By combining cross-modal alignment with RNA/protein expression profiles in a two-stage pre-training paradigm, CARE achieves optimal average performance across 33 downstream tasks while using only approximately 1/10 of the data required by mainstream models.
Multimodal Protein Language Models for Enzyme Kinetic Parameters: From Substrate Recognition to Conformational Adaptation: The authors propose ERBA (Enzyme-Reaction Bridging Adapter), which reframes enzyme kinetic parameter prediction as a staged multimodal conditional generation problem. The model injects substrate information via MRCA to capture substrate recognition specificity, integrates active site 3D structures through G-MoE to capture conformational adaptation, and utilizes ESDA distribution alignment to maintain the biochemical semantic priors of the PLM.
FEAST: Fully Connected Expressive Attention for Spatial Transcriptomics: FEAST transforms the task of "predicting spatial gene expression from large H&E pathology images" from a GNN paradigm relying on predefined sparse graphs into a fully connected attention framework. It utilizes self-attention to naturally model pairwise interactions between all spots, supplemented by negative attention to express "inhibitory relationships" and off-grid sampling to complete information in grid gaps. It achieves SOTA on 7 out of 9 metrics across three public ST datasets.
From Spots to Pixels: Dense Spatial Gene Expression Prediction from Histology Images: This paper reframes the task of "predicting spatial gene expression from pathology slides" from a spot-wise regression task to a dense prediction task. It proposes PixNet: first, a pathology foundation model extracts pyramid features; then, a U-Net-style decoder progressively generates a full-image dense gene expression map; finally, expression values for spots of any position or radius are obtained through circular region aggregation. This approach outperforms existing SOTA methods across multiple spatial scales (from 2µm single-cell level to 100µm).
HINGE: Adapting a Pre-trained Single-Cell Foundation Model to Spatial Gene Expression Generation from Histology Images: The HINGE framework is proposed to adapt a pre-trained expression-space single-cell foundation model (sc-FM, CellFM) into a histology image-conditioned spatial gene expression generator. This is achieved by lightweight injection of visual context via identity-initialized SoftAdaLN modulation, alignment with pre-training objectives through an expression-space masked diffusion process, and training stabilization via a warm-start curriculum. It achieves SOTA results across three ST datasets while maintaining superior gene co-expression consistency.
Hyperbolic Busemann Neural Networks: This work utilizes Busemann functions to intrinsically lift Multinomial Logistic Regression (MLR) and Fully Connected (FC) layers into hyperbolic space. It introduces two unified components, BMLR and BFC, which are applicable to both the Poincaré ball and Lorentz models. These components outperform existing hyperbolic layers across four task categories: image classification, genomic sequences, node classification, and link prediction.
TRIDENT: A Trimodal Cascade Generative Framework for Drug and RNA-Conditioned Cellular Morphology Synthesis: TRIDENT proposes a cascade framework of "VAE encoding (drug + pre-perturbed RNA) \(\rightarrow\) latent condition \(z\) \(\rightarrow\) Diffusion Transformer cellular morphology generation," explicitly modeling the causal chain of "RNA \(\rightarrow\) morphology" for the first time. On the custom MorphoGene trimodal dataset, it reduces FID by 5–7 times compared to the SOTA, and generalizes to unseen compounds.
Advancing Cancer Prognosis with Hierarchical Fusion of Genomic, Proteomic and Pathology Imaging Data from a Systems Biology Perspective: HFGPI explicitly models the "gene → protein → tissue morphology" systems biology cascade as a hierarchical fusion pipeline. It utilizes graph-aware cross-attention to characterize gene-to-protein regulation and hypergraphs to link proteins to pathology patches. On 5 TCGA cohorts, it achieves an average C-index of 0.753 for survival prediction, outperforming all Prev. SOTA.
Cell-Type Prototype-Informed Neural Network for Gene Expression Estimation from Pathology Images: Ours proposes CPNN, which leverages public single-cell RNA-seq data to construct cell-type prototypes. It models slide/patch-level gene expression as a weighted combination of these prototypes, achieving Prev. SOTA performance in gene expression estimation while providing interpretability.

Browse all 21 Computational Biology papers →

⚛️ Physics & Scientific Computing (2)¶

AviaSafe: A Physics-Informed Data-Driven Model for Aviation Safety-Critical Cloud Forecasts: AviaSafe embeds the "localization before quantification" hierarchical strategy and the long-validated "Icing Condition (IC) index" into a Swin Transformer backbone. It achieves the first global, 6-hourly, phase-separable (ice/liquid/rain/snow) cloud microphysics forecast, outperforming the FuXi baseline on 93.7% of variable-lead time combinations and matching or exceeding the operational NWP ECMWF HRES on key background variables up to a 7-day lead time.
Spatial-Spectral Residuals Informed Diffusion Neural Operator for Pan-sharpening: SRINO replaces the attention-based denoising backbone of diffusion models for pan-sharpening with a Galerkin-type Neural Operator (transferring the generation process to a continuous function space to significantly save FLOPs and memory). It treats pixel-level spatial/spectral consistency residuals directly as conditions fed into each step of the reverse sampling process for closed-loop guidance. On WV3/GF2/QB datasets, it outperforms current SOTA methods while being several times more computationally efficient than attention-based diffusion models.

🧮 Scientific Computing (3)¶

NESTOR: A Nested MOE-based Neural Operator for Large-Scale PDE Pre-Training: Ours proposes NESTOR, a nested MoE neural operator. It captures global features of different PDE types through image-level MoE and local correlations within physical fields through token-level Sub-MoE. It achieves large-scale pre-training across 12 PDE datasets and effectively transfers to downstream tasks.
Continuous Exposure-Time Modeling for Realistic Atmospheric Turbulence Synthesis: Ours proposes the Exposure-Time dependent Modulation Transfer Function (ET-MTF), modeling exposure time as a continuous variable. A large-scale synthetic turbulence dataset, ET-Turb (5,083 videos, 2 million frames), is constructed, significantly improving the generalization of turbulence restoration models on real-world data.
EHETM: High-Quality and Efficient Turbulence Mitigation with Events: EHETM is proposed as the first method to leverage the microsecond temporal resolution of event cameras to break the accuracy-efficiency bottleneck of traditional multi-frame Turbulence Mitigation (TM). By discovering two key physical phenomena—the correlation between polarity alternation of turbulence-induced events and sharp gradients, and the formation of spatio-temporally coherent "event tubes" by dynamic objects—the authors design the Polarity-Weighted Gradient and Event Tube Constraint modules. EHETM reduces data overhead by 77.3% and system latency by 89.5%, significantly surpassing SOTA methods, especially in dynamic scenes.

🌍 Earth Science (2)¶

SIGMA: A Physics-Based Benchmark for Gas Chimney Understanding in Seismic Images: This work proposes SIGMA, the first physics-based synthetic seismic image dataset with ground truth labels. By combining wave equation forward modeling and Reverse Time Migration (RTM), velocity models containing gas chimneys are converted into seismic images. The dataset provides pixel-level gas chimney masks (for detection) and paired "degraded-clean" images (for enhancement). Benchmarking multiple baselines reveals that existing methods collectively struggle on this data.
PhyOceanCast: Global Ocean Forecasting with Physics-Informed Diffusion: PhyOceanCast models global ocean forecasting as a residual diffusion problem. It utilizes a Spherical Graph Attention Network (SGAN-MOC) to address "high-latitude projection distortion + variable coupling" and a Physics-informed Wavelet Temporal Connection module (PWTC) to handle "multi-scale dynamics + conservation constraints". The framework predicts 145 ocean variables across 36 depth layers simultaneously, reducing the 30-day forecast RMSE by approximately 13.7% compared to the strongest baseline.

📡 Signal & Communications (2)¶

CLAY: Conditional Visual Similarity Modulation in Vision-Language Embedding Space: CLAY proposes a training-free method for conditional visual similarity calculation, which modulates similarity by constructing a text-conditioned subspace within the VLM embedding space. This approach adapts to different retrieval conditions without recomputing database features and supports multi-condition retrieval.
AcTTA: Rethinking Test-Time Adaptation via Dynamic Activation: This paper proposes AcTTA, a test-time adaptation framework based on dynamic activation function modulation. By reparameterizing traditional fixed activation functions into a learnable form—incorporating activation center shifts and asymmetric gradient slopes—the model adaptively adjusts activation behavior during inference to handle distribution shifts. AcTTA consistently outperforms normalization-based TTA methods on CIFAR10-C, CIFAR100-C, and ImageNet-C.

👥 Social Computing (3)¶

Instance-level Visual Active Tracking with Occlusion-Aware Planning: OA-VAT constructs discriminative "instance prototypes" offline from a single reference image to resist similar distractors. It utilizes online EMA-enhanced prototypes and confidence-adaptive Kalman filtering to maintain stable tracking, while training a target-box-conditioned diffusion trajectory planner to actively bypass obstacles and recover the target upon occlusion—achieving an average SR of 0.93 on UnrealCV, 90.8% average CAR on real images, and 81.6% TSR on real UAVs, reaching 35 FPS on an RTX 3090.
Bridging Pixels and Words: Mask-Aware Local Semantic Fusion for Multimodal Media Verification: The MaLSF framework is proposed, utilizing mask-label pairs as semantic anchors to achieve active local semantic conflict detection through Bidirectional Cross-modal Verification (BCV) and Hierarchical Semantic Aggregation (HSA) modules, achieving SOTA on DGM4 and fake news detection tasks.
Revisiting Unknowns: Towards Effective and Efficient Open-Set Active Learning: Ours proposes E2OAL, a detector-free open-set active learning framework that discovers latent structures of unknown classes via label-guided clustering, jointly models known and unknown categories using a Dirichlet-calibrated auxiliary head, and designs a two-stage adaptive querying strategy to simultaneously achieve high accuracy, high query purity, and high training efficiency across multiple benchmarks.

🛡️ AI Safety (145)¶

Skyra: AI-Generated Video Detection via Grounded Artifact Reasoning: Skyra transforms AI-generated video detection from black-box binary classification into interpretable artifact reasoning. It utilizes a cold-start SFT phase on the manually annotated ViF-CoT-4K dataset to teach MLLMs to spatio-temporally locate and explain artifacts. This is followed by GRPO reinforcement learning with asymmetric rewards to encourage active artifact discovery, achieving a 26.73% absolute accuracy improvement over the second-best method on the ViF-Bench.
When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models: Ours proposes the UPA-RFAS framework to learn a single physical adversarial patch that achieves universal, transferable black-box attacks on VLA robot policies through a three-pronged approach: feature-space shifting, attention hijacking, and semantic misalignment.
The Coherence Trap: When MLLM-Crafted Narratives Exploit Manipulated Visual Contexts: This work reveals a core threat where existing multimodal manipulation detection neglects MLLMs' ability to generate semantically consistent deceptive narratives. It constructs the MDSM dataset with 441k semantic-aligned forged samples and proposes the AMD framework based on Artifact Tokens and manipulation-oriented reasoning. With only 0.27B parameters, it achieves SOTA generalization performance in cross-domain detection: 88.18 ACC / 60.25 mAP / 61.02 mIoU.
UniGame: Turning a Unified Multimodal Model Into Its Own Adversary: UniGame proposes the first self-adversarial post-training framework for Unified Multimodal Models (UMM). By installing a lightweight perturber at the shared visual token interface, it enables the generation branch to actively create semantically consistent adversarial samples to challenge the understanding branch. This forms a min-max self-play that significantly improves consistency (+4.6%), understanding (+3.6%), generation, and robustness.
RemedyGS: Defend 3D Gaussian Splatting Against Computation Cost Attacks: RemedyGS proposes the first black-box defense framework against "computation cost attacks" on 3DGS (such as Poison-splat, which triggers Gaussian explosion by poisoning input images to exhaust GPU resources and cause Denial-of-Service). Utilizing a two-stage "detector + purifier + adversarial training" pipeline, it purifies only images identified as poisoned, restoring computation costs to normal levels while maintaining reconstruction quality for legitimate users.
V-Attack: Targeting Disentangled Value Features for Controllable Adversarial Attacks on LVLMs: It is discovered that Value features in ViTs possess more disentangled local semantic representations compared to Patch features. V-Attack is proposed to achieve precise and controllable local semantic attacks on LVLMs through self-enhanced Value features and text-guided semantic manipulation, improving ASR by an average of 36%.
RunawayEvil: Jailbreaking the Image-to-Video Generative Models: This paper proposes RunawayEvil—the first multimodal jailbreak attack framework targeting "Image-to-Video (I2V)" models. It utilizes a "Strategy-Tactic-Action" paradigm to coordinate attacks across text and image modalities, evolving through Reinforcement Learning and LLMs to increase attack success rates on COCO2017 by 58.5%–79% compared to existing methods.
LaSM: Layer-wise Scaling Mechanism for Defending Pop-up Attack on GUI Agents: Through a systematic analysis of how pop-up injection attacks distort the layer-wise attention of GUI Agents, the authors found that deep-layer attention diverges significantly between "correct" and "incorrect" samples. They propose LaSM—a training-free, plug-and-play layer-wise scaling mechanism that amplifies attention and MLP weights specifically in middle semantic layers. This mechanism improves the defense success rate (DSR) of Qwen2-VL-7B from approximately 19% to over 66% under pop-up attacks, with minimal impact on normal task performance.
POUR: A Provably Optimal Method for Unlearning Representations via Neural Collapse: Aiming at the problem where existing machine unlearning methods only modify the classification head while forgotten class information remains in the features, this paper elevates "unlearning" to the representation level. Using the geometry of the simplex-ETF in Neural Collapse, it proves that "removing a class = orthogonal projection along its direction results in an ETF remaining an ETF." This result yields the closed-form projection operator POUR-P and its distillation variant POUR-D. These methods refresh both class-level and representation-level unlearning metrics on CIFAR-10/100 and PathMNIST, and provide a formal proof of optimality under the definition of representation-level weak unlearning.
Towards Robust Multimodal Large Language Models Against Jailbreak Attacks: SAFEMLLM is the first adversarial training framework designed specifically for Multimodal Large Language Models (MLLMs). By injecting compact, learnable perturbation matrices into the token embedding layer to simulate cross-modal attacks (CoE-Attack) and iteratively updating model parameters to neutralize these perturbations, SAFEMLLM reduces the Attack Success Rate (ASR) of six jailbreak methods to near 0 in white-box scenarios while maintaining standard multimodal task performance.

Browse all 145 AI Safety papers →

📂 Others (105)¶

UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition: UniMERNet redefines the task of converting formula images to LaTeX: it constructs the UniMER-1M dataset covering four real-world scenarios and, based on the observation that "decoder attention naturally follows a raster-scan (horizontal then vertical) pattern," proposes Raster-Scan Attention. This decomposes 2D attention into two orthogonal 1D computations, reducing complexity from \(O(NH^2W^2D)\) to \(O(NHWD(H+W))\). With 313M parameters, it achieves ~10× VRAM savings and 5× speedup, while its CDM consistently outperforms Texify, GOT, and even 72B/78B multimodal large models across four real-world scenarios.
FlashVSR: Towards Real-time Diffusion-Based Streaming Video Super Resolution: FlashVSR is the first to establish a "one-step + streaming" framework for diffusion-based video super-resolution (VSR). It compresses a full-attention teacher into a single-step block-sparse causal student via a three-stage distillation pipeline, eliminates the training/inference resolution gap using locally-constrained sparse attention, and introduces a tiny conditional decoder that leverages low-resolution (LR) frames as conditions to bypass the 3D VAE decoding bottleneck (which usually consumes 70% of the runtime). This achieves a processing speed of approximately 17 FPS for \(768\times1408\) videos on a single A100 GPU (11.8× faster than the fastest one-step diffusion VSR), while generating stable reconstructions at 1440p resolutions.
PAI-Bench: A Comprehensive Benchmark For Physical AI: PAI-Bench decomposes "Physical AI" into two capability tracks—perception and prediction—and maps them to three tracks: video generation, conditional video generation, and video understanding. Using 2,808 real-world samples paired with task-aligned physical plausibility metrics, the authors systematically evaluate 15 video generation models, 4 controllable generation models, and 16 Multimodal Large Language Models (MLLMs). The findings indicate that while these models produce aesthetically pleasing visuals, they generally fail to learn physical laws, and their understanding capabilities lag significantly behind human performance.
Upsample Anything: A Simple and Hard to Beat Baseline for Feature Upsampling: Upsample Anything unifies classical Joint Bilateral Upsampling (JBU) and 2D Gaussian Splatting into a per-pixel anisotropic Gaussian kernel. These kernels are learned via a 50-step "RGB self-reconstruction" test-time optimization for each image, then directly transferred to the low-resolution features of foundation models for pure mixture upsampling. It requires no dataset-level training, takes approximately 0.419 seconds for a 224×224 image, and achieves or approaches SOTA across segmentation, depth estimation, and depth/probability map upsampling.
ViT3: Unlocking Test-Time Training in Vision: Systematically explores the design space of Test-Time Training (TTT) for vision tasks, summarizes six practical design insights, and proposes ViT3—a pure TTT vision architecture with linear complexity that matches or exceeds Mamba and linear attention methods in classification, generation, detection, and segmentation tasks.
SimRecon: SimReady Compositional Scene Reconstruction from Real Videos: The SimRecon framework proposes a "Perception → Generation → Simulation" pipeline to automatically construct simulator-ready compositional 3D scenes from real-world videos. The core innovations include Active View Optimization (AVO) to find optimal projection views for single-object generation and a Scene Graph Synthesizer (SGS) to guide physically plausible hierarchical assembly.
The SA-FARI Dataset: Segment Anything in Footage of Animals for Recognition and Identification: SA-FARI is the largest in-the-wild multi-animal tracking (MAT) dataset to date, aggregating 11,609 camera trap videos spanning 10 years across 4 continents, 741 stations, and 99 species. It provides the first large-scale human-verified spatiotemporal segment masklets (16,224 individual trajectories, 940k boxes/masks). Experiments demonstrate that training on SA-FARI improves SAM 3 on HOTA-like metrics by over 20 points.
A2GC: Asymmetric Aggregation with Geometric Constraints for Locally Aggregated Descriptors: Addressing the failure of the "symmetric Sinkhorn" assumption in feature aggregation for Visual Place Recognition (VPR), A2GC reformulates the Optimal Transport solver into an asymmetric version (averaging row/column normalization + independent source/target marginal calibration) and overlays a geometric constraint branch (using learnable coordinate embeddings to bias spatially adjacent features towards the same cluster), achieving 95.6% Recall@1 on Pitts30k.
Customized Fusion: A Closed-Loop Dynamic Network for Adaptive Multi-Task-Aware Infrared-Visible Image Fusion: This paper proposes CLDyN, a closed-loop dynamic network that enables a frozen fusion network to adapt to downstream tasks (detection/segmentation/saliency) without retraining. By utilizing a "Request-driven Semantic Compensation (RSC)" module with only 0.46M parameters, the system receives semantic feedback and dynamically customizes convolutional structures for task-specific compensation. It maintains high fusion quality while achieving superior multi-task adaptability on M3FD, FMB, and VT5000 datasets.
NAF: Zero-Shot Feature Upsampling via Neighborhood Attention Filtering: NAF reformulates the task of "upsampling low-resolution features from Vision Foundation Models (VFMs)" as a neighborhood attention filtering process that only observes the high-resolution input image, ignoring the VFM features themselves during guidance. Trained once, it can be applied zero-shot to any VFM (including 7B models) and any magnification factor. It achieves new SOTA results across downstream tasks such as semantic segmentation, depth estimation, open-vocabulary segmentation, and video propagation, while operating approximately 4x faster than comparable methods.

Browse all 105 Others papers →