ECCV2026 ECCV2026 accepted papers ECCV2026 paper list AI paper notes top conference papers 3D Vision Image Generation Multimodal VLM Video Generation VLM Reasoning Autonomous Driving Robotics & Embodied AI Human Understanding

🎞️ ECCV2026 Accepted Papers¶

2837 ECCV2026 paper notes covering 3D Vision (512), Image Generation (328), Multimodal VLM (225), Video Generation (169), VLM Reasoning (164), Autonomous Driving (159), Robotics & Embodied AI (129), Human Understanding (119) and other 45 areas. Each note has TL;DR, motivation, method, experiments, highlights, and limitations — 5-minute reads of core ideas.

🦾 LLM Agent (36)¶

MolmoWeb: Open Visual Web Agent and Open Data for the Open Web: MolmoWeb combines trajectories from structured-web teachers, human demonstrations, and GUI perception supervision into screenshot-conditioned action learning, reaching 78.2% single-run success on WebVoyager with an 8B model and 94.7% pass@4 through parallel attempts; its main contribution is a reproducible data and training recipe rather than a new planning architecture.
QuantV2X: A Fully Quantized Multi-Agent System for Cooperative Perception: QuantV2X proposes the first fully quantized multi-agent V2X cooperative perception system. By utilizing end-to-end quantized neural network models and quantized communication message representations, it maintains 99.8% of the perception accuracy of full-precision models under low-bit INT4/INT8 quantization. Simultaneously, it reduces end-to-end system latency by 3.2 times and surpasses the full-precision baseline in mAP30 by +9.5 on the V2X-Real dataset.
GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding: GUI-AIMA turns a multimodal model's intrinsic attention into a GUI click localizer through a trailing anchor token, instruction-adaptive attention-head weighting, and patch-wise supervision; with 509k training instructions, its 3B model achieves 53.8% single-step and 61.5% crop-and-zoom two-step accuracy on ScreenSpot-Pro.
PPTArena: A Benchmark for PowerPoint Editing: PPTArena turns in-place editing of real presentations into a benchmark that checks both structural correctness and visual quality, while its code/OOXML hybrid agent PPTPilot achieves full-benchmark IF/VQ of 2.57/2.69 with reflection and verification, leaving a substantial gap to reliable complex editing.
A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilation: CineBench evaluates instruction-driven cinematic compilation, while the training-free CineAgents system analyzes source footage before planning and executing an edit, achieving a shot-level F1 of 64.13, temporal correctness of 52.09%, and adversarial rejection of 87.23% on this benchmark.
Automatic Method Illustration Generation for AI Scientific Papers via Drawing Middleware Creation, Evolution, and Orchestration: FigAgent turns recurring visual components into callable, evolvable drawing functions and orchestrates them through multi-agent lookahead search to produce editable SVGs, improving node-alignment F1 on FigAgentBench from PaperBanana's 46.3 to 51.4.
Narrative-Driven Paper-to-Slide Generation via ArcDeck: ArcDeck preserves argumentative dependencies through discourse trees, constrains multi-agent outline revision with a global commitment, and then constructs editable slides; with GPT-4o generation and GPT-5 judging on ArcBench, its Narrative Flow score is 43.25 versus SlideGen's 30.75, although overall quality remains below author-prepared slides.
GUIDE: Resolving Domain Bias in GUI Agents through Real-Time Web Video Retrieval and Plug-and-Play Annotation: GUIDE is a training-free, plug-and-play framework that automatically retrieves and extracts domain-specific planning and grounding knowledge from YouTube tutorial videos. By injecting this knowledge into corresponding modules of GUI agents to eliminate domain bias, it brings performance improvements of +4.47 to +7.48 percentage points for three agents with different architectures on OSWorld.
Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulation: REAL trains Qwen3-VL-8B to explore, ask when necessary, and then manipulate through a tool interface that withholds privileged object state; SFT followed by online GSPO reaches 56.9% success on the interactive REAL-Bench split, and the high-level policy transfers zero-shot to a physical robot with 78.3% success over 60 episodes.
HippoCamp: Benchmarking Contextual Agents on Personal Computers: HippoCamp organizes heterogeneous personal-computer files into three isolated archetypal environments and evaluates factual retention and user profiling with 581 file-grounded questions; the strongest evaluated system achieves only 48.3% profiling accuracy, exposing a substantial gap between finding relevant files and reliably understanding a user.

Browse all 36 LLM Agent papers →

⚖️ Alignment & RLHF (6)¶

Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis: Omni-RRM synthesizes preference data with five-criterion justifications through dual-teacher consensus, then uses SFT→GRPO to learn pairwise response discrimination across images, video, and audio, raising the 7B backbone's five-benchmark average accuracy from 60.2% to 70.4% and improving response selection without updating the generator's parameters.
Don't Let the Video Speak: Audio-Contrastive Preference Optimization for Audio-visual Language Models: For the cross-modal hallucination in which audio-visual language models (AVLMs) invent sounds from what they see, this paper proposes ACPO: preference pairs built along two orthogonal axes — an output-contrastive axis that penalizes visual captions offered as answers to audio-focused prompts under an audio-swapped input, and an input-contrastive axis that requires an answer to become untenable once its supporting audio track is replaced — which, while fine-tuning only the audio projection layer, lifts AVHBench audio-hallucination F1 from 79.0 to 80.4 and unimodal audio-captioning CIDEr from 33.0 to 43.6.
D2PO: Optimizing Diffusion Samplers via Dynamic Preference: D2PO reformulates diffusion sampler parameter optimization from a rigid student-teacher regression task into a preference alignment problem using an energy-based surrogate and dynamic self-refining teachers, markedly enhancing perceptual fidelity under ultra-low NFE budgets.
HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark: HumanTracker introduces a 150-hour optical mocap benchmark across four motion families and HumanScore, a temporal Transformer trajectory reward model trained on 12K human preference pairs, achieving 92.91% preference alignment and revealing contact failures missed by kinematic metrics.
Reward Modeling for Computer-Using Agent from Video Execution: Addressing the reliance of computer-use agent (CUA) evaluation on handcrafted rules or internal reasoning traces, this paper introduces ExeVRM, a model-agnostic reward model evaluated directly on interface execution videos, enabled by the 53k-scale ExeVR-53k dataset, adversarial instruction translation for hard negative synthesis, and spatiotemporal token pruning (STP+TTP) that keeps 720p long-horizon evaluation tractable while outperforming GPT-5.2 and Gemini-3 Pro.
VERTIGO: Visual Preference Optimization for Cinematic Camera Generation: VERTIGO introduces the first visual preference optimization post-training framework for camera trajectory generators by rendering 3D camera paths into 2D previews via a real-time graphics engine (Unity), scoring them using a cinematically adapted VLM via inverse captioning consistency, and applying DPO to align camera motions with shot composition, reducing target off-screen rates from 38.7% to 0.8%.

🔒 LLM Safety (20)¶

BBQ-V: Benchmarking Visual Stereotype Bias in Large Multimodal Models: BBQ-V evaluates unsupported group-based inferences using real multi-person images and under-informative open-ended questions: its 4,497 images support 14,144 image–question pairs, and an input ablation shows that seeing more does not necessarily produce more cautious judgments.
DualTAP: A Dual-Task Adversarial Protector for Mobile MLLM Agents: DualTAP perturbs mobile screenshots before upload, using contrastive attention to locate privacy-sensitive regions and a dual-task objective to preserve normal operations, reducing GPT-5 leakage on PrivScreen from 97.14% to 23.33% while retaining 91.00% task accuracy.
Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning: This paper proposes a new paradigm of "benign memory forgetting" and the S-MLLMUn Bench evaluation benchmark, and designs the SMFA (Sculpted Memory Forgetting Adapter) framework. By utilizing parameter masks guided by retaining anchors, it precisely erases privacy-sensitive knowledge in MLLMs without compromising the models' foundational visual understanding capabilities.
Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignment: MoRAS improves multimodal risk assessment with a short image description, then uses the estimated risk to steer early response activations for the original query toward refusals, reducing LLaVA-1.5-7B's MM-Safety attack success rate from 40.1% to 2.6% while retaining its reported general-task aggregate scores.
SlowBA: An efficiency backdoor attack towards VLM-based GUI agents: SlowBA proposes the first efficiency backdoor attack against VLM-based GUI agents. Through a two-stage Reward-level Backdoor Injection (RBI), the agent is trained to generate ultra-long responses during SFT, and a trigger-aware reward function is then used in RL to distinguish between triggered and clean inputs. Consequently, the agent exhibits extremely high response latency when encountering pop-up triggers, while the task accuracy remains largely unaffected.
Rethinking Visual Privacy: A Compositional Privacy Risk Framework for Severity Assessment with VLMs: This paper turns visual privacy from a binary "private or not" label into a compositional, graded risk estimate: it introduces the four-level Compositional Privacy Risk Taxonomy (CPRT), ordered by standalone identifiability and compositional harm potential, together with a continuous severity scoring function with lexicographic dominance and a 6.7K-image / 22-attribute annotated dataset; benchmarking eight open-weight and frontier VLMs shows that frontier models align well with compositional severity once the taxonomy is supplied (Gemini 3 Flash ρ=0.872) yet systematically underestimate aggregation-driven intermediate risk, and that an 8B LoRA-tuned model distills this ability to a deployable scale.
Safe Responses Matter: Output-Aware Safety Guardrail Mitigate Over-Refusal in MLLMs: Instead of blocking a query merely because its input appears sensitive, OutGuard trains hidden-state classifiers on actual-response safety labels and combines multi-instance contrastive learning with cross-layer voting, reducing in-distribution ARSP to 0.0749 and 0.0634 on two MLLMs while retaining missed risks, distribution-shift degradation and substantial refusal rates on difficult benign queries.
GEO-Detective: Unveiling Location Privacy Risks in Images with LLM Agents: This paper introduces GEO-Detective, an autonomous agent that mimics human reasoning and tool-use behaviors for image geolocation inference; by combining adaptive difficulty assessment, experience-augmented prompting, visual segmentation, and visual reverse search, it systematically demonstrates that agentic LVLMs expose severe location privacy risks in everyday shared images.
Learning to Compose: Revisiting Proxy Task Design for Zero-Shot Composed Image Retrieval: FoCo (Focus-then-Complete) models vision-language composition in zero-shot composed image retrieval (ZS-CIR) as a learnable two-stage "focus-then-complete" process. By jointly training two proxy tasks—text-anchored visual aggregation and context-conditioned semantic completion—along with a cross-instance contrastive loss to prevent shortcut learning, it comprehensively outperforms existing methods across four ZS-CIR benchmarks, without relying on LLMs during inference.
Neural Gate: Mitigating Privacy Risks in LVLMs via Neuron-Level Gradient Gating: This paper proposes Neural Gate, a neuron-level gradient gating method for mitigating privacy risks in LVLMs. The core mechanism is to first measure the consistency of each neuron's contribution to privacy targets on privacy subject features using a learnable vector, classifying neurons into three categories: strongly active, weakly active, and inactive. Then, during model editing, parameter updates are applied only to the gradients of strongly active privacy neurons, while gradients of other neurons are truncated. On MiniGPT and LLaVA, Neural Gate achieves a rejection rate of over 94% on sensitive queries with almost no loss in general task performance, while displaying robust generalization on out-of-distribution privacy categories.

Browse all 20 LLM Safety papers →

👻 Hallucination Detection (20)¶

Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Models: Cognitive Anomaly Detection (CAD) summarizes a vision-language model's evidence-to-answer process with three interpretable information measures and detects departures from a Gaussian-mixture model of nominal behavior, reaching a mean detection AUC of 0.858 on adversarial POPE while distinguishing perceptual instability, evidence-dependence anomalies, and decision uncertainty.
HIVE: Understanding Post Hallucination Reasoning in Vision Language Models: HIVE uses paired faithful and hallucinated captions to study how vision-language models continue reasoning after hallucinations enter their context, finding substantial accuracy gains on some visual tasks without establishing either universal benefits or the trustworthiness of fabricated evidence.
Looking Back and Forth: Cross-Image Attention Calibration and Attentive Preference Learning for Multi-Image Hallucination Mitigation: This paper attributes multi-image hallucination to the unidirectional cross-image information flow induced by autoregressive causal attention, and proposes CAPL: it calibrates attention so that only key tokens become mutually visible across images, then builds preference pairs by generating positives and negatives under the full cross-image mask and a fully truncated one, trained with DPO plus NLL — yielding consistent 1–3.6 point gains on BLINK/MUIRBench without hurting single-image ability.
No Place to Hide: Benchmarking Video Hallucination with Background-Controlled Pairs: This paper proposes VidPair-Halluc, a video hallucination benchmark that utilizes adversarial video pairs with "highly similar backgrounds but significantly different foreground semantics" to decouple model errors from background variations and cleanly attribute them to foreground hallucinations. The accompanying PairFlow three-stage generation pipeline leverages T2I and video generation models to automatically synthesize 1K high-quality video pairs and 11K spatiotemporal QA pairs. Evaluations indicate that mainstream large video models generally suffer from a severe degradation in performance under this controlled setting.
Seeing to Ground: Visual Attention for Hallucination-Resilient MDLLMs: This paper attributes hallucination in multimodal diffusion large language models (MDLLMs) to an objective mismatch in parallel masked decoding — the decoder ranks candidate tokens purely by textual likelihood and never verifies localized visual support — and proposes VISAGE, a training-free decoding-time re-ranking framework that estimates "grounding evidence" from the spatial entropy of last-layer cross-attention over image tokens, aggregates it across heads via a β-quantile consensus into a penalty multiplier, and replaces the ranking score \(c_i\) with \(c_i(1+H_i)^{-\alpha}\), gaining 8.59% relative on MMMU-val and 7.75% on HallusionBench while barely hurting general capability.
TARS: MinMax Token-Adaptive Preference Strategy for Hallucination Reduction in MLLMs: TARS perturbs prompt tokens with weak image relevance during training and combines DPO with spectral preference alignment, reducing LLaVA-v1.5-7B's AMBER hallucination rate from standard DPO's 26.4% to 13.2% using only 4.8k preference samples while improving descriptive coverage.
Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization: Reveals the "context blindness" phenomenon in existing direct preference optimization (DPO) for MLLM alignment, and proposes Context-Calibrated Direct Preference Optimization (C2-DPO), which explicitly maximizes Contextual Preference Gain (CPG) while anchoring preference ranking under degraded context, significantly cutting object hallucination without compromising general reasoning.
Disentangling Hallucinations: Orthogonal Semantic Projection for Robust Interpretability: Discloses that the root cause of widespread "explanation hallucination" in multimodal XAI saliency methods is linear semantic leakage in high-dimensional embedding spaces, and proposes Orthogonal Semantic Projection (OSP), a plug-and-play geometric intervention based on OMP residuals that eliminates ghost signals by orthogonalizing query vectors against distractor concepts.
Do Not Leave a Gap: Hallucination-Free Object Concealment in Vision-Language Models: Addressing the issue where suppression-based object concealment induces compensatory hallucinations in vision-language models (VLMs), this paper proposes Background-Consistent Re-encoding (BCR), a framework that statistically aligns target features and projects them onto background feature manifolds in deep vision encoder layers to conceal objects without leaving representational voids.
From Hallucination to Grounding: Diagnosing Visual Spatial Intelligence via CRISP: CRISP is a structural diagnostic benchmark for visual spatial intelligence. Through a dual-task paradigm of Spatial QA + 3D Scene Graph combined with a cross-task consistency protocol, it reveals whether VLM spatial reasoning represents genuine 3D geometric understanding or merely semantic shortcuts relying on language priors.

Browse all 20 Hallucination Detection papers →

📊 LLM Evaluation (3)¶

SVGEval: A Vision-Grounded Framework for Perceptual-Quality Benchmarking and Evaluation in Text-to-SVG Generation: SVGEval moves the evaluation target of text-to-SVG from the SVG code to the rendered outcome that users actually see: it builds a 500-instance binary-diagnosis plus 500-instance 1–5 scoring benchmark over a four-dimension rubric (Aesthetic, Semantic, Spatial, Structural) with multi-round human annotation and expert adjudication, then distills an explainable scorer that reads both the rendering and the code and outputs dimension-wise scores with evidence-grounded rationales — cutting MAE from 0.71 (best general model) to 0.58 and lifting ±1 adjacent accuracy from 66.6% to 72.0%.
ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos: Addressing the lack of paired novice-expert data and explicit editing prompts, ExpertEdit learns an expert motion prior strictly from unpaired expert videos via masked motion infilling, selectively projecting novice executions at kinematic peaks onto the learned expert manifold to enable guidance-free skill refinement.
LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression: To tackle visual forgetting and attention sink collapse during long-horizon reasoning in LVLMs, LASER introduces a GRPO-based post-training framework that pairs a visual grounding reward with a sink suppression reward, simultaneously regulating temporal attention decay and spatial intra-visual allocation to boost multimodal reasoning and perceptual grounding.

📚 Pretraining (15)¶

360Anything: Geometry-Free Lifting of Images and Videos to 360°: 360Anything proposes a geometry-free diffusion Transformer framework that treats perspective inputs and panoramic targets uniformly as token sequences. By concatenating these sequences, the model autonomously learns the geometric correspondences between them without requiring any camera metadata. It lifts arbitrary-view images/videos into gravity-aligned, seamless 360° panoramas, completely outperforming prior state-of-the-art methods that rely on ground-truth camera parameters in both image and video tasks.
FAIR: Feature-Augmented Implicit Regularization for AI-generated Fake Image Detection: FAIR incorporates texture-orthogonal Scene Composition Structure (SCS) as a privileged geometric prior during training to smooth decision boundaries and penalize texture-biased shortcut learning, subsequently discarding the prior at inference to achieve massive cross-generator generalization gains with zero computational overhead.
GryphOne: Symbol-Aware Masked Diffusion for Structural Refinement in Offline Handwritten Mathematical Expression Recognition: GryphOne redefines handwritten mathematical expression recognition (HMER) from autoregressive sequence generation to an iterative symbol refinement process via discrete masked diffusion. It maintains syntactic consistency under local editing using Symbol-Aware Tokenization (SAT) and enhances refinement stability through Random-Mask Mutual Learning (RMML). This approach comprehensively outperforms re-implemented autoregressive baselines and commercial HMER systems on MathWriting, achieving a 5.51% CER and a 59.9% ExpRate.
Invisible Shortcuts: Why Vision Encoders Know Your Camera: Reveals that vision encoders encode imperceptible camera acquisition and image processing metadata traces because pretraining datasets introduce spurious metadata-semantics correlations exploited as shortcuts, and provides training augmentation and post-hoc adversarial mitigation strategies that generalize across unseen attributes.
Map2World: Segment Map Conditioned Text to 3D World Generation: To overcome data scarcity and narrow domain restrictions in 3D world synthesis, Map2World introduces an arbitrary-shaped segment map conditioned framework that scales pretrained TRELLIS asset priors into complete, layout-controllable, scale-consistent, and detailed large-scale 3DGS worlds using 3D MultiDiffusion, spectral initial noise optimization, and a latent detail enhancer.
PolyLayout: Multi-room Manhattan Layout Estimation: PolyLayout parameterizes multi-room indoor layouts as 3D Manhattan polygons with shared orientation and floor/ceiling heights, formulating a differentiable objective over DINOv2 feature, edge, and confidence maps solved via coarse-to-fine Levenberg-Marquardt optimization with dynamic wall splitting and simplification.
Reflecting Process Expertise in Procedural Material Generation: MaterialApprentice reformulates procedural material creation from static graph-space synthesis into retrieval-time process reasoning over expert tutorial demonstrations, synthesizing intermediate process traces that a compiler grounds into executable Blender shader graphs.
Robustness Emerges Early in Training Dynamics, but Is Not Preserved: This paper discovers that shallow layers spontaneously develop robust representations and flat loss landscapes in early training but lose them during standard convergence; to retain these priors without modifying architectures, it introduces two parameter-free trajectory interventions—Early-Phase Stabilization (EPS) and Asymmetric Weight Reversion (AWR).
SceneDiff: A Benchmark and Method for Multiview Object Change Detection: Addressing object change detection across non-overlapping camera trajectories, this paper introduces the SceneDiff Benchmark with 350 video pairs and dense instance annotations, alongside a training-free framework combining 3D geometry co-registration and region-level feature comparison.
SyntheticDoc: A Large Synthetic Dataset for Document Unwarping and Illumination Correction: To break through the decade-long bottleneck of scarce, low-resolution training data in document restoration, SyntheticDoc introduces a massive dataset of 1,000,000 high-resolution synthetic samples generated via ArcSim physics simulation and Blender path tracing, equipped with pixel-perfect UV and shading annotations, enabling a simple lightweight baseline to achieve new state-of-the-art results on DocUNet.

Browse all 15 Pretraining papers →

🔍 Information Retrieval & RAG (8)¶

TSEmbed: Unlocking Task Scaling in Universal Multimodal Embeddings: TSEmbed distributes conflicting multimodal objectives across input-dependent MoE-LoRA experts, then weights hard negatives by routing similarity after expert warm-up, reaching overall scores of 70.5/74.7 with MMEB-only 2B/7B models and surpassing the corresponding B3 baselines by 2.4/2.7 percentage points.
LightSTAR: Efficient Visual Document Retrieval via Lightweight Selection with Vision-Adaptive Refinement: To address the issue where MLLMs are too slow to encode all pages in visual document retrieval, LightSTAR proposes first filtering out candidates with high recall using an LLM-free lightweight visual selection module, and then performing vision-adaptive semantic refinement only on those candidates. This achieves 89.1 NDCG@5 on ViDoRe while reducing the end-to-end latency for 5,000 pages to 123.9s.
FlowCIR: Semantic Transport via Flow Matching for Zero-Shot Composed Image Retrieval: FlowCIR reformulates zero-shot composed image retrieval as conditional semantic transport within vision-language embedding space, leveraging conditional flow matching and top-K hard-negative retrieval supervision for fast one-step transport, supplemented by an inference-time multi-negative steering strategy to mitigate negation collapse while requiring only 0.5 hours of single-GPU training.
HyFL-CLIP: Hyperbolic Fine-Tuning of CLIP for Robust Long-Context Understanding: HyFL-CLIP distills the Euclidean image-text alignment of pretrained CLIP into hyperbolic space (Lorentz model). By modeling the "global description - local component" hierarchical entailment relationship via Einstein midpoint aggregation, the model stably retrieves correct images even when long texts are perturbed (reordered, deleted, word-dropped), achieving up to a 19.5% improvement over the strongest baseline under word perturbations.
ME-IQA: Memory-Enhanced Image Quality Assessment via Re-Ranking: To resolve the severe discrete collapse issue in reasoning-induced vision-language models for image quality assessment, ME-IQA introduces a training-free, test-time memory-enhanced re-ranking framework that retrieves aligned neighbors from a hybrid memory bank and fuses pairwise VLM preference probabilities with the initial score under Thurstone's Case V model.
What Images Cannot Say: Language-Guided Olfactory Representation Learning: Addressing the fundamental challenge that camera images cannot observe out-of-frame diffusing odor sources and ambient contexts, SCENT leverages Vision-Language Models (VLMs) to generate structured language descriptors across objects, context, and inferred smells as a semantic bridge, achieving state-of-the-art cross-modal retrieval and interpretable odor mixture disentanglement.
MMAgent-R2: Learning to Rerank and Reject for Agentic mRAG: MMAgent-R2 unifies visual reranking and active rejection into an internal verification mechanism for agentic mRAG, optimized via GRPO with step-level verification rewards to break free from the constraints of static candidate pools and noisy global visual retrieval.
Multi-Anchor Distillation with Text-Guided Analytic Classifier for Continual Learning: To tackle both cumulative feature drift in the visual encoder and gradient-based decision boundary overwriting in the classifier for exemplar-free continual learning, this paper proposes an integrated framework combining multi-anchor historical distillation with a closed-form text-guided analytic classifier, establishing new state-of-the-art benchmarks especially on long task sequences.

💻 Code Intelligence (2)¶

Coding with Eyes: Visual Feedback Unlocks Reliable GUI Code Generating and Debugging: The paper first curates 984 executable multi-page GUI development tasks from real Flathub desktop applications together with a programmable Interactive Evaluation Script (IES) to form InteractGUI Bench, then proposes VF-Coder, a three-agent framework in which an MLLM directly looks at and operates the running interface; it lifts Gemini-3-Flash from 21.68% to 28.29% task success rate and from 0.4284 to 0.5584 visual similarity.
PrintAnything: Learning Geometric Plan Map for 3D Printing G-code Generation from Unoriented Point Clouds: Addressing the vulnerability of conventional 3D printing slicing that strictly mandates watertight meshes and easily fails under point-cloud reconstruction artifacts, PrintAnything introduces slice-wise multi-conditioning point projection and a unified 2D Geometric Plan (G-plan) map representation, achieving the first end-to-end framework that directly generates executable 3D printing G-code from unoriented point clouds.

💬 LLM (Other) (13)¶

ProAct: Agentic Lookahead in Interactive Environments: ProAct moves lookahead out of inference time and into data construction: it first probes the real environment with MCTS and compresses the resulting futures into observation-analysis-conclusion causal chains for SFT (GLAD), then stabilizes PPO/GRPO training with a parameter-free Monte-Carlo Critic (MC-Critic) that estimates values from cheap random-policy rollouts, letting a 4B model substantially outperform open-source baselines of the same and larger scale on 2048 and Sokoban.
Embedding Rotation Invariance for Provable Multi-Oriented Scene Text Recognition: To eliminate the error accumulation and inference overhead stemming from explicit orientation prediction and rotation data augmentation in multi-oriented scene text recognition, RISTER incorporates rotation equivariance into the encoder (via the RELG backbone) and rotation invariance into the decoder (via cross-attention in RITD), creating an end-to-end rotation-invariant recognition system with theoretical guarantees that outperforms prior SOTA by 4.0% on multi-oriented benchmarks.
ExpoMotion: A Large-Scale Benchmark and A Householder Projection Network for Multi-Exposure Fusion: To tackle the lack of high-fidelity dynamic benchmarks and the propensity of correlation-based fusion to admit ghosting artifacts, this paper presents ExpoMotion, a 1,738-sequence real-world dynamic MEF benchmark, and HOP, a decoupled architecture using Householder orthogonal projections to filter out motion artifacts geometrically while harmonizing global exposures.
GRE-Diff: Gaussian Room Embeddings for Structured Layout Diffusion: GRE-Diff models individual rooms as continuous 2D Gaussian distributions (Gaussian Room Embeddings, GRE) to guide a vector diffusion process, achieving an FID of 4.36 and a 99.21% F1BC-RC on RPLAN while enabling robust multi-modal and GUI-based interactive floor plan editing.
Learning Probabilistic Prompt for Continual Learning: To tackle the ubiquitous "prompt collapse" issue where deterministic prompt vectors become highly correlated and fail to represent diverse distributions across incremental tasks, this paper models prompts as Gaussian distributions, samples diverse prompt tokens from a query-conditioned mixture distribution, and employs a distribution regularization loss to stabilize continuous learning.
Prior-Conditioned Gaussian Discriminants for Generalizable AI-generated Image Detection: Formalizes AI-generated image (AIGI) detection as a tripartite transfer system of training prior, frozen representation, and decision rule, establishing a closed-form Gaussian discriminant ladder with nested covariance assumptions as a diagnostic baseline that reveals learned classifier heads often fail to surpass frozen low-order feature geometry.
SAM2Matting: Generalized Image and Video Matting: SAM2Matting proposes a decoupled "tracker-to-matting" framework that divides video matting into high-level tracking (where a frozen VOS tracker like SAM2/SAM3 ensures temporal consistency) and low-level matting (where a trainable ROI detector + progressive Alpha predictor estimates fine-grained transparency). Trained solely on image matting data, it achieves SOTA in a zero-shot manner on video matting benchmarks. It also supports multiple prompt types, maintains strong temporal consistency, and generalizes well to both human and open-world scenarios.
Debiased Textual Prompt Tuning for Enhancing Unknown Class Discovery: Addressing known-class model bias and cross-modal graph sparsity in text-guided open-world semi-supervised learning, this paper proposes an optimal-transport-debiased prompt tuning scheme paired with an asymmetric bimodal graph label propagation framework, achieving a 20.7% relative accuracy improvement on unknown classes across large-scale benchmarks.
Direct Preference Optimization for Perceptual Alignment via Vision-Language Consistency: VLC-DPO is an unsupervised multimodal alignment framework that constructs preference pairs using complementary image- and description-based responses guided by a joint vision-language consistency score, incorporating consistency discrepancies as adaptive penalties in DPO to effectively suppress hallucinations and advance zero-shot VQA performance.
Leveraging Cross-Modal Knowledge Transfer for Knowledge-Aware Concept Customization: Addressing the instability and lack of semantic reasoning in conventional rare-token concept customization, this paper introduces the knowledge-aware concept customization task alongside MoKus, a two-stage framework that leverages cross-modal knowledge transfer to project diverse natural language knowledge onto a visual anchor via closed-form model editing, achieving efficient and high-fidelity customized generation.

Browse all 13 LLM (Other) papers →

🎨 Image Generation (328)¶

MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE: MixGRPO concentrates stochastic exploration and GRPO updates in a short window that moves from coarse to fine denoising stages, using deterministic sampling elsewhere; on FLUX.1-dev with HPDv2 evaluation, it improves preference scores with fewer optimized steps, while Table 1 reports approximately 49% and 71% lower training iteration time for the base version and the fastest Flash* version, respectively.
TIIF-Bench: How Does Your T2I Model Follow Your Instructions?: TIIF-Bench combines hierarchical short/long prompt pairs, attribute-level binary checks, and specialized text/style metrics, adapting Qwen3VL-8B into an interpretable evaluator that achieves 91.07% accuracy on a human-annotated validation set.
Distribution Matching Distillation Meets Reinforcement Learning: DMDR couples few-step diffusion distillation and preference optimization in a two-stage training process, allowing 4-step students to outperform multi-step teachers on several overall metrics, although preference gains still reduce generation diversity.
Towards Scalable Pre-training of Visual Tokenizers for Generation: VTP trains a visual tokenizer for reconstruction, visual semantics, and image-text alignment together, allowing additional tokenizer pretraining to improve downstream diffusion generation and achieving guided gFID 1.11 on ImageNet 256×256 with a 675M generator trained for 600 epochs.
Continuous Speculative Decoding for Autoregressive Image Generation: This paper is the first to extend speculative decoding from discrete to continuous distributions. It enhances the draft-target acceptance rate through denoising trajectory alignment, and resolves the sampling bottleneck of the correction distribution (which lacks an analytical expression) via acceptance-rejection sampling. It achieves over 2x wall-clock speedup on multiple continuous visual AR models including MAR, xAR, and Harmon while maintaining generation quality, completely training-free.
SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models: SRUM proposes a self-rewarding post-training framework where the comprehension module of a Unified Multimodal Model (UMM) acts as an internal evaluator, providing fine-grained corrective signals to the generation module through a dual global-local reward system, thereby significantly improving the quality of complex text-to-image generation.
GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning: GenAgent treats the generator as a frozen tool and trains one multimodal model through cold-start training and agentic reinforcement learning to rewrite prompts, inspect images, and reflect, raising FLUX's GenEval++ score from 0.325 to 0.561 with at most two generation rounds in the main experiments.
Query-Kontext: An Unified Multimodal Model for Image Generation and Editing: Query-Kontext transfers semantic and coarse visual planning from a VLM to a diffusion model through trainable multimodal query tokens, adds low-level image conditioning for detail, and unifies generation and editing with 0.88 on GenEval using rewritten prompts and 7.66 / 7.65 on the full English / Chinese GEdit-Bench splits.
RealGen: Photorealistic Text-to-Image Generation via Detector-Guided Rewards: RealGen turns synthetic-image detectors into realism rewards, first optimizing Qwen3 prompt rewriting and then FLUX image generation, achieving a 50.15% model-judged win rate against real photographs on RealBench with the full system, which does not establish indistinguishability for all human observers.
Learning on the Manifold: Unlocking Standard Diffusion Transformers with Representation Encoders: RJF replaces straight-line flow matching in representation latent spaces with spherical geodesic flow matching and weights training by curvature-dependent error propagation, enabling a standard-width, 131M-parameter DiT-B to achieve FID 4.95 without guidance and 3.37 with guidance on ImageNet 256 class-conditional generation, both after 200 epochs.

Browse all 328 Image Generation papers →

🎬 Video Generation (169)¶

End-to-End Training for Autoregressive Video Diffusion via Self-Resampling: Resampling Forcing trains a causal diffusion predictor on histories containing its own resampled model errors while retaining clean video targets, removing auxiliary teachers and discriminators and improving long-video temporal quality through native 15-second training.
ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytelling: ShotStream reformulates multi-shot video generation as next-shot prediction conditioned on historical visuals, combining sparse conditioning, boundary-aware dual caches, and two-stage self-forcing distillation to achieve 15.95 FPS on one H200 and a transition-control score of 0.978.
MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens: Without updating generator weights, MemRoPE continuously folds evicted context into long- and short-term EMA memory and applies RoPE to unrotated keys only at attention time, enabling hour-scale generation with a fixed cache and a VBench-Long average of 87.99 versus 87.01 for the comparison method on one-hour LongLive videos.
AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation: AnyFlow replaces endpoint-only video distillation with transitions across arbitrary time intervals, then corrects student rollouts using three-segment shortcut simulation and distribution matching, reaching 84.05 VBench at 4 NFEs and 84.41 at 32 NFEs with its 14B causal model.
PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generation: PhyGDPO proposes a group-wise direct preference optimization framework for physically consistent video generation. It utilizes real-world videos as winning samples and a Plackett-Luce group-wise probability model instead of pairwise comparisons to capture global preferences. Co-designed with VLM-guided physical reward reweighting and a LoRA-switching reference mechanism, it efficiently achieves superior physical consistency over Sora2 and Veo3.1 at the 14B parameter scale.
VGGRPO: Towards World-Consistent Video Generation with 4D Latent Reward: VGGRPO connects video latents directly to a geometry foundation model and performs GRPO post-training with camera smoothness and reprojection consistency rewards, achieving a 66.00% motion-quality win rate on dynamic scenes with Wan2.2-5B while reducing reward computation time from 54.73 s to 41.33 s.
Taming Text-to-Sounding Video Generation via Advanced Modality Condition and Interaction: For the text-to-sounding video generation (T2SV) task that simultaneously generates visual frames and synchronized audio, this paper proposes a dual-agent caption rewriting framework (CRR) to provide separate, interference-free, modality-pure captions for the video tower and the audio tower. It then employs a dual-tower diffusion model, BridgeDiT, using dual cross-attention (DCA) to bridge and exchange features between the two towers, achieving state-of-the-art (SOTA) performance on most metrics across three benchmarks.
Anchor Forcing: Anchor Memory and Tri-Region RoPE for Interactive Streaming Video Diffusion: Anchor Forcing addresses evidence loss at prompt switches with anchor-guided re-caching and positional distribution shift during long rollouts with tri-region RoPE, improving the LongLive baseline from 26.37 to 73.00 in dynamic degree and from 78.92 to 82.25 in quality on interactive 60-second videos.
HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising: HiAR conditions each video block on earlier blocks at the current denoising step's output level, rather than fully clean predictions, and uses bidirectional teacher-trajectory regularization to prevent motion collapse, achieving a VBench Total of 0.821, Drift of 0.257, and reported throughput of 30 fps on 20-second videos.
PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference: PackForcing divides video history into intact initial anchors, compressed and dynamically retrieved intermediate history, and intact recent context, combining these with temporal RoPE correction to generate 120-second videos from a Wan2.1-based model trained with roughly 5-second windows; it achieves the highest 60-second and 120-second VBench Dynamic Degree among the evaluated methods, although a bounded active context must be distinguished from constant storage for all history.

Browse all 169 Video Generation papers →

🧩 Multimodal VLM (225)¶

ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models: ViewSpatial-Bench diagnoses perspective-taking failures through five camera- and person-centered spatial localization tasks, while approximately 43K targeted supervision samples raise Qwen2.5-VL-3B accuracy from 35.85% to 82.09%, a gain of 46.24 percentage points rather than a 46.24% relative improvement.
Visual Spatial Tuning: Without adding a specialized 3D encoder, VST first teaches a VLM spatial perception through multimodal geometric data, then develops spatial reasoning with BEV-assisted textual reconstruction traces and reinforcement learning, raising Qwen2.5-VL-7B from 25.9 to 34.8 on MMSI-Bench and from 38.9 to 61.2 on VSI-Bench.
Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedback: Render-in-the-Loop lets Qwen3-VL-8B observe the cumulative canvas after each SVG fragment, combining specialized multi-turn supervised training with inference-time repetition filtering to outperform several autoregressive baselines trained on larger datasets on MMSVGBench using 0.85M training samples, at approximately 1.50 times the inference latency for VSF.
Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild: Real5-OmniDocBench reconstructs all 1,355 OmniDocBench v1.5 pages under five physical acquisition scenarios, using 6,775 content-matched images to test parsing robustness; digital-document proficiency does not guarantee physical-world reliability, and the 0.9B PaddleOCR-VL-1.6 leads the listed large generalist models with an Overall score of 93.19 in this evaluation.
Towards Interactive Global Geolocation Assistant: MG-Geo trains GaGA with geographic metadata, image-grounded clues, and multi-turn reasoning dialogs to turn one-shot street-view coordinate prediction into an interaction that accepts factual constraints, achieving 63.06% country accuracy and 6.28% city accuracy on GWS15k without uniformly leading at fine-grained coordinate localization.
How to Teach Large Multimodal Models New Skills: The paper attributes part of multimodal-model forgetting to task-induced response bias and shows that tuning only attention projections or MLP Gate&Up yields current-task gains of +24.9 / +30.5 percentage points on LLaVA-OneVision across five sequential tasks, while limiting held-out changes to -0.6 / -2.1 percentage points.
MMR-Bench: A Comprehensive Benchmark for Multimodal LLM Routing: MMR-Bench provides reusable offline evaluation for selecting among heterogeneous multimodal large language models on image-text queries, comparing cost-performance curves across features and routing policies; the authors report an operating point that matches the strongest single model at approximately 33% of its cost, not a uniform saving across all methods and tasks.
UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameters: UniRec-0.1B trains a 0.1B encoder-decoder from scratch using approximately 40 million multi-level samples, explicit line and paragraph supervision, and decoupled text/formula tokenization, achieving an average edit distance of 0.100 on UniRec-Bench and an average block recognition time of 0.37 seconds under the paper's specified single-GPU conditions.
DiaDem: Advancing Dialogue Descriptions in Audiovisual Video Captioning for Multimodal Large Language Models: DiaDem combines dialogue-aware synthetic data and difficulty-partitioned GRPO on AVoCaDO-7B to make complete audiovisual captions more accurate about "who said what," achieving REF 65.9 and ASR 79.3 on DiaDemBench while retaining competitive general caption quality.
OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models: OmniMamba shares an autoregressive Mamba-2 backbone while decoupling visual encoding, output vocabularies, and task adapters, achieving 5.50 FID-30K on COCO with approximately 2M image-text pairs as reported for unified-model training, a data accounting boundary that excludes inherited pretrained components.

Browse all 225 Multimodal VLM papers →

🧠 VLM Reasoning (164)¶

Video-Holmes: Can MLLM Think like Holmes for Complex Video Reasoning?: Video-Holmes tests whether models can independently locate and connect scattered clues through 1,837 questions about 270 suspense short films; Gemini-3.0-Pro leads the main table at 49.6%, compared with 96.3% for humans watching each film once.
Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens: CoVT teaches a VLM to generate continuous visual tokens supervised by segmentation, depth, and semantic experts before answering, replacing purely verbal intermediate observations with 16 latent perceptual states and raising Qwen2.5-VL-7B from 74.5 to 80.0 on CV-Bench, including a 14.0-percentage-point gain on its depth subtask.
LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?: LEGO-Puzzles progressively evaluates elementary spatial questions, assembly planning with distractors, and multi-turn intermediate-state generation, revealing substantial gaps across 29 multimodal models: GPT-5 trails humans by 21.8 percentage points on the Lite subset, while its planning accuracy falls from 90% at one step to 0% at eight steps.
TIR-Bench: A Comprehensive Benchmark for Agentic Thinking-with-Images Reasoning: TIR-Bench expands visual reasoning evaluation beyond crop-based search to 13 image-operation tasks with 1215 examples, showing substantial benefits from tool execution while the strongest o3-TU configuration still scores only 46.0 overall and struggles with complex spatial planning and tool selection.
Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning: VIGA repeatedly writes code, renders, actively inspects, and edits, using semantic skills and bounded historical memory to maintain editable scenes without training, and reports a 124.70% average improvement over one-shot generation on BlenderBench.
Reinforcing Video Reasoning with Focused Thinking: TW-GRPO reallocates learning strength using within-group token distributional differences at aligned positions and grants partial credit to multi-answer predictions under a strict false-positive constraint, improving Qwen2.5-VL-7B's CLEVRER accuracy from 41.1% with matched GRPO to 50.4% using only 1K CLEVRER training samples, although gains do not extend to every video benchmark.
UniREditBench: A Unified Reasoning-based Image Editing Benchmark: UniREditBench extends image editing evaluation to real-world interactions and game-rule reasoning, constrains the evaluator with textual and target-image references, and trains UniREdit-Bagel with synthetic chain-of-thought data to achieve an in-domain overall score of 78.15, while remaining behind the strongest closed-source models on out-of-distribution RISEBench.
Embed-RL: Reinforcement Learning for Reasoning-Driven Multimodal Embeddings: Embed-RL uses retrieval feedback from a frozen embedder to train a separate reasoner and supplies the embedder with image boxes, video keyframes, and text keywords from its chain-of-thought, allowing Embed-RL-4B to reach 68.1 overall on MMEB-V2, 3.6 points above UME-R1-7B.
Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously: VST moves video reasoning ahead of the user query, continually updating textual memory through causally trained streaming thoughts, enabling VST-7B to reach 79.5% on StreamingBench and 41.9% accuracy with 0.56-second query-response latency on VideoHolmes.
LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning: LaViT makes Qwen2.5-VL-3B generate four continuous visual tokens before answering, distills visual semantics and attended regions from a 32B teacher, and gradually opens the direct image-access path to prevent bypassing those tokens, improving BLINK Relative Depth from 61.29% to 78.23%.

Browse all 164 VLM Reasoning papers →

⚡ VLM Efficiency (57)¶

FlashVLM: Text-Guided Visual Token Selection for Large Multimodal Models: At the vision encoder–language model interface, FlashVLM selects important regions using fused query relevance and visual saliency, then adds non-redundant background tokens; reducing LLaVA-1.5-7B from 576 to 128 visual tokens yields a reported 100.6% relative performance, but does not preserve every task's performance.
ResilPhase: Plug-and-Play Phase Mapping and Noise-Resilient Macro-Trajectory Extrapolation for Diffusion Acceleration: ResilPhase reformulates training-free acceleration for DiT from a layer-wise derivative extrapolation paradigm to macro-trajectory extrapolation in the ODE space. It eliminates layer-wise error cascades using an end-to-end Global Drift, bypasses chaotic derivative noise using derivative-free Barycentric Lagrange interpolation, and suppresses the Runge phenomenon by projecting discrete timesteps into a bounded phase space via Chebyshev/Balanced phase mapping. It maintains state-of-the-art generation quality on FLUX.1-dev even at an approximate 5x acceleration.
Fast-dVLA: Accelerating Discrete Diffusion VLA to Real-Time Performance: Fast-dVLA adapts discrete diffusion vision-language-action models to cache completed prefixes and denoise multiple blocks concurrently through lightweight asymmetric distillation, increasing Dream-VLA decoding throughput from 98.8 to 313.1 tokens/s on LIBERO and reporting 30 Hz execution in real-world tasks.
Why and When Visual Token Pruning Fails? A Study on Relevant Visual Information Shift in MLLMs Decoding: The paper finds that static visual token pruning cannot keep pace with shifting visual evidence during multi-step reasoning and introduces DSTP, a training-free add-on that detects attention shifts and temporarily recovers relevant tokens during decoding, raising Qwen3-VL-4B's MathVerse accuracy at 33.3% initial retention from FastV's 32.23% to 52.54%, still below the full model's 61.29%.
DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space: DC-Gen aligns input and output interfaces before using LoRA to adapt a pretrained diffusion backbone to a deeply compressed latent space with fewer tokens, retaining comparable quality across generation tasks; the paper reports up to 53.8 times acceleration for FLUX at 4K, but times only the Transformer backbone.
Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training: Spatial-TTT combines online fast-weight updates, local sliding-window attention, and a small set of full-attention layers, using dense 3D scene descriptions to teach spatial evidence retention; its 2B model scores 64.4 on VSI-Bench while using less memory and computation than Qwen3-VL-2B at 1024 input frames.
ToDRE: Effective Visual Token Pruning via Token Diversity and Task Relevance: ToDRE greedily selects diverse visual tokens before the large language model (LLM), then removes all remaining visual tokens at a deeper layer where cross-modal interaction has subsided, preserving 95.0% mean relative performance across eight image tasks on LLaVA-NeXT-7B with only 10% of input visual tokens retained and achieving about 2.6x speedup in the four-task ablation timing without training.
DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression: DASH treats discontinuities between adjacent audio representations as semantic chunking cues and combines boundary, representational uniqueness, and attention scores to retain tokens, allowing Qwen2.5-Omni-7B to reach 44.9% WorldSense accuracy at 25% target retention and 14.9T FLOPs, compared with OmniZip's 44.7% at 35% retention and 21.4T FLOPs.
MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation: MMLoP factorizes deep prompts in both CLIP encoders into a shared low-rank factor and modality-specific factors, then protects generalization with text drift correction and zero-shot consistency, achieving 79.70% base-to-novel harmonic mean accuracy with about 11.5K trainable parameters without claiming the highest absolute accuracy.
BATQuant: Outlier-Resilient MXFP4 Quantization via Learnable Block-wise Optimization: BATQuant confines learnable non-orthogonal transformations to MXFP4's 32-element quantization blocks, then uses shared factorization and local clipping to reduce deployment overhead and residual outlier effects, achieving 96.43% multimodal performance recovery on Qwen3-VL-8B-Instruct under W4A4KV16.

Browse all 57 VLM Efficiency papers →

🎵 Audio & Speech (23)¶

MGM-Omni: Scaling Omni LLMs to Personalized Long-Horizon Speech: MGM-Omni separates multimodal understanding and speech synthesis into brain-mouth tracks, combining dual audio encoders with chunk-based parallel decoding to achieve 4.98% English WER and 0.19 RTF with a 2B SpeechLM on its Long-TTS-Eval benchmark while conditioning generation on a reference voice.
JAM-Flow: Joint Audio-Motion Synthesis with Flow Matching: JAM-Flow treats LivePortrait's low-dimensional mouth expression keypoints and F5-TTS's mel-spectrograms as two mutually maskable modalities inside a single flow-matching framework, trained jointly through a dual-stream MM-DiT with half-fused joint attention, RoPE temporal alignment, and modality-asymmetric attention masks; one ~500M model covers talking head generation, TTS, motion-to-audio, and automated dubbing, producing a 20-second video in 45 seconds on a single RTX A6000.
Progressive Representation Learning for Multimodal Sentiment Analysis with Incomplete Modalities: PRLF decomposes "which modality is trustworthy right now" into two complementary signals — classification confidence and Fisher information — to pick a per-sample dominant modality, then lets the auxiliary modalities self-refine first and align to the dominant one through projection and residual denoising over four iterations, reaching average F1 of 77.02 / 76.24 / 81.19 on CMU-MOSI / CMU-MOSEI / SIMS under inter- and intra-modality missingness.
CoSyncDiT: Cognitive Synchronous Diffusion Transformer for Movie Dubbing: CoSyncDiT organizes voice-style learning, lip-rhythm calibration, and text alignment sequentially inside a flow-matching Transformer, then regularizes alignment with frame-level contrastive learning and CTC, achieving 47.24% speaker similarity, 7.53% WER, and 0.319 Sync-KL on CinePile-Dub zero-shot Setting 2 (original Table 5).
Music-to-Dance Generation via Atomic Movements: The paper models dance explicitly as a sequence of semantically readable, reusable "atomic movements": a discrete diffusion model first plans the type, duration, and temporal position of every atomic movement conditioned on the full music (a symbolic dance score), and a transition-aware continuous diffusion generator then fills in the transitions, retrieves instances by duration-nearest matching and re-creates them under masked noise, achieving the lowest FID (25.26 / 9.03), the highest BAS (0.2470), and the highest structural consistency R (26.6) on AIST++.
Step-by-Step Video-to-Audio Synthesis via Negative Audio Guidance: Addressing the limitation where existing video-to-audio (V2A) models can only generate a complete audio track all at once instead of adding sounds layer-by-layer like Foley artists, this paper proposes Negative Audio Guidance (NAG). By training a branch conditioned on "already-generated audio tracks" and utilizing it inversely during sampling, the current generation is pushed away from existing sounds. This enables step-by-step generation of complementary audio tracks to mix into high-quality synthesized sound, using only standard single-reference audio-visual datasets.
HumanOmni-Speaker: Identifying Who said What and When: To tackle the fundamental failure of Omni-modal LLMs in multi-person dynamic conversations ("who said what and when") and their over-reliance on static visual shortcuts, this paper introduces the Visual-Registered Speaker Diarization and Recognition (VR-SDR) benchmark and proposes HumanOmni-Speaker, featuring a 25 fps Visual Delta Encoder that compresses inter-frame motion residuals into just 6 tokens per frame for robust cross-modal spatio-temporal alignment.
JoVA: Unified Multimodal Learning for Joint Video-Audio Generation and Editing: JoVA introduces a streamlined, unified diffusion framework for joint video-audio generation and editing, replacing fragmented cross-modal attention layers with native joint self-attention and channel-wise conditioning alongside an adaptive mouth-area flow matching loss to achieve state-of-the-art multimodal synchronization and instruction-guided editing.
MG-RWKV: Multi-Grained Context-Aware RWKV for Temporal Forgery Localization: MG-RWKV proposes a temporal forgery localization (TFL) framework based on the linear-complexity recurrent architecture of RWKV. By capturing global temporal context via bidirectional RWKV, adaptively selecting explicit temporal receptive fields through a Multi-Grained Mixture of Experts (MG-MoE), and eliminating contradictory predictions across multi-scale features using a Cross-Grained Consistency constraint (CGC), it comprehensively outperforms previous state-of-the-art methods on four benchmarks: Lav-DF, TVIL, Psynd, and AV-Deepfake1M, while maintaining \(O(T)\) linear complexity with an inference time of only 73.4ms.
MJEPA: A Simple and Scalable Joint-Embedding Predictive Architecture for Audio-Visual Learning: MJEPA introduces an elegant and scalable audio-visual self-supervised learning framework that trains a single, unified ViT encoder using only joint-embedding predictive objectives in latent space; by combining intra-modal multi-level feature prediction with cross-modal bidirectional alignment, it sets new state-of-the-art records across seven frozen evaluation benchmarks without requiring contrastive objectives, pixel/spectrogram reconstruction, or heavy data augmentations.

Browse all 23 Audio & Speech papers →

🔎 AIGC Detection (8)¶

Beyond Artifacts: Real-Centric Envelope Modeling for Reliable AI-Generated Image Detection: Instead of primarily tracking artifacts from known generators, REM learns a feature envelope of real images using controlled reconstructions and stabilizes it across degradations, achieving 84.2% balanced accuracy on RealChain with chain degradations, 18.4 percentage points above DDA.
Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection: The paper proposes ForenAgent: it packs 12 low-level forensic operators (FFT, DWT, resampling periodicity, SRM, Bayar, PRNU, JPEG Ghost, and others) into a toolbox that an MLLM can call directly, lets the model generate and execute Python code over multiple turns and feed the sandboxed forensic visualizations back as observations, and trains it in two stages with cold start plus a GRPO stage driven by four process rewards — raising overall accuracy on the self-built 100k-image three-class FABench from 81.2% (best baseline) to 89.5%, and reaching 83.1% on SIDA-Test for the best cross-dataset generalization.
SPLIT: Training-Free AI-Generated and Partially Edited Video Detection via Spatial Patch-Level Incoherence and Temporal Roughness: SPLIT is a fully training-free detector that computes two statistics over the patch tokens of a frozen vision encoder — Two-step Temporal Roughness (TTR, comparing accumulated 1-step and 2-step feature displacements) and Local Spatial Motion Incoherence (LSMI, the spatial gradient of a feature-space motion field) — fuses them multiplicatively with gamma correction, and calibrates its threshold on real videos only, reaching the highest Fake Recall at FPR = 0.1% on all three benchmarks (74.45% on FakeParts, far above the best supervised baseline ReStraV at 35.67%).
From Masks to Pixels and Meaning: A New Taxonomy, Benchmark and Metrics for VLM Image Tampering: Existing tampering benchmarks annotate "where the edit is" with object masks, which are badly misaligned with the true generative edit signal; this paper reformulates VLM image tampering as a pixel-grounded, semantics- and language-aware task, releases PIXAR — 420K image pairs with a taxonomy of 8 edit primitives, thresholded pixel labels \(M_\tau\) and semantic class labels — together with a five-loss training framework and pixel/semantic/language metrics, raising tamper-localization IoU from 11.8% to 18.1% under the same backbone.
Efficient Document Tampering Localization with Multi-Level Discrepancy Features and Unified DCT-Quantization Embedding: DiffNet introduces two complementary designs: a multi-level discrepancy transform that converts the feature pyramid from content signals into sign-invariant discrepancy intensity signals, and a unified DCT-quantization joint embedding that replaces the traditional, high-overhead Frequency Perception Head (FPH) with discrete embeddings. This achieves approximately a 30% F1 improvement on cross-domain and human-manipulated document localization at a lower computational cost, with up to a 7-fold increase in throughput.
Trustworthy Image Authentication using Forensic Knowledge Graphs: This work combines the "evidence-seeking" capability of image forensics with the natural language generation of VLMs into a single system: first, a self-supervised forensic fingerprint backbone is used to segment the image into forensically consistent regions and predict the source identity, post-processing history, and compression parameters for each region to construct a structured "Forensic Knowledge Graph (FKG)"; subsequently, a VLM is guided to generate step-by-step verifiable forensic explanations based strictly on the graph, consistently outperforming pure forensic models and pure VLMs in terms of detection, forgery typing/localization, and forensic explanation.
Asymmetric Anchoring: Opening the Black Box of MLLMs for Forgery Detection: AAP aligns intermediate multimodal representations with a frozen visual reference only for real images, then uses patchwise alignment errors to guide localization, achieving 97.1% detection accuracy and 56.8% localization IoU with AAP-lisa on SID-Set.
Q-REAL: Towards Naturalness and Distortion Evaluation for AI-Generated Content: To overcome the uninterpretable single-scalar score paradigm in text-to-image quality assessment, this work establishes Q-Real—a 10K-image fine-grained benchmark with 32.8K entities annotated across Naturalness and Distortion dimensions—alongside a progressive three-stage MLLM training pipeline that achieves entity-level judgment, spatial grounding with causal attribution, and consistent quality score prediction.

🧊 3D Vision (512)¶

RoMa v2: Harder Better Faster Denser Feature Matching: RoMa v2 combines multi-view coarse matching over frozen DINOv3 features, separately trained efficient refiners, and pixel-wise uncertainty prediction to improve challenging-viewpoint matching and subpixel precision, reducing RoMa's AerialMegaDepth EPE from 25.05 to 4.12 pixels without claiming universal superiority in extreme cross-modal generalization or absolute speed.
Ultra3D: Efficient and High-Fidelity 3D Generation with Part Attention: Ultra3D first establishes a coarse object layout with compact VecSet tokens, then generates sparse voxel latents using attention organized along part boundaries, supporting 1024-resolution meshes with 6.7× faster part self-attention and 3.3× faster DiT inference than a full-attention baseline.
P3-SAM: Native 3D Part Segmentation: P3-SAM learns a single-point-prompted, two-stage multi-mask network from approximately 3.7 million native 3D assets and combines quality prediction with deduplication for automatic segmentation, reaching an average IoU of 59.88 on PartObj-Tiny without connectivity, compared with PartField's 53.93.
VolSplat: Rethinking Feed-Forward 3D Gaussian Splatting with Voxel-Aligned Prediction: VolSplat shifts the primitive prediction paradigm of feed-forward 3D Gaussian Splatting from pixel-aligned to voxel-aligned: it first projects multi-view 2D features back to 3D space to form a voxel feature grid through depth maps, refines them using residual learning via a sparse 3D U-Net, and then predicts Gaussian parameters per voxel, thus fundamentally decoupling the 3D representation from the rigid binding of the input image resolution to achieve adaptive Gaussian density allocation.
LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory: LoGeR reconstructs video chunk by chunk, preserving adjacent-chunk detail through sliding-window attention and propagating global geometry through test-time-updated fast weights; its feedforward-aligned LoGeR* variant reduces average KITTI ATE from TTT3R's 72.86 m to 18.65 m, although very-long-sequence evaluation includes periodic state resets.
MVGS: Multi-view Regulated Gaussian Splatting for Novel View Synthesis: MVGS replaces single-view iterative fitting in Gaussian Splatting with joint multi-view updates, cross-intrinsic guidance, and two densification strategies, improving 3DGS PSNR on Mip-NeRF 360 from 28.69 to 29.61 while increasing training time from 0.3 to 2.4 hours.
Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery: Skyfall-GS anchors coarse urban geometry and appearance in satellite observations, then progressively transfers diffusion-generated facade details into a 3D Gaussian scene through a descending view curriculum, achieving FIDCLIP 27.03 and CMMD 2.110 on DFC2019 and enabling real-time exploration after offline optimization.
XYZ-IBD: Benchmarking Robust 6D Object Pose Estimation under Real-World Industrial Complexity: XYZ-IBD combines multi-sensor real-world bin data, coating-assisted geometric annotation, and noise-matched simulation to benchmark industrial 6D pose estimation across 15 parts, exposing a substantial performance gap: SAM6D achieves 0.578 6D AP on XYZ-IBD versus its BOP-Core 5 average of 0.704.
Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding: VEGA-3D repurposes frozen Wan2.1 as a spatial feature extractor, combining moderate noise injection, intermediate-layer features, and token-level gated fusion to raise ScanRefer [email protected] from 51.7 to 56.2, with varying gains on spatial reasoning and manipulation.
LoMa: Local Feature Matching Revisited: LoMa retains the basic division between the DaD detector, DeDoDe descriptor, and LightGlue matcher, but scales training data, updates the training recipe, and widens the matcher to reach 54.3 mAA@10px on the new HardMatch test set, 18.6 points above ALIKED+LightGlue.

Browse all 512 3D Vision papers →

🎯 Object Detection (79)¶

RT-DETRv4: Painlessly Furthering Real-Time Object Detection with Vision Foundation Models: RT-DETRv4 supervises deep AIFI features with a frozen vision foundation model during training and adapts supervision strength to the module's gradient share, raising the L variant from DEIM-L's 54.7 to 55.4 COCO AP while retaining 8.07 ms inference latency.
M4-SAR: A Multi-Resolution, Multi-Polarization, Multi-Scene, Multi-Source Dataset and Benchmark for optical-SAR Object Detection: This paper proposes M4-SAR—the first large-scale optical-SAR paired oriented object detection dataset (112,174 image pairs, 981,862 instances, 6 categories), accompanied by a unified evaluation tool MSRODet and an end-to-end fusion detection framework E2E-OSDet (FAM + CMIM + AFM). After fusion, mAP improves by up to 5.1% compared to the best single-source modality, with particularly significant gains in complex environments.
Low-latency Event-based Object Detection with Spatially-Sparse Linear Attention: SSLA decomposes the global state of linear attention into geometrically routed local substates, enabling parallel training and event-by-event recurrent inference for low-latency detection; SSLA-L achieves 0.375 mAP on Gen1 with 0.724 MFLOPS per event and a measured single-core processing latency of 7.20 microseconds.
VLOD-TTA: Test-Time Adaptation of Vision-Language Object Detectors: To address the performance degradation of vision-language object detectors (such as YOLO-World and Grounding DINO) under test-time distribution shifts, VLOD-TTA utilizes "IoU-weighted entropy minimization" to focus adaptation on spatially dense clusters of overlapping bounding box proposals, and "image-conditioned prompt selection" to select the text prompts that best match the current image. By updating only a minimal number of adapter parameters, it consistently outperforms existing TTA baselines and previous specialized methods under diverse shifts—including artistic styles, adverse driving conditions, low-light, and common degradations—at a much lower computational cost.
DroneFINE: Domain-Aware Parameter-Efficient Fine-Tuning of Vision-Language Detectors for Drone Images: Building on a frozen GroundingDINO with only 1.54M trainable parameters (5.6% of the 27.5M used by full fine-tuning), DroneFINE combines two domain-aware modules — the foreground-aware dynamic multi-path convolution adapter HyperAdapter and the background-vocabulary gated query re-ranking module SemanticGate — and reaches 38.4 mAP / 61.2 mAP50 on VisDrone and 26.5 mAP / 43.6 mAP50 on UAVDT, matching (and on UAVDT surpassing) full fine-tuning while preserving generalization on COCO.
RGBT-GroundBench: Visual Grounding Beyond RGB in Complex Real-World Scenarios: The paper builds RGBT-GroundBench, the first large-scale RGB-Thermal visual grounding benchmark for complex real-world scenarios (21,535 RGB-TIR pairs, 38,760 referred instances, three-level annotations), defines a unified protocol supporting RGB-only, TIR-only, and RGB+TIR inputs, benchmarks 11 VG models under it, and proposes the reference baseline RGBT-VGNet, which attains the best [email protected] on all three sub-datasets.
What is the Right Embedding Space for Contrastive Learning in Referring Expression Counting?: The paper moves contrastive learning for Referring Expression Counting (REC) out of the image-text embedding space and entirely into the image embedding space, using bipartite matching to define positive and negative sets on the fly, expanding the negative pool from a handful of referring expressions to hundreds of in-image visual tokens, and using only positive-class tokens as anchors; the resulting plug-in module, C-REX, improves three detection-based REC architectures consistently, cutting MAE by up to 28% and RMSE by up to 24.5% on REC-8K, and also transfers to class-agnostic counting.
DetPO: In-Context Learning with Multi-Modal LLMs for Few-Shot Object Detection: DetPO converts a few annotated images into repeatedly validated textual class definitions, using false positives to narrow concepts and false negatives to broaden them, improving few-shot detection with frozen multimodal large language models and optionally refining detection rankings through per-box VQA re-scoring.
MOCHA: Multi-modal Objects-aware Cross-arcHitecture Alignment: MOCHA has a frozen LLaVa teacher produce object-level multimodal embeddings conditioned on "this box plus this class name," maps the multi-scale region features of a YOLO student into that space with a channel-attention translation module, and supervises the result with a dual objective combining pointwise alignment and relational geometry preservation, lifting a 3.2M-parameter detector by 10.1 points on average over the YOLOv8n baseline across four personalized detection benchmarks at only 3 ms/image of extra inference cost.
Rethinking IRSTD: Single-Point Supervision Guided Encoder-only Framework is Enough for Infrared Small Target Detection: SPIRE replaces contour segmentation with centroid response regression, using single points to construct structured local supervision, a lightweight encoder-only network to predict heatmaps, and direct peak extraction to achieve 97.05% F1 on SIRST-UAVB with approximately 0.29M parameters.

Browse all 79 Object Detection papers →

✂️ Segmentation (94)¶

LlamaSeg: Image Segmentation via Autoregressive Mask Generation: LlamaSeg uses a frozen VQGAN to turn masks into discrete visual tokens and a LLaMA-style model to generate them from images and text, reaching 55.9 mIoU on ADE20K at 384 pixels, while still trailing specialized discriminative methods in referring segmentation accuracy and latency.
Delayed Bidirectional Alignment via Disentangled Audio Semantics for Audio-Visual Segmentation: DDAVS disentangles multi-source audio semantics using learnable queries anchored by a prototype memory bank, and introduces delayed bidirectional cross-attention in deep layers to align audio-visual modalities, comprehensively outperforming previous state-of-the-art (SOTA) methods on AVSBench and VPO multi-source/multi-category/multi-instance scenarios.
EM3M: An Electron Micrograph Dataset for Microstructural Segmentation and Generation: EM3M establishes a materials microstructure benchmark with 5,091 real electron micrographs, 2,985,660 instance annotations, and nine-dimensional structured descriptions, while SDXL augmentation and the adapted flow-based UniAIMS baseline raise full-dataset [email protected] from 0.800 with real-data training to 0.809 with mixed training.
Segmenting Visuals With Querying Words: Language Anchors For Semi-Supervised Image Segmentation: HVLFormer converts pretrained vision-language class semantics into dataset- and image-aware multi-scale textual queries, then stabilizes mask decoding through layer-wise cross-view consistency, reaching 89.1% and 58.8% mIoU with 92 VOC labels and 232 COCO labels, respectively, using SigLIP2.
Beyond Aesthetics: Quantifying Information Loss in Turbid Scenes: The paper introduces the controlled TUB dataset and Phase-Congruency Delentropy (PCD) to distinguish unattractive underwater images from images that have lost task-relevant structure; PCD correlates most strongly with segmentation performance in the reported real and synthetic experiments, but is not a universal measure of recognizability across arbitrary scenes.
DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation: DGSeg has the MLLM emit two complementary cues — a textual semantic description of what the target is and a bounding-box spatial localization of where it is — routes them through two separate segmentation branches, and uses a lightweight dynamic gating module to estimate pixel-wise fusion weights, reaching 69.6 (val) / 67.3 (test) gIoU on ReasonSeg under the zero-shot setting and surpassing strong baselines such as Seg-Zero, CoPRS and SAM-Veteran.
ET-SAM: Efficient Point Prompt Prediction in SAM for Unified Scene Text Detection and Layout Analysis: ET-SAM employs a lightweight Point Decoder to predict word-level heatmaps for extracting sparse foreground points, replacing the convention in Hi-SAM of randomly sampling thousands of points from pixel-level segmentations. Coupled with learnable task prompts and a joint training strategy to utilize multi-granularity heterogeneous annotation data, it achieves competitive performance on HierText with approximately 3\(\times\) inference speedup, and improves the average F-score by 11.0% on Total-Text/CTW1500/ICDAR2015.
Fast and Flexible Robustness Certificates for Semantic Segmentation: This paper reduces certified robustness for semantic segmentation to "one forward pass plus one sort": a Lipschitz-constrained network yields per-pixel invariance radii, and "how many pixels can an attacker flip" is recast as a one-dimensional knapsack with capacity \((L\epsilon)^2\), delivering deterministic worst-case lower bounds on pixel accuracy, FNR, stability and class IoU for 1024×1024 Cityscapes images in about 0.1 s — roughly 600× faster than randomized smoothing.
MobileManiBench: Simplifying Model Verification for Mobile Manipulation: This paper proposes MobileManiBench, a large-scale mobile manipulation simulation benchmark based on Isaac Sim. By training reinforcement learning policies, it automatically generates 300K diverse manipulation trajectories (containing language instructions, multi-view RGB-D images, object/robot states, and actions) across 2 robot platforms, 630 objects, 5 skills, and 100 scenes. Multiple VLA models are systematically evaluated on this benchmark, providing a standardized platform for verifying VLA architectures in mobile manipulation tasks.
PanoSAM2: Lightweight Distortion- and Memory-aware Adaptions of SAM2 for 360 Video Object Segmentation: PanoSAM2 addresses projection distortion, left-right seams, and sparse object memory in panoramic videos through SAM2 decoder and memory adaptations plus geometry-aware supervision, raising Hiera-S from 60.2 to 65.8 \(\mathcal{J}\&\mathcal{F}\) on the 360VOTS test set at the cost of some inference speed.

Browse all 94 Segmentation papers →

🖼️ Image Restoration (73)¶

Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision: Built on Lumina-Next generative priors, OmniLV combines separate text and image encoding, projection-addition of visual exemplars, and a co-trained shallow condition adapter to cover four low-level vision categories, with strong task alignment and several perceptual results but no across-the-board advantage over specialists.
Denoising the Deep Sky: Physics-Based CCD Noise Formation for Astronomical Imaging: Rather than redesigning the denoising network, this work estimates CCD-specific spatial response and readout noise from calibration frames and synthesizes training pairs from high-SNR stacked proxies, enabling a standard U-Net to reach average injected-source FLUX-SNR values of 39.91 and 39.87 dB on real MuSCAT-3 and cross-instrument MuSCAT-4 observations, respectively.
QualiTeacher: Quality-Conditioned Pseudo-Labeling for Real-World Image Restoration: QualiTeacher turns teacher pseudo-label quality into a control condition for the student, then constrains quality ordering and cropped consistency so that imperfect supervision supports controllable real-world restoration, reducing RTTS dehazing FADE from 0.824 for CORUN with Colabator to 0.590.
Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion: Stream-DiffVSR combines four-step diffusion denoising, flow-aligned historical output guidance, and temporal decoding to restore video frame by frame without future frames, achieving REDS4 LPIPS of 0.099 and 0.328-second time-to-first-frame at 720p on an RTX 4090, although it does not yet provide conventional real-time HD video processing.
SparkVSR: Interactive Video Super-Resolution via Sparse Keyframe Propagation: SparkVSR lets users restore a few keyframes and propagate their visual priors under the motion constraints of the low-resolution video, reaching MUSIQ 68.88 and DOVER 0.6212 on MovieLQ with PiSA-SR references while exposing a guidance control for choosing between greater fidelity and richer generated detail.
Spectral and Trajectory Regularization for Diffusion Transformer Super-Resolution: StrSR proposes a joint framework of asymmetric adversarial distillation and frequency-domain distribution matching. By replacing the DiT discriminator with a CLIP-ConvNeXt discriminator, it avoids model collapse, and uses a Frequency Distribution Loss (FDL) to suppress grid-like periodic artifacts inherent in DiT. It achieves high-quality single-step real-world image super-resolution on DiT architectures for the first time.
λSplit: Self-Supervised Content-Aware Spectral Unmixing for Fluorescence Microscopy: λSplit learns cellular structural priors with a hierarchical VAE and remixes predicted fluorophore concentration maps through known spectra for training without unmixed ground truth; on simulated CellAtlas data with 5 ms exposure and 32 bands, it achieves 34.03 dB PSNR versus 28.91 dB for RLU, the strongest comparator by PSNR in that setting.
Calibrated Harmonic Overlaid Implicit Neural Representations for Multi-Dimensional Data: CHOIR replaces the layer-wise function composition of deep periodic INRs with β-gated additive harmonic superposition, then uses the 1/f power-law prior of natural images to calibrate both the frequency distribution and the output amplitude of every neuron, enabling stable periodic INR training at 12–15 layers and surpassing SIREN, FINER, FreSh, LRTFR, and CRNL on 2D image fitting, NeRF novel view synthesis, and hyperspectral / multispectral / video missing-data completion and mixed-degradation restoration.
RL-AWB: Deep Reinforcement Learning for Auto White Balance Correction in Low-Light Night-time Scenes: RL-AWB delegates illuminant estimation to the nighttime gray-pixel estimator SGP-LRD and uses a SAC policy trained on few images to adjust the gray-pixel percentage and aggregation exponent per image, achieving a median angular error of 1.99° when trained on LEVI and tested on NCC.
Experts-Guided Unbalanced Optimal Transport for ISP Learning from Unpaired and/or Paired Data: EGUOT leaves the deployed ISP network unchanged and combines a content anchor, unbalanced optimal transport, and color/structure/frequency experts during training, reaching 21.69 dB PSNR with a fixed Restormer in unpaired ZRR experiments while also supporting paired supervision.

Browse all 73 Image Restoration papers →

🛰️ Remote Sensing (39)¶

OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents: OpenEarthAgent connects visual perception, GIS, spectral indices, and georeferenced raster operations to a unified executor, then fine-tunes Qwen3-4B on replay-validated trajectories, improving strict tool-order matching from 14.71% to 67.24% on its test set while non-generation answer accuracy remains only 45.26%.
MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Training: MMEarth-Bench evaluates geographic generalization with five global environmental tasks and twelve aligned modalities, then adapts encoders by reconstructing these modalities at test time; relative to joint training, TTT-MMR improves aggregate results across tasks and both test splits, with geographic batching particularly helpful for long-tail samples.
Moonstone: A Multimodal Foundation Model and Benchmark for Lunar Remote Sensing: Moonstone aligns 28 channels from seven instrument families aboard five lunar orbiters into a single global map at 128 pixels per degree (~237 m/pixel), pretrains the self-supervised masked autoencoder MG-MAE with modality-grouped tokenizers, a shared ViT, coverage-adaptive masking and missing-modality attention masking, and reports the first multimodal lunar downstream benchmark (six classification/regression/segmentation tasks), beating scratch, ImageNet, Earth-observation foundation models and vanilla MAE by large margins — +16.2% geology accuracy and +14.7 mIoU crater segmentation.
TerraDiT-Ω: Unified Spatial Control for Satellite Image Synthesis with Any Geospatial Primitive: TerraDiT-Ω lets a diffusion transformer consume native geospatial primitives (polygons, polylines, bounding boxes, points) directly as spatial control: a Unified Primitive Encoder maps all four formats into one shared instance token, and Geometry-Aware Local Attention multiplies explicit geometric priors — a rotated anisotropic Gaussian plus a dense spatial geometry field — into the attention logits, beating both dense-raster and sparse-prompt baselines across all conditioning formats and serving as a single model for synthetic data augmentation on four remote sensing tasks.
Beyond 2D Matching: A Unified Single-Stage Framework for Geometry-Aware Cross-View Object Geo-Localization: GAGeo adapts the 3D foundation model π3 using CMA-Loc building correspondences and camera annotations, jointly decodes prompted objects and cameras in one forward pass, and uses satellite-anchored contrastive learning to reach 43.86 mAcc and 57.88 mIoU on the untrained ground-to-drone configuration.
CROSS: Cascaded Distillation and Dual-Constraint Grounding for Remote Sensing Referring Segmentation: CROSS distills text-filtered patch-to-patch geometric affinities from the SAM 2 encoder into the shallow, deep, and final layers of SigLIP 2 as a soft structural regularizer, then adds dual-constrained contrastive learning (perceptive hard negatives plus spatial counterfactual texts) to break the object-centric shortcut, reaching 83.25 cIoU and 41.32 [email protected] on RefSegRS.
G-ZAP: A Generalizable Zero-Shot Framework for Arbitrary-Scale Pansharpening: G-ZAP uses a scale-conditioned implicit neural representation (INR) as its pansharpening fusion backbone, trained by a three-level collaborative scheme — an unsupervised spectral consistency loss at full resolution plus two levels of constructed supervision at reduced resolutions — so that a model trained on nothing but the test image pair itself produces HRMS output at any magnification, and the learned weights transfer directly to unseen image pairs, scenes, and sensors.
GeoSolver: Scaling Test-Time Reasoning in Remote Sensing with Fine-Grained Process Supervision: GeoSolver trains the token-level GeoPRM through entropy-guided tree search and visual hallucination injection, then integrates it into Process-Aware Tree-GRPO and test-time search to constrain intermediate visual evidence as well as final answers; in the paper's Table 3, visual grounding improves from 58.19 to 68.04 with a generation budget of 32.
PKINet-v2: Towards Powerful and Efficient Poly-Kernel Remote Sensing Object Detection: PKINet-v2 combines heterogeneous kernels to address object geometry and scale in remote sensing, then equivalently merges its training-time aggregation branches for deployment, achieving 80.46% mAP and 54.60 FPS on single-scale DOTA-v1.0 versus 78.39% and 14.05 FPS for PKINet-v1-S.
Recurrent Cross-View Object Geo-Localization: ReCOT repeatedly reads satellite features with learnable tokens carrying the queried object's semantics, supported by SAM distillation and reference feature enhancement, achieving 48.10% Ground-to-Satellite and 72.05% Drone-to-Satellite [email protected] on the CVOGL test set with 29.9M parameters.

Browse all 39 Remote Sensing papers →

🔍 Anomaly Detection (8)¶

Pistachio: Towards Synthetic, Balanced, and Long-Form Video Anomaly Benchmarks: Pistachio uses scene-constrained storylines to drive continuous video generation for a synthetic detection-and-understanding benchmark, exposing difficulties with complex normal behavior and long-term understanding while showing that CLIP-TSA trained with synthetic data and 10% of UCF-Crime training data outperforms training on the complete UCF-Crime training set alone.
Anomaly Factory 3D: A Modular Framework for Diverse Pseudo-Anomaly Synthesis in Unsupervised 3D Anomaly Detection: AF3AD is a standalone geometric pseudo-anomaly generator that combines local coordinates, spatial kernels, directional gating, and stochastic presets; with a modified offset-prediction detector, it reaches object-level AUROC of 91.5% on AnomalyShapeNet and 85.2% on Real3D-AD.
DeltaDeno: Zero-Shot Anomaly Generation via Delta-Denoising Attribution: DeltaDeno drives two synchronized diffusion trajectories with normal and anomaly prompts, accumulates post-denoising latent differences into a defect localization map, and uses that map to constrain local generation, increasing downstream MVTec AD image-level AUROC from AnomalyAny's 77.6 to 84.7 without generator fine-tuning or real anomalous samples.
IMMoE: Incomplete Multi-View Anomaly Detection via Mixture of View Experts Fusion: IMMoE uses local feature dropout to reduce overfitting to masking patterns and view-expert fusion to guide normal-feature reconstruction, achieving 84.7 I-AUROC / 33.1 P-AP on RIMAD with 50% of the product area masked, without fully establishing the recoverability of hidden defects under real occlusion.
ArcAD: Anomaly-Rectified Calibration for Cold-Start Supervised Anomaly Detection: ArcAD calibrates the latent space of a reconstruction-based industrial anomaly detector using balanced normal prototypes, prototype-filtered synthetic anomalies, and a few real defects, raising Dinomaly's image-level AUROC from 88.8% to 92.5% on multi-class cold-start Real-IAD while retaining its original reconstruction-based inference score.
BAAF: Universal Transformation of One-Class Classifiers for Unsupervised Image Anomaly Detection: BAAF cleans contaminated training images through disjoint bags, out-of-bag anomaly scoring, and majority voting before retraining one unchanged one-class detector, raising average image AUROC across eight detectors from 0.762 to 0.933 on MVTec AD with 10% training contamination without adding test-time models.
A Comprehensive Analysis about Unsupervised Outlier Detection for Images: VUOD obtains reliable inlier/outlier pseudo-labels through fine-grained clustering and distance-ranking agreement, learns a lightweight feature mapping, and applies OCSVM to the improved representation, raising aggregated CIFAR-10 AUROC from FlexUOD's 0.942 to 0.972 while extending the same framework to industrial and medical images.
Beyond Common Sense: Grounding Logical Anomaly Detection in Inspection Criteria: SCAN turns industrial logical anomaly detection into explicit rule-by-rule visual inspection, combining FLAW training data, structured supervised fine-tuning, and rule-level reinforcement learning to raise an 8B model's strict scoring accuracy on MVTec-LOCO from 56.73% to 64.58%, while leaving substantial missed-detection concerns.

🧑 Human Understanding (119)¶

FlowerDance: MeanFlow for Efficient and Refined 3D Dance Generation: FlowerDance combines MeanFlow (a flow matching variant that replaces instantaneous velocity prediction with interval-average velocity) with Physical Consistency Constraints (PCC), a BiMamba backbone, and channel-level cross-modal fusion. It generates high-quality 3D dance motions in only 5–20 sampling steps. While achieving state-of-the-art (SOTA) quality on FineDance and AIST++, its inference speed (2008 FPS) far exceeds the previous best method (MatchDance at 345 FPS), leaving ample computational budget for real-time 3D rendering.
ReMoMask: Retrieval-Augmented Masked Motion Generation: ReMoMask gives motion retrieval, training masks, and generative evidence fusion a shared body structure through HBM, TSM, and SSTA, achieving FID 0.026 and R-Precision Top-1 0.566 in the main HumanML3D experiment.
Human Mesh Modeling for Anny Body: Anny converts MakeHuman's artist-designed deformation assets into a differentiable body model without learning its shape space from scans, calibrates sampling with WHO statistics, and generates 780k Anny-One images, retaining millimeter-level adult scan fitting while improving child human mesh recovery.
Forge4D: Feed-Forward 4D Human Reconstruction and Interpolation from Uncalibrated Sparse-View Videos: Forge4D decomposes reconstruction of dynamic humans and interacting objects into scale-consistent streaming 3D Gaussian prediction and bidirectional dense motion matching, then produces intermediate times through occlusion-aware fusion; it achieves 29.0378 dB PSNR on joint novel-time and novel-view evaluation on DNA-Rendering without requiring input camera calibration parameters.
VersatileMotion: A Unified Framework for Motion Synthesis and Comprehension: VersatileMotion uses the world-space causal body tokenizer VerMoVQ to formulate text, audio, and multi-person motion as unified next-token prediction, covering nine motion tasks in one model with competitive generation and understanding under the authors' evaluation protocol.
Fast Sam 3D Body: Accelerating SAM 3D Body for Real-Time Full-Body Human Mesh Recovery: Fast SAM 3D Body keeps SAM 3D Body weights frozen while restructuring body-hand dependencies and simplifying decoding, then accelerates MHR-to-SMPL conversion with a separately trained MLP, bringing reconstruction closer to interactive rates with gains that must be interpreted alongside datasets, hardware, and accuracy changes.
ECHO: Ego-centric Modeling of Human-Object Interactions: ECHO treats human motion, object trajectories, and contacts as mutually constrained diffusion variables to recover full interactions from head and wrist tracking with known object geometry, combining mixed-data training and smooth inpainting to achieve human MPJPE of 6.8±0.1 cm and object vertex error of 33.5±0.5 cm on BEHAVE.
ActionPlan: Future-Aware Streaming Motion Synthesis via Frame-Level Action Planning: ActionPlan generates temporally aligned semantic action plans before progressively synthesizing 3D human motion, combining offline generation and streaming output in one model and reducing streaming FID from MotionStreamer's 11.790 to 5.735 on the HumanML3D-272 test set.
UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation: UniMotion treats human motion as a continuous modality with equal status to RGB within a shared LLM. By replacing discrete tokens with a Continuous Motion Aligned VAE (CMA-VAE) and a dual-path embedder, and incorporating Dual Posterior KL Alignment (DPA) to inject visual semantic priors along with Latent Reconstruction Alignment (LRA) to resolve the motion path cold-start issue, UniMotion unifies and achieves comprehensive state-of-the-art (SOTA) performance across seven understanding, generation, and editing tasks on Motion-Text-RGB tri-modalities.
CLIP-AUTT: Test-Time Personalization with Action Unit Prompting for Fine-Grained Video Emotion Recognition: CLIP-AUTT first learns expression classification from similarities between Action Unit text and videos, then selects the lowest-prediction-entropy contiguous segment in an unseen subject's video and updates only the AU prompt embeddings, improving BioVid WAR from 78.0 for the non-personalized CLIP-AU to 81.5.

Browse all 119 Human Understanding papers →

📹 Video Understanding (110)¶

EventMemAgent: Hierarchical Event-Centric Memory for Online Video Understanding with Adaptive Tool Use: EventMemAgent compresses video history into hierarchical event-centric memory and uses reinforcement learning to teach Qwen3-VL-8B when to retrieve, perform OCR, or detect objects, achieving an overall score of 60.75% on OVO-Bench with at most 32 frames in short-term memory.
History-Aware Transformation of ReID Features for Multiple Object Tracking: HATReID-MOT treats established tracks as online pseudo-classes, constructs a video-specific ReID subspace using temporally weighted Fisher linear discriminant analysis, and fuses original similarities, raising the default appearance baseline's SportsMOT test HOTA from 67.3 to 78.1 without additional network training.
MegaFlow: Zero-Shot Large Displacement Optical Flow: MegaFlow builds cross-frame features from pretrained visual geometry priors, locates large motions through global matching, and refines them with local and temporal evidence, achieving 1.83 EPE in four-frame zero-shot Sintel Final evaluation while supporting transfer to point tracking without tracking supervision.
TAR: Temporal Anchor-Constrained Reasoning for Video Temporal Grounding: TAR inserts intermediate anchors verifiable against ground-truth intervals into video reasoning, constrains reinforcement learning with progressive refinement and an anchor-count penalty, and bootstraps cold-start data with a same-scale model, reaching 61.1 mIoU on Charades-STA with a 7B model.
Bounding-Box Trajectories Matter for Video Anomaly Detection: TrajVAD promotes bounding-box trajectories already produced by detection and tracking to the primary anomaly signal, learning normal motion with a class-aware normalizing flow; its trajectory-only variant reaches 87.7 AP on ShanghaiTech, while reliability-gated pose fusion reaches 88.6 AUROC and 90.9 AP, although pose does not help on every dataset.
CurveStream: Boosting Streaming Video Understanding in MLLMs via Curvature-Aware Hierarchical Visual Memory Management: CurveStream detects semantic changes through the turning behavior of visual feature trajectories and uses online dual thresholds to retain frames at high resolution, retain them at low resolution, or discard them, improving Qwen2.5-VL-7B from 73.31% / 59.90% to 84.00% / 73.48% on StreamingBench / OVOBench without training.
EgoExo-Con: Exploring View-Invariant Video Temporal Understanding: This paper introduces the EgoExo-Con benchmark (1,148 pairs of synchronized ego-exo videos + 2,269 human-refined temporal queries) to systematically evaluate the cross-view consistency of Video-LLMs in temporal understanding for the first time, revealing that existing models' cross-view consistency barely reaches half of their single-view performance. Furthermore, it proposes the View-GRPO reinforcement learning framework, which leverages dual reasoning rewards of semantic alignment and structural consistency to significantly enhance the robustness and consistency of cross-view temporal understanding.
Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding: Em-Garde reformulates the complex "whether to answer" reasoning performed at every frame into a one-time visual proposal generation during query time and lightweight in-stream matching. It achieves an average F1 of 30.99 on OVO-Bench, 38.0% proactive response accuracy on StreamingBench, and maintains constant-length streaming computation at approximately 13 fps.
OmniScript: Towards Audio-Visual Script Generation for Long-Form Cinematic Video: OmniScript transcribes cinematic videos into temporally grounded hierarchical scripts, combining character-memory-assisted data construction, deep audio-visual fusion, plot reasoning, and event-level segmented rewards to achieve 37.7 event Overall F1 and 69.3 [email protected] with an 8B model on 5-minute videos, without uniformly surpassing proprietary models.
Small Vision-Language Models are Smart Compressors for Long Video Understanding: Tempo uses a 2B small vision-language model to compress video segments into query-conditioned memories for a 4B global language model, allocating tokens by relevance at inference time; with at most 1024 frames and a 4K visual budget, it scores 52.7 on LVBench versus 48.2 for VideoChat-Flash.

Browse all 110 Video Understanding papers →

🚗 Autonomous Driving (159)¶

Masked Depth Modeling for Spatial Perception: MDM turns missing sensor depth into physically motivated masks and learns geometric representations through joint attention over complete RGB and remaining depth, reducing NYUv2 Extreme RMSE from PromptDA's 0.324 to 0.167 while improving real-world robotic grasping.
MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learning: MindDrive uses a shared language model with separate LoRA adapters for decision and action experts, restricting online exploration to discrete driving intentions and optimizing decisions through rewards from executed trajectories; its 3B version reaches 80.59 DS and 58.26% SR on Bench2Drive, although validation remains limited to CARLA simulation.
UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving: UniDrive-WM jointly achieves scene understanding, trajectory planning, and trajectory-conditioned future image generation within a single VLM framework. Through a differentiable bidirectional coupling of planning and generation, it improves the Driving Score from 77.74 to 79.31 in Bench2Drive closed-loop evaluation, reduces the L2 trajectory error by 7.3%, and decreases the collision rate by 10.4%. Furthermore, it systematically analyzes the speed-fidelity trade-off between two complementary image generation branches (discrete autoregressive and continuous diffusion).
DriveVA: Video Action Models are Zero-Shot Drivers: DriveVA proposes a unified video-action world model that jointly decodes future video prediction and trajectory planning within a single DiT diffusion generation process. Leveraging the spatio-temporal priors of a large-scale video generation model (Wan2.2-TI2V-5B), it achieves strong zero-shot cross-domain generalization, obtaining 90.9 PDMS on NAVSIM, and reducing the average L2 error and collision rate by 78.9%/52.5% and 83.3%/52.4% on nuScenes and Bench2Drive, respectively.
SparseDriveV2: Scoring is All You Need for End-to-End Autonomous Driving: SparseDriveV2 replaces dynamic trajectory generation with a super-dense static space composed of paths and velocity profiles, selecting actions through factor-level filtering and trajectory-level scoring to reach 92.0 PDMS on NAVSIM v1 and 90.1 EPDMS under the corrected NAVSIM v2 protocol with ResNet-34.
Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout: Modeling "how the in-cabin driver will move in the future" as a rollout problem in a latent space, causally driven by out-of-cabin traffic events: utilizing frozen Qwen3-VL features as the perception interface, a dual-stream architecture separately encodes traffic and the driver. A "gated causal injection" mechanism allows the external environment to unidirectionally modulate the autoregressive evolution of the driver's internal state. This is ultimately decoded into the human skeleton trajectory for the next 5 steps (augmented by behavioral/emotional semantics as auxiliary regularization), achieving robust long-term prediction on safety-critical, high-motion sequences in the AIDE dataset.
UniDriveDreamer: A Single-Stage Multimodal World Model for Autonomous Driving: UniDriveDreamer calibrates video and LiDAR latents before feeding them into one generative Transformer, jointly synthesizing multisensor sequences through bidirectional interaction and achieving FID 2.81 and FVD 11.44 on nuScenes, with augmentation gains in a controlled detection experiment.
ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving: A unified understanding-generation framework, ExploreVLA, is proposed. It introduces future RGB + depth image generation as a dense world modeling objective within the VLA driving model. The image prediction uncertainty of the world model is leveraged as an intrinsic exploration reward. Under a safety-gated GRPO, the policy is enabled to surpass the imitation learning distribution and discover diverse yet safe driving behaviors, achieving state-of-the-art results of \(93.7\) PDMS and \(88.8\) EPDMS on NAVSIM.
Beyond Imitation: Learning Safe End-to-End Autonomous Driving from Hard Negatives: BeyondDrive trains a planner to approach expert trajectories while avoiding generated trajectories that are nearby but unsafe, improving optimized LTF from 88.7 to 89.7 PDMS on NAVSIM v1 and from 89.1 to 90.1 EPDMS on NAVSIM v2, with additional evidence from other planners and zero-shot HUGSIM evaluation.
CausalDrive: Real-time Causal World Models for Autonomous Driving: CausalDrive generates reactive driving videos from only an initial front-view image, an ego trajectory, and a global interaction prompt; a causal autoregressive teacher and context-forced DMD reduce inference to 1-4 steps, reaching 12.4 FPS on one A100 and an 82.0% yielding rate under the Polite condition on SocioDrive-Bench, without eliminating all false collisions.

Browse all 159 Autonomous Driving papers →

🤖 Robotics & Embodied AI (129)¶

VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model: VLA-JEPA generates latent actions from a vision-language model that sees only initial images and instructions, learns manipulation semantics by predicting future V-JEPA2 states, and generates robot actions through flow matching; Tables 1 and 3 report success rates of 97.2% on LIBERO and 78.1% on LIBERO-Plus.
VLA-R1: Enhancing Reasoning in Vision-Language-Action Models: Built on Qwen2.5-VL-3B, VLA-R1 first learns geometrically grounded embodied reasoning from VLA-CoT-13K and then uses GRPO to optimize region, trajectory, and format rewards, reaching 36.51% IoU and 91.74 average trajectory error in-domain, although its real-robot results require caution about small evaluation samples and inconsistent reported numbers.
RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Tasks: RoboClaw uses the same vision-language model (VLM) agent for robot data collection and execution, pairing forward actions with inverse reset behaviors and supervising multiple vision-language-action (VLA) policies online; real-robot experiments show reduced human collection time and improvements across four manipulation policies as closed-loop data accumulate.
Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain: RoboBench evaluates 18 models using 6092 questions across five embodied cognitive dimensions and a DAG-guided MLLM world-simulator judge, showing that strong general question answering and verbal planning do not guarantee implicit-intent understanding, spatial grounding, or real execution failure diagnosis.
EchoVLA: Robotic Vision-Language-Action Model with Synergistic Declarative Memory for Mobile Manipulation: EchoVLA separately maintains voxel scene memory for what exists where and episodic memory for recent interactions, retrieves both at different granularities to condition base and arm diffusion policies, and achieves 0.31 mean success on RoboCasa mobile manipulation versus 0.20 for π0.5 in the same table.
World Models for Learning Dexterous Hand-Object Interactions from Human Videos: DexWM conditions a latent world model on hand-keypoint differences and camera motion, using hand-consistency supervision to preserve local detail; after pretraining on human and parallel-jaw robot videos and fine-tuning on about 4 hours of target-robot simulation exploration, model-based planning achieves 10/12 successful real-robot grasps, where zero-shot means unseen test skills on the target embodiment and no real-robot fine-tuning—not the absence of all target-embodiment data.
RL from Physical Feedback: Aligning Large Motion Models with Humanoid Control: RLPF fine-tunes a large motion model with both simulated humanoid tracking success and text-motion semantic correspondence, while Hierarchical RLPF alternately improves the generator and controller to make motions executable without collapsing into instruction-independent standing.
Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignment: Without changing the deployed policy's parameters, CoVer-VLA uses a contrastive verifier trained on successful demonstrations to jointly select instruction rephrases and action chunks, raising average task progress on PolaRiS from 40.0% to 53.9% and success rate from 3.8% to 13.1%, showing that verification offers an effective alternative to further policy training, subject to candidate quality and additional compute costs.
MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robots: MobileVLA-R1 establishes a parseable reasoning-and-action interface through multi-granularity chain-of-thought supervision, then uses offline GRPO rewards for continuous movement direction, discrete behaviors, and output format, reaching 68.3% SR on R2R-CE Val-Unseen and demonstrating quadruped closed-loop execution through fixed onboard/remote hybrid deployment.
Towards Generalizable Robotic Manipulation in Dynamic Environments: To address the limitation that existing VLAs can only grasp static objects and fail when facing moving targets, this work constructs DOMINO, a dynamic manipulation benchmark containing 117k trajectories across 35 tasks. Furthermore, it proposes the PUMA architecture, which equips VLAs with "historical optical flow + object-level future feature prediction," achieving a 6.3% absolute success rate increase on dynamic tasks.

Browse all 129 Robotics & Embodied AI papers →

🎮 Reinforcement Learning (12)¶

ELVA: Exploring Ranking-Driven Universal Multimodal Retrieval: ELVA proposes a rule-based verifiable reward RL framework that co-optimizes the retrieval-ranking capability of MLLMs through ranking rewards and margin rewards. This addresses the "grain blindness" issue that arises when adapting contrastive learning to retrieval tasks. ELVA achieves SOTA performance on both M-BEIR and a self-built multi-grained benchmark MRBench, with a 13.1% improvement on MRBench.
Teaching an Agent to Sketch One Part at a Time: The paper automatically annotates vector sketches with semantic parts, then trains a vision-language agent with SFT and multi-turn process-reward GRPO to observe the canvas and add curves part by part; users prefer its final outputs over SketchAgent in 77.5% of comparisons.
GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Tracking: GoStop formulates event stream accumulation as a sequential decision-making process governed by a lightweight reinforcement learning agent that dynamically chooses between accumulating events (go) and triggering tracking inference (stop), significantly enhancing tracking accuracy and robustness while substantially lowering latency under abrupt dynamic motions.
FedDO: Dynamic Client Optimization for Adaptive Federated Learning: FedDO formulates client coordination in federated learning as a continuous control problem, leveraging a Distributional Soft Actor-Critic with Three Refinements (DSAC-T) agent to dynamically allocate per-client data-usage ratios and integrating low-rank parameterization to scale to thousands of clients while mitigating gradient drift.
From Illusion to Intention: Visual Rationale Learning for Reliable Evidence Acquisition: To eliminate the disconnect between visual actions and outcomes caused by outcome-only rewards in "thinking with images", this paper proposes Visual Rationale Learning (ViRL), an end-to-end reinforcement learning paradigm that combines evidence-grounded curation, signed-plus-refinement fidelity rewards, and fine-grained credit assignment for faithful and efficient evidence acquisition.
HERO: Enhancing Multimodal Faithfulness via Dynamic Entropy-Aware Reinforcement Learning: Proposes HERO, an on-policy reinforcement learning framework that identifies the "Confidence Trap" in CoT-tuned LVLMs—where models retreat into language priors under visual ambiguity to produce low-entropy confident errors—and overcomes it using dynamic entropy-aware weighting and variance-gated hard-negative sampling.
Molecular Identifier Visual Prompting and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsing: To resolve the semantic misalignment between visual chemical entities and pre-trained priors, alongside the optimization discrepancy between token-level training and reaction-level evaluation, this paper introduces Identifier as Visual Prompting (IdtVP) and Re3-DAPO verifiable reinforcement learning, substantially enhancing zero-shot parsing accuracy and out-of-distribution robustness across historical scanned literature.
Reinforcement Learning for Multimodal Diffusion Language Models via Bidimensional Trajectory and Thought Optimization: Bi-VRL presents the first bidimensional reinforcement learning framework tailored for multimodal diffusion language models, coupling temporal trajectory-level value estimation across denoising steps with spatial thought-level visual alignment rewards on critical culmination tokens, delivering substantial reasoning gains across both discrete and continuous visual architectures.
Table-MCR2TR: Merged-Cell-Aware Table Recognition via Reinforced Multimodal Language Models: Table-MCR2TR addresses the critical failure of table recognition models on complex merged cells by constructing the large-scale MCT-350K dataset and introducing a progressive multi-task paradigm coupled with GRPO-based collaborative reinforcement learning (M-CRL), enabling a 3B model to achieve state-of-the-art performance and outperform leading proprietary models such as Gemini-2.5-Pro across multiple benchmarks.
TiCRL: Textual Image Classification with Reinforcement Learning-Based Curriculum Learning: TiCRL introduces an OCR-free visual-textual hybrid difficulty measurer and a continuous-action DDPG scheduler that dynamically adjusts sample difficulty thresholds using real-time training feedback, achieving 83.67% accuracy on CNN with 82% data and 93.02% accuracy on DiT with only 58% data.

Browse all 12 Reinforcement Learning papers →

🔄 Self-Supervised Learning (21)¶

V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning: V-JEPA 2.1 extends video latent-prediction supervision to visible positions and multiple network depths, improving local spatiotemporal structure while recovering global semantic capabilities; its ViT-G reaches NYUv2 depth RMSE of 0.307 and SSv2 action recognition accuracy of 77.7% in frozen-feature evaluations.
The Prism Hypothesis: Harmonizing Semantic and Pixel Representations via Unified Autoencoding: UAE separates pretrained visual representations into a low-frequency base that preserves semantics and high-frequency residuals that can learn pixel detail, using selective distillation and masked reconstruction to combine both capabilities; Table 1 reports ImageNet-1K PSNR 32.17, SSIM 0.92, and rFID 0.35 for its DINOv2-B configuration.
DSeq-JEPA: Discriminative Sequential Joint-Embedding Predictive Architecture: DSeq-JEPA adds two inductive biases on top of I-JEPA's latent prediction objective — an attention-derived saliency map that selects and ranks regions by discriminativeness (where to predict), and a predictor that then predicts the embedding of the next region along that order (in what order to predict) — improving ImageNet linear probing by 1.1/0.8/1.3 points over I-JEPA (ViT-B/L/H) with consistent gains on fine-grained recognition, COCO/ADE20K, and CLEVR, at zero inference overhead.
GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure: GhostPoint observes that the learning signal in LiDAR self-supervised pretraining is almost exclusively defined on returns from visible surfaces and leaves occluded and no-return regions unconstrained; it therefore uses instance voxel dilation to generate neighborhood voxel sets around pseudo-instances that contain visible, masked, and no-return voxels, trains a lightweight predictor to hallucinate features at those positions from observed context, and supervises them with asymmetric teacher targets (occupied voxels match the teacher encoder, no-return voxels match the teacher predictor), unifying pseudo-occlusion and true occlusion under one self-distillation objective — reaching state-of-the-art downstream 3D detection on nuScenes and Waymo (67.5 mAP / 71.2 NDS on nuScenes after full fine-tuning).
Human-like Object Grouping in Self-supervised Vision Transformers: This paper scales the classical two-dot same/different object judgment paradigm into a human behavioral benchmark of 255 natural COCO scenes and 1,020 trials, reads out same/different decisions and trial-wise reaction times from a simple MLP over two patch features, finds that DINO-family self-supervised ViTs align best (up to 91.9% grouping accuracy), proposes an object-centricity AUC based on ROC analysis of patch affinity maps that correlates strongly with behavioral alignment across nine models (Spearman r=0.950), and finally raises supervised models' grouping accuracy by roughly 8–18 percentage points through Gram matrix distillation.
Implicit Neural Representation Facilitates Unified Universal Vision Encoding: The paper trains an image implicit neural representation (INR) hyper-network into a unified vision encoder, HUVR: a single ViT backbone emits standard tokens, compressed tokens (TinToks, up to 96x smaller), and per-patch INR modulation quantities, and is pretrained self-supervised with pixel reconstruction plus teacher distillation, making one representation usable simultaneously for classification, semantic segmentation, depth estimation, and image reconstruction.
Vision-TTT: Efficient and Expressive Visual Representation Learning with Test-Time Training: The paper brings Test-Time Training (TTT) from NLP into vision: it treats an image's token sequence as a dataset and updates a per-head hidden state online by gradient descent on a self-supervised reconstruction loss, then adds a dual-dataset strategy and Conv2d dataset preprocessing to turn the originally unidirectional TTT into a 2D-aware vision backbone — Vittt-T/S/B reach 77.7/81.8/82.7 Top-1 on ImageNet and, at 1280×1280, save 79.4% FLOPs, run 4.72× faster and use 88.9% less memory than DeiT-T.
Bottom-up modeling of repeated elements via single image analysis-by-synthesis: A shared two-dimensional prototype with adjustable appearance, single-image crop-based curriculum optimization, and full-image selection and refinement turn repeated-element discovery into interpretable reconstruction; on 116 curated FSC-147 images, the method achieves an image PSNR of 23.76 and a counting MAE of 14.11, without implying that the entire system is free of pretraining or scale priors.
DP-BOA: Dirichlet-Process Birth-or-Assign for On-the-Fly Category Discovery: DP-BOA reformulates per-sample online category discovery as a Bayesian comparison between existing-category assignment evidence and new-category birth evidence; support-set prior calibration, full-covariance prediction, and online statistical updates yield 53.4% overall and 51.6% novel-class clustering accuracy on CUB, without leading on every dataset or known-class metric.
ExPLoRe: Expert Patch-Level Loss Routing for Multi-Objective Masked Image Modeling: ExPLoRe repurposes the dispatch weights of Soft-MoE as patch-level, learnable loss coefficients in multi-objective masked image modeling. Through a loss-coupling mechanism, the router dynamically adjusts the target weights based on the semantic characteristics of different regions, achieving 80.6% linear probe and 85.3% finetuning accuracy on ImageNet-1K with a lower inference computation cost (11.93 GFLOPs vs. 17.45). It also proposes three downstream transfer strategies (Freeze Routing / Expert Dropout / Freeze Attention) to completely bridge the 2.5–2.9 mIoU gap on semantic segmentation.

Browse all 21 Self-Supervised Learning papers →

📐 Optimization & Theory (13)¶

LDC-MTL: Balancing Multi-Task Learning through Scalable Loss Discrepancy Control: LDC-MTL explicitly controls task-loss discrepancies through bilevel optimization and removes an empirically small gradient term to enable single-loop first-order training, achieving an aggregate relative performance-drop score of -1.31 on CelebA at 1.07 times the training time of linear scalarization.
HSFM: Hard-Set-Guided Feature-Space Meta-Learning for Robust Classification under Spurious Correlations: HSFM freezes an ERM-pretrained backbone and treats the feature-space embeddings of a small support set as learnable parameters, optimizing them through a bilevel meta-objective driven by the model's own loss on hard examples (inner loop adapts the linear head, outer loop minimizes hard-set loss); in a few minutes on a single GPU it reaches worst-group accuracy on par with methods that use group labels on Waterbirds and Dominoes, and the resulting feature edits can be decoded back into images via unCLIP to expose the model's spurious dependencies.
Why Can Accurate Models Be Learned from Inaccurate Annotations?: The paper turns "why can accurate models be learned from inaccurate annotations?" into an analyzable question: label inaccuracy only deposits noise in the lower singular components of the classification weight matrix, and at moderate inaccuracy rates the principal subspace barely rotates, so task-relevant knowledge survives. This is argued empirically through singular-value spectra and Grassmann subspace similarity and theoretically through a Davis-Kahan angle bound, and it motivates LIP — a training-free post-processing plug-in that keeps the top-\(k\) singular components and re-estimates the flattened tail singular values from the training set — which improves 11 baselines on 6 real-world partial-label datasets and on CIFAR/CUB noisy settings.
Coordinate Singularities Break Conformal Coverage for Gaze and Head Pose: Reveals that using flat chart coordinates (yaw-pitch or Euler angles) for conformal prediction nonconformity scores causes catastrophic 30–50 pp slice-conditional coverage collapse near singularities on \(\mathcal{S}^2\) and \(\mathrm{SO}(3)\) despite nominal 90% marginal coverage, proves that scalar adaptive calibration cannot resolve this shape distortion, and introduces a retraining-free, zero-overhead geodesic scoring remedy.
Dive into the implicit biases of low-rank vision-language alignment: Challenging the prevailing dogma that vision-language alignment demands full-parameter fine-tuning, this paper demonstrates that low-rank alignment significantly slashes training overhead while outperforming full-parameter baselines, and theoretically as well as geometrically proves that low-rank adaptation preserves per-token linear separability (averting the LS-curse) and biases optimization toward flat, perturbation-robust parameter subspaces.
RBE-Flow: Recurrent Bayesian Estimation on Feature Manifolds for Cross-Modal Registration: Reformulates cross-modal dense optical flow estimation as a closed-loop recurrent Bayesian state estimation problem on feature manifolds. Relying on Recurrent Manifold Optimization (RMO) to generate flow observations with uncertainty, and Uncertainty-Adaptive Probabilistic Update (UAPU) to perform MMSE-optimal Bayesian fusion via Sigma-point projections while feeding back the posterior covariance to regulate the next optimization step. It consistently achieves SOTA on three cross-modal datasets (OSdataset, WHU-OPT-SAR, and RoadScene), reducing the AEPE by up to 45.4% compared to previous best methods.
SUM: Unified Geometric Surgery on Spatio-Temporal Adaptation Vectors for Federated Class Incremental Learning: Reinterpreting Federated Class Incremental Learning (FCIL) as a unified multi-task learning problem, SUM performs purely server-side geometric surgery on spatio-temporal adaptation vectors during aggregation—eliminating cross-client and cross-task directional interference without any client-side computational or communication overhead.
AnaPFL: When Closed-Form Solutions Meet Generalization and Personalization in Personalized Federated Learning: AnaPFL analytically aggregates a global primary stream and then fits a local residual refinement stream on frozen visual features, requiring one aggregation round and improving accuracy over the strongest baseline in each of 18 benchmark settings by 1.57–16.71 percentage points.
Can Vision Models Truly Forget? Mirage: Representation-Level Certification of Visual Unlearning: Addressing the critical vulnerability that existing visual unlearning evaluations rely solely on output-level metrics and mask persistent internal traces, this paper proposes Mirage, a representation-level certification framework combining linear probe recovery, centered kernel alignment, feature separability, and layer-wise tracking, revealing the widespread "forgetting illusion" and an unlearning trilemma across utility, output forgetting, and representational erasure.
H-SFP: Hierarchical Federated Learning with Decoupled Split-Model Prototyping: Addressing the bandwidth bottlenecks and synchronous locks of transmitting high-dimensional activations and backward gradients in split federated learning, H-SFP decouples cross-tier backpropagation by exchanging compact class-wise first- and second-order feature statistics, synthesizing Gaussian feature distributions at upper tiers to cut communication overhead by over two orders of magnitude while substantially curbing client drift.

Browse all 13 Optimization & Theory papers →

📐 Learning Theory (1)¶

Pay Attention to Attention Distribution: A New Local Lipschitz Bound for Transformers: The paper incorporates attention probability distributions into a local bound on the self-attention Jacobian, uses tighter softmax spectral estimates to construct JaSMin regularization, and improves ViT-B accuracy under some adversarial attacks while exposing trade-offs among robustness, clean accuracy, and gradient propagation.

🔗 Causal Inference (3)¶

CausalVAE as a Plug-in for World Models: Towards Reliable Counterfactual Dynamics: The paper detaches the CausalVAE causal layer into a latent-space plug-in that sits between any encoder and any action-conditioned transition backbone and organizes latents through a learnable DAG adjacency matrix; without touching the base prediction architecture it sharply improves intervention-aware counterfactual retrieval (CF-H@1 on Physics 3-body rises from 11.0 to 41.0 for GNN_NLL) while largely preserving factual prediction.
SPARC: Scalable Path-Specific Counterfactual Fairness via Causal Conditional Independence: SPARC turns path-specific counterfactual fairness for high-dimensional inputs into a label-conditioned sensitive-attribute independence objective, using mutual-information pretraining to retain diagnostic information and conditional adversarial masks to suppress unfair pathways, improving fairness on three medical datasets while retaining a predictive-performance cost relative to unconstrained models.
FD\(^2\): A Dedicated Framework for Fine-Grained Dataset Distillation: Addressing the issue that existing decoupled dataset distillation methods ignore discriminative local regions on fine-grained datasets, FD\(^2\) introduces Counterfactual Attention Learning (CAL) to extract discriminative attention maps and class prototypes. Integrating fine-grained feature constraints and intra-class sample similarity constraints during the distillation phase, it serves as a plug-and-play module that significantly improves the distillation quality of SRe2L++ and FADRM+ on fine-grained datasets such as CUB-200-2011, FGVC-Aircraft, and Stanford Cars, achieving a maximum improvement of +15.1% at IPC=1.

🔬 Interpretability (30)¶

CFM: Language-aligned Concept Foundation Model for Vision: CFM decomposes language-aligned visual features into sparse concepts with spatial locations, hierarchical relations, and textual names, making downstream predictions explainable and steerable while largely preserving classification, segmentation, and captioning performance.
On the Reliability of Cue Conflict and Beyond: This paper systematically questions the reliability of the widely used cue-conflict benchmark on both the stimulus-construction and the measurement side, identifying four problems — stylization entangles the cues, cue informativeness is imbalanced, ratio-based bias hides absolute cue sensitivity, and a restricted label space distorts predictions — and offers REFINED-BIAS as a replacement, built from human-perceptual cue definitions, controlled purified cue generation, and a full-label-space MRR sensitivity metric; on 32 training-strategy models, multiple architecture families, OpenCLIP/DINOv2 large models, and multi-level quantized models it resolves contradictory conclusions that the original benchmark could not explain.
Seeing Through Circuits: Faithful Mechanistic Interpretability for Vision Transformers: The paper formalizes a ViT as a directed graph over residual-stream connections, builds clean/corrupted input pairs via semantic segmentation plus inpainting, and prunes edges sequentially with target logit difference as the criterion (Vi-CD) to automatically recover class-specific circuits in ViT-B and OpenCLIP that are faithful yet roughly 10x sparser than the strongest baseline, and it shows these circuits act as causal switches that defend CLIP against typographic attacks and lower the danger retrieval score on RoCOCO.
Caption Bottleneck Models: CaBM converts images into taxonomy-censored captions, recognizes classes with an independent text-only classifier, and discovers concepts from its textual evidence, removing the visual-feature bypass while achieving competitive but not uniformly leading classification results.
DEX-AR: A Dynamic Explainability Method for Autoregressive Vision-Language Models: DEX-AR treats the gradient of each generation step's attention maps with respect to intermediate logits as the attribution signal, then dynamically filters attention heads and generated tokens by their "visual minus textual" maximum-gradient difference, producing both per-token and sequence-level image heatmaps for autoregressive VLMs that consistently beat Grad-CAM, attention-based, and perturbation-based baselines on ImageNet/VQAv2 perturbation and PascalVOC localization.
From Drop-off to Recovery: A Mechanistic Analysis of Segmentation in MLLMs: Without adding any new module to the MLLM, this paper dissects its segmentation competence through three interventional experiments: layerwise linear probing reveals a representation drop-off at the adapter when visual features are projected into the language space, followed by progressive recovery of mIoU across LLM layers; attention knockout shows that this recovery is not a passive by-product of residual connections but is actively mediated by cross-token attention, in which correctly classified tokens act as semantic anchors pulling misclassified neighbours toward the correct label; finally, causal attention is shown to starve the earliest image tokens of global context, a cost that bidirectional attention among image tokens eliminates.
This Looks Distinctly Like That: Grounding Interpretable Recognition in Stiefel Geometry against Neural Collapse: AMP replaces class prototypes with Stiefel-constrained orthonormal bases and uses sparse capacity and spatial constraints to select and localize distinct evidence, achieving 88.4% accuracy with ResNet50 on CUB, although its strict guarantee concerns basis rank rather than necessarily distinct semantic parts.
Towards Robustness against Typographic Attack with Training-free Concept Localization: This paper localizes typographic-attack vulnerability to a small set of "text-reading" attention heads in CLIP ViTs: it samples concept directions in the multi-head subspace, scores each head with a gradient-based text-focus metric (nTAS) to mine those heads, and then applies a training-free test-time intervention (attention reweighting or zero ablation) to them — raising object classification accuracy by more than 6 points on average across five CLIP backbones and four typographic-attack benchmarks while cutting the text confusion rate by 10–36 points.
World Knowledge in the Weights: Reading Concept Circuits of Vision Transformers: The work ports Cross-Layer Transcoders (CLTs) from language models to ViTs, reading a global concept circuit — the model's "world knowledge" — directly from input-invariant encoder/decoder weights and instance concept circuits from sparse activations; co-activation-reweighted global weights are validated by mean ablation to carry genuine functional importance, enabling fully automatic spurious-correlation discovery (graffiti for Freight Car, bird feeder for Hummingbird) and a +23% worst-group intervention gain on Waterbird, while CLT circuits beat SAE baselines on faithfulness and completeness across the board.
Blind to Position, Biased in Language: Probing Mid-Layer Representational Bias in Vision-Language Encoders for Zero-Shot Language-Grounded Spatial Understanding: B2G probes frozen vision-language encoders and reuses spatially sensitive visual features and more language-consistent text features for region ranking, increasing average multilingual referring segmentation mIoU from 42.42 to 49.62 across nine benchmarks, with additional multilingual inference cost.

Browse all 30 Interpretability papers →

📦 Model Compression (89)¶

GenRecal: Generation after Recalibration from Large to Small Vision-Language Models: GenRecal uses a training-time Recalibrator to connect a small vision-language model's hidden features to the teacher's language head, bypassing direct cross-tokenizer correspondence and improving InternVL2.5-8B from 56.0 to 68.1 on MMMU, while removing all auxiliary modules for deployment and retaining the student's original architecture.
Towards Consistent and Efficient Dataset Distillation via Diffusion-Driven Selection: This work turns a pre-trained diffusion model from an image generator into a real-image patch selector, combining class-conditioned localization with DIFT clustering to build reusable compressed training sets and achieving \(61.0\pm0.3\%\) classification accuracy on ImageNet-1K at IPC=50 in the main experiment.
Enhancing Pretrained Model-based Continual Representation Learning via Guided Random Projection: SCL-MGSM uses first-session labels and residuals to select blocks of random projection bases, then freezes the representation and recursively updates a ridge-regression classifier, improving expressivity and numerical stability in exemplar-free class-incremental learning and reaching 77.12% final accuracy on ImageNet-R B-0 Inc-10.
Quick ViTs: Speeding up Vision Transformers through Equivariance: Quick ViTs turns rotation and reflection equivariance into structured sparsity and weight sharing in Fourier-domain linear layers, allowing H8(ViT-H/14) to reach 85.0% rather than 84.6% supervised ImageNet-1K Top-1 accuracy with 102.3G rather than 167.8G FLOPs, while explaining why theoretical FLOP savings do not imply an equal runtime speedup.
Audio-Visual Continual Test-Time Adaptation without Forgetting: AVReCAP retrieves historical fusion-layer parameters using input statistics instead of continually overwriting one parameter state, achieving 43.51% mean accuracy on VGGSound-2C, 6.28 percentage points above the source model, while substantially reducing source-domain forgetting.
LANCE: Low Rank Activation Compression for Efficient On-Device Continual Learning: LANCE runs a single higher-order singular value decomposition (HOSVD) before fine-tuning to calibrate a fixed low-rank projection subspace per layer activation; during training it stores only the low-rank core tensor and reprojects it in the backward pass with the same orthonormal factors, cutting peak activation memory to 1/25–1/380 of full backpropagation, and the same fixed subspaces can be partitioned across tasks into mutually orthogonal null spaces at offline calibration time, enabling on-device continual learning without replay buffers or resident task-specific matrices.
Pathryoshka: Compressing Pathology Foundation Models via Multi-Teacher Knowledge Distillation with Nested Embeddings: Pathryoshka compresses global and spatial knowledge from three large pathology foundation models into a prefix-truncatable student, achieving a median accuracy of 85.6 across ten classification benchmarks with 86M parameters while reducing encoding and feature-storage costs.
Shared LoRA Subspaces for almost Strict Continual Learning: Share integrates incoming data or LoRA adapters into an evolving shared basis and replaces separate adapters with lightweight task coefficients, reaching 94.20% final accuracy in replay-free CIFAR-100 class-incremental learning; however, its general formulation retains task coefficients and Share-full permits old-data access, so not all results represent strict continual learning.
Stay Unique, Stay Efficient: Preserving Model Personality in Multi-Task Merging: Rather than forcing every task to use identical weights, DTS stores reconstructable task residuals through low-rank decomposition and four-group thresholding, achieving 90.17% average accuracy in eight-task ViT-B/32 merging with 0.98% additional storage per task and combining residuals by task semantic similarity for unseen tasks.
C2E: Boosting Ego-Only 3D Object Detection via Multi-Teacher Contrastive Knowledge Distillation: C2E uses the M2S multi-teacher distillation framework to transfer dense knowledge from multi-agent training observations into a student receiving only ego LiDAR, raising AttFuse's V2XSet test [email protected] from 58.06 to 66.70 without inter-agent communication at deployment.

Browse all 89 Model Compression papers →

🕸️ Graph Learning (1)¶

MG2-RAG: Multi-Granularity Graph for Multimodal Retrieval-Augmented Generation: Addressing the issues of flat vector retrieval overlooking structural dependencies and traditional graph methods discarding visual details through costly MLLM triplet extraction, MG2-RAG integrates lightweight dependency parsing with entity-driven open-vocabulary segmentation into unified multimodal nodes, enabling low-cost, high-fidelity multi-hop retrieval and reasoning via dense similarity aggregation and Personalized PageRank.

🤝 Federated Learning (2)¶

Neural Collapse-Inspired Multi-Label Federated Learning under Label-Distribution Skew: FedNCA-ML extracts class-wise image features and aligns their geometry across clients using a shared fixed equiangular tight frame and positive and negative feature regularization, raising macro-AUC from the strongest comparator's 83.63 to 87.55 in one skewed multi-label CIFAR-10 setting, without improving every micro metric.
Prevention over Correction: Learning Aligned Representations in One-shot Federated Learning: FACE moves cross-client representation alignment into local training through shared fixed class directions and an anti-collapse regularizer, allowing feature fusion without server-side recalibration and improving CIFAR-10 accuracy from the strongest comparator's 60.28% to 69.21% under extreme heterogeneity.

📈 Time Series (5)¶

Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild: Addressing the challenge of predicting diverse and non-stationary viewer emotional responses to video content in the wild, this paper presents Video2Reaction—the first large-scale benchmark comprising 10,348 movie clips (389 hours) paired with 21-class fine-grained audience reaction distributions derived from social media comments—supported by a scalable two-stage multi-agent LLM annotation pipeline and comprehensive evaluations showing that finetuned vision-language models substantially outperform classical distribution baselines.
Forecasting Animal Motion: By establishing dense point trajectories as visual tokens for behavior modeling and combining local DINOv3 visual features with a camera motion compensation pipeline, this work introduces a diffusion transformer (DiT) architecture over unordered trajectory sets to forecast complex non-rigid wild animal motion across diverse species without category-specific 3D priors.
Geometry-Aware Visual Representation for Remaining Useful Life Prediction: Addressing the failure of conventional time-frequency representations to capture intrinsic system dynamical geometry, this paper introduces Phase Space Density Images (PSDI) with Globally Anchored Reconstruction (GAR) to model degradation as attractor dispersion, achieving robust RUL prediction via sliding-window temporal aggregation and compact knowledge distillation.
NeuralDMD: Interpretable Untrained Neural Network for Imaging from Sparse and Noisy Observations: Proposes NeuralDMD, a simulation-free, untrained per-instance reconstruction framework that parameterizes dynamic mode decomposition (DMD) spatial modes as continuous neural fields with spectral time evolution, reconstructing and extrapolating continuous spatio-temporal dynamics directly from extremely sparse and noisy observations via differentiable forward operators.
PMGC-SimVP: Parametric Multi-scale Gated Convolution for Global Ionospheric TEC Prediction: To tackle severe non-stationarity and pronounced latitudinal heterogeneity in global ionospheric Total Electron Content (TEC) under space weather disturbances, this paper proposes PMGC-SimVP, a purely convolutional framework featuring a two-stage low-rank temporal adapter, latitudinally structured parameter modulation, and dual-baseline residual learning, slashing parameters by over 90% while significantly boosting storm-time forecasting accuracy and long-horizon stability.

🏥 Medical Imaging (110)¶

Comprehensive language–image pre-training for 3D medical image understanding: COLIPRI combines chest CT image–text alignment, affirmative/negative short-statement discrimination, report generation, and multiresolution masked reconstruction so that one visual encoder captures both global semantics and local structure; the full model reaches 73.23% zero-shot AUROC on RAD-ChestCT, although not every task requires the report-generation branch.
MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolution: MedSynapse-V compresses anatomical priors from a medical segmentation encoder into continuous memory, trains the model to use that memory through region-masking counterfactual rewards, and distills it into an autonomous branch without the external encoder, achieving a reported seven-benchmark average of 59.6 on offline medical evaluations, not evidence of clinical effectiveness.
FlexiBrain: Resolution-Agnostic Voxel-Level Encoding for Native fMRI: FlexiBrain defines fMRI patches in millimeters and seconds, combines dynamically resized shared kernels with physical positional encoding and Mamba-JEPA, and learns directly from heterogeneous native-space data, achieving the highest accuracy on four of five disease classification tasks while avoiding expensive template registration.
MOOZY: A Patient-First Foundation Model for Computational Pathology: MOOZY learns spatial slide representations through masked self-distillation, then jointly trains patient-level representations with a case transformer and 333 clinical tasks, achieving macro-average weighted F1 of 0.769 and balanced accuracy of 0.702 across 16 held-out tasks, while its macro-average AUC remains slightly below TITAN.
Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering: Render-FM proposes a feedforward model based on an nnU-Net-style 3D U-Net that directly regresses 6D Gaussian Splatting (6DGS) parameters from CT volume data in a single 2.8-second forward pass. This eliminates the per-scan optimization bottleneck (~500x speedup). By introducing Anatomy-Guided Prior Initialization (AGP), it utilizes segmentation masks and transfer functions as structural and appearance priors, bridging the domain gap between natural scene reconstruction and medical volume rendering. It supports real-time interactive rendering (328+ FPS) and organ-composition visualization with zero extra overhead.
ECHO: Efficient Chest X-ray Report Generation with One-step Block Diffusion: ECHO converts a medical autoregressive model into a block diffusion model and distills conditional distributions from multi-step teacher trajectories into a one-step-per-block student, retaining strong clinical-content scores on the paper's normalized chest X-ray report test sets while reaching 274.21 tokens/s with block size 8.
RAU: Reference-based Anatomical Understanding with Vision-Language Models: RAU trains Qwen2.5-VL-7B to compare anatomical spatial relationships between reference and target images, then connects segmentation tokens to SAM2 reference memory as soft prompts, achieving Dice scores of 0.7503 on CAMUS and 0.7010 on LERA when segmentation training uses only RAOS and Arcade.
Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image Segmentation: AtlasSegFM converts one annotated anatomical atlas into automatic prompts for a frozen segmentation foundation model and learns voxel-wise fusion at test time, raising SegRap Dice from 50.02 for five-click nnInteractive to 69.19 while retaining a dependence on anatomical correspondence between support and query.
Discrete Diffusion Models with MLLMs for Unified Medical Multimodal Generation: MeDiM adapts pretrained Liquid into a discrete diffusion model with bidirectional attention and timestep conditioning, using one denoising network for medical report-to-image, image-to-report, and joint image-report generation, with image-generation FID scores of 16.60 on MIMIC-CXR and 24.19 on PathGen.
MedSPOT: A Workflow-Aware Sequential Grounding Benchmark for Clinical GUI: MedSPOT is the first workflow-aware sequential grounding benchmark for clinical software interfaces: it records 216 interactive tasks with 597 annotated keyframes across 10 real medical imaging platforms, recasts GUI grounding as a 2-3 step state-dependent sequence of click decisions, and evaluates 16 MLLMs under a strict first-error-terminates protocol with a six-class failure taxonomy — the best model, GUI-Actor, reaches only 43.5% task completion accuracy, while human annotators reach 82.22%.

Browse all 110 Medical Imaging papers →

🧬 Computational Biology (6)¶

CellFluxRL: Biologically-Constrained Virtual Cell Modeling via Reinforcement Learning: Without touching the generative architecture, CellFluxRL treats seven automatically computable biological evaluators (MoA consistency, nucleus-in-cytoplasm containment, nuclear roundness, and size/count statistics of nuclei and cytoplasm) as rewards, and post-trains the flow-matching virtual cell model CellFlux with RL to push "visually realistic" cell images toward "biologically meaningful" ones; all seven rewards improve over the baselines, and best-of-N test-time scaling amplifies the gains further.
FPicker: Topology-Guided Evolution for Filament Tracing in Low-SNR Microscopy: Tsinghua University researchers present FPicker, which constructs a coarse topological skeletal prior via a box-free center-endpoint representation and iteratively deforms it through an open-boundary graph convolutional snake module with bridge curriculum training, eliminating ghost center drift and tip shrinking in low-SNR Cryo-EM filament tracing (SNR = -20 dB).
Revisiting Scene Graph Generation from the Perspective of Detector-Conditioned Reachability: This paper quantitatively analyzes prediction discrepancies between detector-based and query-based SGG methods through the lens of detector-conditioned reachability, and proposes Dual-SGG—a unified framework combining top-down and bottom-up triplet queries within a single decoder to achieve state-of-the-art relationship detection without complex post-hoc calibration.
Score-Based Matching with Target Guidance for Cryo-EM Denoising: Addressing the extreme low-SNR and lack of clean ground-truth supervision in cryo-electron microscopy (cryo-EM), this paper introduces a target-guided score matching (TSM) framework leveraging low-pass reference density projections, which stabilizes near-manifold score learning and suppresses structured background to consistently improve downstream particle picking and 3D reconstruction resolution.
Together, Then Apart: Balancing Alignment and Distinctiveness for Multimodal Survival Analysis: Addressing the issue of over-alignment collapse where aggressive cross-modal fusion suppresses modality-specific evidence in multimodal survival analysis, this paper proposes the Together-Then-Apart (TTA) framework: it first discovers shared prognostic patterns via shared prototypes and semi-relaxed unbalanced optimal transport with a curriculum mass schedule, and then preserves modality-specific features using learnable anchors and contrastive regularization, consistently outperforming recent SOTA models across five TCGA cohorts.
Trust-Region Noise Search for Black-Box Alignment of Diffusion and Flow Models: Addressing GPU memory overhead, inapplicability to non-differentiable/expensive rewards, and off-manifold drift in existing inference-time alignment methods, this paper proposes Trust-Region Noise Search (TRS), which adaptively searches and optimizes the source noise using multi-center hypercubes and coordinate masks, significantly outperforming prior black-box, gradient-based, and trajectory-search baselines across text-to-image, molecule, and protein generation tasks.

⚛️ Physics & Scientific Computing (5)¶

High-speed Imaging through Turbulence with Event-based Light Fields: Addressing the severe motion blur and geometric wobble when imaging dynamic scenes through atmospheric turbulence, this work presents the first event-only light field imaging system. By exploiting kaleidoscope optics to capture 9 sub-aperture views, it physically decouples cross-view correlated scene motion from uncorrelated turbulence perturbations, enabling crystal-clear video reconstruction at up to 12,000 fps under strong turbulence.
Resolution-Agnostic Neural Operators for Multi-Rate Sparse-View CT: This paper introduces Computed Tomography neural Operator (CTO), the first neural operator framework for sparse-view CT reconstruction that utilizes DISCO continuous convolutions and dual-domain spatial-frequency learning to achieve high-fidelity zero-shot generalization across measurement sampling rates and grid resolutions.
ELHINN: Unifying Dense Crowd Simulation Across Scales via Eulerian–Lagrangian Hydrodynamics: To bridge the fundamental disconnect between macroscopic collective coherence and microscopic individual realism in dense crowd simulation, ELHINN introduces a cross-scale hydrodynamics framework where an enhanced Eulerian velocity model (HINN++) guides a microscopic Lagrangian trajectory network (PINN) augmented with KAN residual correction and entrance-aware resampling.
Event-based Sparse-view Background-Oriented Schlieren Tomography: Addressing the high data bandwidth and intense illumination demands of frame-based background-oriented schlieren (BOS) tomography, this paper proposes the first event-based sparse-view BOS tomography framework (EventBOS), coupling a continuous 4D neural implicit fluid field with differentiable refractive ray tracing and thermal-convection Navier-Stokes regularization to reconstruct high-speed airflow dynamics that outperform 1000 fps frame-based baselines under single- and orthogonal dual-view setups.
TurboMPLE: Joint Infrared Turbulence Mitigation and Physical Fields Estimation via Mutual Progressive Layered Extraction: To tackle optical distortions, blur, and grayscale drift induced by atmospheric turbulence in thermal infrared imaging, TurboMPLE establishes an end-to-end multi-task learning framework that achieves bidirectional feature-level mutual guidance and physical law constraints, jointly delivering state-of-the-art video turbulence mitigation and accurate 2D dynamic physical fields estimation.

🌍 Earth Science (4)¶

Lessons and Open Questions from a Unified Study of Camera-Trap Species Recognition Over Time: The paper reframes camera-trap species recognition from cross-site generalization to streaming evaluation at a fixed site over time, builds StreamTrap (17 datasets, 546 camera traps), and answers four deployment questions — showing that naive accumulated fine-tuning can fall below zero-shot, offering a LoRA + Balanced Softmax recipe with post-processing, and laying out a list of open questions.
Mapping Dark-Matter Clusters via Physics-Guided Diffusion Models: The paper releases DARKCLUSTERS-15K, the largest galaxy-cluster mass-mapping dataset to date (15,000 paired surface-mass-density maps and simulated three-band HST photometry), trains a photometry-conditioned diffusion prior on it, and turns strong- and weak-lensing physics into likelihood guidance terms driven by DAPS annealing posterior sampling — producing 512×512 non-parametric mass maps with calibrated per-pixel uncertainties in minutes, beating Kaiser-Squires, Napier et al. and a UNet baseline on the TNG/SIMBA held-out test sets and approaching expert-tuned reconstructions on the real cluster MACS 1206.
RainODE: Continuous-Time Precipitation Forecasting with Latent Neural ODEs: RainODE learns continuous precipitation trajectories with a discrete-teacher-guided latent Neural ODE and refines intensity and detail through a Brownian Bridge, supporting unseen temporal queries while raising RAPID-60min CSI-M from SimVP's 0.141 to 0.220, without guaranteeing a corresponding reduction in pixel-wise error.
Less Tokens, Better Forecasts: Sparse Residual Routing for Efficient Weather Prediction: Addressing the heavy intermediate redundancy and memory bottlenecks of uniform token computation in ViT-based meteorological forecasting, Sparse-Reslim proposes a parameter-free sparse residual routing module that routes only 25% of spatial tokens through expensive middle blocks and scatters the residual delta back, achieving ~2.5× training speedup, over 2.2× peak VRAM reduction, and superior forecast skill across all variables.

📡 Signal & Communications (7)¶

FreqPhys: Repurposing Implicit Physiological Frequency Prior for Robust Remote Photoplethysmography: FreqPhys introduces a cardiac frequency-band constraint, learnable spectral modulation, and in-band noise selection into every conditional diffusion denoising step, then recovers pulse waveforms through cross-domain attention, demonstrating robustness across six datasets with a BUAA heart-rate RMSE of 1.31 bpm, although it does not lead every metric on every dataset.
MixCompress: Mixture of Experts for Variable Rate Learned Image Compression: Addressing severe gradient conflict between high and low bit-rates in dense variable-rate codecs, MixCompress integrates rate-conditioned Mixture-of-Experts (MoE) and Mixture-of-Depths (MoD) with Conditional Auxiliary Transforms (CAT), surpassing individually optimized single-rate models under a single unified checkpoint.
Raw-JPEG Adapter: Efficient Raw Image Compression with JPEG: To tackle the massive storage footprint of linear raw sensor images and their incompatibility with standard 8-bit JPEG pipelines, this paper introduces Raw-JPEG Adapter—a lightweight, learnable, and invertible pre-processing pipeline that predicts channel lookup tables, frequency DCT scaling, and spatial gamma maps embedded into the JPEG comment segment (<64 KB), enabling high-fidelity raw recovery with negligible decoding runtime.
Incentive Noise and Structural Prior Infusion for Multi-Modal Object Re-Identification: Addressing textual noise and cross-spectral structural discrepancies in multi-modal ReID, this work reformulates ambiguous language descriptions into task-aware positive-incentive noise to modulate visual global tokens, combining DINOv3 structural prompt priors with context-guided sparse fusion to achieve superior retrieval robustness under adverse conditions.
MaterialFlow: Attribute-Disentangled Material Transfer via Trajectory-Aware Velocity Modulation: To eliminate structural distortion and trajectory drift caused by feature entanglement and inversion accumulation in material transfer, MaterialFlow introduces an inversion-free trajectory-aware velocity modulation framework alongside a tri-attribute disentanglement paradigm for color, texture, and structural patterns on pre-trained Rectified Flow models.
Pol-CACTI: A System and dataset forHigh-Speed Polarized Video Compressive Imaging: To overcome the frame-rate bottleneck of conventional DoFP sensors and prevent severe error accumulation in decoupled two-stage pipelines, Pol-CACTI introduces a hardware-software co-designed polarization video SCI prototype, a curated 8,889-frame dynamic polarized video dataset, and a physics-driven joint reconstruction network featuring a Polarization-Aware Learnable Back-Projection (PA-LBP) module to achieve high-fidelity high-speed polarized video from a single compressive snapshot.
Prototype-Conditioned Imagination for Compositional Zero-Shot Learning: To tackle ambiguous cross-modal primitive alignment and the lack of explicit attribute-object interaction in Compositional Zero-Shot Learning (CZSL), this paper proposes Prototype-Conditioned Imagination (PCI), a two-stage framework that couples optimal-transport-regularized bidirectional disentanglement with prototype-driven affine modulation to synthesize virtual compositional representations, establishing new state-of-the-art results across four benchmarks under both closed-world and open-world settings.

👥 Social Computing (7)¶

Dual-Margin Embedding for Fine-Grained Long-Tailed Plant Taxonomy: Addressing the co-occurring challenges of fine-grained morphological ambiguity, extreme long-tailed taxa distribution, and open-world domain shifts, this paper proposes TaxoNet, which couples a mathematically grounded dual-margin objective to suppress repulsive head-class gradients on rare prototypes with an adaptive norm-guided sample selection strategy.
Evidence Triangulation for Multimodal Fact-Checking in the Wild: Addressing the lack of in-the-wild training benchmarks and the structural limitations of existing fact-checking models, this paper introduces X-POSE, a real-world crowdsourced benchmark derived from X Community Notes, alongside TRENT, a lightweight architecture that triangulates image, text, and external evidence via three dedicated cross-attention streams and relational NLI fusion.
Fourier Self-Supervision for Fine-Grained Generalized Category Discovery: To overcome the failure of standard contrastive learning in fine-grained Generalized Category Discovery caused by subtle inter-class differences, this paper introduces Fourier Self-Supervision that decouples the latent space into low- and high-frequency sub-dimensions based on adaptive SNR thresholds, harnessing low frequencies for abstract hierarchical discovery and high frequencies for discriminative fine details.
HCSU: A Dataset and Benchmark for Fine-Grained Historical Calligraphy Style Understanding: Targeting the critical cognitive gap where Large Vision-Language Models (LVLMs) recognize characters but fail to perceive brushwork dynamics, HCSU introduces the first large-scale benchmark that decouples authentic ink manuscripts (Tie) from stone rubbings (Bei) across 39,307 character images from 49 historical masters, incorporating hierarchical aesthetic descriptions to enable dual evaluation on 8-way style discrimination and interpretable aesthetic reasoning.
Coarse-to-fine Contrast: A Hybrid Self-supervised Method for Non-rigid 3D Shape Matching: This paper introduces a coarse-to-fine hybrid self-supervised framework for non-rigid 3D shape matching that enforces consistency between coarse soft maps and multi-scale refined hard correspondences across symmetric Laplacian and elastic basis branches, eliminating computationally expensive least-squares solvers and test-time fine-tuning while achieving state-of-the-art accuracy.
MED-LCDS: Multi-Expert-Domain CLIP Classification via Logit Calibration: To resolve cross-domain interference and artificial overconfidence caused by heterogeneous logit scales when merging independently trained LoRA experts into a unified vision-language model, MED-DSLC (MED-LCDS) combines domain-supervised mixture-of-experts routing with learnable domain-wise temperature calibration, effectively restoring global logit comparability across unified label spaces.
PanoRec: Spatially-Structured Sequence Modeling for Multi-Granularity Panoramic Retrieval: Addressing the severe semantic dilution caused by compressing 360-degree scenes into a single global embedding in conventional vision-language models, PanoRec serializes distortion-free cubemap faces, a downsampled global panorama, and spatial anchor tokens into a unified sequence, combining dynamic MaxSim routing with a joint spatial InfoNCE objective to achieve efficient multi-granularity panoramic retrieval in a single forward pass.

🛡️ AI Safety (44)¶

Scaling Laws for Black-box Adversarial Attacks: After mitigating gradient conflict among surrogate models, visual adversarial transfer success grows approximately linearly with the logarithm of ensemble size within a finite regime, a conditional empirical relationship examined on classifiers, defended models, and multimodal models.
Contextual Image Attack: How Visual Context Exposes Multimodal Safety Vulnerabilities: CIA rewrites a harmful query into a visually coherent scene — a teaching demonstration on a blackboard, steps along a roadmap, an entry in a paper, a comic dialogue panel — so that a multimodal large language model mistakes the harmful content for a routine task to complete, reaching attack success rates of 86.83% on GPT-4o and 91.02% on Qwen2.5-VL-72B.
Stealthy Multi-task Adversarial Attacks: The paper proposes SMTA2, which casts "degrade one task of a multi-task model while leaving the other tasks intact" as a constrained multi-objective optimization problem, relaxes it into an ordinary first-order iterative attack via a weighted loss with positive weights on the targeted task and negative weights on the non-targeted tasks, and resolves the remaining weight-selection difficulty with an per-iteration automated weight search; on NYUv2 and Cityscapes it degrades the targeted task heavily while holding non-targeted tasks at their clean baselines, on both undefended and adversarially trained models.
SPQR: A Multi-Dimensional Benchmark for Safety Alignment under Benign Model Adaptation: SPQR formalizes post-deployment benign fine-tuning as an unintentional attack and turns it into an evaluation axis, combining four normalized metrics — Safety, Prompt adherence, Quality and Robustness after fine-tuning — into a single harmonic-mean score; benchmarking 11 safety-alignment methods on SD v1.5/XL/v3 exposes a "utility rises while safety silently collapses" failure mode that single-axis benchmarks cannot see.
Unsafe by Reciprocity: How Generation–Understanding Coupling Undermines Safety in Unified Multimodal Models: This paper treats the reciprocity between understanding and generation in Unified Multimodal Models (UMMs) as a structural attack surface, and proposes RICE — a framework that needs no external model and no gradient optimization, shipping unsafe intermediate signals along the internal G→U and U→G loops to reach the highest ASR on five jailbreak benchmarks and three unsafe-image-generation benchmarks (89.62% / 92.69% on AdvBench for Bagel / Janus-Pro-7B).
A Classifier-Agnostic Zero-Shot Adversarial Attack Detection via CLIP: Proposes A4D, a completely black-box, zero-shot adversarial attack detection framework. By exploiting the sensitivity of CLIP to micro-perturbations, the framework compares the cosine similarity of image embeddings with a set of carefully crafted text prompts, and then aggregates them into a single detection score via PCA. This detects adversarial examples without needing to know the attack type or classifier architecture, achieving SOTA performance across multiple attacks, datasets, and classifiers.
Adversarial Attack and Disturbance Detection by Hadamard-Coded Output Representations for Object Detection and Semantic Segmentation: HadamardNet replaces conventional class outputs with redundant codewords and uses probability-simplex projection to produce both class probabilities and an inconsistency residual, enabling single-pass perturbation detection without guaranteeing that predictions on disturbed inputs are correct.
Anti-Prompt: Image Protection against Text-Guided Image-to-Video Generation: Anti-Prompt protects an owner's images with small perturbations that weaken text conditioning, increase the relative dominance of visual pathways, and disrupt image encoding; on white-box CogVideoX evaluation, its video assessment average is 3.25 versus 3.92 for reproduced I2VGuard, where lower scores indicate stronger protection.
Capacity Overflow: A Blind Spot for Backdoor Attacks in Vision MoE: This paper uncovers that the token dropping mechanism in Vision Mixture-of-Experts (Vision MoE), governed dynamically by batch size and capacity factor, constitutes an overlooked attack surface, and proposes the first load-conditioned supply-chain backdoor that stays dormant during small-batch audits and achieves high activation success under large-batch deployment.
Defending from GeoLocalization through Adversarial Road Trips: This paper introduces RoadTrip Attack (RTA), an adversarial defense method against retrieval-based visual geolocalization that reconceptualizes the attack as finding an optimal journey composed of intermediate geographic targets, utilizing adaptive-radius sampling and beam search to achieve state-of-the-art targeted deception and black-box transferability at minimal perturbation budgets.

Browse all 44 AI Safety papers →

📂 Others (57)¶

Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlation: Addressing the long-standing issue where optical tactile sensors struggle to distinguish fine-grained contact states, this paper reveals that the dot product of "transient motion" and "cumulative motion" can explicitly differentiate states such as "making contact / stable contact / releasing contact." Based on this, a Tactile Motion Correlation (TMC) representation is proposed. Using a Mixture-of-Transformers, TMC is integrated as an independent modality alongside vision, raw touch, and proprioception, achieving higher success rates across four real-world contact-rich manipulation tasks.
Articulat3D: Reconstructing Articulated Digital Twins From Monocular Videos with Geometric and Motion Constraints: Articulat3D proposes a two-stage optimization framework to reconstruct interactive digital twins of articulated objects from casual monocular videos. The first stage performs structured, part-level motion decomposition on 3D point trajectories using low-dimensional motion bases. The second stage constrains the motion to satisfy physical rigid-body rules using explicit kinematic primitives (axes, pivots, and frame-wise scalars of revolute/prismatic joints). It achieves state-of-the-art (SOTA) accuracy on both synthetic and real-world data without requiring static pre-scans.
Match-Any-Events: Zero-Shot Motion-Robust Feature Matching Across Wide Baselines for Event Cameras: This paper proposes the first event camera wide-baseline feature matching model that achieves zero-shot generalization across datasets. It learns highly efficient multi-temporal-scale event features using a separable spatial-temporal attention backbone (TAg) and sparse-aware token adaptive pruning (SETS). Trained on a large-scale synthetic (E-MegaDepth) and real-world (ECM) wide-baseline dataset, it outperforms the previous state-of-the-art by 37.7% on multiple benchmarks.
FedLAS: Feature-Modulated Bidirectional Label Smoothing for Neural Network Calibration: Proposes FeDLaS (Feature-modulated Bidirectional Label Smoothing), which utilizes the L1 norm of hidden layer features as a confidence proxy to bypass the validation limits of softmax saturation. Combined with bidirectional calibration gating, it dynamically determines the overconfidence/underconfidence state of each sample in real-time, adaptively modulating the label smoothing strength per sample to significantly reduce ECE and AECE while maintaining Top-1 accuracy.
Multi-Hypothesis Test-Time Adaptation to Mitigate Underspecification: This paper reinterprets entropy-based test-time adaptation (TTA) as a posterior inference problem under underspecification — many low-entropy solutions exist with drastically different decision boundaries — and instead of committing to a single point estimate it maintains K adaptation particles that only update normalization layers, explores different low-entropy basins, applies diversification at the output, parameter, optimizer, and input levels to prevent collapse, and aggregates the multi-hypothesis predictions at inference; it consistently beats Tent / SAR / DeYO on ImageNet-C under batch size one, label shifts, and mixed shifts, and works as a plug-and-play wrapper for existing TTA methods.
Quantile‑Adaptive Temperature Scaling for Confidence Calibration: QaTS chooses a sample's temperature from its confidence quantile in the calibration set, fitting only two positive parameters to improve calibration while preserving class predictions; on CIFAR-100 with RN-50, for example, ECE falls from 5.32 with standard TS to 2.56.
Color Pass-Through via Camera-Display Coupling: Addressing noticeable chromaticity and lightness gaps when capturing real-world scenes on smartphone cameras and viewing them on screens, this paper proposes an end-to-end Color Pass-Through framework that couples camera and display as a unified system, utilizing a lightweight neural projector and rank-1 camera-null space color correction with a single-step observer calibration to faithfully reproduce perceived scene colors.
Combining Discrepancy-Confusion Uncertainty and Calibration Diversity for Active Fine-Grained Image Classification: To tackle the failure of conventional active learning caused by severe inter-class visual overlap in fine-grained image classification, DECERN decouples structural stability and category directionality via local feature fusion, coupled with uncertainty-weighted clustering and calibration diversity to achieve state-of-the-art sample selection under tight annotation budgets.
CubicSplat: Differentiable Vector Graphics via Error-Bounded Forward Relaxation: CubicSplat resolves the gradient seesaw between forward geometric exactness and backward gradient conditioning in differentiable vector graphics through error-bounded forward relaxation, replacing Bézier closest-point root solvers with an \(\mathcal{O}(S^{-2})\) polyline surrogate to enable closed-form distance evaluation, achieving up to 4× faster training and over 2 dB PSNR gains on closed-fill benchmarks.
Estimating Velocity and Spin of Spherical Objects from Rolling-Shutter Image(s): Rather than treating rolling-shutter distortions as artifacts to be eliminated, this paper introduces a correspondence-free back-projection framework that decouples translation from rotation, enabling high-precision estimation of 3D velocity and spin for high-speed spherical objects from a single rolling-shutter image.

Browse all 57 Others papers →

📄 transfer_learning (1)¶

Dual Masked Generative Adversarial Transformer for Unsupervised Domain Adaptation: Addressing the issue where masked image modeling only provides sample-level consistency regularization while neglecting distribution-level discrepancies under large domain gaps, Dual-MGAT decouples and eliminates information gaps and domain distribution discrepancies within the global [CLS] token feature space via a bidirectional generative adversarial adaptation pipeline.