Skip to content

🧩 Multimodal VLM

🧠 NeurIPS2026 · 175 paper notes

📌 Same area in other venues: 🎞️ ECCV2026 (227) · 📷 CVPR2026 (428) · 🔬 ICLR2026 (214) · 💬 ACL2026 (82) · 🧪 ICML2026 (91) · 🤖 AAAI2026 (75)

🔥 Top topics: Multimodal/VLM ×102 · LLM ×13 · Alignment/RLHF ×10 · Reasoning ×8 · Agents ×6

3D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models

3D-PLOT-LLM introduces a decoder-free 3D multimodal large language model that reorganizes point patch tokens into \(K\) spatially coherent regions, inserting reserved vocabulary tokens <part_k> and continuous markers updated by Marker-Space Refinement (MSR) to enable native, bidirectional part-level reading and citation in the LLM's own token stream.

A Closer Look at Dynamic Scene Graph Generation In the Era of Multimodal Large Language Models

This paper revisits Dynamic Scene Graph Generation (DSGG) from task setup and model design, uncovering severe recall-oriented evaluation flaws, introducing 5 practicality metrics, and establishing the first strong top-down reason-then-locate MLLM baseline equipped with Temporal Relation Set (TRS) prediction and Importance-Aware Finetuning (IAF) to achieve SOTA across Action Genome, VidVRD, and PVSG.

A Matched-Budget Audit Framework for Recaptioned Image-Text Supervision Distributions

This paper formalizes recaptioned image-text corpora as supervision distributions induced by captioning policies, introducing a matched-budget five-axis audit framework anchored on Controllable Basic Units (CBUs) alongside an audited ~490M multi-source recaption corpus.

AdaCodec: A Predictive Visual Code for Video MLLMs

AdaCodec introduces an MLLM-oriented predictive visual code interface that allocates full ViT tokens solely to unpredictable reference frames and compact motion-residual P-tokens to intermediate frames, slashing visual token counts by over 84% while cutting latency and outperforming per-frame RGB baselines.

Aligning Forest and Trees in Images: Long Captions for Visually Grounded Understanding

Proposes CAFT, a hierarchical vision-language learning framework that combines a fine-to-coarse superpixel image encoder and a part-whole text chunking encoder to align sub-captions with intermediate visual regions and whole-captions with final scene features, achieving state-of-the-art long-text retrieval and zero-shot referring segmentation without dense region supervision.

AMPS: Adaptive Modality Preference Steering via Functional Entropy

AMPS addresses the fundamental stability-efficiency dilemma of uniform activation steering in multimodal large language models by introducing a functional entropy-derived Modality Contribution Ratio via the logarithmic Sobolev inequality, training a lightweight MLP to adaptively scale steering vectors per instance and achieving robust preference modulation without generation collapse.

Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation

Animation2Code introduces the first benchmark comprising 1,069 web animations for evaluating video-to-code de-rendering into executable HTML/CSS/JS, paired with human-aligned metrics that disentangle appearance fidelity from temporal alignment, revealing that frontier VLMs achieve strong appearance similarity (~0.84) but struggle severely with temporal motion reconstruction (~0.31).

Are Multimodal Benchmarks Really Useful? Item-Level Multimodal Benchmark Diagnosis via Structure-Response Co-Calibration

This paper introduces SRCoD, a structure-response co-calibrated diagnosis framework that anchors modality-conditioned Item Response Theory with sample-wise partial information decomposition priors, delivering reliable item-level modality dependence diagnosis and interpretable attribution across multimodal benchmarks.

Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study

Establishes MMDG-Bench, the first unified benchmark covering 6 datasets, 3 task families, 6 modality combinations, and 9 representative methods; training 7,402 models reveals that specialized MMDG methods marginally outperform tuned ERM, trimodal fusion suffers from unresolved competition, and models severely degrade under input corruptions and missing modalities while decoupling accuracy from trustworthiness.

Attention Alignment Between Humans and Vision-Language Models

By implementing a \(2 \times 2\) factorial framework crossing visual encoders (CNN vs. ViT) with language decoders (LSTM with additive bottleneck vs. Transformer with multi-head cross-attention) alongside modern VLMs, this work demonstrates that the top-down decoder attention bottleneck is the primary determinant of human gaze alignment (yielding a 40–50 percentage-point gain), while revealing a striking dissociation between behavioral fixation alignment and cortical neural predictive power.

Automotive-Env: Benchmarking Multimodal Models in Automotive Cockpit Environments

Addressing the inability of traditional GUI agent benchmarks to evaluate in-vehicle interaction dynamics, this paper introduces Automotive-ENV, an interactive cockpit benchmark built on Android Automotive OS (AAOS) with 420 deterministic tasks, revealing severe performance degradations of modern general multimodal models and GUI-specialized VLMs in implicit intent understanding, vehicle-control common sense, and safety-critical reasoning.

BalCapRL : A Balanced Framework for RL-Based MLLM Image Captioning

Addressing the systematic trade-offs in RL-based MLLM image captioning caused by optimizing narrow evaluation criteria, BalCapRL jointly optimizes utility-aware correctness, reference coverage, and linguistic quality via continuous reward-decoupled policy optimization (c-GDPO) and length-conditional reward masking.

BayesRAG: Probabilistic Mutual Evidence Corroboration for Multimodal Retrieval-Augmented Generation

To tackle cross-modal isolation and spurious relevance in visually rich documents, BayesRAG reformulates multimodal retrieval as Bayesian evidence fusion, combining Dempster-Shafer evidence theory with graph topology and layout priors to dynamically penalize modality conflicts and significantly enhance retrieval recall and downstream QA accuracy.

Behavior Pack Optimization for Video MLLM Post-Training

Addressing the issue where Video MLLMs rely on static appearance and language priors to output unchanged answers under visual evidence interventions, this paper proposes Behavior Pack Optimization (BPO), which replaces the single-response post-training unit with a behavior pack of outputs across question-type-conditioned counterfactual views scored under cross-view contracts, achieving significant gains in temporal reasoning and honest abstention under matched compute budgets.

Beyond Modality Fusion: Deep Ensembles for Multimodal Classification

This paper challenges the prevailing belief that multimodal classification requires explicit feature fusion, proving that heterogeneous deep ensembles (HeDEs) of independently trained unimodal networks consistently outperform state-of-the-art late-fusion, intermediate-fusion, and hybrid baselines at equal parameter counts.

Beyond Prediction: Steering VLM Agents with Retrospective World Modeling

RWM makes VLM agents infer the previous action from textual belief states derived from consecutive observations and converts action-transition consistency into training feedback; the full configuration achieves the paper's reported 0.81 Overall success rate across four task families, but its gains jointly involve retrospective reasoning, Bi-Level GAE, external judging, and SCR rather than SCR alone.

Beyond Scalar Distances: Semantic Attribute Gradients from Frozen MLLMs for Visual Embeddings

Introduces SAGA, a framework that employs a frozen multimodal large language model as a training-time supervisor to convert binary class labels into attribute-resolved policy gradients via GRPO, combining attention distillation and metric learning to significantly enhance zero-shot retrieval.

Beyond SFT-to-RL: Pre-alignment via Black-box On-policy Distillation for Multimodal RL

Addressing the severe distributional drift introduced by standard SFT that distorts both visual perception and reasoning in multimodal models, PRISM introduces a black-box adversarial on-policy distillation stage between SFT and RLVR with a decoupled MoE discriminator, achieving significant downstream RL performance gains.

Beyond SFT-to-RL: Pre-Alignment via Black-Box On-Policy Distillation for Multimodal LLMs

To eliminate the severe distributional drift and heterogeneous perception-reasoning degradation introduced by supervised fine-tuning, PRISM inserts an intermediate black-box adversarial on-policy distillation stage with a decoupled MoE discriminator before RLVR, consistently boosting downstream multimodal reasoning across diverse RL algorithms on Qwen3-VL.

Beyond Single Expert: Harmonizing Diverse Visual Priors in MLLMs for Spatial Understanding

Addressing the representation bottleneck of single-expert priors and the linear inference overhead of multi-model ensembles, this paper proposes ViPS, which unifies multiple heterogeneous foundation priors via an Efficient Prior Proxy (distilling single-backbone features into lightweight MLPs) and task-adaptive Dynamic Prior Fusion with zero-initialized convolutions, establishing new state-of-the-art results on spatial reasoning and 3D embodied understanding benchmarks while preserving single-encoder inference cost.

Beyond Thinking: Imagining in 360\(^\circ\) for Humanoid Visual Search

This paper introduces Imagining in 360\(^\circ\), a decoupled humanoid visual search framework where a lightweight Imaginator generates panoramic semantic layouts and probabilistic spatial priors in language space, guiding an executive Actor to break out of single-model reasoning deadlocks while cutting trajectory annotation costs.

Binding Multiple Modalities via Multimodal Wasserstein Barycenter

BaryBind replaces a fixed modality anchor with a learnable Wasserstein barycenter map and applies volumetric contrastive learning to modality gap vectors around that anchor, improving zero-shot retrieval and classification on a VAST backbone, although its published dual derivation, potential cancellation, and some experimental numbers require clarification.

Binding Visual Features Point by Point

Rooted in cognitive science and mechanistic interpretability, this paper demonstrates that visual language model (VLM) failures in multi-object scenes stem from attentional interference among compositional representations, and proves that pointing-via-text induces an internal serial visual search routine driven by dedicated "search heads," eliminating feature binding errors and enabling robust out-of-distribution compositional generalization.

CAC: Advancing Video Reward Models via Hierarchical Spatiotemporal Concentrating

Addressing subtle, spatiotemporally sparse structural, physical, and motion anomalies in generated videos, CaC introduces a coarse-to-fine two-turn concentrating reward model optimized with three-stage progressive training and verifiable spatiotemporal IoU rewards, reducing video anomalies by 11.7% in downstream generation alignment.

Can Complementary Signals Bridge Similarity Islands? Manifold-Augmented Graph Embedding for Multimodal Recommendation

Addressing the fragmented "Similarity Islands" where complementary items remain geometrically isolated in multimodal recommendation, MAGE decouples complementarity modeling into offline manifold-based geometric augmentation and online asymmetric dual-stream graph propagation, outperforming leading multimodal GNN baselines across four benchmark datasets.

Clarification as Supervision: Reinforcement Learning for Vision-Language Interfaces

Addressing the interface mismatch in decoupled vision-reasoning pipelines where generic visual captions omit crucial quantitative details, Adaptive-Clarification Reinforcement Learning (AC-RL) uses clarification interactions during training as implicit supervision with tiered rewards, compelling captioners to front-load self-sufficient information and boosting benchmark accuracy by 4.4 points while preserving single-pass inference.

CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments

CogArena translates 10 canonical experimental paradigms from the social and behavioral sciences into interactive browser-based tasks, introducing a three-level scoring framework (completion, accuracy, behavioral signatures) to reveal that multimodal agents frequently complete tasks without reproducing characteristic human cognitive signatures.

CoMet: Context and Multiplicity Decomposition for Multimodal Uncertainty Estimation

To tackle entangled uncertainty sources and unbounded open-ended answer spaces in multimodal large language models (MLLMs), CoMet decomposes predictive posterior uncertainty into a context-specific term and a multiplicity-specific term, enabling reliable, generation-free uncertainty estimation via lightweight post-hoc regression heads without autoregressive generation or repeated sampling.

COMET: Decoupled Distillation, Routing, and Capacity Control for Task-Agnostic Continual Vision-Language Learning

To tackle the coupled pressures of modal-consistency drift, representation interference, and capacity shortfall in task-agnostic continual vision-language learning, COMET introduces an architectural role-disjoint framework combining Tri-Affinity Distillation, Student-Aware Cross-Expert Distillation, and Contextual Bandit capacity control, achieving state-of-the-art retention and forward transfer with sublinear expert pool growth.

Conceal, Reconstruct, Jailbreak: Exploiting the Reconstruction–Concealment Tradeoff in MLLMs

The paper reveals the fundamental "reconstruction–concealment tradeoff" in intent-obfuscation jailbreak attacks on multimodal large language models (MLLMs) and proposes concealment-aware variant construction alongside keyword-related distractor images, exploiting the victim model's own reconstruction capabilities to achieve jailbreak attack success rates of up to 99.7%.

CondenseVLA: Learnable History Condensation for Efficient Multi-Frame VLA

CondenseVLA introduces a Learnable History Condensation (LHC) module at the diffusion action-decoder interface of a multi-frame VLA policy, distilling FIFO-cached historical observation features into a fixed budget of slot tokens with attention diversity and middle-only decorrelation regularizations, significantly boosting multi-stage manipulation success while maintaining low inference latency.

Cross-Attentive Bayesian Low-Rank Adaptation for Multimodal Uncertainty Estimation

SPECTRA introduces a multimodal Bayesian low-rank adaptation framework that confines stochasticity to compact rank-\(r\) latent matrices inside frozen LoRA adapters, modulating variational posterior parameters through per-layer token-level text-audio cross-attention to achieve calibrated, heteroscedastic uncertainty estimation.

CSI-TextBench: A Dataset and Benchmark for Language-Grounded Ambient Sensing Perception

Addressing the lack of naturally paired text for ubiquitous WiFi Channel State Information (CSI), this paper presents CSI-TextBench—a large-scale benchmark unifying 285,790 samples across 7 tasks and 2.0M CSI–text pairs—demonstrating that label-free contrastive alignment matches supervised classifiers while enabling open-vocabulary prompting and compositional scene understanding.

DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation

This paper exposes the "zero-delta trap" where up to 69% of prompts in standard multimodal datasets yield identical teacher-student answer distributions, and introduces DeltaPrompts—a staged prompt synthesis pipeline driven by answer divergence that achieves up to 15% relative gains across 10 VLM reasoning benchmarks.

Diagnosing and Mitigating Modality Interference in Multimodal Large Language Models

Addressing the "cross-modality competency" limitation where multimodal large language models fail to distinguish task-irrelevant signals, this paper introduces a causal perturbation diagnostic framework and a unified fine-tuning strategy combining modality-masked adversarial perturbations with consistency regularization, achieving superior unimodal robustness without sacrificing standard VQA performance.

Do Composed Image Retrieval Benchmarks Require Multimodal Composition?

This paper audits four major CIR benchmarks using eleven Generalist Multimodal Embedding models, discovering that 32.2% to 83.6% of queries can be solved via a single modality, and introduces CIRCUS, a shortcut-cleaned, human-validated evaluation benchmark.

Do Multimodal Models Imagine Electric Sheep?

This paper discovers that vision-language models (VLMs) spontaneously form internal "mental imagery" encoding evolving world states when trained via open-loop behavior cloning on spatial puzzles, and showing that augmenting reasoning with compact visual chain-of-thought tokens or auxiliary visual prediction losses boosts average solve rates from 83% to 89.5%.

DocAtlas: Long-Document Understanding as Mutable-State Interaction

DocAtlas formulates long-document understanding as an interactive, mutable-state information-seeking process where an agent dynamically writes grounded findings back into a hierarchical document tree and maintains structured working memory, achieving state-of-the-art results with frontier VLMs and enabling end-to-end RL for compact models.

DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translation

DocPTBench establishes the first end-to-end benchmarking suite for parsing and translating real-world photographed documents across 1,300+ high-resolution images and 8 translation directions, revealing that photographic degradation induces an 18%-25% accuracy collapse in parsing and a 12% BLEU score drop in translation across existing SOTA MLLMs and specialized systems.

Does the Question Really Matter? Training-Free Data Selection for Vision-Language SFT

To tackle prevalent linguistic shortcuts and weak multimodal supervision in visual instruction tuning, this paper proposes CVS, a training-free data selection method leveraging a frozen VLLM's conditional verdict shifts to filter semantic-conflict noise and prioritize hard positive samples near the decision boundary, outperforming full-data training with only 10%–15% data.

doomed to re-annotate forever the imagenet story

Through an iterative refinement pipeline combining in-house human annotators with vision models and MLLMs, the ImageNet-1k validation set is fully re-annotated (ReImageNet) with multilabel bounding boxes and five semantic attributes, revealing that ~12% of original labels are incorrect, 33.3% of images are multilabel, and errors severely propagate into derivative benchmarks.

Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning

Addressing the failure of static module composition to capture within-task sample diversity in multimodal continual instruction tuning, Drape introduces an instance-level soft prompt generation framework via instruction-guided visual cross-attention, combined with null-space gradient projection on the shared projector and task-label-free CLIP prototype routing to suppress catastrophic forgetting.

Dynamic Resolution Routing for Efficient Egocentric Grounding

SmartRes resolves the heavy visual encoding bottleneck of egocentric visual grounding by shifting from heuristic latent token pruning to proactive pixel-space dynamic resolution routing, achieving an 86.4% performance retention rate with a 67% visual token reduction and up to 1.66x faster inference.

EchoPrune: Interpreting Redundancy as Temporal Echoes for Efficient VideoLLMs

EchoPrune is a lightweight, training-free token pruning framework for VideoLLMs that conceptualizes inter-frame redundancy as "temporal echoes" reconstructable from previous frames, combining query-guided relevance with spatial-temporal reconstruction errors to scale visible frames by up to 20× under fixed token budgets while yielding up to 5.6× prefilling speedup.

EDMA: Entropy-Driven Multimodal Answering

EDMA formalizes multimodal question answering as an iterative partition refinement and entropy reduction process over candidate answer spaces via logical entropy, providing explicit modality-level attribution while significantly outperforming CoT and planning prompting baselines.

EgoBench: An Interactive Egocentric Multimodal Benchmark for Tool-Using Agents

EgoBench is the first interactive multimodal benchmark grounded in egocentric video for tool-using agents, covering 1,590 tasks across five daily scenarios with deterministic process-and-result joint validation.

ES-Merging: Biological MLLM Merging via Embedding Space Signals

ES-Merging introduces a representation-aware model merging framework that derives layer-wise (sliced Wasserstein distance differences) and element-wise (gradient-based alignment sensitivity) merging coefficients from probe-induced embedding space signals, enabling tuning-free biological MLLM unification with superior cross-modal reasoning and single-modal retention.

Exemplar2VQA: A Scalable Exemplar-Driven Visual Question Answering Generation Framework via Multi-Agent Coding

Exemplar2VQA compiles spatial QA exemplars into reusable simulator capture and geometric annotation programs, using four-role collaboration and execution feedback to reduce generation errors; approximately 10K synthetic examples raise the author-reported average score of Qwen2.5-VL-3B on the multiple-choice subset of VSI-Bench from 35.3 to 42.9.

Exploring MLLM-Diffusion Information Transfer with MetaCanvas

MetaCanvas introduces learnable multidimensional canvas tokens appended to MLLM inputs, enabling multimodal LLMs to draft spatial and spatiotemporal visual plans directly in latent space and inject them patch-wise into diffusion noisy latents via a lightweight two-block connector, substantially improving structured visual control and editing accuracy across images and videos.

FineVision: Open Data Is All You Need

FineVision curates and unifies over 200 public multimodal sources into an open 24.3M-sample, 88.9M-turn instruction dataset with a standardized GUI action space and rigorous SSCD-based decontamination, consistently outperforming existing open-data mixtures across 11 standard benchmarks.

FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learning

FLAME introduces an adaptive sparse MoE architecture for joint pretraining across heterogeneous modality combinations and post-deployment continual learning, decoupling modality dispatch from task definitions via per-modality routing and achieving structural no-forgetting via low-rank expert compression.

From Failure Taxonomy to Intervention: A Diagnostic Methodology for Industry-Scale AVLM in Video and Live-Streaming Platform Moderation

Addressing the challenge that generic multimodal models fail to systematically isolate and correct policy-level errors in trillion-scale video and live-streaming moderation, this paper proposes a diagnostic methodology mapping observable failure signatures to targeted development interventions, successfully training and deploying the first industrial-scale AVLM foundation model.

From Senses to Decisions: The Information Flow of Auditory and Visual Perception in Multimodal LLMs

This paper provides the first comprehensive map of how audio and visual signals travel through Audio-Visual Large Language Models (AVLLMs): video inputs adhere to a sequential Modalities \(\to\) Question \(\to\) Prediction routing, interleaved multi-inputs shift into multi-stream parallel paths, late-layer video attention spikes are unmasked as non-semantic vision sinks, and discarding multimodal tokens once transferred yields lossless prefill latency reductions.

Fusion or Confusion? Multimodal Complexity Is Not All You Need

Under unified standardized experiments and Bayesian hierarchical testing, this paper rigorously benchmarks 19 high-impact multimodal architectures, revealing that complex multimodal fusion, gradient regulation, and structural modifications fail to reliably outperform best-performing unimodal baselines or a simple late-fusion transformer baseline (SimBaMM).

FusionAudit: Pathway-Conditioned Robustness Auditing for Native Multimodal Models

To address the defense dynamics introduced by deep cross-modal fusion in native vision-language models, this paper introduces PaRMA and FusionAudit—an evidence-driven auditing methodology that uncovers the gradient dilution pitfall of standard PGD and the "see-but-not-deceived" visual rejection signature of rendered text, introducing diagnostic-motivated attacks MANTIS and Nat-Evid to reveal substantial hidden residual risks across 18 open-weight VLMs.

GeoRouter: Dynamic Paradigm Routing for Worldwide Image Geolocalization

Addressing the complementary error profiles between retrieval-based instance matching and generation-based semantic reasoning in worldwide image geolocalization, this paper proposes GeoRouter, a dynamic routing framework that uses a vision-language model with continuous distance-aware preference optimization to adaptively select the optimal paradigm for each query.

Hidden Forgetting in Continual Multimodal Learning: When Accuracy Survives but Grounding Fails

This paper uncovers and formalizes "hidden evidence-use forgetting" in multimodal continual learning—where answer accuracy survives but grounding evidence drifts—and introduces Reliance-Constrained Continual Learning (RCL), a replay-free framework that preserves evidence-reliance profiles via counterfactual interventions and confidence gating with zero inference-time overhead.

Hierarchical Semantic Tree Anchoring for CLIP-based Class-Incremental Learning

Addressing catastrophic forgetting and fine-grained feature drift caused by flat class assumptions in CLIP-based class-incremental learning, Hasten anchors multimodal features in hyperbolic Lorentz space via an incrementally grown semantic tree with partial-order cone constraints and contrastive learning, while stabilizing the shared mapper using covariance null space gradient projection.

Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs

Based on the psychometric Factor-Referenced Cognitive Test (FRCT), this paper introduces VisFactor—a benchmark spanning 4 cognitive domains and 20 vision-centric subtests along with controllable parametric generation algorithms—revealing foundational visual cognition deficiencies and a "Middle Score Anomaly" in frontier MLLMs.

HYDRA: Representation-Harmonized Tokenization for Multimodal Generation and Understanding

Addressing the fundamental representational divergence between high-level visual understanding and fine-grained visual generation, HYDRA introduces HYDRA-TOK—a pure ViT progressive tokenizer with a Generation-Semantic Bottleneck (GSB)—enabling bidirectional co-promotion and establishing new SOTA benchmarks across understanding and generation within a single parameter space.

Hydra-X: Native Unified Multimodal Models with Holistic Visual Tokenizers

Hydra-X presents the first native unified multimodal model that seamlessly tokenizes images and videos within a single Vision Transformer (Hydra-XTok) via tubelet causal attention, hierarchical temporal compression, and decompressor distillation, while unlocking identity-faithful image editing through tokenizer-stage source-target interaction.

I Don’t Miss You, but I Do: Self-Explanation Faithfulness of Modality Missingness in Vision-Language Models

This paper introduces an executable interventional evaluation protocol to assess whether vision-language models faithfully explain their reliance on missing versus available modalities, revealing a systematic bias where models heavily overstate evidence sufficiency and fail to anticipate prediction changes caused by restoring absent inputs.

I-Perceive: A Foundation Model for Vision-Language Active Perception

I-Perceive proposes the first foundation model for open-instruction active perception across large-scale indoor scenes, fusing multi-layer intermediate semantic features from a VLM into a multi-view geometric transformer backbone to directly regress physically feasible and instruction-aligned 6D target camera poses end-to-end.

InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories

InduceKV freezes all multimodal backbone parameters and externalizes task increments into a budgeted, attention-compatible external KV memory via bilevel inducing-set selection and minimal retrieval calibration, achieving superior continual adaptation without parameter expansion or forgetting.

Instruction Anchor: Dissecting the Mechanistic Dynamics of Modality Arbitration

This paper investigates multimodal instruction following from an information flow perspective, discovering that instruction tokens serve as structural anchors where modality arbitration crystallizes—transitioning from undifferentiated shallow-layer buffering to sparse, deep-layer attention head arbitration.

IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams

This paper introduces IPIBench, the first benchmark for evaluating the interactive proactive intelligence of MLLMs under streaming video settings, and presents IPI-Agent, a training-free framework featuring an interaction-control policy and a temporal-gating mechanism that significantly enhances proactive triggering stability and multi-turn coordination.

ITO: Multi-View Alignment and Training-Time Fusion for Image-Text Pretraining

Addressing the modality-induced separation problem in dual-encoder vision-language pretraining, ITO decouples instance discriminative learning from global representation regularization by combining many-to-one multimodal multiple alignment with a lightweight training-time fusion module that is discarded at deployment, delivering consistent gains across classification, retrieval, and MLLM benchmarks at zero inference cost.

JMed48k: A Multi-Profession Japanese Medical Licensing Benchmark for Vision-Language Model Evaluation

Constructed from 20 years of official Japanese Ministry of Health, Labour and Welfare licensing examination archives across 11 healthcare professions, JMed48k aggregates 48,862 questions with 20,142 images under an 8-type taxonomy, and reveals through a paired image-removal audit that current medical-specific VLMs exhibit minimal true reliance on clinical imagery compared to proprietary frontier models.

K9-Bench: Evaluating Multimodal LLMs on Canine-Centric Videos

K9-Bench introduces a benchmark of 4,744 five-way multiple-choice questions across 907 curated real-world videos to evaluate multimodal LLMs on fine-grained canine posture, action sequences, and social interactions, revealing that frontier models still struggle significantly with long-horizon animal behavioral reasoning.

LAION-BVD: A 10-Million-Hour Open Video Dataset for Multimodal Pre-training

Constructs and releases LAION-BVD, a 10-million-hour open multimodal web video dataset with automated pipelines producing 55M video/audio captions and 300M scene-transition frames, outperforming previous open datasets across video, audio, and image pretraining benchmarks.

Language-Assisted Image Clustering Guided by Discriminative Relational Signals and Adaptive Semantic Centers

Addressing the weak discriminability of constructed textual representations and the rigid constraints of pre-built alignment spaces in language-assisted image clustering, this paper proposes ridge-regression-based cross-modal relational matrix modeling and adaptive continuous semantic center prompt learning, outperforming state-of-the-art baselines by 2.6% on average across eight benchmarks.

LARGO: Low-Rank Hypernetwork for Handling Missing Modalities

LARGO shifts from conventional feature-space imputation to weight space by reparameterizing the convolutional weights of all \(2^N-1\) dedicated missing-modality models via Canonical Polyadic (CP) tensor decomposition, delivering dedicated-model specialization across all modality subsets at the parameter budget of a single network.

Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Grounding

Addressing the expensive multiple rollouts and sparse reward bottlenecks of reinforcement learning in GUI grounding, GUI-SD introduces the first on-policy self-distillation framework tailored for GUI grounding, combining visual privileged guidance with entropy-guided token weighting to provide dense token-level supervision from a single rollout, achieving superior accuracy and ~4x faster training than GRPO baselines.

Less Language, More Latents: Annotation-Efficient VLAs for Driving

To alleviate the severe annotation bottleneck in driving Vision-Language-Action models (VLAs), LADA introduces a three-stage decoupled pipeline that requires only ~5% high-level commands and counterfactual annotations, outperforming fully supervised baselines with an 87.98 Driving Score on closed-loop Bench2Drive.

LoMo: Local Modality Substitution for Deeper Vision-Language Fusion

Addressing the severe "carrier sensitivity" problem where Vision-Language Models collapse when textual queries are rendered as images, this paper proposes LoMo—an architecture-agnostic data curation paradigm that reformulates text-only instructions into fine-grained interleaved text-visual sequences, providing implicit supervision for cross-modal representational invariance.

Look-Before-Move: Narrative-Grounded World Visual Attention in Dynamic 3D Story Worlds

Proposes Look-Before-Move, a framework that establishes an observation contract and searches for feasible viewpoints before grounding them into continuous, collision-free camera motion, solving narrative-grounded active visual attention planning in dynamic 3D story worlds.

M*: A Modular, Extensible, Serving System for Multimodal Models

Addressing the architectural mismatch where traditional autoregressive serving engines struggle with the diverse computational topologies of composite multimodal models, M* formulates multimodal inference as traversal over a Walk Graph, decoupling model graph topology from physical cluster placement via composable primitives and an asynchronous runtime to achieve flexible loops, parallel branching, and dynamic streaming.

Magnifying What Matters: Attention-Guided Adaptive Rendering for Visual Text Comprehension

Addressing the "localization-without-utilization" bottleneck in Visual Text Comprehension (VTC) where VLMs pinpoint evidence but fail to answer correctly, AGAR introduces a training-free adaptive rendering framework that uses middle-to-late layer attentions to magnify key evidence spans on a re-rendered page, achieving substantial accuracy gains at minimal token overhead.

Magnifying What Matters: Attention-Guided Adaptive Rendering for Visual Text Compression

Addressing the "localization-without-utilization" failure mode in visual text comprehension (VTC)—where vision-language models accurately attend to evidence patches but still produce incorrect answers—this paper introduces AGAR, a training-free and model-agnostic adaptive rendering method that uses middle-to-late layer attention to identify critical visual patches, maps them back to character spans, and magnifies those spans on a re-rendered page, consistently boosting performance across 9 benchmarks.

MARS: Harmonizing Multimodal Convergence via Adaptive Rank Search

Addressing the performance bottlenecks and training oscillations caused by asynchronous multimodal convergence during full-module LoRA fine-tuning in vision-language models, MARS introduces an automated adaptive rank search algorithm governed by dual scaling laws (Scaling Law-C and Scaling Law-P) that prunes the search space via convergence alignment and identifies optimal rank pairs under lightweight calibration, achieving aligned convergence and outperforming heuristic baselines while accelerating search by over 11.5x.

MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarios

MDPBench establishes the first comprehensive benchmark evaluating document parsing across both clean digital-born and physically degraded photographed documents in 17 languages with 3,400 rigorously annotated images, demonstrating a severe performance collapse in open-source parsers on non-Latin scripts and real-world photographic distortions.

Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability

This paper demonstrates that conventional unimodal perturbation metrics trigger severe evaluation collapse due to cross-modal redundancy and language priors, introduces Synergistic Faithfulness (\(\mathcal{F}_{syn}\)) based on the Shapley Interaction Index and 2-way Harsanyi dividends (achieving a 24× speedup with \(\rho=0.92\)), and benchmarks 8 XAI methods across 3 VLM architectures to reveal that attention-based explainers significantly outperform VLM-native explainers that over-index on visual salience.

MemLeak: Diagnosing Information Leaks in Multimodal Agent Memory

Introduces the Information Provenance Graph (IPG) taxonomy and the MemLeak benchmark, uncovering that after deleting text entries and tagged images, VLMs still recover 12.0% of forgotten facts via implicit visual cues from retained images, while content-aware semantic deletion cuts this residual to 2.0%.

Meow-Omni 1: A Multimodal Large Language Model for Feline Ethology

Meow-Omni 1 is the first open-source, quad-modal multimodal large language model purpose-built for computational ethology, natively co-embedding video, audio, and high-frequency biological sensor time-series into the LLM latent space to achieve a state-of-the-art 71.16% intent-recognition accuracy on the expert-curated MeowBench benchmark, successfully overcoming semantic aliasing in animal behavior.

MicroWorld: Empowering Multimodal Large Language Models to Bridge the Microscopic Domain Gap with Multimodal Attribute Graph

MicroWorld constructs a multimodal attributed property graph (MAPG) comprising approximately 111K nodes and 346K typed edges from large-scale scientific image–caption literature, injecting structured biomedical domain knowledge during inference without fine-tuning and enabling an open-source 8B model to outperform GPT-5 by 13.0% on MicroVQA.

MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome

Addressing the critical limitations of static, report-only benchmarks that lack multimodal grounding, MiroEval establishes an evolving benchmark of 100 real-world tasks (70 text-only, 30 multimodal) alongside a decoupled three-dimensional evaluation framework covering adaptive synthesis quality, agentic multi-source factuality checking, and process-centric causal trajectory auditing across 13 frontier research systems.

Mirror, Mirror on the Wall: Can VLM Agents Tell Who They Are at All?

The paper introduces a controlled 3D simulation benchmark inspired by animal mirror self-recognition, requiring first-person VLM agents to infer a hidden body attribute from an unlabeled reflective surface and navigate to the matching physical target, systematically exposing deep dissociations between visual evidence gathering, self-attribution, and embodied action.

Mitigating Saliency Collapse: Robust Saliency-Aware Long-Text Image-Text Alignment

Existing long-text vision-language models suffer from "saliency collapse" by over-relying on visually prominent cues while neglecting informative background context; RoSA resolves this via modality-independent salient/contextual view decomposition and auxiliary contrastive objectives without introducing any inference overhead.

Mixture of Layers: Dynamic Layer Routing for Visual Reasoning

Addressing the drawback where Multimodal Large Language Models discard low-level visual details by relying exclusively on the final vision-encoder layer, this paper proposes Mixture of Layers (MoL), an instruction-conditioned, patch-level sparse routing mechanism with disagreement-gated fallback that dynamically aggregates intermediate visual representations without inflating the visual token budget.

MLLMs Fail to Refuse when Using Tools Agentically

Reveals that agentic multimodal large language models (MLLMs) experience severe safety alignment degradation when iteratively invoking tools like zooming and OCR, increasing relative refusal failure rates by up to 68.7%.

MM-IssueLoc: A Controlled Benchmark for Evaluating Visual Evidence in Multimodal Issue Localization

Addressing the limitation where existing software engineering benchmarks conflate code repair with localization and obscure the real impact of visual inputs, this paper introduces MM-IssueLoc—the first controlled benchmark for repository-level multimodal issue localization across 23 languages and 652 instances—featuring strict file- and function-level gold labels, 7 image evidence categories, 4 relevance levels, an image-to-text diagnostic protocol (VCE), and a two-stage multimodal retriever (MM-IssueLoc-VL-Embedding).

MMA-SafetyBench: A Benchmark for Multimodal Agent Safety Evaluation

MMA-SafetyBench establishes a benchmark of 1,083 adversarial trajectories across five interactive agent domains to evaluate Visual Semantic Hijacking, introducing a 1–3–5 scoring rubric and Semantic Compromise Rate (SCR) that separate spatial target-matching from semantic deviation, revealing severe vulnerability in frontier vision-language models (ASR up to 87.96%, SCR up to 98.09%).

MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods

This paper introduces MMFineReason, a large-scale open-source multimodal reasoning dataset (1.77M samples with 5.15B CoT tokens) distilled from Qwen3-VL-235B across a structured 3-stage pipeline; fine-tuning Qwen3-VL-Instruct yields the MMFineReason-2B/4B/8B model family that outperforms significantly larger reasoning models, uncovering a "less is more" phenomenon where a 7% difficulty-filtered subset matches full-corpus training.

MMLongCite: A Benchmark for Evaluating Faithfulness of Long-Context Vision-Language Models

MMLongCite is the first comprehensive benchmark evaluating the fine-grained citation faithfulness of long-context vision-language models (scaling from 16K to 128K tokens across 8 diverse datasets), demonstrating across 28 leading models that high answer correctness frequently decouples from citation faithfulness and collapses under spatial grid re-stitching (MMLongCite-HR).

MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos

Addressing the critical absence of long-horizon joint audio-visual reasoning evaluations in current MLLMs, this paper introduces MMOU, a massive benchmark comprising 20,000 expert-annotated questions spanning 11,877 real-world videos (up to 121 minutes) and 13 foundational reasoning skills, revealing severe performance gaps where the strongest closed model reaches only 64.2% accuracy and top open-source models stall at 46.8%.

MMSkills: Towards Multimodal Skills for General Visual Agents

Addressing the core bottleneck of conventional agent skill libraries that rely solely on textual specifications while missing state visual cues, MMSkills couples textual procedures with runtime state cards and multi-view keyframes, leveraging an automated trajectory generation pipeline and branch loading to substantially improve visual agent success and execution efficiency across GUI and game benchmarks.

Model Guides You How to Draw: Adaptive Visual Gating for Unified Multimodal Reasoning

To resolve harmful intermediate visual steps and excessive computational costs in unified multimodal models, this paper proposes AdaViG, a training-free adaptive visual gating method that leverages early diffusion attention signals (Generation Intent and Visual Fidelity at step 5) to dynamically abort harmful visual generations, improving mathematical reasoning accuracy by up to 5.7% while cutting generation FLOPs by up to 91.0% and wall-clock latency by up to 45.6%.

MPDocBench-Parse: Benchmarking Practical Multi-Page Document Parsing

To overcome the limitations of existing benchmarks that focus on single-page, text-centric, or isolated subtasks, MPDocBench-Parse introduces a practical multi-page document parsing benchmark encompassing 15 diverse English and Chinese document categories across 3,246 pages, establishing an eight-dimensional evaluation protocol spanning text/table/formula recognition, truncated text and table merging, fine-grained visual figure extraction, reading order, and heading hierarchy tree recovery.

MSP: Modality Self-Play for Training Multimodal LLMs without Paired Cross-Modal Data

Modality Self-Play (MSP) trains multimodal LLMs without real paired non-text supervision by generating anchor-supporting proxy textual descriptions for unseen modalities and projecting them through a ridge-regression bridge, achieving state-of-the-art zero-shot cross-modal reasoning.

MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image

MulTaBench constructs a benchmark of 20 image-tabular and 20 text-tabular tasks by requiring both complementary multimodal signal and gains from target-aware representations over frozen embeddings, finding that adaptation gains generalize to new learners but are not consistently significant on every selected dataset.

MUNI: Multimodal Unified Latent Diffusion for Coherent Any-to-Any Generation

MUNI extends unified latent diffusion to multimodal any-to-any generation end-to-end through non-mixture posterior aggregation, target-detached self-reconstruction, and leave-one-out prior routing, achieving superior cross-modal conditional fidelity and state-of-the-art unconditional joint sampling coherence in a single shared stochastic latent space.

Neural Bayesian Filtering

Neural Bayesian Filtering (NBF) represents belief states as compact distribution embeddings and sequentially updates them by drawing fresh particles from a conditional flow-matching generator, propagating and reweighting them via physical dynamics, and re-embedding the weighted set, effectively eliminating particle impoverishment while enabling zero-shot generalization to unseen dynamics.

Not Another Text Benchmark: Putting the “Visual” Back in Visual Question Answering for Large Video Models

Challenging the dominant paradigm where video understanding benchmarks rely exclusively on text answer choices—allowing models to exploit language shortcuts—this paper introduces the first vision-centric evaluation benchmark featuring visual frame queries and options across temporal frame retrieval, physical future prediction, and cognitive causal memory distortion, exposing critical vulnerabilities in frontier video models.

Object-Uni: A Unified Model for Object-Centric Spatial Understanding and Controllable Generation

Object-Uni formulates object-centric spatial intelligence as a unified understanding-and-generation framework, bridging continuous 3D object poses into language space via viewpoint-based orientation abstraction, and coupling object-token-grounded pose anchors (PoseAnchor) with CNOCS dense geometric guidance to achieve state-of-the-art pose understanding, spatial reasoning, controllable generation, and object-centric novel view synthesis in both single- and multi-object environments.

OmniGF: A Dual-Branch Vision-Language Framework for Unified Gaze Following

OmniGF introduces a unified vision-language framework for gaze following that combines head-conditioned visual token injection with a dual-branch architecture—coupling a structured language branch with a continuous spatial heatmap decoder—to enable scalable multi-person spatial localization, semantic target identification, and social gaze reasoning in a single forward pass.

OmniSpace: Efficient Geometry Awareness for Autonomous Vehicles MLLMs

OmniSpace introduces a Camera Pose Injector (Plücker ray embeddings), Multi-view Epipolar Attention, and temporal 3D geometric distillation, enabling multimodal LLMs to achieve strong spatial intelligence and trajectory planning from pure 2D observations without relying on external 3D auxiliary models at inference time.

OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Vision-Language Models

This paper introduces OneVision-Encoder (OV-Encoder), an HEVC-aligned visual encoder that uses motion vectors and prediction residuals from video codecs to selectively encode only 3.1%–25% of visual patches rich in signal entropy, combined with 3D-RoPE and million-scale semantic cluster discrimination to achieve substantial gains over Qwen3-ViT and SigLIP2 across 16 multimodal and video understanding benchmarks under identical token budgets.

OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents

To overcome proprietary training barriers and cascading multi-turn failures in frontier multimodal search agents, OpenSearch-VL introduces an entirely open recipe encompassing Wikipedia graph-based multi-hop VQA synthesis, a unified active perception and retrieval tool suite, and a multi-turn fatal-aware GRPO algorithm featuring one-sided advantage clamping.

OpenView: Empowering MLLMs with Out-of-view VQA

Addressing the limitation of Multimodal LLMs confined to reasoning within visible frames, OpenView introduces an automated four-stage pipeline leveraging panoramic imagery to synthesize Out-of-view (OOV) multi-choice VQA data, delivering the 158k-sample OpenView-Dataset and the rigorously validated OpenView-Bench to boost open-source MLLM joint accuracy from 48.6% to 64.1%.

P\(^3\)-VLM: A Point-based Alternative for Grounded 3D Vision-Language Models

Reveals severe bounding-box semantic leakage in existing 3D VLMs where models exploit geometric aspect ratios rather than visual evidence, introduces a strict point-based spatial grounding protocol, and develops a tri-pathway detector-free 3D VLM achieving state-of-the-art grounded captioning accuracy and superior out-of-distribution robustness.

PathView-Bench: Can Multimodal Large Language Models Achieve Fine-Grained Multiscale Pathology Analysis?

PathView-Bench is the first vision-anchored pathology MLLM benchmark that spans both high-magnification Region-FOV and macro Slide-FOV scales across 14 deterministic tasks and 308k samples, revealing that current models frequently mask deep visual grounding and spatial estimation failures beneath fluent diagnostic text.

Pathways of Visual Information Flow in Vision-Language Models

By integrating layer-wise counterfactual causal patching and attention knockout interventions, this work reveals that vision-language models route visual information through two distinct pathways—direct readout and text-mediated transfer—demonstrating that pathway selection is dynamically modulated by task complexity and data distribution, with text mediation acting as an adaptive fallback circuit when the direct route is disrupted.

PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loop

PhysVista addresses fragmented physical evaluation and the neglect of AI-generated content by introducing a closed-loop benchmark spanning physical state perception, dynamics reasoning, and plausibility assessment, systematically exposing major bottlenecks in fine-grained scale estimation, counterfactual prediction, and calibrated plausibility judgment across modern VLMs.

ProCLIP: Progressive Vision-Language Alignment via LLM-based Embedder

ProCLIP introduces a curriculum-learning-inspired progressive vision-language alignment framework that distills native CLIP textual priors into an LLM adapter during Representation Inheritance and regularizes contrastive tuning with visual momentum self-distillation, boosting zero-shot classification by 6.8%–13.5% over LLM2CLIP while unlocking long-text and multilingual capabilities.

PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping

To tackle the coarse-grained credit assignment bottleneck where sparse visual tokens are overwhelmed by language priors in multimodal reasoning, PRPO introduces Robust Visual Dependency (RVD) via dual-perturbation probing and Perceptual Advantage Reshaping (PAR) via an S-shaped synaptic gating curve, delivering significant gains across seven benchmarks.

RankAlign: Vision-Language Representation Alignment via Rank Transformation

Addressing the challenging class-level unpaired matching problem between independently trained vision and language models without cross-modal supervision, RankAlign proposes a solver-agnostic rank transformation strategy that converts within-modality similarity kernels into monotone-invariant ordinal representations, substantially boosting matching accuracy and robustness.

ReaLM: A Unified Red-Teaming Benchmark for Physical-World VLMs

Establishes ReaLM, the first unified red-teaming benchmark for physical-world vision-language models, introducing an agentic target-generation pipeline to derive scenario-specific physically grounded attack objectives and evaluating 12 red-teaming methods, 3 defenses, and 13 VLMs under a strict black-box threat model.

ReefNet: A Large-Scale Dataset and Benchmark for Fine-Grained Coral Reef Recognition

ReefNet presents a large-scale, WoRMS-taxonomically standardized coral reef benchmark with ~925K point annotations across 39 hard-coral classes from 76 CoralNet sources and the Red Sea, systematically exposing key bottlenecks of vision-language and multimodal large language models under zero-shot, few-shot, and cross-source shifts.

Reinforcing VLAs in Task-Agnostic World Models

RAW-Dream establishes a task-agnostic world model reinforcement learning paradigm that decouples environment dynamics and reward evaluation from downstream task dependencies, leveraging broad play-data video diffusion models and zero-shot VLM rewards with a dual-noise verification mechanism to post-train VLA policies purely in imagination.

Representation Forcing for Bottleneck-Free Unified Multimodal Models

To eliminate the structural bottleneck and quality upper bound imposed by frozen VAEs in unified multimodal models, this paper proposes Representation Forcing (RF), which trains the decoder to autoregressively predict understanding-derived discrete representation tokens as in-context scaffolding before pixel-space flow matching, matching state-of-the-art VAE-based generation while boosting visual understanding.

Rethinking Projector Training in Multimodal LLMs

Revealing that Stage-I projector training in multimodal LLMs essentially performs coarse geometric conditioning rather than learning fine-grained semantic mappings, and introducing PORTAL, a training-free closed-form projector based on Gaussian optimal transport that eliminates paired image-caption data while matching or exceeding 558k-trained baselines.

ReToken: Improving Long-Context VLMs with Visual Retrieval Token

To overcome attention breakdown and GPU memory exhaustion in long-context vision-language models with extensive visual distractors, ReToken introduces a single learnable retrieval token with a single projection matrix to perform accurate in-cache retrieval directly over the frozen VLM's final-layer Value projections, achieving substantial performance gains across image and hour-long video benchmarks.

Rosetta: Composable Native Multimodal Pretraining

To overcome severe representation overwriting and routing collapse when unified foundation models jointly learn discrete autoregressive language understanding and continuous visual diffusion, Rosetta introduces modular plug-and-play experts, a Global Shared Expert semantic bridge, and Momentum-Anchored Orthogonal Projection (MAOP) with zero memory overhead, achieving non-destructive and mutually reinforcing native multimodal pretraining.

Safety Geometry Collapse in Multimodal LLMs and Adaptive Drift Correction

From a representation-geometric lens, this work reveals that modality-induced drift under multimodal inputs compresses the usable separation along the refusal direction (Safety Geometry Collapse), and proposes ReGap—a training-free inference-time intervention that adaptively neutralizes modality drift guided by forward self-rectification dynamics to recover safety without degrading general multimodal capabilities.

SALART-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images

SalArt-VQA introduces a fine-grained diagnostic benchmark comprising 950 images and 3,681 human-authored multiple-choice questions that decomposes artifact analysis into a four-stage chain—presence detection, semantic localization, bounding box grounding, and evidence description—coupled with matched real photographs and paired unedited generated reference controls, exposing that high image-level artifact detection recall frequently masks severe localization and evidence failures as well as an acute sensitivity-calibration trade-off across 20 leading VLMs.

Scaling Laws for Multimodal Data Mixtures

For natively unified multimodal pretraining across text, vision, and audio, this paper derives a unified scaling-law framework based on sparse Mixture-of-Experts (MoE) architectures that decouples capacity dilution, repetition saturation, and directional cross-modal interference, revealing that audio favors parameter scaling over tokens, non-text data saturates after two epochs, and vision acts as the primary interference source against audio.

See it to Place it: Evolving Macro Placements with Vision Language Models

VeoPlace introduces a training-free visual evolutionary framework that utilizes the spatial reasoning and in-context learning of vision-language models to propose macro bounding regions, guiding learning-based policies via action masking and analytical solvers via soft anchor constraints to achieve up to 32% wirelength reduction and superior post-route PPA.

Seeing Is Not Screening: Multimodal Hidden Instruction Attacks on Agent Skill Scanners

This paper exposes a critical security blind spot where existing agent skill scanners rely almost exclusively on text and source code while ignoring bundled visual resources; it introduces SkillCamo, an attack that embeds operational malicious payloads into images with naturally rewritten documentation, and ExecScan, an execution-grounded multimodal scanning framework that recovers hidden directives and simulates runtime action chains to eliminate the threat.

SemMSA: Latent Semantic-Aided Robust Multimodal Sentiment Analysis with Incomplete Data

SemMSA feeds incomplete language, visual, and acoustic evidence into a frozen LLM, recursively appends continuous hidden states as auxiliary semantics, and learns robust representations through within-instance kernel spectral alignment and cross-instance separation, achieving average Acc-2 scores of 74.36/73.91 on MOSI, 79.61/79.38 on MOSEI, and 75.46 on SIMS across ten intra-modal missing rates.

SenseBench: A Benchmark for Remote Sensing Low-Level Visual Perception and Description in Large Vision-Language Models

SenseBench establishes the first comprehensive diagnostic benchmark spanning 6 major categories and 22 fine-grained degradation types across 10,815 instances, systematically evaluating both objective low-level visual perception and subjective diagnostic description capabilities of 29 vision-language models on remote sensing imagery.

Simple yet Effective Semi-supervised Knowledge Distillation from Vision-Language Models via Dual-Head Optimization

To tackle severe gradient conflicts between supervised and distillation losses in semi-supervised VLM knowledge distillation under few-shot scenarios, Dual-Head Optimization (DHO) decouples the prediction heads and performs lightweight inference-time interpolation, achieving retraining-free hyperparameter tuning and consistently outperforming single-head baselines and teacher models.

SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows

SLVR establishes a structured latent visual reasoning framework that first induces strict visual dependency by contrasting answers against image-grounded distractors, and then decomposes continuous reasoning into typed latent stages for planning, grounding, evidence selection, and integration, delivering substantial gains across fine-grained benchmarks without textual CoT decoding latency.

SMMBench: A Benchmark for Source-Distributed Multimodal Agent Memory

Addressing the limitation of existing benchmarks confined to pre-assembled single contexts, SMMBench establishes the first source-distributed multimodal agent memory benchmark across 264 independent sources and 1,877 instances, evaluating cross-source evidence composition, conflict resolution, and action generation while revealing major bottlenecks in current systems.

Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests

Spatial-IQ decomposes 3D stacked object counting into 9 perceptual-cognitive sub-tasks across developmental stages, exposing pervasive shortcut behaviors in multimodal LLMs and demonstrating that hierarchical chain-of-thought supervision with verifiable-reward RL significantly improves grounded spatial reasoning.

Spherical Interpolation for Backward-Compatible Multimodal Representations

Without rebuilding the old gallery index, Procrustes first aligns new-model queries to the old space, then spherical interpolation combines them with old-model queries for the same input; endpoint complementarity accounts for most average retrieval gains, while support-set weight selection further improves compatibility success rates.

Squeezing Capacity from Multimodal Large Language Models for Subject-Driven Generation

To resolve the multimodal reasoning deficit and severe copy-paste artifacts in existing subject-driven image generation, this paper introduces a Dual Layer Aggregation (DLA) module that decouples layerwise feature pooling across MLLM layers, harmonized with VAE high-frequency identity details via a two-stage training scheme and multi-stage timestep-aware denoising.

STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation

STARFlow2 introduces the Pretzel architecture to vertically interleave a frozen VLM stream with an autoregressive normalizing flow (TARFlow) stream, enabling high-fidelity continuous image generation and multimodal understanding under a unified causal mask and shared KV-cache without visual re-encoding.

Steering Visual Generation in Unified Multimodal Models with Understanding Supervision

Tsinghua University and Kuaishou propose Understanding-Oriented Post-Training (UNO), a lightweight framework that freezes the understanding expert to provide dual supervision—via linguistic recaptioning and MetaQuery-based dense visual feature regression—over intermediate noised representations, substantially boosting image generation and editing quality with zero inference overhead.

SynDORBench: Evaluating LVLM Perceptual Robustness Under Physically Constrained Visibility Conditions

SynDORBench is the first physically grounded multimodal benchmark calibrated to the BS EN 62676-4 DORI standard, evaluating 18 models across 54k QA pairs under controlled variations in distance, illumination, and viewpoint, revealing that pixel density and imaging physics govern perceptual breakdown rather than model parameter scale alone.

SynIB: Informational Bottleneck for Maximizing Synergy in Multimodal Learning

SynIB penalizes confident predictions under modality-corrupted counterfactual passes to force fusion networks away from unimodal shortcuts and maximize synergistic cross-modal information, yielding consistent gains across MultiBench, Hateful Memes, and a controllable CREMA-D irony benchmark.

TasteBench: multimodal benchmark for sensory prediction, from molecules to sustainable foods

TasteBench introduces the first multimodal sensory prediction benchmark bridging microscopic flavor molecules to macroscopic consumer foods, evaluating within-category similarity ranking across 215 plant-based products against 21K+ human evaluations and 15K flavor molecules, where an ensemble combining multimodal LLMs with physical features achieves 0.683 pairwise accuracy, exceeding the median untrained human panelist (0.650).

Text as Partial Constraint: Core–Residual Alignment for Robust Vision–Language Learning

Reframing natural text as a partial constraint on visual instances, TPC aligns image embeddings to a multi-view consensus semantic core while suppressing orthogonal unsaid residuals, significantly improving zero-shot robustness and reducing LVLM hallucinations under single-query deployment.

The Alignment Illusion in Multimodal Large Language Models

Using norm-matched noise and irrelevant images in 13 multimodal large language models, this paper shows that shared MLP weights can produce high similarity and introduces the principal-angle gap to distinguish one-directional collapse from multi-directional structure, while emphasizing that geometry alone cannot establish the use of question-relevant visual content.

The Design Space of Tri-Modal Masked Diffusion Models

This paper introduces the first unified tri-modal discrete masked diffusion model (MDM) pretrained from scratch across text, image, and audio, eliminates optimal batch-size tuning via stochastic differential equation (SDE) reparameterization, and derives empirical compute-optimal scaling laws showing asymptotic data efficiency.

The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric

Addressing the limitation of classic perceptual metrics (such as LPIPS and DreamSim) that collapse visual similarity into an ambiguous scalar, this paper introduces a dataset of 1 million human relative perceptual judgments across free-form visual aspects and proposes TPIPS, a text-prompted perceptual metric that significantly narrows the gap to human consensus.

The Value of Being Wrong: Self-Mined Visual In-Context Learning from Errors

Addressing the brittle and complex pipelines of existing counterfactual demonstration selection, SMILE mines a target VLM's own prediction errors offline as natural counterfactuals and uses zero-shot predictions to guide semantic negative retrieval online, delivering an average gain of +3.09 points over SOTA across four benchmarks at \(5\times\) lower latency.

THEIA: A Multimodal Dataset and Benchmark for Vision-Language Analysis of Layout

Addressing the long-standing reliance on manual verification and rigid scripts in analog IC physical back-ends, this paper introduces THEIA—the first multimodal dataset and benchmark pairing 30,034 DRC/LVS-verified GDSII physical layouts with 77,354 multi-tier Q&A conversations—demonstrating that fine-tuned VLMs achieve 81%–100% accuracy on complex layout reasoning, outperforming off-the-shelf general-purpose VLMs by up to 73 percentage points.

Topology-Aware Representation Alignment for Semi-Supervised Vision-Language Learning

Addressing the issue where semi-supervised vision-language adaptation with scarce paired data relies strictly on pairwise alignment and disrupts global representation manifold structures, this paper proposes Topology-Aware Multimodal Representation Alignment (ToMA), which decomposes relations into \(H_0\)-death connectivity backbones and \(H_1\)-birth loop structures via 1-skeleton persistent homology without constructing 2-simplices, aligning cross-modal edge directions to significantly enhance zero-shot classification accuracy and hard-query retrieval performance.

TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-training

Presents TorchUMM, the first standardized codebase spanning evaluation, mechanistic analysis, and post-training across 15 unified multimodal models, 12 benchmark suites, and 5 post-training methods, demonstrating that strong architectural unification does not guarantee representational unification.

Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agents

This paper introduces a visual-native agent harness supporting persistent intermediate visual state reuse and an On-policy Data Evolution (ODE) closed-loop data generation framework, enabling tools to consume intermediate visual outputs while steering task synthesis with policy rollout feedback, boosting Qwen3-VL across 8 deep search benchmarks and outperforming Gemini-2.5 Pro.

TRACER: Verifiable Generative Provenance for Multimodal Tool-Using Agents

Addressing the "provenance gap" where multimodal tool-using agents lack explicit support links between generated claims and intermediate tool observations, TRACER co-generates structured provenance records alongside sentence-level claims and uses verifiable checks plus local credit reinforcement learning to achieve 78.23% task accuracy while reducing test-set tool calls by 29.56%.

Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context

This paper systematically investigates multimodal long-context continued pre-training (LongPT) recipes, introducing MMProLong which extends a 7B LVLM from 32K to 128K context window using only 5B tokens and generalizes robustly up to 512K and long video benchmarks.

TRIAD: Benchmarking Omni-Modal Ambiguity in Multimodal Large Language Models

Introduces TRIAD, the first diagnostic benchmark for strict tri-modal mutual disambiguation across text, vision, and audio, revealing that current leading omni-modal LLMs collapse to single-digit group accuracy once option scaffolding is removed and frequently rely on textual shortcuts while neglecting acoustic prosody.

Triggering Generalist Reasoning via Predictive Uncertainty for Dual-System VLA

To address the severe compute waste and delayed reaction inherent in fixed-interval generalist invocation for dual-system Vision-Language-Action (VLA) models, this paper proposes TUD, an adaptive inference framework that uses cross-step dispersion of future action-plan re-predictions as a predictive uncertainty signal, triggering deliberative reasoning without auxiliary models or manual labels while significantly cutting VLM calls and boosting success on both simulation and a real robot.

Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets

Argus compares uncertainty methods using unified single-step GUI click records under different observable interfaces, finding stronger cross-dataset method-ranking transfer at a fixed model than across models or from open weights to API-only systems, while error discrimination cannot replace probability calibration or spatial coverage checks.

Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volume

To tackle confabulation errors where Multimodal Large Language Models (MLLMs) produce plausible yet ungrounded responses, this paper proposes UMPIRE, a training-free framework that quantifies uncertainty via an incoherence-adjusted semantic volume over sampled responses without external tools, simultaneously capturing global semantic diversity and local model confidence across image, audio, and video modalities.

Uncovering and Shaping the Latent Representation of 3D Scene Topology in Vision-Language Models

This paper demonstrates that vision-language models (VLMs) internally possess a 3D topology-preserving cognitive map overshadowed by non-geometric visual semantics (e.g., color and shape); by extracting the clean spatial subspace via cross-scene averaging and applying a Dirichlet-energy regularizer on middle layers, a minimal 500-step fine-tuning yields up to a 12.1% performance gain on real-world 3D spatial reasoning benchmarks.

Unlocking Fine-Grained Perception in CLIP via Structurally-Aware Latent Masked Modeling

Addressing the fundamental bottleneck where CLIP loses local spatial and geometric details due to its contrastive focus on global semantic consistency, this work introduces SALM and its self-distillation variant SALM-Self, utilizing second-order dual-matrix spatial alignment and cross-guided latent masked modeling to significantly enhance dense prediction and MLLM spatial perception while preserving zero-shot capabilities.

V-GIFT: Boosting Visual Instruction Tuning with Self-Supervised Guidance

V-GIFT reformulates classic self-supervised pretext tasks into standard natural language instruction-following triplets, injecting a modest 3%–10% of automatically synthesized samples into visual instruction tuning to enforce reliance on visual evidence and eliminate language shortcuts without auxiliary losses or architectural modifications.

ViCO: A Training Strategy towards Semantic Aware Dynamic High-Resolution

ViCO proposes Visual Consistency Learning and a lightweight Visual Resolution Router (ViR) that dynamically allocates compression rates to image patches based on semantic complexity, reducing vision tokens by up to 50% with negligible performance drop and doubling first-token inference throughput.

VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents

VisGym introduces 17 diverse, long-horizon visually interactive environments with fine-grained parameterization and heuristic multi-step oracle solvers, providing controlled cross-domain diagnoses of 12 frontier VLMs that reveal long-context reverse-scaling and visual grounding bottlenecks, while substantially boosting generalization via information-revealing SFT demonstrations.

VistaQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence

Introduces VistaQA, a benchmark for jointly evaluating free-form textual answer correctness and pixel-level supporting evidence masks, along with Grove, an uncompensable geometric-mean joint metric, exposing a severe modality gap in aligning reasoning and visual evidence among frontier multimodal models.

Visual-Advantage On-Policy Distillation for Vision-Language Models

Addressing the issue where standard on-policy distillation with uniform KL loss dilutes critical visual supervision signals in VLMs, this paper proposes visual advantage counterfactual probing and two-level on-policy distillation (VA-OPD) to reweight trajectories and group tokens, significantly boosting visual reliance and reasoning.

Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models

To tackle the issue where parallel decoding in discrete diffusion-based MLLMs commits tokens with overlapping visual grounding, this paper proposes VRCD, a training-free inference-time decoding method that reranks positions using cross-attention residual saliency, significantly lowering predictive entropy and enhancing multimodal reasoning.

Visual Grounding First, Multimodal In-context Learning Follows

Addressing the issue where multimodal in-context learning degrades due to weak visual grounding and shortcut textual over-reliance, this paper introduces ProCoRe, a proposition-level contrastive reinforcement learning framework trained without ICL-formatted data that substantially boosts cross-task generalization, alongside the PICL benchmark for evaluating genuine visual concept binding.

Visual Instruction Tuning Aligns Modalities through Abstraction

Reveals that visual instruction tuning in Vision-Language Models primarily aligns visual features directly into the intermediate semantic abstraction layers of the LLM backbone, and demonstrates that restricting fine-tuning exclusively to these intermediate layers preserves full fine-tuning performance while slashing training time by up to 24%.

VisualWorldBench: A Fine-Grained Multi-Task Benchmark for Evaluating Visual World Knowledge in MLLMs

VisualWorldBench establishes a hierarchical taxonomy of 21,291 leaf-node classes across 8 visual domains and an evaluation suite of 32,541 instances covering 15,479 ground-truth categories across four complementary tasks (closed-set recognition, contrastive selection, precise localization, and open-world naming), systematically exposing and mitigating sharp capability dissociations in MLLMs' fine-grained visual entity knowledge.

VLA-Corrector: Lightweight Detect-and-Correct Inference for Adaptive Action Horizon

Addressing error accumulation within open-loop blind spots of action-chunked Vision-Language-Action (VLA) models, VLA-Corrector introduces a lightweight inference-time detect-and-correct framework that monitors latent visual dynamics deviations, event-triggers action truncation, and guides flow-matching recovery via Online Gradient Guidance without retraining the VLA backbone.

Wasserstein Gradient Flows and Forward-Only Diffusion Are Not Enough for Multimodal Sampling

By establishing the marginal density equivalence between Wasserstein gradient flows and forward-only diffusion through the JKO scheme and Otto calculus, this paper demonstrates that purely local gradient-driven sampling provably suffers from exponential mixing times in multimodal settings, a bottleneck that standard log-linear annealing cannot resolve.

When Are Multimodal Predictions Biologically Supported? A Diagnostic Evaluation Framework

This paper introduces DECAT, a model-agnostic post-hoc evaluation framework that uses frozen representations and five permutation null-calibrated metrics to diagnose whether multimodal predictions are driven by shared biology, modality-specific biology, or spurious confounders without requiring confounder labels.

When Language Overrules: Revealing Text Dominance in Multimodal Large Language Models

This paper provides the first systematic cross-modal investigation revealing pervasive "text dominance" across images, videos, audio, time-series, and graphs in MLLMs, introduces the Modality Dominance Index (MDI) and Attention Efficiency Index (AEI), proves that attention dilution stems from non-text token redundancy, and demonstrates that training-free token compression rebalances cross-modal attention.

When to Align, When to Predict: A Phase Diagram for Multimodal Learning

Addressing the lack of theoretical guidelines for choosing between cross-modal alignment (CA) and prediction (CP), this paper derives closed-form separation ratios under a spiked model with structured cross-modal nuisance correlation, establishes a four-region phase diagram, and provides a lightweight pre-training diagnostic tool.

Where Do Long Captions Fail? Position-Aware Diagnosis and Reinforcement Learning for Detailed Image Captioning

Addressing the position-dependent failures where autoregressive vision-language models suffer from mid-caption unsupported fine-grained specificity and tail generic escape in long captioning, this work introduces a position-aware rubric diagnostic framework alongside PAGER reinforcement learning, achieving targeted error repair and superior grounded quality.

Who Says What: Symbolic Trimodal Binding Mechanisms in Audio-Visual LLMs

Through representational analysis and activation interventions on controlled successful trajectories, this paper explains how audio-visual LLMs bind “who says what” through temporal and position IDs, then uses an existing active speaker detector for visual prompting and optional lightweight fine-tuning to improve conversation understanding, with training-free gains depending on visual-marker grounding capabilities.

WorldMemArena: Evaluating Multimodal Agent Memory Through Action–World Interaction

Introduces WorldMemArena, a benchmark that formulates multimodal agent memory as an Action–World Interaction Loop with an observable four-stage lifecycle (Write, Maintain, Retrieve, and Use), enabling the first fine-grained diagnostic comparison across long-context, manually designed, and harness-based memory paradigms on 461 multi-session tasks.