🧩 Multimodal VLM

🎞️ ECCV2026 · 225 paper notes

📌 Same area in other venues: 🧠 NeurIPS2026 (2) · 📷 CVPR2026 (419) · 🔬 ICLR2026 (211) · 💬 ACL2026 (82) · 🧪 ICML2026 (89) · 🤖 AAAI2026 (75)

🔥 Top topics: Multimodal/VLM ×109 · Alignment/RLHF ×14 · Adversarial Robustness ×10 · LLM ×10 · Few-/Zero-Shot Learning ×10

ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models

ViewSpatial-Bench diagnoses perspective-taking failures through five camera- and person-centered spatial localization tasks, while approximately 43K targeted supervision samples raise Qwen2.5-VL-3B accuracy from 35.85% to 82.09%, a gain of 46.24 percentage points rather than a 46.24% relative improvement.

Visual Spatial Tuning

Without adding a specialized 3D encoder, VST first teaches a VLM spatial perception through multimodal geometric data, then develops spatial reasoning with BEV-assisted textual reconstruction traces and reinforcement learning, raising Qwen2.5-VL-7B from 25.9 to 34.8 on MMSI-Bench and from 38.9 to 61.2 on VSI-Bench.

Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedback

Render-in-the-Loop lets Qwen3-VL-8B observe the cumulative canvas after each SVG fragment, combining specialized multi-turn supervised training with inference-time repetition filtering to outperform several autoregressive baselines trained on larger datasets on MMSVGBench using 0.85M training samples, at approximately 1.50 times the inference latency for VSF.

Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild

Real5-OmniDocBench reconstructs all 1,355 OmniDocBench v1.5 pages under five physical acquisition scenarios, using 6,775 content-matched images to test parsing robustness; digital-document proficiency does not guarantee physical-world reliability, and the 0.9B PaddleOCR-VL-1.6 leads the listed large generalist models with an Overall score of 93.19 in this evaluation.

Towards Interactive Global Geolocation Assistant

MG-Geo trains GaGA with geographic metadata, image-grounded clues, and multi-turn reasoning dialogs to turn one-shot street-view coordinate prediction into an interaction that accepts factual constraints, achieving 63.06% country accuracy and 6.28% city accuracy on GWS15k without uniformly leading at fine-grained coordinate localization.

How to Teach Large Multimodal Models New Skills

The paper attributes part of multimodal-model forgetting to task-induced response bias and shows that tuning only attention projections or MLP Gate&Up yields current-task gains of +24.9 / +30.5 percentage points on LLaVA-OneVision across five sequential tasks, while limiting held-out changes to -0.6 / -2.1 percentage points.

MMR-Bench: A Comprehensive Benchmark for Multimodal LLM Routing

MMR-Bench provides reusable offline evaluation for selecting among heterogeneous multimodal large language models on image-text queries, comparing cost-performance curves across features and routing policies; the authors report an operating point that matches the strongest single model at approximately 33% of its cost, not a uniform saving across all methods and tasks.

UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameters

UniRec-0.1B trains a 0.1B encoder-decoder from scratch using approximately 40 million multi-level samples, explicit line and paragraph supervision, and decoupled text/formula tokenization, achieving an average edit distance of 0.100 on UniRec-Bench and an average block recognition time of 0.37 seconds under the paper's specified single-GPU conditions.

DiaDem: Advancing Dialogue Descriptions in Audiovisual Video Captioning for Multimodal Large Language Models

DiaDem combines dialogue-aware synthetic data and difficulty-partitioned GRPO on AVoCaDO-7B to make complete audiovisual captions more accurate about "who said what," achieving REF 65.9 and ASR 79.3 on DiaDemBench while retaining competitive general caption quality.

OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models

OmniMamba shares an autoregressive Mamba-2 backbone while decoupling visual encoding, output vocabularies, and task adapters, achieving 5.50 FID-30K on COCO with approximately 2M image-text pairs as reported for unified-model training, a data accounting boundary that excludes inherited pretrained components.

UEval: A Benchmark for Unified Multimodal Generation

UEval uses 1,000 questions requiring both images and text and 10,417 human-validated rubric criteria to test whether unified models can answer jointly across modalities; GPT-5-Thinking scores only 66.4 overall, while the strongest open-source model, Emu3.5, scores 49.1.

DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Models

DiffusionVL preserves a strong autoregressive backbone and generates visual question-answering text through block diffusion fine-tuning and cached block decoding, reaching 25.0 on MMMU-Pro vision with only 738K additional fine-tuning samples for its 7B variant, although speed and quality gains depend on the task and decoding configuration.

MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding

MinerU-Diffusion treats document OCR as inverse rendering from images to structured symbols, combining autoregression across blocks, diffusion within blocks, and a two-stage curriculum to reach 93.37 Overall on OmniDocBench v1.5 with ground-truth layout and approximately 2.1 times faster decoding at its default threshold.

Personalize Your Large Vision-language Models With In-context Prompt Tuning

ICPT uses a shared Adaptive Concept Projector to compress multiple reference images into label anchors and variable-length visual prompts, with contextual debiasing and cross-concept separation enabling immediate personalization of a frozen LVLM; on LLaVA-NeXT-7B, weighted existence-recognition recall rises from MC-LLaVA's 0.811 to 0.868.

StarDojo: Benchmarking Open-Ended Behaviors of Agentic Multimodal LLMs in Production–Living Simulations with Stardew Valley

This paper introduces StarDojo, the first open-world MLLM agent benchmark that evaluates coupled "production activities" and "social life" based on Stardew Valley. Featuring 1,000 human-annotated tasks, the strongest model (GPT-4.1) achieves a success rate of only 12.7%, revealing severe bottlenecks of current MLLMs in visual perception, multimodal reasoning, and low-level control.

UniTac: A Unified Multimodal Model for Cross-Sensor Tactile Understanding and Generation

UniTac is the first unified multimodal large model for touch that integrates both "understanding" and "generation" into a single framework. By modeling dual-level information of both "object-level semantics" and "sensor-level configurations", it can infer physical properties of objects and synthesize realistic tactile signals across heterogeneous tactile sensors including GelSight, DIGIT, and Duragel.

VisNec: Measuring and Leveraging Visual Necessity for Multimodal Instruction Tuning

VisNec quantifies the dependence of each training sample on visual inputs by comparing the cross-entropy loss difference of a multimodal model between "with-image vs. without-image" forward passes. This enables the selection of a high-value subset that truly requires cross-modal reasoning, recovering or even exceeding full-dataset training performance with only 15% of the LLaVA-665K data.

Magic-MM-Embedding: Towards Visual-Token-Efficient Universal Multimodal Embedding with MLLMs

Magic-MM-Embedding introduces a parameter-free visual token compression mechanism (reducing tokens to 25%) based on InternVL3, coupled with a three-stage progressive training pipeline (generative recovery \(\rightarrow\) contrastive pre-training \(\rightarrow\) MLLM judge fine-tuning). This approach not only overcomes the performance degradation caused by compression but also completely outperforms prior SOTA models on MMEB, VisDoc, and cross-modal retrieval, while significantly reducing inference latency.

PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Grounding

This work utilizes "rendered panoramas" as an intermediate representation between 2D and 3D. This allows a pre-trained 2D VLM (CogVLM-17B) to directly output 2D bounding boxes on a single 360° panorama, which are then lifted and fused into 3D bounding boxes via multi-view predictions. By doing so, the strong language reasoning capabilities of 2D VLMs are successfully transferred to 3D visual grounding, achieving SOTA results on ScanRefer and Nr3D while demonstrating robust generalization to unseen scenes and paraphrased texts.

StrucTab: A Structured Optimization Framework for Table Parsing

StrucTab trains a 1B table parser to count rows and columns, identify merged cells, and then generate HTML, using Uni-TabRL to target the first structural error and normalize content-expression styles, improving TableVerse-5K TEDS from 84.59 for its SFT model to 87.81.

Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs

UG turns MLLM response entropy or affirmative confidence into a visual evidence selection signal for local search, video sampling, and temporal grounding without updating parameters, improving InternVL2.5-8B V*Bench accuracy from 71.7% to 91.1% while requiring additional candidate-scoring computation.

360° Image Perception with MLLMs: A Comprehensive Benchmark and a Training-Free Method

360Bench exposes failures in fine-grained and spatial understanding of high-resolution panoramas, and training-free Free360 combines question-relevant entities, local attributes, inter-entity relations, and viewer directions into a scene graph, raising Qwen2.5-VL-7B overall accuracy from 38.1% with CMP input to 45.3%.

Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs

Targeting two failures of multimodal large language models (MLLMs) in gastrointestinal endoscopy — reasoning that ignores the expert cognitive pathway and diagnoses driven by spurious background correlations — this paper curates a dataset whose reasoning chains follow an explicit "anatomical localization → morphological evaluation → micro-detail analysis" hierarchy, internalizes it with SFT, formally shows that standard supervised tuning necessarily converges to background shortcuts, and corrects the bias with a counterfactual-driven GRPO that synthesizes lesion-masked normal samples and optimizes clinical-cognition-centric rewards, reaching 67.67% average accuracy across five endoscopy benchmarks and surpassing Qwen3-VL-Plus by roughly 26 points.

Entropy-Gradient Grounding: Training-Free Evidence Retrieval in Vision-Language Models

Backpropagating first-token predictive entropy to visual embeddings retrieves multiple image regions for adaptive zooming without updating model weights, raising Qwen2.5-VL 7B V* accuracy from 73.30 to 86.91 at the cost of additional forward and backward computation.

Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents

This work proposes a failure-driven inference-time self-improvement loop: an LLM acts as a meta-controller to analyze the failure trajectories accumulated by an agent in GUI tasks, identifying four categories of bottlenecks: grounding errors, capability gaps, insufficient knowledge, and redundant loops. It designs corresponding inference-time optimization strategies for each (visual search / terminal execution / knowledge support / repetition warnings), which are injected into the agent workflow as code patches. This approach improves the success rate of OpenCUA-72B on OSWorld from \(42.3\%\) to \(48.9\%\) (\(+6.6\) points) with zero training overhead, requiring only a \(\sim 8\%\) increase in inference computation.

LiveK12Bench: Have Large Multimodal Models Truly Conquered High School-level Examinations?

LiveK12Bench combines continuously refreshed authentic high-school papers with evaluation of answers, reasoning processes, efficiency, and full-page visual inputs, showing that strong question-answering models are not necessarily reliable exam takers: its complex-layout table reports GPT-5 accuracy falling from 82.7% with standard inputs to 50.4% with full-page inputs, although the paired-sample basis of that table requires clarification.

Skin-R1: Clinical Knowledge-Guided Dermatological Diagnosis Using Vision-Language Models

Skin-R1 proposes a three-stage training framework: first, it automatically constructs a hierarchical perception and differential diagnosis reasoning trajectory dataset, SkinRationale, from authoritative dermatology textbooks; next, Supervised Fine-Tuning (SFT) is utilized to initialize the model's clinical reasoning capabilities; finally, Group Relative Policy Optimization (GRPO) reinforcement learning, paired with a hierarchical reward design, generalizes the reasoning capability to large-scale, sparsely annotated datasets, comprehensively outperforming existing Med-VLMs across multiple dermatological diagnosis benchmarks.

Towards Trustworthy Dermatology MLLMs: A Benchmark and Multimodal Evaluator for Diagnostic Narratives

This paper builds a clinically grounded evaluation infrastructure for image-to-diagnostic-narrative generation in dermatology: DermBench pairs 4000 DermNet images with dermatologist-certified reference narratives and has a DeepSeek-R1 judge score candidates on six clinical dimensions from 0 to 5, while DermEval is a reference-free multimodal evaluator built on LLaVA v1.5-7B and reinforcement-aligned to physician scores, producing a structured critique plus six scores from an image and a narrative alone; their mean absolute errors against expert annotations are 0.363 and 0.178 out of 5, and they rank models almost identically.

UniDDT: Unifying Multimodal Understanding and Generation with Decoupled Diffusion Transformer

UniDDT unifies the semantic encoding of understanding and generation behind a single Noisy ViT encoder plus an LLM backbone, decouples diffusion decoding from text decoding with a separate diffusion decoder, and turns the same image-text pairs into dual-format training data for joint training and duality-based post-training — reaching 1699.5 on MME, 0.87 on GenEval and 86.9 on DPG.

VersaViT: Enhancing MLLM Vision Backbones via Task-Guided Optimization

VersaViT jointly post-trains one MLLM vision backbone with language, geometry, and pixel-localization supervision, raising Qwen2-VL-ViT's frozen ADE20k linear-probing performance from 33.6 to 49.6 mIoU while improving the OpenCompass average from 64.8 to 66.4 under a shared evaluation configuration.

VisCoP: Visual Probing for Video Domain Adaptation of Vision Language Models

VisCoP adds a layer-wise probing branch beside a frozen vision encoder to adapt vision-language models to new viewpoints, modalities and control tasks, raising the depth-video target average from 45.86 to 65.13 while increasing the RGB source average from 72.79 to 74.63.

AutoV: Loss-Oriented Ranking for Visual Prompt Retrieval in LVLMs

Instead of using one visual prompt for every question, AutoV labels candidate preferences with a frozen LVLM's reference-answer losses and trains a lightweight instance-level ranker, raising LLaVA-OneVision 7B's VizWiz score from 58.2 to 68.4 while allowing the selection strategy to transfer to other models.

Cheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation

Cheers routes compressed visual semantics through a shared language backbone and injects patch details according to generation progress through a cascaded flow matching head, achieving 0.78 on GenEval with a 1.5B language backbone and 83M training samples while retaining competitive image understanding.

Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs

CaT organizes user concepts into editable trees, generates positive, easy-negative, and hard-negative samples, and retains concept-relevant information through perturbation-based filtering; with an increased positive-sample budget, MC-LLaVA recognition accuracy on its namesake dataset rises from 0.917 to 0.963.

Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMs

DDE freezes CLIP, calibrates classification with class inclusion/exclusion Gaussians, and selects negative labels through positive-negative image response differences, improving both classification and rejection in zero-shot noisy test streams and raising average ImageNet harmonic accuracy from AdaND's 73.09% to 76.79%.

EpiBench: Benchmarking Multi-turn Research Workflows for Multimodal Agents

EpiBench instantiates a short literature study as a multi-turn question episode: the task provides no target paper and only an image snippet or a short bibliographic cue, so the agent must search out several papers on its own, extract figures and tables as evidence, append that evidence to an episode-level memory pool, and perform cross-paper multi-evidence fusion in a final turn where all tools are disabled; beyond final correctness, the protocol scores evidence correctness (EC) and the minimality gap (MG) so that "did it reach the right evidence" and "did it get there efficiently" are measured separately.

FingerCap: Fine-grained Finger-level Hand Motion Captioning

FingerCap establishes finger-level captioning using gesture and hand-object interaction videos, and injects dense 2D hand poses into a sparse-RGB video model through FiGOP, improving the same fine-tuned Qwen3-VL-8B baseline from 107.28 to 155.27 CIDEr and from 2.13 to 2.74 HandJudge.

Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translation

GLoTran recasts high-resolution text-rich image translation as "detect text regions, cut them into local slices in reading order, then let an MLLM translate region by region under the contextual reference of a low-resolution global image," injecting global semantics into local tokens through early-layer cross-attention with source-type and spatial biases and keeping regions consistent via an \(\eta\)-window translation replay and a four-part structured prompt; it lifts average BLEU on MCiTon English-to-Chinese translation from 46.7 (Qwen3-VL 8B) to 50.2.

LaVPR: Benchmarking Language and Vision for Place Recognition

LaVPR uses Gemini 2.5 Flash to attach 651,865 dense natural-language descriptions to established VPR datasets (GSV-Cities, Pitts30K, MSLS, AmsterTime) and turns both language-vision late fusion and text-to-image "blind" localization into a controlled benchmark on identical visual backbones and original geographic splits: language reliably recovers accuracy on visually degraded subsets (a compact backbone plus one language channel matches much larger vision-only models), while cross-modal localization only becomes viable with LoRA plus Multi-Similarity fine-tuning, lifting R@1 from under 3% zero-shot to 12-38%.

OpenSpatial: A Principled Data Engine for Empowering Spatial Intelligence

OpenSpatial turns "how to build spatial training data" into an open, reproducible data engine: it uses world-coordinate oriented 3D bounding boxes (OBBs) as a unified primitive, produces scene-level boxes through two complementary routes (human annotation and automated 3D lifting), converts them into an object–frame index via projection, filtering and mask refinement, and finally enumerates scene graphs into single-view and multi-view QA, yielding the 3M-sample OpenSpatial-3M dataset that lifts several open-source VLMs by 14.1% on average (up to 19%) across BLINK, AllAngles, VSI and MMSI while largely preserving general multimodal ability.

ProactiveBench: Benchmarking Proactiveness in Multimodal Large Language Models

This paper proposes ProactiveBench, the first benchmark to evaluate the "proactiveness" of Multimodal Large Language Models (MLLMs). By adapting 7 existing datasets into multi-turn interactive environments where "the initial frame is blurry and user intervention is required to find the correct answer," the authors evaluate 22 MLLMs. The results reveal that they rarely seek help proactively (either abstaining or hallucinating) and that proactiveness does not emerge with model scale. However, fine-tuning via GRPO with customized rewards can teach models to be proactive and generalize to unseen scenarios.

RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning

RSICCLLM is the first post-training framework for remote sensing image change captioning (RSICC). It first uses binary change masks as geometric priors to automatically generate a large-scale instruction dataset (RSICI) utilizing Qwen-VL-Max. It then explicitly injects change representations through difference-aware supervised fine-tuning (CDC geometric texture extraction + Hough voting + Cross-Attention fusion). Finally, it refines output quality via double negative preference optimization (DNPO). With only 7B parameters, it surpasses several hundred-billion-parameter general models (such as Qwen3-VL-235B and InternVL-3.5-241B) and specialized RSICC models across all 8 evaluation metrics.

T-REN: Learning Text-Aligned Region Tokens Improves Dense Vision-Language Alignment and Scalability

T-REN learns multiple regions per point and joint region-text alignment on frozen DINOv3/dino.txt, reaching 30.6 mIoU on ADE20K open-vocabulary segmentation and, in a separate video localization setting, 52.4% Recall@1 with 187.5-fold representation compression.

UrbanAlign: Post-hoc Semantic Calibration for VLM-Human Preference Alignment

UrbanAlign uses a frozen vision-language model to extract interpretable urban concepts and calibrates them against similar human-labeled image pairs, achieving 70.8% average accuracy across six Place Pulse 2.0 categories, 14.1 percentage points above zero-shot GPT-4o.

ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering

ViTexQA admits questions only when answering requires integrating text across frames, while FrameThinker learns explicit perception through spatiotemporal evidence supervision and reinforcement learning, achieving 73.5 ROUGE-L without ViTexQA training data and 83.5 with that data in the full configuration.

Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO

CycleGRPO trains one multimodal model to describe a region and then localize its own description, using reconstructed-mask overlap to improve both directions without caption ground truths in its main training loop; SAMTok-4B improves from 61.9 to 67.7 on DLC-Bench and from 57.5 to 67.6 overall mask gIoU on GroundingSuite.

ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs

ADAPT distills semantically aligned visual anchors from the multi-layer cross-attention of early decoding steps. During inference, it uses these anchors as references to monitor attention fidelity online and applies sparse corrections only when significant degradation is detected. Additionally, it strengthens the model's reliance on visual evidence through visually-conditioned contrastive preference optimization, reducing hallucination rates by 40–60% on multiple hallucination benchmarks with only a 1.42x inference overhead.

Art Beyond Semantics: Sheaf-Informed Contrastive Learning for Multi-Relational Representations

CANVAS modulates each artwork or text into relation-specific representations and uses a training graph's line-graph heat kernel to soften false-negative penalties, achieving the strongest image-to-text retrieval results across three art datasets without requiring full graph connectivity at inference.

Beyond Language: Grounding Referring Expressions with Hand Pointing in Egocentric Vision

The paper introduces EgoPoint-Ground with 15,338 images and SV-CoT, which separates deictic grounding into direction parsing, ray-based filtering, and semantic verification, achieving real-world EDG [email protected] of 0.818 versus Qwen3-VL-8B's 0.701, a gain of 11.7 percentage points.

Beyond Sequential Distance: Inter-Modal Distance Invariant Position Encoding

DIPE uses sequential queries for intra-modal attention and segment-anchored queries for cross-modal attention, preserving image geometry and text order while mitigating visual fading and raising the MRoPE aggregate benchmark score from 44.69 to 48.79 under 8K textual distractors.

CLIMP: Contrastive Language-Image Mamba Pretraining

CLIMP replaces both towers of CLIP with state space models — VMamba with its SS2D four-direction cross-scan for vision and a Mamba-1/Mamba-2 language model with last-token pooling for text — and trains them on CC12M with CLIP's original symmetric contrastive objective, trading quadratic complexity for linear cost while gaining better zero-shot retrieval, stronger out-of-distribution robustness (surpassing CLIP-ViT-B/16 trained on 167× more data on ImageNet-O), and native support for arbitrary resolutions and arbitrarily long text.

CoSPlan: Corrective Sequential Planning via Scene Graph Incremental Updates

The paper introduces CoSPlan, a benchmark that evaluates VLMs on corrective sequential planning in the image-plus-text setting (four tasks, each with exactly one erroneous step injected into the already-performed actions, requiring the model both to identify the error and to select the remaining actions that correct it and reach the goal), together with SGI (Scene Graph Incremental update), a training-free method that reads images into textual scene graphs, rewrites their nodes, attributes and edges action by action, then simulates each MCQ option and picks the one whose resulting graph is judged closest to the goal graph — yielding an average gain of about 4.4% on CoSPlan that also generalizes to VQA and PlanBench.

DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning

DecepGPT combines structured audiovisual report supervision with SICS and DMC to improve benchmark deception classification, achieving 73.23% accuracy on DOLOs, but these results do not establish reliable real-world lie detection.

Delineating Knowledge Boundaries for Honest Large Vision-Language Models

This paper probes a target model's own knowledge boundary with high-temperature multi-sample consistency, labels each knowledge-intensive question as mastered or unmastered, and then builds preference pairs whose chosen/rejected direction flips with that label; after SFT plus DPO/CPO/ORPO alignment, LLaVA-1.5-7B learns to refuse honestly on parametric knowledge gaps without over-refusing what it does know, lifting the Truthful rate on V-Idk from 57.9% to 67.3%.

Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams

Diagram-MMU jointly evaluates TikZ parsing, editing, and open-ended question answering across six scientific diagram domains and 12 multimodal models: Gemini-3.0 Pro reaches 86.46% question-answering accuracy but only 54.64% parsing object F1, showing that answering correctly does not imply faithful reconstruction.

EGM: Efficient Visual Grounding Language Models

EGM first distills a "how to find the target" reasoning trace for every grounding sample from a proprietary VLM for SFT, then applies GRPO with a reward mixing IoU and grounding success, letting an 8B model reach 91.4 average accuracy on RefCOCO and surpass the 235B model's 90.5 while cutting end-to-end latency from 4320ms to 737ms (5.9x).

ESC: Emotional Self-Correction for Reliable Vision-Language Models

ESC finds that prepending a hand-written emotional expression to a query makes a VLM slow down and re-walk its own reasoning, so it uses a separate verifier VLM to decide whether the initial response is suspicious, injects an emotional cue to trigger one training-free self-correction pass, and finally arbitrates between the original and the revised answer; it reduces attack success rate or improves accuracy consistently across safety, hallucination, visual perception, and multimodal reasoning benchmarks without any degradation.

From Macro to Micro: Benchmarking Microscopic Spatial Intelligence on Molecules via Vision-Language Models

This paper introduces MiSI-Bench: it decomposes "microscopic spatial intelligence" into four elementary operations (translation, rotation, zooming, interaction) and renders roughly 4,000 PDBbind crystal complexes into 163K QA pairs and 588K orthographic-projection images across nine unit and composite tasks; state-of-the-art VLMs average only 16-34 (humans: 81.18), while a 7B model fine-tuned on this training split reaches 62.96 and surpasses humans on rotation tasks, yet still lags far behind on knowledge-grounded tasks such as hydrogen-bond recognition.

From One-to-One to Many-to-Many: Dynamic Cross-Layer Injection for Deep Vision-Language Fusion

Two parameter-efficient modules — Adaptive Multi-Projection (per-layer LoRA projections) and Adaptive Gating Fusion (a gate computed from the decoding context) — turn the single-point "last vision layer to first LLM layer" connection of a VLM into a dynamic many-to-many bridge between multiple vision layers and multiple decoder layers, yielding consistent gains across 28 benchmarks on LLaVA-OneVision and LLaVA-1.5 (LLaVA-OV-7B: +9.7 overall, LLaVA-W +6.5, OCR +4.7, MME +3.3).

Generating a Paracosm for Training-Free Zero-Shot Composed Image Retrieval

Paracosm introduces the image editing capabilities of LMMs to zero-shot composed image retrieval for the first time. It directly generates a "mental image" for the multimodal query while simultaneously creating synthetic counterparts for the database images. By aligning the query and database into a unified synthetic space (paracosm) for matching, it substantially outperforms all previous zero-shot methods in a training-free manner, even rivaling supervised approaches.

Incentivizing Vision Language Models to Search for Long Video Question Answering

VSeek turns long-video question answering from "sample frames uniformly and answer in one pass" into a multi-turn natural-language retrieval process, and VETL compiles the question plus reference answer into a checkable checklist of visual primitives that supplies a dense reward for GRPO post-training, reaching up to +8% Pass@1 and +15% Pass@4 as reported by the paper on five long-video benchmarks with a visual budget of about 32 frames per query.

Learning Sample-wise Rank-Aware Interpolation Weights for Composed Visual Data Retrieval

SRAIN estimates query-specific spherical interpolation weights from within-batch target ranks, then replaces query-time multimodal large-model fusion with two-stage training and prototype-conditioned prediction, achieving 61.07% R@1 on WebVid-CoVR while retaining a substantial gap to an oracle that uses true test targets.

MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Models

MMBU uses 410 public datasets, 11 modalities (35 submodalities), 78K samples and 13 structured metadata attributes per sample to operationalize biomedical visual perception as a unified evaluation matrix of four task types (ungrounded classification, two forms of grounded classification, object detection) under open and closed answer formats; benchmarking 15 open-weight and 2 frontier VLMs shows that even the best result reaches only 0.693 F1 (closed, segmentation-grounded classification), the closed-to-open gap averages 0.26, no model beats the random baseline at object detection, and medical adaptation yields limited gains that fail to transfer to MMBU's out-of-distribution data.

Multi-label Instance-level Generalised Visual Grounding in Agriculture

The paper builds gRef-CW, the first generalised visual grounding benchmark for agricultural field scenes (8,034 high-resolution images, 82,592 template-based referring annotations including image-level and instance-level negatives), and proposes Weed-VG, a modular framework on top of GroundingDINO that splits grounding into global existence detection and instance relevance scoring bound by a hierarchical constraint, reaching 62.42% Top-1 and 78.35% Neg-Acc on the test set while four general grounding baselines peak at 34.88% Top-1 and below 26% Neg-Acc in zero-shot evaluation.

Read or Ignore? A Unified Benchmark for Typographic-Attack Robustness and Text Recognition in Vision-Language Models

The paper introduces the Read-or-Ignore VQA (RIO-VQA) task and the RIO-Bench benchmark, which jointly evaluate typographic-attack robustness and scene-text reading in LVLMs through a same-scene counterfactual grid crossing question intent with text condition; it exposes the hidden cost of object-centric defenses that buy robustness by broadly down-weighting textual cues, and provides a data-driven RIO-RT fine-tuning baseline that improves both sides.

ReflectCAP: Detailed Image Captioning with Reflective Memory

ReflectCAP uses an offline multi-agent pipeline to distill the recurring hallucinations and omissions of a target LVLM into two sets of "Structured Reflection Notes" (Avoid / Include), then injects them separately at inference time to produce a grounded base caption and a detail-focused caption that are conservatively merged with the image as an anchor—improving factuality and coverage simultaneously across 8 LVLMs with no training at all, at lower compute than model scaling or existing multi-agent pipelines.

Solving Semi-Supervised Few-Shot Learning from an Auto-Annotation Perspective

This paper investigates semi-supervised few-shot learning (SSFSL) from a realistic perspective of auto-annotation. It reveals that directly applying existing SSL methods to fine-tune VLMs fails completely due to "flat" softmax probability distributions generated by VLMs (leading to zero utilization of unlabeled data and insufficient training supervision). The authors propose using two simple temperature parameters (a confidence temperature \(T_{conf}\) and a loss temperature \(T_{loss}\)) to sharpen the softmax output and boost training signals. Combined with a three-stage step-by-step training scheme and external open data retrieval, the proposed method outperforms existing baselines by approximately 5 percentage points across 5 fine-grained benchmarks, even rivaling fully supervised learning.

SPAR: Semantic-Pixel Self-Alignment and Adaptive Routing for Unified Multimodal Models

SPAR first reconciles semantic preservation and pixel reconstruction with an asymmetric dual-stream tokenizer, then conditions diffusion through token-wise cross-layer routing and internal representation alignment, reaching 0.91 on GenEval and 0.64 on WISE with SPAR-3B while largely preserving the original visual understanding capabilities.

Towards Spatial Supersensing in the Wild

VSI-Super-Wild uses 442 real long-form videos and 6,980 human-verified question-answer pairs to test continual world-state modeling around the agent, objects, and environment, with even the strongest model, Gemini-3.1-Pro, reaching only 44.36 overall.

Transferability Between Understanding and Generation in Unified Multimodal Models

Using single-task fine-tuning and cross-task evaluation, this paper finds the strongest bidirectional capability transfer in Lumina-DiMOO, which shares its backbone and visual encoder, and improves counting, spatial relations, and several text-generation metrics through understanding-only training with less distribution shift than direct generation fine-tuning.

Unleashing Multimodal Large Language Models for Training-free HOI Detection in the Wild

AgentHOI recasts HOI detection as an agentic pipeline where an MLLM performs open-ended interaction reasoning and a grounding model places the boxes: context-aware multi-round reasoning recovers missed interactions and instance-level descriptive prompting resolves localization ambiguity, all without training a single HOI-specific parameter, surpassing supervised and weakly supervised methods on HICO-DET zero-shot and under distribution shifts.

What CLIP Knows but Cannot Say: Recovering Negation from Frozen Intermediate Features

The paper tracks two layer-wise curves in CLIP's text encoder — negation separability and visual alignment — and finds that negation information peaks in the middle layers before being erased by the final layers as "Representational Collapse"; it then proposes PeakPatch, a lightweight post-hoc system that keeps CLIP entirely frozen (an Embedding Correction Network plus a Score Correction Network, only 5.2M parameters, 3.5% of the backbone), raising NegBench COCO MCQ accuracy from 39.2% to 74.3%.

Why Do Vision Language Models Struggle To Recognize Human Emotions?

Using video-based dynamic facial expression recognition (DFER) as a probe task, this paper traces the emotion-recognition failures of contemporary VLMs to two root causes — head-class bias inherited from long-tailed pretraining data and a breakdown of temporal modeling under a fixed token budget — via three evidence chains (a lexical-frequency proxy, frame shuffling, and a frame-rate density sweep), and validates the diagnosis with balanced-data decoupled fine-tuning and an inference-time Multi-Stage Context Enrichment (MSCE) pipeline, reporting consistent F1 gains on the balanced test splits of MAFW and DFEW.

Why Far Looks Up: Probing Spatial Representation in Vision-Language Models

The paper builds a representation-level analysis framework that extracts delta vectors from the final-token hidden states of minimal contrastive question pairs (same image, swapped object order) to measure how the three spatial axes are organized inside VLMs, finds a pervasive vertical-distance entanglement in which image-plane height is used as a proxy for depth, and uses SpatialTunnel — a synthetic benchmark that geometrically decouples vertical position from depth — to show the bias is model-intrinsic and that models with a more coherent distance axis are the ones that stay robust on counter-heuristic examples.

3D FaceShell: Attribute Transfer in 3D Face Avatars as a VLM Defense Mechanism

3D FaceShell adds a learnable Gaussian shell to a frozen 3D face and uses multi-view semantic alignment to redirect VLM attribute predictions, reaching 48.7% injection rate on VideoLLaMA3 with LPIPS 0.1499 and identity similarity 0.7629 on frontal views; its contribution is the trade-off between semantic intervention and visual fidelity, not uniformly maximal attack success.

3D-Aware VLMs with Implicit and Explicit Geometries

VLM-IE3D injects both implicit geometric features and explicit tokens derived from reconstructed depth into Qwen2.5-VL, raising 3D video detection [email protected] from the backbone's 30.9 to 42.8, although its average spatial reasoning gain over an existing geometry-enhanced model is modest.

AdaBoosting Text Prompts for Vision-Language Models

This paper proposes Text Prompt Boosting (TPB), which introduces the AdaBoost framework into the construction of text prompts for VLMs. By treating the collection of prompts for each category as a weak classifier, it iteratively reweights hard samples, constructs new weak prompt classifiers round by round, and ensembles them into a strong classifier. This achieves sustained, sample-driven performance improvements (shot scalability) in few-shot scenarios. Moreover, this natural language prompt ensemble can be directly re-embedded and transferred across heterogeneous VLMs, preserving the performance gains after cross-model transfer.

Adapting MLLMs for Nuanced Video Retrieval

TARA turns a generative MLLM into a unified retrieval encoder by fine-tuning its shared language model on just 20,000 carefully constructed text triplets, raising Tarsier 2's CiA-SSv2 Chiral mAP from 77.7 to 88.9 while also improving negated queries and composed video retrieval.

AnchorGUI: Asymmetric Memory for Dual-Scale Learning in GUI Navigation

AnchorGUI uses cognitive state anchors to compare expected action effects with actual interface changes, compressing ordinary steps into text while retaining visual evidence for mismatches, so that one shared VLM supports both intra-trial correction and cross-trial distillation, achieving 57.3% single-attempt and 69.2% three-attempt cumulative success on AndroidWorld.

Attention is Case-Sensitive

Controlled casing interventions across 13 models show that uppercase and alternating-case targets attract attention without guaranteeing better accuracy; in VLMs, stronger local target concentration can coexist with reduced whole-image attention, so attention enhancement should not be equated with better understanding.

BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension

BaFCo benchmarks layout analysis and information extraction on carefully annotated Bangladeshi government forms, varying label granularity, prompting, and reasoning effort to show that Gemini 3 Pro reaches only 0.1177 fine-grained [email protected] with high-reasoning zero-shot prompting, despite achieving 0.848 Bangla extraction Macro-F1, exposing the gap between reading field values and precisely reconstructing form structure.

Benchmarking Vision-Language Models for Microscopic Plant Image Understanding

PlantMicro organizes 5,410 plant microscopy images into 9,718 question-answer pairs across 10 tasks, showing that general-purpose VLMs can approach perfect modality recognition while struggling with host and pathogen identity; task-restricted visual retrieval substantially improves recognition, whereas reasoning prompts and a single demonstration do not reliably help.

Beyond Atomic Layouts: Compositional Design Understanding with Vision-Language Models

The paper introduces CoDeLayout and the MASON post-training approach, combining element grounding supervision with explicit layer-aware spatial metadata to improve Qwen2.5-VL 7B from 88.80% to 91.66% weighted compositional pairing accuracy over full-data direct finetuning.

Bridging Visual Representation and Reinforcement Learning from Verifiable Rewards in Large Vision-Language Models

To resolve the structural representation bottleneck in Large Vision-Language Models trained via Reinforcement Learning from Verifiable Rewards (RLVR) where dense tokenization mismatches sparse diagrammatic signals, this paper proposes KAWHI, a plug-and-play reward reweighting mechanism that extracts geometric key regions with a Structure-Guided Union-Find algorithm, performs asymmetric Q-K spatial alignment using vision-critical attention heads, and reallocates credit at the paragraph level, consistently improving multimodal reasoning benchmarks with only 3.6% training overhead.

CMDR: Contextual Multimodal Document Retrieval

Addressing the fundamental limitation where vision-based document retrievers encode pages in isolation and discard cross-page dependencies, this work introduces CMDR-Bench (the first benchmark requiring multi-page contextual reasoning) and CMDR-Embed, which combines chunk-then-split joint encoding with context-aware contrastive learning to significantly outperform non-contextual baselines.

COMPASS: Grounding Composition-Intent Guidance in Unified Multimodal Models

COMPASS adds a composition-expert branch and an expert token to Ming-Lite-Uni, then guides generation with grayscale pixelated references and restricted information pathways, reporting composition-recognition mAP of 90.6 and generation composition consistency of 0.74 on Comp-11, although its mAP aggregation and self-evaluated composition metrics require caution.

Controlling Embedding Spaces with Text-Conditioned Transformations

By employing a lightweight three-layer MLP hypernetwork that predicts global affine transformations conditioned on natural language attribute descriptions, this approach achieves state-of-the-art attribute-aware retrieval over frozen galleries and unsupervised multi-clustering at near-zero inference cost.

CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA

Addressing the limitation of conventional knowledge-intensive VQA benchmarks that only report end-task accuracy and obscure internal failure modes, this work introduces CRAG-MM-Diagnostics with target ROI and disambiguation metadata, isolating visual grounding, object identification, and knowledge retrieval & reasoning, while demonstrating the superiority of a grounded bimodal RAG pipeline.

Curvature-Guided Mixing for MLLM Adaptation

This paper proposes the Curvature-Guided Mixing (CGM) framework. By performing second-order Taylor expansion and Hessian diagonal approximation on both pre-training and fine-tuning loss surfaces, a closed-form soft mixing ratio based on relative curvature (CGM) is derived. Furthermore, a sparse hard mixing strategy based on curvature-aware scores (CGM†) is designed. Together, they achieve an optimal balance between downstream adaptation and general knowledge preservation in MLLM fine-tuning.

Decompose, Compare, and Decide: Multimodal LLMs are Implicit Few-Shot Learners

Proposes DeCoDe, a training-free inference framework that decomposes few-shot classification into independent pairwise support-query binary comparisons and ranks candidate classes via the unnormalized "Yes" token logit, resolving the vulnerability of off-the-shelf MLLMs to context distraction and label semantic priors.

DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cues

DiCoBench proposes the first high-resolution multi-image fine-grained perception benchmark driven by implicit visual cues (difference and commonality), comprising 765 high-quality samples and 8 types of tasks. It utilizes a multiple-choice question (MCQ) paradigm to eliminate n-gram evaluation bias. Among 18 evaluated MLLMs, the best-performing model, Gemini-3-Pro, achieves only 58.1% (compared to 98.3% for humans), revealing a severe deficiency in current models' ability to autonomously discover visual cues.

Disentangling Pictorial Cue Understanding from Language Bias in VLMs via Depth Ordering Task

By introducing O3-D—a controlled odd-one-out depth ordering benchmark that isolates 9 pictorial depth cues across 37K images and 147K questions—and proposing the SDGM metric, this paper demonstrates that 12 leading VLMs perform at chance level on monocular depth perception while remaining heavily biased by language variations.

Domain Arithmetic: One-Shot VLA Adaptation under Environmental Shifts

DART proposes a one-shot domain adaptation method for VLA models based on weight-space analogy arithmetic. By fine-tuning separately with a single source-domain and target-domain demonstration, it extracts a pure domain vector via subtraction after subspace alignment filtering and adds it back to the base model. Under LIBERO multi-view, visual perturbations, cross-embodiment, and real robot scenarios, it significantly outperforms existing adaptation baselines using only a single demonstration.

Domain Generalization via Text-Anchored Information Bottleneck

This work demonstrates that expressive visual guidance in vision-language distillation inevitably propagates domain-specific spurious cues, proposing a purely text-anchored Conditional Entropy Bottleneck framework that enforces domain invariance by distilling semantic anchors and compressing intra-class stylistic variations.

Dynamic Cluster Data Sampling for Efficient and Long-Tail-Aware Vision-Language Pre-training

Addressing the excessive computational cost and severe underrepresentation of rare concepts in vision-language pre-training on noisy web data, this paper proposes DynamiCS, a framework combining power-law cluster scaling with cross-epoch dynamic resampling that slashes GPU pre-training hours to ~3% while dramatically boosting long-tail recognition.

EgoSAT: A Comprehensive Benchmark of Egocentric Streaming Interaction Understanding

EgoSAT is the first systematic benchmark for egocentric streaming interaction understanding. It unifies three types of temporal reasoning—past (retrospective), present (real-time), and future (prospective)—under a strict online protocol. Through answerability quantification and confidence diagnosis, it reveals that current VLMs are not only weak at temporal reasoning but also suffer from severe miscalibration problems (being "confidently wrong").

Enhancing Alignment for Unified Multimodal Models via Semantically-Grounded Supervision

Addressing the core granularity mismatch and supervisory redundancy in Unified Multimodal Models (UMMs), SeGroS filters discriminative text tokens via dual intra- and inter-modal affinities and builds a visual grounding map to provide semantic visual hints while restricting masked reconstruction loss exclusively to text-aligned core regions, substantially boosting cross-modal alignment and generation fidelity.

EraseLoRA: MLLM-Driven Foreground Exclusion and Background Subtype Aggregation for Dataset-Free Object Removal

EraseLoRA proposes a dataset-free object removal framework: it first utilizes an MLLM to perform three-way segmentation on a single image (target foreground / non-target foreground / background subtypes), and then reconstructs and aggregates multiple background subtypes into the masked region via test-time LoRA adaptive optimization. Without requiring any training data or performing explicit self-attention blocking or redirection, it improves background fidelity by at least 23% and nearly halves the foreground regeneration rate compared to previous dataset-free methods, while also outperforming all dataset-driven methods.

Evaluating and Enhancing Negation Comprehension in Remote Sensing MLLMs

Addressing the severe performance degradation of remote sensing MLLMs under negative queries, this paper constructs the first remote sensing negation comprehension benchmark, RS-Neg (22K samples covering four tasks from regional to scene levels), and proposes a test-time adaptation method, NeFo. By explicitly modeling the logical role of negation (truth-value inversion operator) as an optimization objective, NeFo significantly enhances the negation comprehension capabilities of multiple base models using only around 5% of unlabeled test samples, demonstrating strong generalization on unseen tasks.

Evaluating and Understanding Model Editing for Medical Vision Language Models

Addressing post-deployment errors in medical vision-language models, this paper introduces M3Bench, the first clinically grounded multimodal model editing benchmark comprising 10 evaluation tasks across 16,276 questions, systematically uncovering fundamental tradeoffs between gradient-based and memory-based editing paradigms and attributing their failures to latent space cone geometry.

Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Tasks

An Explicit Logic Channel (ELC) is proposed to run in parallel with the MLLM black-box channel (ILC), generating interpretable decisions through a three-step process of LLM semantic fact extraction, VFM visual grounding, and probabilistic logical reasoning. Consistency Rate (CR) is defined as a proxy metric for model quality under unsupervised/unlabeled settings. Through cross-channel alignment and fusion, performance on zero-shot VLC tasks is enhanced, which is validated on 11 MLLMs across 3 benchmarks.

GAP-MLLM: Geometry-Aligned Pre-training for Activating 3D Spatial Perception in Multimodal Large Language Models

GAP-MLLM introduces the first geometry-aligned pre-training paradigm for pure RGB-based multimodal large language models, activating metric 3D spatial perception under extremely sparse point supervision through visual-prompted joint pointmap-semantic prediction and layer-wise gated fusion with multi-modal DeepStack injection.

GeMoE: Gating Entropy is All You Need for Uncertainty-aware Adaptive Routing in MoE-based Large Vision-Language Models

Frames MoE dynamic routing as a Minimum Description Length (MDL) optimization problem, using gating entropy as a proxy for uncertainty and information gain to train an expert assignment predictor via monotonic loss, preserving 99.5% average performance while improving expert activation sparsity by 36.5%.

General Incomplete Multimodal Learning via Dynamic Quality Perception

To tackle the coexistence of complete inter-modality absence and severe intra-modality noise degradation, this paper proposes GIML, a unified framework modeling heterogeneous missing patterns as continuous quality degradation, combining noise-semantic decoupled probabilistic representations with a calibrated noise-aware quality estimator for adaptive fusion.

Generalize LMMs to Versatile Visual Modalities via Fabricated Modality Synthesis

To overcome the heavy reliance of Large Multimodal Models (LMMs) on in-modality data for non-RGB vision, this work introduces VVM-Tuning, a training framework that synthesizes appearance-varied images from RGB data to disentangle invariant semantics from photographic features, achieving zero-shot generalization to unseen visual modalities via textual modality contexts.

GKDT: General Keypoint Detection Transformer

This paper constructs a large-scale unified keypoint dataset, MegaKPT, containing 1.3 million instances, and proposes GKDT, a General Keypoint Detection Transformer based on DINOv3. Through a Kernel Generation mechanism, visual/textual keypoint prototypes are dynamically converted into convolutional kernels. GKDT achieves 90%+ [email protected] accuracy on most categories across 22 test sets using a single model.

Global Graph-Validated Optimization for VLM-based 3D Indoor Scene Generation

Addressing the core pitfalls in VLM-based 3D scene generation—where local pairwise modeling causes global semantic contradictions and gradient optimization gets trapped in physically infeasible local minima—this paper explicitly decouples semantic coherence from physical feasibility by introducing Global Semantic Verification (GSV) over scene graphs and Global Physical Feasibility Search (GPFS) via hybrid evolutionary-gradient optimization, synthesizing globally consistent and collision-free 3D scenes without relying on VLM pose initializations.

Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Models

To resolve the widespread evaluation failure caused by safety guardrails refusing attribute-inferring prompts, this work decouples person images into background user context and evaluates societal bias across three person-irrelevant tasks, achieving zero refusals and uncovering persistent stereotypes across 20 state-of-the-art LVLMs.

HAS: Highlight-guided Attention Steering for Multimodal LLM Video Summarization

To overcome the context fragmentation and irreversible information loss caused by hard keyframe selection, HAS introduces an inference-time continuous attention steering mechanism that smoothly biases frozen video MLLM cross-attention heads toward salient moments using prompt-conditioned highlight distributions while preserving complete background context.

Holistic Optimal Label Selection for Robust Prompt Learning under Partial Labels

Addressing severe candidate ambiguity and scarce supervision in few-shot partial-label prompt tuning, HopS introduces a Local Density-based Filter (LDF) via k-NN consensus voting and a Global Optimal Transport Planner (GOP) via Sinkhorn-Knopp optimization, providing robust dual-perspective pseudo-label supervision with frozen vision encoders and achieving SOTA across 8 benchmarks.

Hybrid Advantage Estimation with Unified Critic for VLM Agentic Reinforcement Learning

To tackle nearsightedness and fractured credit assignment in multi-turn VLM agents, this paper introduces HyGAE, which unifies token- and turn-level advantages via a single critic and broadcasted mixed returns to achieve low-bias, low-variance policy optimization.

Identifying and Resolving Pitfalls of Knowledge-Based VQA Benchmarks: Auditing, Repairing, and Augmenting

A systematic audit of the knowledge-based visual question answering (KB-VQA) benchmarks InfoSeek and E-VQA reveals three widespread deficiencies: answer-evidence misalignment, question under-specification, and single-entity visual shortcuts. To address this, a four-stage repair protocol and a controlled multi-entity augmentation protocol are proposed. Experimental results demonstrate model ranking reversals after repairing, along with substantial drops in retrieval recall and QA accuracy after augmentation, indicating that current KB-VQA evaluations significantly distort the measurement of knowledge grounding and reasoning capabilities.

Isotropic Embedding Perturbations for Robust Vision Language Encoders

Addressing the saturation of conventional pixel-space data augmentations and their tendency to induce anisotropic noise while disrupting cross-modal alignment, this paper proposes Aether—a plug-and-play embedding-space augmentation method utilizing variance-preserving alpha-mixing to inject isotropic perturbations, substantially smoothing representations and enhancing the robustness and generalization of vision-language encoders.

Learning to Deny: Action Denial in Multimodal Large Language Models

This paper proposes the UCF101-AD benchmark (11k paired videos of "Action-Presence/Action-Denial") specifically to evaluate whether MLLMs can correctly deny an action when "the scene, objects, and actors are all present but the defining movement does not occur." The study reveals that while 20 state-of-the-art (SOTA) models achieve >85% accuracy in identifying positive samples, their accuracy on denial samples generally falls below 50%. Consequently, the CausalAct causal graph framework is proposed (representing the scene as a Directed Acyclic Graph (DAG) with seven nodes: P/O/L/S/I/M/A, and fine-tuning with natural language prompts and graph concepts), which significantly reduces false positives and demonstrates that "denial" is a learnable reasoning skill.

LENS: Adaptive Spatio-Temporal Zooming for Keyframe Sampling in Long-Form Videos

LENS is a training-free keyframe sampling framework that adaptively partitions a limited frame budget between spatial zoom-in (patch-level visual prompting for fine-grained details) and temporal zoom-out (video graph reasoning and hyperframe aggregation for long-range context), boosting Video-MME accuracy of 7B MLLMs from 53.3% to 60.7% under a strict 8-frame budget.

Let ViT Speak: Generative Language-Image Pre-training

GenLIP dispenses with contrastive dual encoders and cascaded auxiliary text decoders by training a single Vision Transformer to predict language tokens directly from visual prefix sequences via autoregressive next-token prediction, matching or surpassing SigLIP2 with only one-fifth of the pretraining data.

Linear Scaling Video VLMs for Long Video Understanding

To eliminate the quadratic compute and latency explosion in video VLM prefill, StateKV decouples streaming prefill from decoding on frozen backbones using a fixed-capacity, importance-based recurrent state that preserves temporal sinks, achieving strictly constant per-frame compute and global \(O(N)\) linear complexity.

LongVQUBench: Benchmarking Long-Term Video Quality Understanding of Vision-Language Models

Addressing the limitation of existing video quality benchmarks that focus on short clips and isolated artifacts while neglecting cumulative degradation and deep reasoning, this paper introduces LongVQUBench, the first comprehensive benchmark featuring 1,200+ diverse videos (spanning minutes up to two hours) and 1,500 QA pairs across a three-level hierarchy (LQU, CQR, GQU) with needle distortion QA, demonstrating significant performance drops in 14 state-of-the-art LVLMs as temporal horizon and reasoning depth increase.

MedQ-Deg: A Multidimensional Benchmark for Evaluating MLLMs Across Medical Image Quality Degradations

MedQ-Deg introduces a comprehensive robustness benchmark comprising 24,894 QA pairs across 7 imaging modalities, 18 physics-grounded degradation types (at 3 radiologist-calibrated severity levels), and 30 clinical capability dimensions, revealing a severe "AI Dunning-Kruger effect" where MLLMs maintain dangerously high confidence despite accuracy collapse.

MIRROR: Aligning Semantic Relations from Language to Image via Gromov-Wasserstein

MIRROR proposes the Semi-Inverse Gromov-Wasserstein (SI-GW) geometric regularization framework, which maps the relational structure between concepts in language space to visual space via a closed-form solution. Without introducing additional parameters or inference overhead, it significantly improves the relational reasoning capabilities of MLLMs.

Mitigating Modality and Language-Style Gaps for Zero-Shot Video Moment Retrieval

Addressing the vulnerability of zero-shot video moment retrieval to both modality and language-style gaps in query-video matching, this paper proposes Self-Similarity-based Moment proposal and Scoring (Self-SiMS), which leverages intrinsic intra-video self-similarity to generate proposals and score contextual consistency, followed by MLLM-based query-aware binary re-ranking to achieve new state-of-the-art performance across benchmarks.

Multi-modal Knowledge Preserving Adapter for Embedding Backward Compatibility

To eliminate the prohibitive computational cost of database re-indexing during multimodal retrieval system upgrades, this paper presents MKP-Adapter—the first adapter-only backward compatible training framework for Multimodal Large Language Models (MLLMs)—which enforces compatibility and retains representation quality via multi-level geometric preservation losses and focal re-weighting without touching backbone weights.

Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction

AMG-Fuse proposes a mask-guided multi-modal image fusion framework for adverse weather. By decoupling modal contribution masks from the "Pseudo Ground Truth" (Pseudo GT) and combining the Mask-guided Feature Extraction Module (MFEM) with Mask-guided Cross-modal Cross-Attention (MCCA), Mask-Guided Learning Strategy (MGLS), and Task-coupled Degradation-Aware learning Strategy (TDAS), it simultaneously achieves feature restoration and cross-modal interaction in a unified network. It comprehensively outperforms SOTA methods under rain, snow, and haze, as well as in real-world scenarios, while achieving optimal mAP on downstream object detection tasks.

MultiMem: Measuring and Mitigating Memorization in Multi-Modal Contrastive Learning

Introduces MultiMem, the first principled leave-one-out metric designed to quantify global memorization across arbitrary modality combinations in contrastive learning, demonstrates that cross-modal semantic misalignment drives harmful memorization, and establishes in-training and post-training mitigation strategies that reduce memorization while boosting downstream retrieval and zero-shot accuracy.

NegAS: Negative Label Guided Attention and Scoring for Out-of-Distribution Object Detection with Vision-Language Models

NegAS addresses two unique challenges of VLM-based object detectors in OOD scenarios: the indiscriminate treatment of background regions by text-guided attention, and the incompatibility between sigmoid multi-label outputs and softmax-based OOD scores. By proposing the Negative Label Guided Attention module (NegA) and the Negative Label Guided Scoring function (NegS), NegAS significantly improves OOD detection performance on both YOLO-World and Grounding DINO with zero extra inference overhead.

Parallel Vision Token Scheduling for Fast and Accurate Multimodal LMMs Inference

Addressing the quadratic attention overhead of MLLMs and context loss from aggressive visual token pruning, ParVTS establishes a training-free parallel scheduling framework that routes subject and non-subject tokens through decoupled early transformer layers to migrate their semantics into question tokens, before discarding non-subject tokens mid-inference to achieve an 88.89% visual token reduction and a 1.77× real-world speedup.

Personalizing MLLMs via Reinforced Multimodal Reference Game

Reframing MLLM personalization into a collaborative speaker-listener multimodal reference game, this paper optimizes the speaker using GRPO with verifiable probability-normalized contrastive rewards derived from hard positives and hard negatives, producing state-invariant and discriminative concept descriptions that substantially advance personalization benchmarks.

Prune Once: Retraining-Free Task-Agnostic Pruning for Vision-Language Models

Addressing modality bias and calibration sensitivity in vision-language model pruning caused by LLM-oriented activation outlier assumptions, this paper proposes PORTA, which uses cross-modal stable activation variance to quantify representation utility and couples it with adaptive layer-wise sparsity allocation driven by output feature variability, enabling a single pruning pass on generic data to transfer zero-shot across classification, retrieval, and VQA without retraining.

Qwen-3D: A Generalist 3D Vision-Language Model for Spatial Understanding

Qwen-3D extends Qwen2.5-VL with parameter-free 3D voxel token merging, 3D Rotary Position Embeddings (3D MRoPE), and a shared dense visual representation feeding a Mask2Former-style decoder, achieving state-of-the-art 3D grounding and segmentation while preserving strong 2D vision-language reasoning.

Rank-Aware Hyperbolic Alignment for Vision–Language Dataset Distillation

Lifts vision-language features to the Lorentz hyperbolic space for contrastive alignment (hITC), decomposes the cross-covariance of real batches via SVD into range (dominant shared directions) and residual (weakly coupled directions) subspaces based on cumulative energy, and matches correlation distributions using Sinkhorn optimal transport with asymmetric regularization. This selectively aligns highly informative shared structures while preserving modality-specific diversity under extreme compression budgets, achieving superior cross-architecture transferability and robustness.

RegRet: Enhancing Region-Level Retrieval in Large Multimodal Models

Addressing the trade-off between fine-grained local details and global background context in large multimodal models for region-level retrieval, RegRet introduces a layer-wise cross-attention Region-Aware Encoder with decoupled projectors alongside a three-stage training pipeline and the 225k REGMB benchmark, boosting regional retrieval accuracy by over 20% while preserving global retrieval capability.

ReGRPO: Reflection-Augmented Policy Optimization for Tool-Using Agents

Addressing the issue where multimodal vision–language agents collapse upon encountering intermediate tool failures and lack explicit recovery signals, ReGRPO constructs a structured reflective data engine and leverages Group Relative Policy Optimization (GRPO) to jointly optimize diagnostic reflection tokens and corrective actions within local trajectories, achieving state-of-the-art open-source performance on GTA and GAIA under zero-verifier single-path inference.

Show Me Examples: Inferring Visual Concepts from Image Sets

Addressing the failure of modern VLMs to infer implicit concepts purely from visual examples without text, this paper introduces the Visual Concept Inference from Sets (VICIS) benchmark along with a framework combining a Set Learner, subspace projection, and rectified flow matching for nonverbal concept induction and diverse target synthesis.

SIMPLER: Efficient Foundation Model Adaptation via Similarity-Guided Layer Pruning for Earth Observation

Addressing the heavy computational and deployment overhead of Earth Observation foundation models, SIMPLER introduces pre-fine-tuning layer pruning guided by CKA representation similarity on unlabeled downstream data, pruning up to 79% of parameters while preserving 94%~97% baseline accuracy with 2.1× training and 2.6× inference speedups.

SPOT-E: Test-Time Entropy Shaping with Visual Spotlights for Frozen VLMs

Addressing the failure of frozen vision-language models on evidence-intensive tasks caused by overlooking fine-grained visual cues, SPOT-E introduces an entropy-shaping principle combining answer-span entropy reduction with low-entropy anchor preservation, optimizing a question-conditioned visual spotlight via test-time GRPO without modifying backbone weights.

Starve to Perceive: Taming Lazy Perception in VLMs with Constrained Visual Bandwidth

Addressing the pervasive "lazy perception" where vision-language models mimic tool-use syntax without functionally depending on visual outputs, this paper introduces a training paradigm that strictly constrains visual token bandwidth throughout post-training, removing shortcut solutions and compelling the agent to master genuine multi-step active visual search using purely sparse task-level rewards.

StochasT: Learning with Stochastic Turn Depth for Visual Instruction Tuning

StochasT proposes randomly pruning the historical dialogue context during visual instruction tuning (VIT) of large visual-language models (LVLMs), unfolding a fixed chain into a dialogue tree. This trains an implicit multi-depth ensemble model that optimizes both single-turn and multi-turn evaluation performance. Along with this, a robustness evaluation framework based on Balanced Latin Squares and the CRA/CRA+ metrics are introduced.

Stream3D-VLM: Online 3D Spatial Understanding with Incremental Geometry Priors

Stream3D-VLM presents the first online 3D vision-language model operating solely on streaming video, combining autoregressive next-token prediction for response timing, lightweight VSFI for incremental geometry prior injection, and GAVC for spatial token compression to enable real-time 3D spatial understanding.

SVI-Bench: A Dynamic Microworld for Strategic Video Intelligence

SVI-Bench is the first large-scale benchmark evaluating the full Strategic Video Intelligence (SVI) stack—from perception through causal reasoning and simulation to agentic synthesis—across 35K hours of multi-agent team sports videos, revealing a severe capability cliff where state-of-the-art models drop from ~74% on fine-grained perception to just 4.6% on autonomous cross-corpus synthesis.

TecoPrompt: Temporal-Conservative Prompt Learning for Vision-Language Models

TecoPrompt is a closed-loop robust prompt learning framework that leverages global optimal transport pseudo-labeling coupled with a K-epoch temporal stability window and EMA confidence gating to conservatively rewrite noisy labels, optimized via a tri-group loss for noise-resilient few-shot adaptation.

To Adapt or Not to Adapt? Selective Adaptation for Vision-Language Models

Addressing the observation that over 90% of adaptations in test-time adaptation (TTA) are negligible or detrimental, this paper introduces the selective adaptation problem alongside a Cross-Augmentation Similarity (CAS) scoring baseline, bypassing unnecessary updates and preserving or improving classification and calibration accuracy even when skipping nearly 85% of adaptations.

Token-Based Affordance Grounding with Large Vision-Language Models

TokAG is a training-free zero-shot affordance grounding framework that aggregates cross-attention activations across all layers and heads of an LVLM and applies an object-aware spatial constraint to select the single dominant token, converting implicit action semantics into high-fidelity affordance heatmaps.

Token-level Response-visual Attention Guidance for Multimodal LLMs Knowledge Distillation

Addressing the limitations of conventional multimodal distillation that rely on prompt-to-vision attention with uniform objectives, TRAG re-targets cross-modal attention distillation to the response generation phase (Response-to-Vision) and dynamically balances forward and reverse KL divergences based on teacher attention entropy, significantly outperforming prior baselines on VQA and compositional reasoning.

Towards Purified Multi-Label Test-Time Adaptation of Vision-Language Models

To overcome shared global representation entanglement and subsequent cache saturation in multi-label test-time adaptation (MLTTA) of vision-language models, PuRF introduces region purification combined with episodic prototype anchoring and temporal decay refreshing, achieving fine-grained multimodal adaptation and persistent streaming adaptability without backbone fine-tuning.

Two Birds, One Projection: Harmonizing Safety and Utility in LVLMs via Inference-time Feature Projection

Addressing the pervasive "safety–utility tradeoff" in large vision-language models where defense mechanisms degrade general reasoning performance, TBOP identifies that modality-induced bias serves as a shared root cause undermining both safety and utility, and removes this bias by projecting cross-modal representations onto its null space in a single forward pass during inference.

UC-VLM: Consistency-Driven Learning for AI-Generated Image Detection with Vision-Language Large Models

Addressing the dual bottlenecks of existing vision-language models in AI-generated image detection—namely, over-reliance on human rationale annotations and neglect of low-level visual forensic cues—UC-VLM introduces a unified multi-stage framework trained solely on binary real/fake supervision. By combining stochastic cropping and patch shuffling for visual adaptation, LLM-driven instruction evolution, and label-conditioned text regeneration on a frozen vision tower, it achieves state-of-the-art accuracies of 96.1% on GenImage and 77.9% on Chameleon.

UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models

Addressing the heavy computational burden and latency bottleneck of vision encoders in edge-deployed LVLMs, UltraViT constructs a macro-block heterogeneous pyramidal vision encoder informed by physical on-device NPU latency measurements, combined with a two-stage generative pre-training framework that delivers 1.7× to 19.1× on-device speedups while establishing state-of-the-art multimodal accuracy.

Unlocking Few-Shot Capabilities in LVLMs via Prompt Conditioning and Head Selection

Proposes Head Ensemble Classifiers (HEC), a training-free framework that conditions LVLM multimodal feature distributions using tiered text prompts and identifies a sparse set of discriminative attention heads via Gaussian Discriminant Analysis and soft accuracy ranking, bridging the longstanding classification performance gap between LVLMs and CLIP.

VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Context

VisReflect proposes a latent space visual reflection mechanism. By generating a special token <VR>, the LVLM directly recalls query-relevant visual features from images or videos using its hidden states. This enables attention refocusing within a single forward pass, bypassing the coordinate prediction errors and multi-forward overhead inherent in existing "zoom-and-re-encode" paradigms. It achieves performance improvements of 4.1% and 1.8% on fine-grained image and video perception tasks, respectively, while accelerating inference speed by 44% compared to multi-turn zoom methods.

Visual Prompt Discovery via Semantic Exploration

Addressing the systemic perception failures of Large Vision-Language Models (LVLMs) and the excessive token overhead of per-sample tool-use pipelines, this paper introduces SEVEX—an automated semantic tree search framework that discovers task-wise visual prompt scripts within an abstract idea space, achieving superior accuracy while cutting inference token consumption by 91.2%.

VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling

VLZip addresses the quadratic attention complexity bottleneck in interleaved image-text long-context modeling by hierarchically distilling visual and textual chunks into layer-specific soft prefixes and injecting them into decoder hidden states, enabling 6x longer training sequences up to 120K tokens and superior performance on the narrative-driven LongVLBench benchmark.

Wake up for Touch! Mask-isolated Tactile Alignment Learning in MLLMs

This paper proposes Splash, a mask-isolated tactile alignment framework. By utilizing a Wanda-style importance scoring metric, the LLM parameters of a small MLLM are partitioned into "dormant" and "critical" subspaces. Only the dormant subspace is updated to encode tactile capabilities, while the frozen critical subspace serves as a stable anchor to preserve original vision-language reasoning.

Where to Look Matters: Learning Influential Views for VLM-based 3D Visual Grounding

Addressing the limitation of visibility heuristics in VLM-based zero-shot 3D visual grounding, IVSGround introduces a learned view selector trained on reasoning feedback from a frozen VLM via rejection sampling, substantially boosting grounding accuracy without modifying the base reasoner.

AMCI: Unlock the Potential of Large Multimodal Models for Fine-grained Open-world Classification via Adaptive Memory Context Injection

AMCI turns previously observed test images and their attribute descriptions into retrievable memory, then uses a language model to refine neighboring context into recognition prompts, improving Qwen2-VL 7B's mean CFI across five fine-grained datasets from 36.6 to 63.0 without updating parameters, while still depending on stream quality and additional language-model inference.

An Inverse-Adversarial and Difficulty-Adaptive Robust Vision-Language Model

IADA-RVLM constrains adversarial training with inverse-adversarial examples moved toward the correct class, learns specialized knowledge through visual prompts and a text adapter, and adjusts text knowledge fusion using class semantic distances, achieving the best Nat/Rob harmonic mean on novel classes across five datasets, including 52.52% PGD-100 robust accuracy on Caltech101.

Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception

This work systematically evaluates the performance of various VLMs on texture-driven slant perception tasks. It reveals that VLM zero-shot predictions are heavily anchored to a few discrete values (\(0^\circ\), \(\pm 25^\circ\), \(\pm 45^\circ\)), behaving completely differently from the continuous yet biased perception exhibited by humans and self-supervised CNNs. Furthermore, this anchoring originates from a readout bottleneck at the language output side rather than a lack of geometric information in the vision encoder.

ATOMIC: A Domain-Specific Vision-Language Model for Transmission Electron Microscopy

ATOMIC retains the LLaVA-v1.5 architecture, converts published TEM subimages and captions into complementary visual and domain supervision, and performs two-stage adaptation, reaching 75.2% TEM-MCQ accuracy and 37.5% TEM-VQA Answer Word Coverage with its 7B model, without yet replacing quantitative microscopy analysis.

ATP-Bench: Towards the Agentic Tool Planning for MLLM Interleaved Generation

ATP-Bench unifies five visual tools for interleaved text-image planning and evaluates 7,702 human-verified QA pairs with three MAM judge agents that separately inspect valid calls, missing visuals, and overall composition; the strongest tested model, Gemini 3 Pro, reaches an average FS of 79.88, with substantial gaps remaining in complex travel and renovation responses.

Attention-Logit Steering to Compositional Generalization for Continual VQA

Q-STEER uses a shared controller to coordinate new expert capacity with corrections to where answer tokens attend, achieving 53.68 AP, 4.51 AF, and 51.00/51.48 Novel/Seen composition scores on continual VQA v2 without inference-time task labels or sample replay.

Background Blurring Matters: Improving Visual Grounding by Merging Text-Irrelevant Tokens

ToB preferentially merges visually similar but text-irrelevant tokens in deeper visual encoder layers, preserving fine-grained target representations and improving a DINOv2-B/BERT-B baseline from 76.05% to 80.36% on RefCOCOg test-u while increasing inference speed from 20.80 to 21.57 fps.

Benchmarking MLLMs on Mistake Recognition and Explanation in Single-Step Components of Cooking

Cook-MRE keeps cooking videos unchanged while replacing one instruction element, using 1,289 samples to test whether models can detect and explain deviations; the best default Full Pass Accuracy among 13 open-source MLLMs is only 57.0%, showing that flagging a mistake is far from seeing and explaining it correctly.

Beyond Categorical Matching: Intra-Class Graded Relevance Estimation for Cross-Modal 3D Retrieval

ReMU3D reparameterizes the joint image-text-point-cloud space with SCI and then aggregates arbitrary query-modality combinations through Q-MoE, raising NDCG@10 from the best baseline's 0.76 to 0.93 on the new INGRE graded-relevance benchmark.

Beyond Disjoint Tasks: Towards More Natural Continual Learning for Vision-Language Models

Addressing the failure of existing MoE-based multimodal continual learning methods under realistic task distributions, SOCiaL uses a Gaussian mixture model to quantify inter-task semantic overlap, consolidates redundant adapters via cross-term-free full-weight-space merging and regularized fine-tuning, and guides an elastic contextual bandit router, cutting deployed adapters threefold while achieving state-of-the-art accuracy across disjoint and overlapping benchmarks.

Beyond Filter Pruning: Top-K Spatial Selection for Efficient Neural Networks

Addressing the overlooked intra-filter spatial redundancy left behind by conventional filter pruning, Top-K Pruning (TKP) introduces a two-stage paradigm combining auxiliary exponential regularization with deterministic spatial selection to transform dense convolutions into efficient sparse sampling operations, delivering superior accuracy-efficiency trade-offs across CNNs, ViTs, LLMs, and extreme QAT regimes.

Beyond Script Family Boundaries: Towards Unified Open-Set Scene Text Recognition

This paper introduces the first unified cross-script-family open-set scene text recognition framework that combines CAM-CTC alignment, dynamic prototype matching, and heterogeneous character-word co-training to enable zero-shot recognition and rejection across CJK, Indic, and low-resource Yi scripts within a single model.

BIP: Bi-level Information Transfer and Completion Prompting for Visual Recognition with Missing Modalities

Addressing feature distribution inconsistency and cross-layer prompt crosstalk in missing-modality visual recognition, BIP introduces a bi-level prompt framework that pairs instance-level semantic alignment completion (AlignPrompt) with missing-type-aware hypernetwork prompting (HyperPrompt), outperforming both feature reconstruction and prior prompt baselines with only 3.8% tunable parameters over a frozen CLIP backbone.

Break Visual-Linguistic Asymmetry: Unleashing VLM's Cross-Modal Potential for General Face Forgery Detection

Addressing the inherent cross-modal and intra-modal visual-linguistic asymmetry of vision-language models (VLMs) in face forgery detection, this paper introduces Asy-Det—an asymmetry-guided parameter-efficient detection framework combining a two-phase Meta Forgery Adapter (MFA) and an Asymmetry-Guided Interactor (AGI) to achieve multi-granularity bidirectional cross-modal alignment, delivering an image-level AUC of 92.94% on unseen Celeb-DF-v2 with only 4.41M trainable parameters.

Ceptor: Vision-Language Model-Infused Diverse Guidance for Detecting Anything

Ceptor draws inspiration from human visual search by employing a pre-trained vision-language model (VLM) as both the image backbone and multi-prompt generator, integrating a spatial Pyramid RoI Extractor and Multi-Route FFN to achieve unified, universal open-set object detection across interactive visual, generic visual, textual, and hybrid prompts.

Co-Steer: Cross-Modal Collaborative Steering for Jailbreaking MLLMs

Co-Steer discovers that visual and textual jailbreak perturbations in MLLMs correspond to misaligned projections of a unified low-rank vulnerability direction in intermediate fusion layers; by extracting a universal steering vector via SVD and coordinating continuous image noise and discrete textual suffixes under a steering alignment loss, it achieves state-of-the-art universal jailbreak performance across MM-SafetyBench, HarmBench, AdvBench, and black-box commercial models.

CoCo-IR: Conversational Composed Image Retrieval

Addressing the single-turn constraint of conventional composed image retrieval, this paper introduces Contextual Composed Image Retrieval (CoCo-IR) alongside TIE, an LMM-based retriever featuring an \(\langle\text{EMB}\rangle\) global bottleneck token, hybrid attention, and an autonomous data engine, achieving a 4-turn R@1 of 44.1% where prior methods collapse.

D-VLAM: Differential Vision and Language Mixing for Rehearsal Free Continual Learning

Addressing the heavy reliance on external LLM descriptors and attention noise in rehearsal-free continual learning, D-VLAM explicitly mixes vision and language modalities within lightweight prefix prompt generators and introduces differential prefix tuning to eliminate vision noise, achieving new state-of-the-art accuracy with minimal parameters.

DeCoPatch: Revealing Causal Latent Subspaces in Vision-Language Models for GUI Grounding

Tsinghua University and Pengcheng Laboratory mechanistically reveal that visual markers enhance GUI grounding via low-rank latent subspaces in late decoder layers, and propose DeCoPatch—a marker-free decode-time causal intervention method that selects critical FFN neurons during prefill and scales their activations during decoding, achieving consistent grounding gains across four benchmarks with near-zero overhead.

DICE: Disentangled Instance-Class knowlEdge prompt tuning via SAE for Vision-Language Models

Addressing the misalignment and visual ungroundedness of predefined LLM class descriptions, DICE reformulates the Sparse Autoencoder (SAE) into a learnable concept dictionary and proposes a co-activation score to dynamically disentangle and select instance-specific concept vectors for prompt fusion, setting a new benchmark for few-shot prompt tuning.

DIGS: Differentiable, Incremental, Global, Scalable Pruning for Language Models

Addressing the long-horizon multiplier explosion and late-stage gradient oscillations in dual-ascent \(L_0\) structured pruning, DIGS introduces per-step gate-gradient norm bounds and closed-form dual updates paired with a single-backward engine GNCL-Opt, scaling structured channel pruning smoothly beyond 100B tokens and yielding monotonic accuracy gains under fixed sparsity.

DiscoVL: Unveiling Disentangled Cross-Modal Representation Learning via Orthogonal Adversarial Regularization for Vision-Language Models

DiscoVL introduces a disentangled cross-modal representation learning framework that decouples shared transferable semantics from task-specific cues via a multi-branch low-rank residual aligner and employs orthogonal adversarial triplet regularization to prevent centroid collapse, achieving superior generalization across 15 benchmarks.

Disentangling and Reusing Interaction Cues for Zero-Shot HOI Detection

To resolve the two-fold bottleneck where fine-grained interaction cues are overshadowed by global semantics and deep layers fall prey to background shortcut bias, DRIC decouples interaction-specific residuals via context-orthogonal projection and aggregates multi-depth features via stratified local fusion to recover essential contact details.

Divide and Align: Disentangled Vision-Language Learning for Text-Based Person Anomaly Search

To tackle representation entanglement among appearance, action, and background in text-based person anomaly search, this paper proposes SeDA, a disentangled vision-language retrieval framework that projects embeddings into factor-specific subspaces via Semantic Token Projection and enforces semantic separation using LLM-decomposed caption supervision, significantly improving anomaly search and out-of-distribution generalization.

Do Vision Language Models Recognize Visual Ambiguity?

Addressing the issue where vision-language models produce overconfident predictions on visually ambiguous inputs, this paper proposes Visual Semantic Entropy (VSE), which perturbs only the image and measures mass-weighted dispersion among semantic answer prototypes, substantially outperforming prior uncertainty estimators across five VLMs and five VQA benchmarks.

DriveFine: Refining-Augmented Masked Diffusion VLA for Accurate and Robust Driving

DriveFine explores masked diffusion LLMs (dLLMs) for autonomous driving trajectory planning, introducing a lightweight block-level MoE to decouple parallel trajectory generation from iterative self-refinement and establishing state-of-the-art accuracy and generalization on NAVSIM and Navhard benchmarks.

Dual-Generalization-aware Minimization for Continual Fine-Tuning of Vision-Language Models

To prevent catastrophic forgetting and zero-shot capability collapse caused by single-task flatness failing under sequential distribution shifts, this paper proposes Dual-Generalization-aware Minimization (DGM), an optimization framework that couples pre-trained zero-shot alignment with task-specific visual uncertainty into adversarial perturbation generation to reach jointly flat minima.

ECC: Encoder-Centric Corruption for Fine-Grained Vision in VLMs

Addressing granularity mismatch, low-frequency bias, and object hallucination in vision-language models, this paper proposes Encoder-Centric Corruption (ECC), which injects Gaussian noise into shallow encoder feature space, retains conjunctive masking, and decouples denoising from mask reconstruction via a disruption loss to significantly improve fine-grained vision and multimodal alignment.

EmbedCopilot: Evaluating Vision-Language Models for Hardware-Aware Embedded System Development

Introduces EmbedCopilot-Bench, the first multimodal benchmark for hardware-aware embedded development, pairing an iterative self-reflective LLM-as-a-Judge with emulator- and hardware-in-the-loop Execution Success Rate (ESR) evaluation to uncover critical pin-level visual grounding bottlenecks in state-of-the-art LVLMs.

Enhancing prompt-image alignment evaluations via cyclic mutual information maximization

To resolve the decoupling between visual quality and prompt alignment as well as the overwhelming of sparse text by dense visual features, this paper proposes CMIM-AIGIQA, a cyclic mutual information maximization framework that filters modality noise forward and preserves critical prompt constraints backward, achieving state-of-the-art alignment evaluation across three benchmarks.

Finding Highlight Images In Your Albums: From Benchmark To Agentic MLLMs

To tackle high subjectivity and data scarcity in Album Highlight Recommendation (AHR), this paper establishes three benchmark datasets encompassing 1,010 albums and 97K images (including the owner-annotated C-AHR-100) and introduces Highlight4U, a two-stage agentic MLLM framework that synergizes local highlight distribution aggregation with global multimodal reference re-recommendation.

FontCopilot: Towards Generalist Multimodal Large Language Models for Holistic Chinese Font Engineering

FontCopilot is the first unified multimodal large language model framework for holistic Chinese font engineering that consolidates glyph raster-to-vector conversion, Bézier curve sparsification, defect localization, and geometric correction into structured SVG XML code understanding and rewriting, achieving production-grade contour compactness and cutting manual cleanup time to 24.5s per glyph.

Fragmented Text Is Insufficient for Image Representation: Fine-Grained Correspondence in Multimodal Dataset Distillation

Addressing the issues where single fragmented captions fail to capture rich image semantics and standard contrastive losses indiscriminately penalize informative unpaired samples, this paper introduces P2DE with visually grounded text fusion and conditional-entropy-guided step size control, achieving substantial gains in multimodal dataset distillation under tight budgets.

Going Deep: Deep Visual Prompting with LoTeP

To tackle the severe parameter explosion and semantic disruption when visual prompting is extended into deep activations, this paper presents LoTeP, a low-rank tensor visual prompting method that factorizes prompts via Tucker and CP decompositions coupled with a layer-wise cosine rank-decaying schedule to achieve seamless adaptation.

Group3D: MLLM-Guided Semantic Grouping for Open-Vocabulary 3D Object Detection

Group3D addresses the irreversible over-merging issue in multi-view RGB open-vocabulary 3D detection by utilizing an MLLM to construct a scene-adaptive vocabulary and semantic compatibility groups, introducing semantically gated merging alongside asymmetric voxel overlap/containment verification to achieve state-of-the-art results without ground-truth point clouds or 3D supervision.

Inductive Visual Logic for Few-Shot Out-Of-Distribution Adaptation in VLMs

Addressing the failure of vision-language models under representational absence in distant out-of-distribution domains, this paper introduces Inductive Visual Logic (IVL), a training-free framework that leverages surviving primitive descriptive capabilities to construct dual-mode trait dictionaries and performs hierarchically grounded few-shot classification.

iSyncTab: Learning Cross-Modal Feature Sequencing for Image-Tabular Data via Neural Synchrony

Addressing the optimization instability of self-attention caused by arbitrary feature permutations in image-tabular fusion, iSyncTab models feature ordering as a Column Permutation Problem via neural synchrony matching and couples it with a linear-complexity memory-augmented Transformer, achieving top classification accuracy across six benchmarks with favorable computational efficiency.

Iterative Perceptual Alignment for VLMs via Deterministic Reconstruction Feedback

This paper introduces a self-supervised vision-language alignment framework based on deterministic visual reconstruction feedback, utilizing a few-step diffusion distillation engine and perceptual distance evaluation to construct a "reconstruct-compare-refine" loop, which trains a highly discriminative reward model using only 20% of typical data and substantially improves VLM fine-grained attribute grounding via DPO.

Kilometer-Vision: A New Frontier for Large-Scale Spatial Awareness in VLMs

KilometerVision introduces the first benchmark probing city-scale spatial intelligence in Vision-Language Models from real-world walking tour videos spanning up to 1km and 10 minutes, using a cognitive landmark-route-map framework across 1000 multiple-choice QAs to demonstrate that state-of-the-art models circumvent genuine geometric reasoning via 2D visual matching and OCR shortcuts.

Layer-Specific Prompt Fusion Discovery via Differentiable Search in Vision Foundation Models

This paper reformulates the prompt-image token fusion strategy (traditionally hardcoded as concatenation/addition in visual prompt tuning) as a bi-level optimization problem. Utilizing differentiable architecture search (DARTS), it discovers a layer-specific fusion operator (concat / add / affine / cross-attention) for each layer of a frozen ViT. At a cost of tuning only 0.75% of parameters, it pushes the VTAB-1k average score to 77.01%, revealing that "fusion mechanism itself" is a previously overlooked first-order variable in prompt tuning.

LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding

LocateAnything introduces Parallel Box Decoding (PBD), framing 2D bounding boxes as atomic units predicted concurrently in a single forward pass, which—powered by 138M multi-domain training queries and an adaptive hybrid fallback mechanism—achieves over 2.5× higher throughput and superior high-IoU localization accuracy compared to state-of-the-art autoregressive VLMs.

MedRegion-CT: Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation

MedRegion-CT proposes a region-aware 3D CT report generation framework. By employing three modules—Region-based SlowFast Tokenizer, Mask-Driven Visual Extractor, and Lesion Attribute Extractor—it explicitly models the hierarchical diagnostic workflow of radiologists into an MLLM, achieving SOTA performance on structured chest CT report generation benchmarks.

Mitigating Sycophancy in Multimodal Chart Understanding via Vision-Grounded Verification

Addressing the safety-utility trade-off where suppressing multimodal chart sycophancy causes severe over-refusal, this paper proposes Re-Check, a training-free inference framework that integrates blind atomic claim decomposition, a KL-divergence-based Contrastive Visual Dependency Score (CVDS) to certify visual causality, and adaptive three-way routing to achieve robust factual correction while preserving general reasoning utility.

Molmo-Point: Better Pointing for VLMs with Grounding Tokens

Addressing the learning inefficiency and high decoding overhead of text coordinates in vision-language models, MolmoPoint introduces a coarse-to-fine grounding mechanism via three specialized tokens (<PATCH>, <SUBPATCH>, <LOCATION>) that directly cross-attend to visual hidden features, setting new state-of-the-art benchmarks on PointBench, PixMo-Points, and ScreenSpotPro across image, GUI, and video tracking tasks.

MotionAtlas: Detailed Region Captioning for Motion-Centric Videos

MotionAtlas narrows down the task of "describing the motion of an entire video"—which is notoriously difficult to evaluate quantitatively—to "only describing the motion within a target region given a video and a spatio-temporal mask." Along with this formulation, it delivers a diagnostic benchmark with 2,073 MCQs, a bootstrapped data pipeline capable of scalably generating 159k high-fidelity motion descriptions, and a training recipe that consistently boosts Qwen3-VL and Molmo2 across eight motion understanding benchmarks.

MoVA: Modular Long Video-Text Alignment via Asymmetric Dual Projection

MoVA models the video-text alignment problem as a latent variable identification problem, proposing an asymmetric dual projection architecture—a temporal mask network on the text side to filter relevant textual concepts frame-by-frame, and a concept mask network on the video side to isolate text-related visual factors—paired with a modular contrastive learning objective to achieve a structured decoupled alignment of long videos and long texts.

NaVLM-PVC: Progressive Visual Compression for Efficient Native-Resolution Encoding in MLLMs

Addressing the core tension between semantic fragmentation in slice-based encoding and excessive compute in global native-resolution encoding, NaVLM-PVC proposes Progressive Visual Compression (PVC)—combining Refined Patch Embedding (RPE) and hierarchical Windowed Token Compression (WTC)—to deliver 1.9× to 2.4× TTFT latency reductions while preserving fine-grained multimodal performance.

PhyMAGIC: Physical Motion-Aware Generative Inference with Confidence-guided VLM

Addressing the severely under-constrained nature of inferring physical properties from a single static image, PhyMAGIC introduces a training-free framework that actively probes physical properties by generating motion videos with an I2V model, iteratively refines parameters via a confidence-guided VLM loop, and simulates physically consistent 3D dynamics using a differentiable Material Point Method (MPM) solver.

Preventing Expert Collapse in MoE-dVLMs via Modality-Wise Norm Alignment

Addressing catastrophic expert selection collapse on visual tokens in sparse mixture-of-experts discrete diffusion vision-language models, this paper uncovers an upstream geometric mismatch where a 167x norm disparity causes Pre-Norm attention drowning and representational inertia, and proposes TowerAlign to globally rescale visual token norms while preserving internal ViT sink topologies with negligible overhead.

Probe, Anchor, and Amend: Active Test-Time Adaptation of Vision-Language Models

The paper presents PAA, a framework that models visual and textual prototypes as an evolving target-domain semantic state and updates it via a cyclic "Probe-Anchor-Amend" routine, turning sparse binary verification feedback into structured and robust test-time adaptation for vision-language models.

ReDesign: Recovering Editable Design Structures from Raster Images via Agentic Decomposition

ReDesign casts the recovery of editable design structures from raster images into a progressive layer tree expansion managed by a VLM controller over heterogeneous tools, equipped with local graceful verification to accept, prune, or retry branches, achieving state-of-the-art editability and a 7.1× parallel speedup.

Region-Aware Multimodal Interleaving for Animal Re-Identification

To overcome background overfitting and the limitations of expressing fine-grained animal coat patterns in plain text, this paper proposes RAMI, a framework that mines biologically representative foreground regions via lightweight statistical image processing and interleaves regional visual tokens with text placeholders in a shallow Transformer to achieve dense cross-modal interaction, substantially outperforming prior SOTA methods on benchmark and in-the-wild datasets.

Rethinking Attention Reallocation for Multimodal Emotion Recognition

Uncovers the counterintuitive phenomenon where attention reallocation from generated to modality tokens helps in two-modality settings but hurts in three-modality settings, and proposes a training-free attention rectification method leveraging shallow attention priors to regularize entangled deep-layer attention.

ROSE: Real-Time Open-World Scene Understanding from Monocular Video via Compact Multimodal 4D Scene Graphs

To address the lack of structured dynamic spatial grounding in vision-language models, ROSE builds compact multimodal 4D scene graphs from monocular RGB video at 10 Hz via modality-separated VISTA node encodings and introduces 4DSGs as structured training-time context, achieving a state-of-the-art 67.5% (+8.6%) on DSR-Bench.

SafeSAE-VLA: Interpreting OpenVLA Progress Dynamics with Sparse Feature Analysis

SafeSAE-VLA presents an interpretable sparse representation analysis pipeline for Vision-Language-Action (VLA) models, demonstrating that decomposing OpenVLA action-token residual streams with a sparse autoencoder enables top-20 sparse features alone to achieve an AUROC of 0.894 in linear progress discrimination, proving that robotic policies internally maintain compact, inspectable, and writable task-progress subspaces.

Same Person, Different Depiction: Counterfactual Evaluation of Vision-Language Models on Individuals with Limb Deficiencies

This paper introduces InclusiveCFImageBias, a matched-pair counterfactual benchmark (462 pairs, 924 images, 22k+ queries) that holds identity, scene, and activity fixed while toggling visible limb-deficiency cues under ability-neutral prompts, demonstrating that frontier VLMs systematically shift toward higher ratings, more affirmative decisions, and less neutral wording.

Same Pool, Different Answer: Stable Best-of-N Selection for Vision-Language Models

Addressing the issue where small scoring perturbations frequently flip the winning answer in likelihood-based Best-of-N selection on a fixed candidate pool, this paper proposes DSPA, which pairs a frozen full-context reference anchored selector with training-time shortcut filters, cutting the perturbation flip-rate from 44.3% to 24.9% and POPE object hallucination from 9.6% to 6.2%.

SAMPLe: A Sharpness Aware Minimization based Optimizer for Prompt Learning in Vision-Language Models

To tackle the generalization degradation caused by limited prompt parameters easily converging to sharp minima in vision-language models, this paper proposes SAMPLe, a plug-in sharpness-aware optimizer that adaptively balances empirical risk gradient alignment (exploitation) and full-batch orthogonal gradient exploration (exploration), preserving base class accuracy while significantly boosting novel-class and cross-domain generalization.

SCoT: Similarity-guided Conflict-aware Task Consolidation for Continual VQA

SCoT introduces an anchor-relative task vector consolidation framework that measures layer-wise cosine alignment to adaptively modulate plasticity, removes destructive directional conflicts via minimal-norm conditional projection, and stabilizes composition-sensitive encoder-decoder interfaces, achieving near-zero forgetting (AF 0.07 and -1.90) and rare positive backward transfer while outperforming joint multi-task training on VQAv2 and NExT-QA.

SDSA: Shallow-Deep Squeezing Adapter for Vision-Language Models

Addressing the issue where dense cross-modal interaction and unconstrained alignment capacity degrade novel-class generalization in vision-language model adapters, SDSA proposes a hierarchical two-stage "shallow-deep squeezing" adapter that imposes structured token sparsity via random masking and constrains alignment capacity via a shared low-rank subspace and cross-attention, significantly boosting few-shot base-to-novel generalization across 11 benchmarks.

Seeing Isn't Orienting: A Cognitively Grounded Hierarchical Benchmark for Object Orientation in MLLMs

This paper introduces DORI, a cognitively grounded hierarchical benchmark that decouples object orientation into four cognitive dimensions across coarse and granular evaluation tiers, revealing that MLLMs highly capable on general spatial tasks perform near-random on geometric orientation and rotational reasoning.

SiGMA: Sign-Guided Merging and Adaptation framework for Multimodal Continual Instruction Tuning

Addressing the overlooked issue of negative interference during weight consolidation in multimodal continual instruction tuning, SiGMA introduces a parameter sign-guided decoupling framework that uses a shared general subspace for low-drift adaptive tuning during training, and selectively filters and amplifies salient specific parameters at inference, significantly outperforming prior methods on both UCIT and DCL benchmarks.

Staying VIGILant: Mitigating Visual Laziness via Counterfactual Visual Alignment in MLLMs

To address the "visual laziness" issue where MLLMs get misled by language priors despite perceiving correct visual evidence, this paper proposes VIGIL. It introduces a "counterfactual blind state" (where visual attention is masked) as a negative anchor into DPO preference optimization. By using geometric constraints to explicitly widen the log-likelihood gap between the "see" and "blind" states, VIGIL forces high-confidence model responses to be causally anchored to pixels. It outperforms state-of-the-art methods using only 25% of preference data and unexpectedly exhibits emergent spatial grounding capability.

Synesthesia via Direct Latent Augmentation: Bypassing the Decode-Encode Loop for Cross-Modal Distillation

By identifying the inefficiencies of generative decode-encode cycles and the capacity interference of rigid \(L_2\) cross-modal distillation, this paper introduces Direct Latent Augmentation (DLA) and Multilayer Explicit Simulated Synesthesia (MESSy), allowing a vision-only student to internalize 3D geometric and acoustic priors without requiring paired physical datasets.

Towards Reliable Medical Large Vision-Language Models via Counterfactual Preference Optimization

To mitigate clinical hallucinations caused by non-diagnostic background shortcuts and disease co-occurrence bias in Medical Large Vision-Language Models (Med-LVLMs), this paper proposes Counterfactual Medical Preference Optimization (CoMedPO), which uses causal mediation analysis to eliminate harmful background direct effects while preserving essential anatomical indirect effects without incurring any inference overhead.

Unbalanced Optimal Transport for Efficient Visual Document Retrieval

To resolve the token explosion and excessive late-interaction (MaxSim) overhead in multi-vector visual document retrieval models like ColPali, this paper presents a training-free token compression framework via Unbalanced Optimal Transport (UOT) that unifies redundancy reduction, attention importance, and coverage capacity into an alternating optimization process while theoretically upper-bounding retrieval error.

Unleashing the Power of Large-Scale ViT in Zero-Shot SBIR: A Strong Baseline with Multi-Layer Feature Aggregation

Addressing the semantic over-compression dilemma in large Vision-Language Models where terminal [CLS] tokens discard essential structural primitives, this paper proposes an anti-compression baseline on ViT-L-14 using Multi-Layer Feature Aggregation (MLFA) and Phased Cross-Modal Interaction (PCMI) for coarse-to-fine alignment, achieving state-of-the-art performance across multiple ZS-SBIR benchmarks.

VISOR++ : VISUAL INPUT BASED STEERING FOR LARGE VISION LANGUAGE MODELS

Introduces VISOR++ (Visual Input based Steering for Output Redirection), an input-space steering framework that replaces invasive runtime activation vectors with optimized visual inputs via differentiable preprocessing and dual-momentum spectral augmentation (CWA-SSA), achieving bidirectional behavioral control across heterogeneous VLMs without requiring internal model access.

VIVAS: Vitalizing Visual Perception in VLM Pre-training via Vision-language Unified Autoregressive Supervision

To overcome the severe deficiency in fine-grained visual perception caused by text-dominant pre-training bias, VIVAS incorporates a dense-structural-semantic vision tokenizer and a unified token space into large-scale pre-training, enabling unified autoregressive supervision over both visual details and text with only 3.8% additional inference latency.

When Sinks Help or Hurt: Unified Framework for Attention Sink in Large Vision-Language Models

This paper categorizes "visual attention sinks" in Large Vision-Language Models (LVLMs) into two types: V-sinks inherited from the vision encoder, and L-sinks emerging in the deep layers of the LLM. It reveals their double-edged sword effect—encoding global scene priors to benefit coarse-grained reasoning while suppressing fine-grained perception. Accordingly, the authors propose LSG, a layer-wise gating mechanism that freezes the backbone and trains only a lightweight MLP. LSG dynamically adjusts Key scaling of sinks based on the input and layer depth, consistently improving performance across various multimodal benchmarks.

World-in-Loop: Online Correction via Event-Triggered World Models for Robust VLA Policies

Addressing the compounding execution errors of open-loop VLA policies and the prohibitive computational burden of continuous world model rollouts, World-in-Loop selectively activates discriminative failure assessment and generative video imagination exclusively at grasp initiation keyframes, rescuing near-failure trajectories before physical execution.

X-Stream: Benchmarking MLLMs as Multiplexers for Multi-Stream Understanding

X-Stream is the first benchmark for streaming understanding over multiple concurrent video streams: a sufficiency+necessity dual-verification pipeline builds 4,220 cross-stream QA pairs, and MLLMs are abstracted as bandwidth-limited naive multiplexers to systematically compare spatial, temporal, and semantic multiplexing strategies; the best model, Gemini 3 Pro, scores only 49.60 overall, and forward (proactive) QA scores generally stay below 21, exposing a large gap in concurrent-stream and proactive response capabilities.