🧩 Multimodal VLM

🎞️ ECCV2026 · 21 paper notes

📌 Same area in other venues: 📷 CVPR2026 (419) · 🔬 ICLR2026 (211) · 💬 ACL2026 (82) · 🧪 ICML2026 (89) · 🤖 AAAI2026 (75) · 🧠 NeurIPS2025 (107)

🔥 Top topics: Multimodal/VLM ×7 · Layout & Composition ×3 · Adversarial Robustness ×2

360° Image Perception with MLLMs: A Comprehensive Benchmark and a Training-Free Method

360Bench exposes failures in fine-grained and spatial understanding of high-resolution panoramas, and training-free Free360 combines question-relevant entities, local attributes, inter-entity relations, and viewer directions into a scene graph, raising Qwen2.5-VL-7B overall accuracy from 38.1% with CMP input to 45.3%.

3D-Aware VLMs with Implicit and Explicit Geometries

VLM-IE3D injects both implicit geometric features and explicit tokens derived from reconstructed depth into Qwen2.5-VL, raising 3D video detection [email protected] from the backbone's 30.9 to 42.8, although its average spatial reasoning gain over an existing geometry-enhanced model is modest.

3D FaceShell: Attribute Transfer in 3D Face Avatars as a VLM Defense Mechanism

3D FaceShell adds a learnable Gaussian shell to a frozen 3D face and uses multi-view semantic alignment to redirect VLM attribute predictions, reaching 48.7% injection rate on VideoLLaMA3 with LPIPS 0.1499 and identity similarity 0.7629 on frontal views; its contribution is the trade-off between semantic intervention and visual fidelity, not uniformly maximal attack success.

Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO

CycleGRPO trains one multimodal model to describe a region and then localize its own description, using reconstructed-mask overlap to improve both directions without caption ground truths in its main training loop; SAMTok-4B improves from 61.9 to 67.7 on DLC-Bench and from 57.5 to 67.6 overall mask gIoU on GroundingSuite.

Adapting MLLMs for Nuanced Video Retrieval

TARA turns a generative MLLM into a unified retrieval encoder by fine-tuning its shared language model on just 20,000 carefully constructed text triplets, raising Tarsier 2's CiA-SSv2 Chiral mAP from 77.7 to 88.9 while also improving negated queries and composed video retrieval.

AMCI: Unlock the Potential of Large Multimodal Models for Fine-grained Open-world Classification via Adaptive Memory Context Injection

AMCI turns previously observed test images and their attribute descriptions into retrievable memory, then uses a language model to refine neighboring context into recognition prompts, improving Qwen2-VL 7B's mean CFI across five fine-grained datasets from 36.6 to 63.0 without updating parameters, while still depending on stream quality and additional language-model inference.

An Inverse-Adversarial and Difficulty-Adaptive Robust Vision-Language Model

IADA-RVLM constrains adversarial training with inverse-adversarial examples moved toward the correct class, learns specialized knowledge through visual prompts and a text adapter, and adjusts text knowledge fusion using class semantic distances, achieving the best Nat/Rob harmonic mean on novel classes across five datasets, including 52.52% PGD-100 robust accuracy on Caltech101.

AnchorGUI: Asymmetric Memory for Dual-Scale Learning in GUI Navigation

AnchorGUI uses cognitive state anchors to compare expected action effects with actual interface changes, compressing ordinary steps into text while retaining visual evidence for mismatches, so that one shared VLM supports both intra-trial correction and cross-trial distillation, achieving 57.3% single-attempt and 69.2% three-attempt cumulative success on AndroidWorld.

Art Beyond Semantics: Sheaf-Informed Contrastive Learning for Multi-Relational Representations

CANVAS modulates each artwork or text into relation-specific representations and uses a training graph's line-graph heat kernel to soften false-negative penalties, achieving the strongest image-to-text retrieval results across three art datasets without requiring full graph connectivity at inference.

ATOMIC: A Domain-Specific Vision-Language Model for Transmission Electron Microscopy

ATOMIC retains the LLaVA-v1.5 architecture, converts published TEM subimages and captions into complementary visual and domain supervision, and performs two-stage adaptation, reaching 75.2% TEM-MCQ accuracy and 37.5% TEM-VQA Answer Word Coverage with its 7B model, without yet replacing quantitative microscopy analysis.

ATP-Bench: Towards the Agentic Tool Planning for MLLM Interleaved Generation

ATP-Bench unifies five visual tools for interleaved text-image planning and evaluates 7,702 human-verified QA pairs with three MAM judge agents that separately inspect valid calls, missing visuals, and overall composition; the strongest tested model, Gemini 3 Pro, reaches an average FS of 79.88, with substantial gaps remaining in complex travel and renovation responses.

Attention-Logit Steering to Compositional Generalization for Continual VQA

Q-STEER uses a shared controller to coordinate new expert capacity with corrections to where answer tokens attend, achieving 53.68 AP, 4.51 AF, and 51.00/51.48 Novel/Seen composition scores on continual VQA v2 without inference-time task labels or sample replay.

Attention is Case-Sensitive

Controlled casing interventions across 13 models show that uppercase and alternating-case targets attract attention without guaranteeing better accuracy; in VLMs, stronger local target concentration can coexist with reduced whole-image attention, so attention enhancement should not be equated with better understanding.

AutoV: Loss-Oriented Ranking for Visual Prompt Retrieval in LVLMs

Instead of using one visual prompt for every question, AutoV labels candidate preferences with a frozen LVLM's reference-answer losses and trains a lightweight instance-level ranker, raising LLaVA-OneVision 7B's VizWiz score from 58.2 to 68.4 while allowing the selection strategy to transfer to other models.

Background Blurring Matters: Improving Visual Grounding by Merging Text-Irrelevant Tokens

ToB preferentially merges visually similar but text-irrelevant tokens in deeper visual encoder layers, preserving fine-grained target representations and improving a DINOv2-B/BERT-B baseline from 76.05% to 80.36% on RefCOCOg test-u while increasing inference speed from 20.80 to 21.57 fps.

BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension

BaFCo benchmarks layout analysis and information extraction on carefully annotated Bangladeshi government forms, varying label granularity, prompting, and reasoning effort to show that Gemini 3 Pro reaches only 0.1177 fine-grained [email protected] with high-reasoning zero-shot prompting, despite achieving 0.848 Bangla extraction Macro-F1, exposing the gap between reading field values and precisely reconstructing form structure.

Benchmarking MLLMs on Mistake Recognition and Explanation in Single-Step Components of Cooking

Cook-MRE keeps cooking videos unchanged while replacing one instruction element, using 1,289 samples to test whether models can detect and explain deviations; the best default Full Pass Accuracy among 13 open-source MLLMs is only 57.0%, showing that flagging a mistake is far from seeing and explaining it correctly.

Benchmarking Vision-Language Models for Microscopic Plant Image Understanding

PlantMicro organizes 5,410 plant microscopy images into 9,718 question-answer pairs across 10 tasks, showing that general-purpose VLMs can approach perfect modality recognition while struggling with host and pathogen identity; task-restricted visual retrieval substantially improves recognition, whereas reasoning prompts and a single demonstration do not reliably help.

Beyond Atomic Layouts: Compositional Design Understanding with Vision-Language Models

The paper introduces CoDeLayout and the MASON post-training approach, combining element grounding supervision with explicit layer-aware spatial metadata to improve Qwen2.5-VL 7B from 88.80% to 91.66% weighted compositional pairing accuracy over full-data direct finetuning.

Beyond Categorical Matching: Intra-Class Graded Relevance Estimation for Cross-Modal 3D Retrieval

ReMU3D reparameterizes the joint image-text-point-cloud space with SCI and then aggregates arbitrary query-modality combinations through Q-MoE, raising NDCG@10 from the best baseline's 0.76 to 0.93 on the new INGRE graded-relevance benchmark.

COMPASS: Grounding Composition-Intent Guidance in Unified Multimodal Models

COMPASS adds a composition-expert branch and an expert token to Ming-Lite-Uni, then guides generation with grayscale pixelated references and restricted information pathways, reporting composition-recognition mAP of 90.6 and generation composition consistency of 0.74 on Comp-11, although its mAP aggregation and self-evaluated composition metrics require caution.