🧠 VLM Reasoning¶

🧪 ICML2026 · 31 paper notes

📌 Same area in other venues: 📷 CVPR2026 (150) · 🔬 ICLR2026 (112) · 💬 ACL2026 (32) · 🤖 AAAI2026 (10) · 🧠 NeurIPS2025 (30) · 📹 ICCV2025 (15)

🔥 Top topics: Reasoning ×27 · Multimodal/VLM ×14 · LLM ×3 · Reinforcement Learning ×2

3D-RFT: Reinforcement Fine-Tuning for Video-based 3D Scene Understanding: This work adapts the LLM-oriented "Reinforcement Learning with Verifiable Rewards (RLVR)" to video-driven 3D scene understanding. By using GRPO to fine-tune a 4B 3D-aware VLM directly with evaluation metrics (such as 3D IoU, F1, and accuracy) as rewards, the training objectives are aligned with evaluation criteria. Consequently, the 4B model outperforms an 8B baseline on 3D video detection, 3D visual grounding, and spatial reasoning tasks.
3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models: 3ViewSense argues that the bottleneck of VLM spatial reasoning is not insufficient visual features or weak linguistic reasoning, but the lack of a stable 3D intermediate representation. Consequently, it requires the model to first induce front, left, and top views from a single image before reasoning based on these orthographic views, significantly outperforming same-scale VLMs in occlusion counting and view-consistent spatial reasoning.
Active Exploring like a Pigeon: Reinforcing Spatial Reasoning via Agentic Vision-Language Models: This work transforms VLM spatial reasoning from a "passive observation" approach into an agentic workflow that actively selects views based on questions, updates a cognitive map, and verifies reasoning using executable spatial assertions. By fine-tuning Qwen2.5-VL-3B with dense rewards, it achieves 80.5% overall accuracy on MindCube-Tiny, specifically improving the Rotation subset to 85.0%.
Bad Seeing or Bad Thinking? Rewarding Perception for Vision-Language Reasoning: This paper enforces a split in VLM output into <recognition> perception blocks and <think> reasoning blocks. It introduces a perception reward \(R_P\) determined by whether a "blindfolded" text reasoning agent (which only sees the VLM's perception text without the image) can correctly answer the question, paired with Structured Verbal Verification (SVV) as an outcome reward \(R_O\). MoCA uses \(R_P\) as a gate for modality-level credit assignment, enabling a 7B model to improve across 9 perception/reasoning/rich-modality benchmarks simultaneously, surpassing GPT-4o on multiple metrics.
Breaking Dual Bottlenecks: Evolving Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasoners: To address the "understanding-generation gap" (capable of understanding but failing to generate) in unified multimodal models for anything-to-image (X2I) tasks, this paper proposes the Self-Adaptive Interleaved Reasoner. Using a hierarchical data synthesis pipeline, 50,000 samples are routed between three modes: direct generation, self-reflection, and multi-step planning. The model is trained via SFT + GRPO with step-wise reasoning rewards and intra-group complexity penalties, enabling Emu3.5 to outperform closed-source models like GPT-4o and Gemini 2.5 Flash on KRIS-Bench and OmniContext.
Decomposed On-Policy Distillation for Vision-Language Reasoning: Steering Gradients for Visual Grounding: The authors decompose the KL loss of multimodal on-policy distillation into "language prior" and "visual grounding" sub-objectives based on a Bayesian chain. They find that the gradients of these two are nearly orthogonal, and standard distillation merely takes a passive bisector. Consequently, they propose Visual Gradient Steering (VGS) to actively bias the update direction toward the visual subspace, achieving average gains of +2.37%/+1.56% across seven multimodal reasoning benchmarks for Qwen3-VL 8B→2B/4B.
Efficient Reasoning with Hidden Thinking: Heima distills each stage (summary / caption / reasoning) of lengthy Multimodal LLM (MLLM) Chains-of-Thought (CoT) into a single special thinking token. This allows the model to "think" in latent space, reducing the token count from the 100-200 range to 13-16 while achieving zero-shot accuracy more stable than LLaVA-CoT. An accompanying LLM "interpreter" is trained to reconstruct the textual reasoning chain from the thinking token's hidden states, empirically validating the information-theoretic upper bound of compression loss.
Find, Fix, Reason: Context Repair for Video Reasoning: Addressing the dilemma in video reasoning where "on-policy RL stagnates at capacity ceilings and off-policy distillation suffers from entropy collapse," this paper introduces a frozen, tool-integrated large teacher model. When a student's rollout fails, the teacher inserts minimal "evidence patches" (e.g., key-frame intervals, error types), enabling the student to re-attempt the same question. These repaired trajectories are then incorporated into GRPO optimization through a chosen-rollout mechanism.
From Correspondence to Actions: Human-Like Multi-Image Spatial Reasoning in Multi-modal Large Language Models: Inspired by human spatial cognition, HATCH designs two complementary training objectives for MLLMs: aligning cross-view patch features using geometric supervision (PaStA), and using reinforcement learning to force models to generate explicit "viewpoint change actions" before answering (ActoR). Using only a 3B base model, it achieves multi-image spatial reasoning performance comparable to GPT-5.2.
From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models: This paper argues that current VLM post-training overemphasizes "long-chain reasoning" while neglecting perception bottlenecks. It explicitly decouples post-training into three independent stages: "Visual Perception \(\rightarrow\) Textual Reasoning \(\rightarrow\) Visual Reasoning," using RLVR (instead of caption SFT) to specifically refine perception. This approach improves Qwen3-VL-8B by approximately +5.9% and +1.2% on visual math and perception benchmarks, respectively, while shortening reasoning traces by 20.8%.
From Shortcuts to Reasoning: Robust Post-Training of Theory of Mind with Reinforcement Learning: The authors first reveal that mainstream ToM datasets are contaminated by "shortcuts" (achieving 99% accuracy via spurious correlations rather than true mental reasoning). They propose a lightweight auditing framework to filter these datasets and systematically demonstrate on four shortcut-free datasets that reinforcement fine-tuning with explicit reasoning chains (Thinking-RFT) outperforms SFT by 6% on average (approximately 10% in higher-order/multimodal scenarios) and exhibits better generalization and counterfactual robustness because RL teaches the model to anchor reasoning on true causal cues.
Imagination Helps Visual Reasoning, But Not Yet in Latent Space: This paper employs causal mediation analysis to decompose "Latent Visual Reasoning (using MLLM hidden states as latent tokens for visual imagination)" into a causal chain \(X\to Z\to Y\). Empirical evidence reveals that latent tokens are neither varied with inputs (Input-Latent disconnection) nor significantly impact the final answer (Latent-Answer disconnection), questioning their necessity. Consequently, a simple alternative, CapImagine, is proposed to explicitly write visual imagination as text, outperforming complex latent-space methods on visual perception benchmarks.
iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement Learning: Addressing the counter-intuitive phenomenon where "explicit visual grounding actually hinders CoT reasoning," the authors propose iVGR—a dual-stream GRPO training framework. It allows textual CoT and grounded CoT (with boxes) to rollout simultaneously. Using a consistency reward, the visual localization capabilities of high-quality grounded trajectories are "internalized" into pure textual CoT, enabling the model to reap the benefits of grounded reasoning without outputting coordinates during inference.
Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training: VISTA transforms self-improvement training for Multimodal Large Language Models (MLLMs) into a two-stage pipeline: "supplementing samples for difficult problems via prefix resampling" and "filtering pseudo-positives through Vision-aware Attention Score (VAS)." This approach achieves an average improvement of +13.66% in mathematical and medical multimodal reasoning on Qwen2.5-VL-3B.
Learning GUI Grounding with Spatial Reasoning from Visual Feedback: Gui-Cursor reformulates GUI grounding from "one-shot coordinate prediction" into an interactive search of "moving the cursor on the screen to find the target." By training the VLM with GRPO using a dense reward with trajectory penalties, the model leverages visual feedback from rendered cursors to align numerical coordinates with screen positions. With only 8K samples, it improves GTA1's performance on ScreenSpot-Pro from 50.1% to 58.1%.
LIMSSR: LLM-Driven Sequence-to-Score Reasoning under Training-Time Incomplete Multimodal Observations: The authors remodel multimodal Action Quality Assessment (AQA) with "incomplete modalities during training" as an "LLM-based conditional sequence-to-score reasoning" problem. By utilizing prompts and special tokens, the LLM completes missing semantics without complete data supervision. Combined with mask-aware dual-path fusion to suppress hallucinations, the method outperforms SOTAs relying on complete training data across three AQA datasets.
CSMR (Look on Demand): A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning: Inspired by Baddeley's working memory theory, CSMR treats "when to introduce visual evidence into reasoning" as a dynamic decision-making process. The LLM maintains the reasoning state and invokes an independent perception module (VLM) for visual evidence on demand until sufficiency is reached. It addresses the flaws of existing paradigms (pre-reasoning textualization losing details / unified VL space being contaminated by language priors), outperforming baselines zero-shot across multiple multimodal reasoning benchmarks.
MentisOculi: Revealing the Limits of Reasoning with Mental Imagery: The authors developed MentisOculi, a procedural, hierarchically difficult multi-step visual reasoning benchmark consisting of five tasks that "can only be solved via internal mental imagery." By systematically testing whether frontier models can utilize "mental imagery" to assist in reasoning like humans, the study concludes that current explicit visual strategies (latent tokens, generated images, video) fail to consistently outperform pure text baselines. More pointedly, Unified Multimodal Models (UMMs) cannot effectively utilize even ground-truth visualizations, exposing a dual bottleneck of "generation errors" compounded by "interpretation errors."
MET-Bench: Multimodal Entity Tracking for Evaluating the Limitations of Vision-Language and Reasoning Models: This paper proposes MET-Bench, a multimodal entity tracking benchmark across three domains (Chess, Shell Game, and Minecraft). It requires vision-language models to track entity state changes from sequences of actions presented in either text or image format. Findings show that "image-based actions" are significantly more difficult than "text-based actions," and this gap originates from visual reasoning rather than perception. While GRPO reinforcement learning yields substantial improvements within a single modality, it fails to transfer across modalities.
Native Active Perception as Reasoning for Omni-Modal Understanding: OmniAgent shifts long video understanding from the passive "watch-it-all" paradigm to an active perception approach of "iterative look-as-needed." Using an Observation-Thought-Action (OTA) loop within a native omni-modal model, it distills audio-visual cues into persistent text memory and immediately discards raw media, thereby decoupling reasoning costs from video duration. Combined with Agentic SFT for cold-starting and Agentic RL with TAURA, the 7B model achieves 50.5% on LVBench, surpassing the 10x larger Qwen2.5-VL-72B (47.3%), and exhibits positive test-time scaling where performance improves with more reasoning turns.
Position: The Systemic Lack of Agency in Visual Reasoning: This position paper argues that current VLMs exhibit a systemic "lack of visual agency"—they can perceive details when explicitly directed but fail to autonomously search for implicit visual cues that are unmentioned in the prompt yet essential for problem-solving. Through a formal framework, a four-quadrant taxonomy, and the specially constructed V-IRD benchmark, the authors demonstrate that even the strongest closed-source models fail primarily because they "did not look for evidence" rather than "could not compute the answer."
R\(^3\)L: Reasoning 3D Layouts from Relative Spatial Relations: R³L attributes two types of systemic errors in multi-hop "relative spatial relation" reasoning of MLLMs (semantic drift and metric drift) to "recurrent reference frame transformations." Through three modules—Invariant Spatial Decomposition (shortening relation chains), Consistent Spatial Imagination (an imagine-and-revise loop to eliminate conflicts), and Supportive Spatial Optimization (global-to-local pose re-parameterization)—it enables GPT-5 to generate open-vocabulary 3D scenes where collision and out-of-bounds rates across 9 scene types are near zero, significantly outperforming LayoutVLM/Holodeck/LayoutGPT in semantic metrics.
Reason, Then Re-reason: Cross-view Revisiting Improves Spatial Reasoning: To address the issue where egocentric video spatial reasoning is "trapped by camera trajectories and relies on semantic priors to guess geometry," this paper proposes the training-free ReRe framework. It first forms a spatial hypothesis on the original video (Reason), then verifies or corrects the hypothesis using a new "oblique overview" video rendered from 3D geometry (Re-reason), enabling open-source MLLMs to approach closed-source SOTA on VSI-Bench / STI-Bench.
ReVSI: Rebuilding Visual Spatial Intelligence Evaluation for Accurate Assessment of VLM 3D Reasoning: This paper systematically reveals structural failures in the widely used VSI-Bench due to 3D annotation drift and frame sampling inconsistency. By re-annotating 381 scenes and 5365 objects and designing frame-budget adaptive QA alongside "dummy video" stress tests (removing frames containing target objects), the authors construct ReVSI, a high-fidelity spatial intelligence benchmark. Evaluations show that open-source VLMs suffer performance drops of up to 40% on ReVSI while exhibiting high hallucination rates on dummy videos, exposing a systematic overestimation of current 3D reasoning capabilities in VSI-Bench.
Spectral-Progressive Thought Flow for Lightweight Multimodal Reasoning: SpecFlow shifts multimodal spatial reasoning from "pixel thinking" to "spectral thinking"—using Block Discrete Cosine Transform + Flow Matching + Progressive Frequency Activation to maintain visual intermediate thoughts in a fixed-size spectral workspace, combined with Classifier-Free Guidance (CFG) to let text guide visual evolution, reducing KV cache by 1.6–2.1× while maintaining spatial reasoning accuracy.
Temporal-Aware Reasoning Optimization for Video Temporal Grounding: This paper proposes TaRO to address the issue of "glitzy but insubstantial reasoning" in RL-trained Video Temporal Grounding (VTG). It constructs high-quality reasoning trajectories using dense captions for warm-starting and introduces a reward based on the "confidence drop after shuffling keyframes" to measure reasoning quality, forcing the model to truly "think with time."
The Perceptual Bandwidth Bottleneck in Vision-Language Models: Active Visual Reasoning via Sequential Experimental Design: This paper formalizes the issue of "VLMs lacking detail perception" as a Sequential Bayesian Optimal Experimental Design (S-BOED) problem and proposes FOVEA, a training-free module based on a computable proxy objective of "Coverage \(\times\) Resolution," which consistently outperforms Direct and ReAct-style baselines on high-resolution and remote sensing benchmarks.
Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces: The authors construct SSI-Bench, a benchmark consisting of 1,000 ordering-based VQA questions focused on "constrained structured spaces" (real 3D structures like roofs, bridges, and towers). It requires VLMs to provide a complete permutation of 3–4 candidate components based on geometric or topological criteria. Evaluations of 31 VLMs reveal that the strongest closed-source model, Gemini-3-Flash, achieves only 33.6%, and the best open-source model, GLM-4.6V, reaches 22.2%, while humans achieve 91.6%. This highlights the lack of consistent spatial reasoning capabilities in current VLMs within real 3D scenes governed by joint geometric, connectivity, and physical feasibility constraints.
VideoKR: Towards Knowledge- and Reasoning-Intensive Video Understanding: VideoKR is the first large-scale post-training corpus specifically oriented towards "knowledge- and reasoning-intensive video understanding." It features 145,000 newly collected CC-licensed professional domain long videos and 315,000 synthesized QAs with Chain-of-Thought (CoT) reasoning. The "human-in-the-loop + skill-oriented" synthesis pipeline ensures difficulty, diversity, and reliability. Additionally, the VideoKR-Eval benchmark is constructed by removing "single-frame answerable" shortcuts. Under a standard SFT→GRPO workflow, data design alone allowed a 7/8B model to outperform previous post-training methods in knowledge-intensive video reasoning.
Vision-aligned Latent Reasoning for Multi-modal Large Language Model: This paper proposes VaLR: a method that inserts several "latent tokens" before each step of CoT reasoning in MLLMs and performs representation alignment (REPA) between these tokens and the patch features of visual encoders like DINOv3, SigLIP, or \(\pi^3\). This continuously "feeds" visual information back into the model during long-chain reasoning, improving the accuracy of Qwen2.5-VL on VSI-Bench from 33.0% to 52.9% and enabling MLLMs to demonstrate "longer reasoning leads to higher accuracy" test-time scaling behavior for the first time.
What You Think is What You See: Driving Exploration in VLM Agents via Visual-Linguistic Curiosity (GLANCE): GLANCE introduces a self-supervised "think-and-see alignment" head for RL in VLM agents: the "next-state prediction" generated in the LLM's CoT is mapped via a lightweight projector to the representation of the actual next frame encoded by an EMA target visual encoder. The gap between prediction and reality serves simultaneously as an intrinsic curiosity reward, a training signal for the visual encoder, and an alignment loss to ground the internalized world model. Combined with a curriculum exploration mechanism that periodically resets the projector to combat curiosity drain, GLANCE consistently outperforms existing exploitation-only VLM-RL methods across 5 agentic tasks.