Learning Active Perception for Pixel-Space Reasoning via Visual-Intent Stratified GRPO

Conference: ECCV 2026
Paper: ECCV Official
Area: Multimodal VLM / LLM Reasoning
Keywords: Pixel-Space Reasoning, Active Perception, Reinforcement Learning, Visual-Intent Stratification, Visual Laziness

TL;DR

Addressing the visual laziness failure mode and degradation of multi-zoom behavior when training pixel-space reasoning models with standard GRPO, this paper introduces Visual-Intent Stratified GRPO (VIS-GRPO), which stratifies rollout groups jointly by zoom depth and spatial-coherence-based visual intent (explore vs. verify) to ensure fair group-relative advantage normalization, boosting performance by 8.7% on HR-Bench 8K and 7.6% on MME-RealWorld.

Background & Motivation

Vision-Language Models (VLMs) have made dramatic strides across complex visual understanding and multimodal reasoning tasks, substantially catalyzed by post-training with reinforcement learning (RL). However, prevailing VLM post-training paradigms frame visual reasoning strictly as autoregressive text generation conditioned on a fixed, pre-computed visual representation. In this passive, single-look setting, the visual representation is frozen at the very beginning of the trajectory. While functional for clean, low-resolution images, this framework encounters fundamental limitations on high-resolution and visually dense inputs where the correct answer hinges on tiny text, subtle symbols, or fine-grained localized details that are inevitably blurred or wiped out by global image resizing.

To overcome these constraints, pixel-space reasoning has emerged as a promising new frontier. Instead of relying on a single coarse snapshot, pixel-space reasoning allows the model to interleave language tokens with explicit zoom-in bounding-box actions, extracting high-resolution local crops on the fly and appending them into the context for subsequent reasoning. Nevertheless, when existing frameworks employ Group Relative Policy Optimization (GRPO) to train pixel-space reasoning policies, an unexpected pathology called visual laziness consistently arises: the policy progressively avoids invoking zoom-in actions as training proceeds and frequently collapses into a zoom-averse state. Furthermore, when multi-zoom rollouts are generated, their empirical accuracy is markedly lower than zero-zoom or single-zoom rollouts, showing that standard GRPO fails to cultivate robust multi-step evidence-gathering capabilities.

This pathology stems directly from a structural mismatch between GRPO's group-relative advantage normalization and the inherently heterogeneous strategy space of pixel-space reasoning trajectories. Trajectories sampled under the same visual prompt vary drastically along two strategic dimensions: zoom depth (the evidence budget measured by the number of zoom-in crops) and visual intent (broad search across disparate regions vs. local verification over overlapping regions). The base model enters RL training with well-mastered single-look textual reasoning abilities inherited from pretraining, achieving relatively high immediate reward on simpler queries without zooming. In contrast, multi-zoom perception is a newly discovered behavior requiring accurate chained localization and multi-turn evidence synthesis; its early execution is noisy and error-prone. Standard GRPO computes advantage normalization across all sampled trajectories in a group regardless of strategy, systematically assigning negative advantages to exploratory multi-step zoom rollouts before they have the chance to be learned. Core idea: Visual-Intent Stratified GRPO (VIS-GRPO) stratifies sampled trajectories jointly by zoom depth and spatial-coherence visual intent, computing advantage normalization exclusively within strategically comparable strata to eliminate visual laziness and foster active perception.

Method

Overall Architecture

In pixel-space reasoning, given a multimodal prompt \(\mathbf{x} = [V, L]\) comprising visual input \(V\) and textual instruction \(L\), a VLM policy \(\pi_\theta\) generates a reasoning trajectory \(\tau \sim \pi_\theta(\cdot \mid \mathbf{x})\) that alternates between textual thought tokens and normalized bounding-box zoom actions \(b_t \in [0, 1]^4\). Each zoom action extracts a high-resolution sub-image \(o_t = f(V, b_t)\) via crop operator \(f\) and appends it to the context.

VIS-GRPO acts as a drop-in algorithmic replacement for standard GRPO. For each prompt \(\mathbf{x}\), a group of \(k\) rollouts is sampled. VIS-GRPO first stratifies trajectories by zoom depth into distinct evidence budgets. For trajectories executing multiple zooms, it further measures spatial coherence among bounding boxes to partition them into explore and verify intents. Finally, empirical return means and variances are computed strictly within each depth-intent stratum to produce standardized advantage estimates for policy optimization.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input x = [V, L]<br/>Sample k rollouts"] --> B["Zoom Depth Stratification<br/>Group by crop count c(τ)"]
    B -->|c = 0| C1["Zero-Zoom Stratum (c=0)"]
    B -->|c = 1| C2["Single-Zoom Stratum (c=1)"]
    B -->|c ≥ 2| D["Visual Intent Partitioning<br/>Compute mean pairwise IoU"]
    D -->|Mean IoU < β| E1["Broad Explore Stratum<br/>Dispersed regional search"]
    D -->|Mean IoU ≥ β| E2["Local Verify Stratum<br/>Overlapping detail inspection"]
    C1 --> F["Stratum-wise Advantage Normalization<br/>Compute within-stratum mean & std"]
    C2 --> F
    E1 --> F
    E2 --> F
    F --> G["Policy Update<br/>Clipped surrogate objective"]

Key Designs

1. Zoom Depth Stratification: Isolating Evidence Budgets from Baseline Distortion Standard GRPO normalizes trajectory returns across the entire group of \(k\) rollouts regardless of how many zoom actions each trajectory takes. Because no-zoom or single-zoom trajectories rely on well-established base model reasoning pathways, they frequently secure modest immediate rewards on easier questions early in training. Conversely, multi-zoom trajectories consume a larger visual evidence budget and involve compound actions that are noisier during initial phases. Blending them into a single global baseline \(\bar{R}\) artificially pulls the baseline above the nascent multi-zoom returns, penalizing exploratory zoom behavior with negative advantages. VIS-GRPO eliminates this distortion by partitioning rollouts by their zoom count \(c(\tau_i)\). Trajectories executing zero, one, or multiple crops are never directly compared against one another, shielding developing multi-step policies from premature suppression.

2. Visual-Intent Spatial Coherence Partitioning: Decoupling Broad Search from Local Verification Restricting comparisons to identical zoom depths is necessary but insufficient. Within the multi-zoom regime (\(c(\tau) \ge 2\)), rollouts bifurcate into two qualitatively distinct cognitive intents: broad search (explore), where the agent samples disjoint areas across the image to locate a missing target, and local verification (verify), where the agent repeatedly inspects overlapping sub-regions to resolve ambiguous details or small text. These two intents possess vastly different spatial dynamics and difficulty profiles. Local verification demands fine spatial precision and is typically harder to master early; when pooled with broad exploratory rollouts at the same depth, verification trajectories receive biased negative updates and wither. VIS-GRPO introduces a lightweight spatial coherence metric using the mean pairwise Intersection-over-Union (IoU) among the sequence of bounding boxes \(\{b_1, \dots, b_{c(\tau)}\}\):

\[\overline{\mathrm{IoU}}(\tau) = \frac{2}{c(\tau)\big(c(\tau)-1\big)} \sum_{1 \le i < j \le c(\tau)} \mathrm{IoU}(b_i, b_j)\]

Trajectories with \(\overline{\mathrm{IoU}}(\tau) \ge \beta\) are assigned to verify, while those below threshold \(\beta\) are assigned to explore. This maps each trajectory to a discrete stratum \(s(\tau) = (c(\tau), m(\tau))\).

3. Stratum-wise Advantage Normalization: Constructing Fair Credit Assignment For an input prompt \(\mathbf{x}\) and a given stratum \(s\), let \(\mathcal{I}_{\mathbf{x}, s} = \{i \mid s(\tau_i) = s\}\) denote the index set of trajectories belonging to that stratum. Rather than standardizing across all \(k\) trajectories, VIS-GRPO computes the stratum-specific empirical mean and standard deviation:

\[\bar{R}_{\mathbf{x}, s} = \frac{1}{|\mathcal{I}_{\mathbf{x}, s}|}\sum_{i \in \mathcal{I}_{\mathbf{x}, s}} R_i, \quad \sigma^2_{\mathbf{x}, s} = \frac{1}{|\mathcal{I}_{\mathbf{x}, s}|}\sum_{i \in \mathcal{I}_{\mathbf{x}, s}} \left(R_i - \bar{R}_{\mathbf{x}, s}\right)^2\]

The standardized advantage for rollout \(\tau_i\) is then computed strictly within its stratum:

\[\tilde{A}^{\text{VIS}}_i = \frac{R_i - \bar{R}_{\mathbf{x}, s(\tau_i)}}{\sigma_{\mathbf{x}, s(\tau_i)} + \epsilon}\]

By evaluating exploratory trajectories only against other exploratory attempts, and verification rollouts only against other verification attempts, competitive execution of difficult multi-step perception is rewarded with positive advantages, unlocking sustained learning of active vision.

Loss & Training

Policy updates are performed using PPO's clipped surrogate objective with a reference policy KL penalty. Training uses Qwen3-VL-8B as the base model on a unified 15,000-sample dataset aggregated from Pixel Reasoner SFT data, InfographicVQA, LLaVA-CoT, and STARQA. The reward signal is binary Exact Match (EM) accuracy on final answers. No heuristic action penalties or artificial zoom bonuses are required, allowing active perception strategies to emerge purely from balanced policy gradients.

Key Experimental Results

Main Results

Evaluation spans four demanding high-resolution visual reasoning benchmarks: V* Bench (fine-grained visual search), HR-Bench 4K, HR-Bench 8K (extreme-resolution visual QA), and MME-RealWorld. All scores report accuracy percentages (data sourced from Table 1 of the paper).

Model Category	Model	Size	V* Bench	HR-Bench 4K	HR-Bench 8K	MME-RealWorld	Avg.
General-purpose VLM	GPT-4o	—	62.8	59.0	55.5	46.4	55.9
General-purpose VLM	LLaVA-OneVision	7B	75.4	63.0	59.8	48.5	61.7
General-purpose VLM	Qwen2.5-VL-7B	7B	73.3	67.3	64.1	44.6	62.3
General-purpose VLM	Qwen2.5-VL-72B	72B	69.1	67.6	68.0	—	—
General-purpose VLM	Qwen3-VL-8B (Base)	8B	86.4	77.8	72.9	48.7	71.5
Pixel-space Reasoning	Pixel Reasoner	7B	84.3	72.6	66.1	—	—
Pixel-space Reasoning	DeepEyes	7B	82.7	74.3	68.8	51.9	69.4
Ours	VIS-GRPO	8B	88.5	81.9	81.6	56.3	77.1

Ablation Study

To isolate the algorithmic impact from differences in base models and datasets, all RL algorithms were evaluated under identical conditions starting from Qwen3-VL-8B using the same 15,000 training examples (data sourced from Table 2 of the paper).

RL Algorithm Configuration	V* Bench	HR-Bench 4K	HR-Bench 8K	MME-RealWorld	Avg.	Note
Qwen3-VL-8B (Un-tuned Base)	86.4	77.8	72.9	48.7	71.5	Pretrained starting checkpoint
+ Standard GRPO	87.4	79.0	77.6	52.7	74.2	Suffers visual laziness; zoom collapses
+ Pixel Reasoner (Heuristic Reward)	83.8	80.6	79.0	54.2	74.4	Hand-crafted reward forcing zoom actions
+ Depth-Stratified GRPO	84.8	79.1	79.4	55.7	74.8	Single-axis ablation (zoom count only)
+ VIS-GRPO (Full Model)	88.5	81.9	81.6	56.3	77.1	Joint depth + intent stratification

Key Findings

• Crucial Role of Dual-Axis Stratification: Depth-Stratified GRPO achieves an average score of 74.8, improving over standard GRPO (74.2) but trailing VIS-GRPO (77.1). On MME-RealWorld multi-zoom trajectories, Depth-Stratified GRPO develops a severe 9:1 imbalance between explore and verify intents, with verify accuracy plunging to 28.0%. In contrast, VIS-GRPO restores a balanced 7:3 distribution and boosts verification accuracy substantially.
• Substantial Gains at High Resolutions: The performance advantage of VIS-GRPO widens as image resolution scales up. On HR-Bench 8K, VIS-GRPO surpasses the base model by 8.7 points (81.6% vs. 72.9%) and beats standard GRPO by 4.0 points. On MME-RealWorld, it posts a 7.6-point gain over the base model (56.3% vs. 48.7%).
• Reversal of Zoom-Decay Dynamics: While standard GRPO exhibits a monotonic decline in average zoom operations per rollout over training steps, VIS-GRPO steadily scales up zoom-in frequency and stabilizes at an elevated plateau, proving genuine acquisition of active perceptual routines.

Highlights & Insights

• Root-Cause Algorithmic Fix over Reward Shaping: Rather than fighting visual laziness by artificially penalizing unzoomed responses or handing out artificial curiosity bonuses, VIS-GRPO targets the mathematical flaw in group normalization, achieving superior exploration without distorting the true environmental objective.
• Unsupervised Spatial Coherence as Semantic Proxy: Deriving cognitive visual intent (broad explore vs. local verify) directly from the mean pairwise IoU of bounding-box trajectories provides an elegant, zero-overhead abstraction that avoids external classifiers or expensive LLM intent annotators.
• Broader Relevance for Multi-Turn Agentic RL: Any sequential decision-making domain exhibiting variable rollout lengths and divergent operational modes (such as web navigation, multi-turn code search, or API tool orchestration) can benefit from this stratified advantage normalization paradigm.

Limitations & Future Work

• Sample Sparsity within Rare Strata: When the rollout group size \(k\) is small, certain niche strata (e.g., high zoom depths or rare verification attempts) may capture only one or two rollouts, increasing the variance of stratum return estimations and necessitating fallback smoothing mechanisms.
• Hard Binary Thresholding on Visual Intent: Utilizing a single scalar threshold \(\beta\) on bounding-box IoU is a coarse approximation of intent; complex multi-turn sequences interleaving broad exploratory sweeps with nested local re-inspections may warrant more dynamic sequence segmentation.
• Future Directions: Exploring hierarchical policy architectures where a high-level manager explicitly emits intent tokens (explore, inspect, re-check) while low-level pixel actuators are trained via stratified policy optimization.

Related Work & Insights

• vs. Standard GRPO (DeepSeekMath): Standard GRPO assumes rollout exchangeability, which holds in text-only symbolic math reasoning but breaks down when spatial perception introduces heterogeneous action modes. VIS-GRPO preserves the critic-free elegance of GRPO while making it compatible with active perceptual environments.
• vs. Pixel Reasoner: Pixel Reasoner relies on heuristic curiosity rewards to force zoom behavior, risking hallucinated bounding-box crops. VIS-GRPO relies entirely on sparse Exact Match task feedback, allowing natural, task-relevant active perception to emerge purely through fair advantage baselines.
• vs. Step-Stratified Search Agents: Previous work on LLM search agents stratified queries based solely on search frequency count. VIS-GRPO establishes that visual domains require an additional spatial-coherence axis to decouple intent from count.

Rating

• Novelty: ⭐⭐⭐⭐☆ [Insightful identification of visual laziness in pixel-space RL with a well-principled dual-axis stratification solution]
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Rigorous evaluation across 4 top-tier benchmarks with controlled RL ablations and detailed intent dynamics]
• Writing Quality: ⭐⭐⭐⭐⭐ [Clear structural narrative, elegant formulations, and thorough qualitative analysis]
• Value: ⭐⭐⭐⭐⭐ [A plug-and-play RL optimization paradigm essential for advancing active perception and test-time visual scaling]

Related Papers

• [ECCV 2026] Vero: Open Reinforcement Learning Recipes for Visual Reasoning
• [CVPR 2026] Act2See: Emergent Active Visual Perception for Video Reasoning
• [ICML 2026] Native Active Perception as Reasoning for Omni-Modal Understanding
• [ICLR 2026] VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning
• [CVPR 2026] Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design