VisCritic: Visual State Comparison as Process Reward for GUI Agents

Conference: ECCV2026
arXiv: 2606.24525
Code: Not open-sourced
Area: GUI Agent / Process Reward Model
Keywords: GUI Agent, Process Reward, Visual State Comparison, Action Verification, Weak Supervision

TL;DR

VisCritic proposes a process reward framework for GUI agents based on visual state comparison. Using a Siamese ViT to compare pre- and post-action screenshots in semantic feature space, it fuses action context to jointly predict action success probability, task progress, and error types. Serving as a plug-and-play inference-time verification module, it consistently improves the task success rate of various GUI agents across five benchmarks.

Background & Motivation

GUI agents (e.g., SeeClick, ShowUI, Qwen2.5-VL Agent) automate digital tasks by interpreting screenshots and executing actions like clicks, text inputs, and scrolls. Supported by multimodal large language models, they have made significant progress. However, in long-horizon tasks, errors in individual actions can cascade and amplify, ultimately leading to task failure. This occurs because agents lack a reliable step-level verification mechanism after executing each action: they cannot confirm whether the action just performed indeed yielded the expected outcome. Existing works attempt to fill this gap through Process Reward Models (PRMs), such as GUI-PRA, BacktrackAgent, and GUI-Critic-R1. However, they share a common limitation: verification signals rely almost entirely on textual reasoning, tool calls, or structured checklists. Since the essence of GUI interaction is visual—buttons change color after being clicked, pop-ups appear, and pages scroll to new content—these state changes occur at the pixel level. Verifying them through textual descriptions introduces a modal mismatch that fundamentally limits the reliability of the verification.

This mismatch between the verification paradigm and the signal modality is precisely the key challenge this paper addresses. Text-based verification struggles to capture purely visual changes like "button highlighting," "icon state switching," or "page navigation." When it comes to describing subtle differences, such as "clicked but the system did not respond" versus "clicked in the wrong place and a different window popped up," textual descriptions are even more inadequate. The key insight of this paper is that since the outcomes of actions are naturally manifested as visual state changes, verification should be performed directly in the visual space. Core Idea: VisCritic is proposed, using a Siamese ViT encoder to compare pre- and post-action screenshot pairs in semantic feature space. By integrating actions and task instructions, it jointly predicts action success probability, task progress, and error types, serving as plug-and-play visual process reward signals.

Method

Overall Architecture

VisCritic is a visual critic that takes screenshot pairs as input: given a pre-action screenshot \(s_t\) and a post-action screenshot \(s_{t+1}\), it outputs three predictions—action success probability \(\hat{y}_{suc} \in [0,1]\), task progress \(\hat{y}_{prog} \in [-1,1]\), and error type \(\hat{y}_{err} \in \{\text{success}, \text{no-op}, \text{wrong-target}, \text{page-error}, \text{timeout}\}\). The overall workflow consists of three stages: first, the Visual Difference Encoder (VDE) extracts patch-level features of both screenshots using a weight-sharing Siamese ViT and computes element-wise differences, which are then aggregated into a semantic-level difference vector via change-region attention with a learnable temperature parameter. Second, the Action-Aware Critic Head fuses this difference vector with the encoded action description and task instructions using cross-attention, producing predictions from three lightweight MLP heads. Finally, when the success probability falls below a threshold \(\gamma\), the downstream agent performs retries or rollbacks based on the critic results.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Pre-action screenshot s_t"] --> B["Shared ViT Encoder"]
    C["Post-action screenshot s_{t+1}"] --> B
    B --> D["Patch Features F_t / F_{t+1}"]
    D --> E["Difference Features ΔF<br/>+ Change Magnitude Map M"]
    E --> F["Change-Region Attention"]
    F --> G["Aggregated Difference Vector v_Δ"]
    G --> H["Cross-Attention Fusion<br/>v_Δ + Action + Instruction"]
    H --> I["Multi-Task Critic Heads"]
    I --> J["y_suc / y_prog / y_err"]
    J --> K["< Threshold γ?<br/>→ Retry / Rollback"]

Key Designs

1. Visual Difference Encoder (VDE) + Change-Region Attention: Comparing Visual States in Semantic Space

The core idea of VDE is to view both pre- and post-action screenshots with the same encoder and then compare their differences patch-by-patch. Specifically, the weight-sharing ViT encoder \(\mathcal{E}_\theta\) extracts patch-level features \(F_t, F_{t+1} \in \mathbb{R}^{P \times d}\) (where \(P\) is the number of patches, and \(d\) is the feature dimension) of \(s_t\) and \(s_{t+1}\), respectively, and then computes the element-wise difference \(\Delta F = F_{t+1} - F_t\). This difference in the ViT semantic space is inherently robust to rendering noise, animation artifacts, and minor layout offsets—a change in top-left pixel values from #FFFFFF to #F0F0F0 may be a large variance in RGB space but virtually zero in ViT semantic space, while a button switching from "unselected" to "selected" leaves a clear difference signal in semantic space. VDE simultaneously computes the change magnitude for each patch \(M_i = \|\Delta F_i\|_2\), forming a change magnitude heatmap.

Not all visual changes are noteworthy—irrelevant advertisement carousels and asynchronous loading animations can severely disrupt simple differencing. Change-region attention applies a softmax with a learnable temperature parameter \(\beta\) to the change magnitude map to obtain attention weights \(\alpha_i = \exp(M_i/\beta) / \sum_j \exp(M_j/\beta)\), and then performs a weighted sum of \(\Delta F\) to produce a compact global difference vector \(\mathbf{v}_\Delta = \sum_i \alpha_i \cdot \Delta F_i\). Through learning, \(\beta\) automatically adjusts the concentration of attention, allowing the model to filter out large-scale but irrelevant visual motions. This mechanism not only contributes an approximate 1.4% improvement in task success rate but also provides natural interpretability: during successful clicks, attention is concentrated on the clicked element and its visual response; during erroneous operations, it focuses on unintended change regions; and during no-op actions, attention is uniformly diffused.

2. Action-Aware Critic Head: Joint Prediction Fusing Visual Difference and Action Context

Screenshot differences alone cannot determine whether a change is "good" or "bad"—the same difference can carry completely opposite meanings under different actions and task contexts (e.g., a "loading" pop-up is normal in a waiting scenario but could indicate an error during a purchase click). The critic head takes the patch-level difference features \(\Delta F\) from the VDE as queries (key/value) and uses the action \(a_t\) and task instruction \(l\), encoded by a text encoder, as key/value. Through cross-attention, the difference features of each patch "know" the current action and task objective. Simultaneously, \(\Delta F\) itself undergoes spatial self-attention to capture spatial relationships among patches. The two branches are added after change-magnitude-weighted pooling to form the fused representation \(\mathbf{z}\). Three sets of MLP heads output from \(\mathbf{z}\), respectively: action success probability \(\hat{y}_{suc}\) activated by Sigmoid, task progress \(\hat{y}_{prog}\) activated by Tanh (positive values indicate moving toward the goal; negative values indicate deviation), and error type \(\hat{y}_{err}\) activated by Softmax. This multi-task design allows the three predictions to share the underlying visual difference understanding, while the error types provide interpretable, fine-grained classification for numerical success values.

3. Unlabeled Critic Training Data Construction: Automated Weakly-Supervised Sample Generation from Existing Trajectories

Training a critic typically requires expensive step-by-step annotation—each action's outcome (success or failure) and error type must be manually labeled. VisCritic ingeniously leverages existing trajectory data from GUI agent training and evaluation to automatically construct training samples. It defines five classes of samples: positive samples are obtained from successful trajectories as consecutive triplets \((s_t, a_t, s_{t+1})\) and labeled with the weak label \(y_{suc}=1\); negative samples are automatically generated through four perturbation strategies: (a) action mismatch: replacing \(a_t\) with a random action from the same trajectory while holding screenshots constant; (b) state mismatch: replacing \(s_{t+1}\) with a screenshot from another trajectory; (c) no-op detection: selecting adjacent screenshot pairs with SSIM > 0.98; and (d) failed trajectory sampling: extracting steps after transition points from failed trajectories and assigning the weak label \(y_{suc}=0\). Capturing error types directly derives from the perturbation strategy types (manually verified with 500 samples, yielding 81.4% consistency), and progress scores are heuristically inferred based on relative positions in trajectories. This approach constructs approximately 200,000 training samples with a positive-to-negative ratio of ~1:1.6 across three public datasets: Mind2Web, AITW, and AndroidWorld. The trained model continues to demonstrate zero-shot generalization capabilities on OSWorld and WebArena (platforms completely unseen during training).

Loss & Training

VisCritic adopts a two-stage training scheme. The first stage is contrastive pre-training, which solely trains the VDE: it uses the InfoNCE loss to pull the difference vector \(\mathbf{v}_\Delta\) close to the text features of the corresponding correct action \(\mathbf{h}_a^+\) and push it away from negative action representations; when samples have reliable action region masks derived from click coordinates, KL divergence is additionally used to constrain attention weights to concentrate on the UI elements affected by the action (samples without reliable masks skip this term). The second stage is multi-task fine-tuning, training the entire model end-to-end: three tasks utilize binary cross-entropy, mean squared error, and cross-entropy losses, respectively, with weights \(\lambda_1=1.0, \lambda_2=0.5, \lambda_3=0.5\). Ablation studies show that contrastive pre-training is the most critical training component (its removal drops performance by 2.8%), allowing the VDE to learn discriminative visual difference representations prior to joint multi-task fine-tuning.

Key Experimental Results

Main Results

VisCritic is evaluated plug-and-play on four base agents (Interactive Average = average of Mind2Web, AndroidWorld, OSWorld, and WebArena):

Base Agent	Configuration	Mind2Web (Step SR)	AndroidWorld (Task SR)	OSWorld (Task SR)	WebArena (Task SR)	Interactive Avg.
SeeClick	Baseline	33.4	14.7	6.8	12.4	16.8
SeeClick	+VisCritic	37.6	19.1	9.5	16.4	20.7
ShowUI	Baseline	43.5	19.8	10.3	17.6	22.8
ShowUI	+VisCritic	48.9	24.8	12.8	23.2	27.4
Qwen2.5-VL	Best Text Baseline	51.8	28.2	17.4	26.5	31.0
Qwen2.5-VL	+VisCritic	54.1	29.8	17.1	29.4	32.6

VisCritic leads to consistent improvements across all base agents, with the most significant gains observed in WebArena and Mind2Web (where error propagation is most severe in long-horizon web tasks). Compared to text-based PRM baselines (GUI-PRA, BacktrackAgent, GUI-Critic-R1), VisCritic performs better in most settings, except for Qwen2.5-VL on OSWorld (a text-dense desktop environment) where a text baseline performs slightly better.

Ablation Study

Configuration	AndroidWorld Success Rate	Note
Full VisCritic	24.8	Full model
w/o Change-Region Attention	23.4	Attention mechanism contribution +1.4
w/o Multi-Task Heads (Success Only)	24.1	Marginal contribution of auxiliary tasks
w/o Contrastive Pre-training	22.0	Most critical training component
w/o Post-action Execution Verification	21.4	Pre-execution prediction alone is insufficient
Replacing VDE with Pixel Differencing	21.2	Raw pixel differences are highly sensitive to noise

Critic Quality Analysis

Evaluating the F1 score of the critic itself on AndroidWorld:

Method	Modality	F1
GUI-PRA	Text	70.4
BacktrackAgent	Text	69.5
VisCritic	Visual	85.2

Control experiments more clearly reveal the modality gap: the text critic achieves only 58.3% F1 on purely visual changes (e.g., button highlighting, icon switching), while the pixel-level critic achieves only 63.8% F1 on semantic changes (e.g., page navigation, content loading). VisCritic achieves >84% F1 across all three categories of changes, demonstrating the comprehensive advantage of semantic visual difference representations.

Key Findings

• Contrastive pre-training is the most critical component in the training pipeline, far exceeding the auxiliary tasks of multi-task heads in importance.
• Semantic features from VDE (84.6% F1) are far superior to raw pixel differencing (70.5% F1), which is heavily disrupted by rendering noise and interface animations.
• Post-action execution verification (based on screenshots after actual execution) is far superior to pre-execution selection (based on predicted states), with a gap of up to 3.4 points.
• Trained once, it can generalize across four different agent architectures without requiring agent-specific adaptation.
• Inference overhead is only 66ms/step (FP16/A100), which is virtually negligible compared to environment interaction delays of 1-3 seconds.

Highlights & Insights

• Approaching Process Reward from a Modality Alignment Perspective: Highly intuitive (GUI interactions are fundamentally visual) yet previously unaccomplished systematically—this insight challenges the core design assumptions of text-based PRMs like GUI-PRA.
• Ingenious Unlabeled Data Construction: Four perturbation strategies + heuristic labeling generate 200,000 training samples at almost zero cost, enabling rigorous critic training and freeing the framework from reliance on manual annotations.
• Change-Region Attention Provides Interpretability: Attention heatmaps intuitively reveal "where the model looks when judging success/failure," unlike the opaque numerical scores of text-based PRMs which are hard to trace.
• Plug-and-Play Design: Requires no modifications to the agent architecture or retraining of the agent, acting purely as an inference-time hook, which lowers the barrier to deployment.

Limitations & Future Work

• Limited Perception of Fine-Grained Text Changes: ViT encoders are less sensitive to extremely subtle text modifications, such as numerical changes (e.g., counting from 3 to 4), which remains a common limitation of ViTs.
• Dynamic Environmental Noise Remains a Challenge: Irrelevant animations, ad carousels, and asynchronous loading can interfere with change detection. Extreme viewport and resolution changes are also difficult to handle.
• Heuristic Weak Labels May Introduce Noise: Divergence point estimation in failed trajectories is completed via trajectory alignment. Label quality degrades when deviations from reference successful trajectories are substantial.
• Recovery Strategies Are Not Yet Independently Quantified: While VisCritic's output interfaces are defined, the design and optimization of recovery strategies (retries/rollbacks) themselves are outside the scope of this paper's quantitative evaluation.
• Future directions include lightweight variants (e.g., SigLIP-400M, which retains 94% of critic F1), combining with deterministic verifiers (such as MiniTap), integrating into MCTS tree search, and fusing multimodal visual + textual PRMs.

Related Work & Insights

• vs GUI-PRA / GUI-Shepherd (Text-based PRMs): These rely on text descriptions or structured checklists to verify actions. VisCritic directly compares state changes in the visual feature space, making them inherently complementary.
• vs BacktrackAgent / GUI-Critic-R1 (Error Detection): These focus on pre-execution predictions or rule-based verification. VisCritic performs post-execution verification based on actual, posterior visual outcomes.
• vs Pixel-level Image Differencing: VisCritic's VDE operates in the ViT semantic space, which is far more robust to rendering noise and UI animations than RGB-space differencing.
• vs MobileDreamer / CUWM (World Models): These predict future states, while VisCritic verifies actual outcomes. They can be used complementarily.

Rating

• Novelty: ⭐⭐⭐⭐ Introducing a visual state comparison system as a process reward for GUI agents is the first complete work of its kind, though the Siamese architecture itself is not entirely new.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Extremely thorough, utilizing five benchmarks, four base agents, comparisons with multiple text baselines, control experiments, ablation studies, and interpretability analyses.
• Writing Quality: ⭐⭐⭐⭐ Clear motivation, comprehensive methodology, and in-depth analysis, although some experimental details are heavily relegated to supplementary materials.
• Value: ⭐⭐⭐⭐⭐ High practical value, as a plug-and-play visual verification module can directly reinforce existing GUI agent systems.

Related Papers

• [ICML 2026] Process Reward Agents for Steering Knowledge-Intensive Reasoning
• [ICLR 2026] WebArbiter: A Principle-Guided Reasoning Process Reward Model for Web Agents
• [AAAI 2026] ProBench: Benchmarking GUI Agents with Accurate Process Information
• [ICLR 2026] ViMo: A Generative Visual GUI World Model for App Agents
• [ICLR 2026] ProRe: A Proactive Reward System for GUI Agents via Reasoner–Actor Collaboration