Skip to content

GUI-PRA: Process Reward Agents for GUI Tasks

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/YuanDaoze/GUI-PRA
Area: LLM Agent
Keywords: GUI Agent, Process Reward Model, Experience Injection, Autoregressive Perception, Active Evidence Seeking

TL;DR

Addressing the evaluative knowledge gap and visual ambiguity inherent to static, single-pass scoring in standard Process Reward Models (PRMs), GUI-PRA decouples what to verify from how to verify via experience-injected criterion synthesis and criterion-guided autoregressive visual tool perception, turning verification into an active investigation that markedly boosts long-horizon GUI task success.

Background & Motivation

Propelled by advances in Multimodal Large Language Models (MLLMs), Graphical User Interface (GUI) agents have made remarkable strides toward automating intricate device interactions across mobile applications, web workflows, and desktop interfaces. However, real-world operating environments require long-horizon, sequential decision-making where individual actions frequently cause irreversible environmental transitions. A minor grounding deviation or a premature conclusion early in the trajectory can trigger compounding errors, ultimately causing irreversible failures. To curb this cascade, Process Reward Models (PRMs) have gained significant attention for their potential to provide dense step-level supervision, pruning unpromising branches and guiding search-based decoding at inference time.

Despite this conceptual appeal, naively deploying standard PRMs to GUI domains yields severely compromised supervision due to two fundamental hurdles. The first barrier is the Evaluative Knowledge Gap: conventional PRMs score candidate actions based on superficial visual alignment rather than operational correctness, lacking domain-specific procedural logic such as prerequisite dependencies, data persistence requirements, and subtle negative constraints. This deficiency induces the "completion illusion," where a model assigns high reward to an agent that has merely typed text into form fields while ignoring an unclicked "Save" button or violating a strict "Do NOT hit save" directive. The second barrier is Visual Ambiguity: standard PRMs operate in a passive, single-pass inspection mode across global screenshots. When adjudicating fine-grained UI details—such as whether a small toggle is switched on, dense tabular rows match target criteria, or list items continue beneath the fold—passive perception relies on coarse visual proxies, culminating in hallucinated reward verdicts.

The central thesis of this paper is that effective GUI verification cannot rely on static intuition; instead, it demands fundamentally decoupling what to verify from how to verify it. Core idea: transform GUI process supervision from a passive single-pass scorer into an active, evidence-seeking Process Reward Agent (GUI-PRA) that synthesizes explicit adjudication criteria from a distilled principle repository and uses these criteria to navigate multi-granularity visual tools in an autoregressive investigation loop before issuing grounded rewards.

Method

Overall Architecture

GUI-PRA converts opaque reward estimation into a structured, evidence-grounded investigation pipeline. Given a natural language user instruction \(I\), current screen state \(S_t\), interaction history \(H_{t-1}\), and a candidate action pool \(A_t = \{(u_{t,j}, a_{t,j})\}_{j=1}^k\) generated by an actor policy, GUI-PRA determines the optimal candidate step \((u_t, a_t)\).

The workflow proceeds across three interconnected stages: 1. Experience-Injected Criterion Synthesis: An offline Meta-Judge distills reusable domain principles from contrastive trajectories into a consolidated repository \(\mathcal{P}\). At inference time, relevant principles are retrieved and instantiated into explicit, state-specific verification criteria \(C_t\) defining what to verify. 2. Criterion-Guided Autoregressive Perception: Guided by \(C_t\), an autoregressive Reason-Act-Update investigation loop dynamically coordinates multi-granularity visual tools—OmniParser, Point, and ZoomInSubfigure—to gather visual observations into an explicit evidence chain \(H^{\text{inv}}_{\text{final}}\), determining how to verify. 3. Evidence-Based Supervision: A downstream Judge Agent evaluates candidate actions against the gathered evidence chain \(H^{\text{inv}}_{\text{final}}\) and synthesized criteria \(C_t\), applying an evidence-aware scoring function \(F_{\text{judge}}\) to rank candidates and select the optimal execution step.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Task Instruction & GUI State<br/>Instruction I / Screenshot St / Candidate Actions (u, a)"] --> B["Experience-Injected Criterion Synthesis<br/>Principle Repository P Retrieval + Instantiation into Ct"]
    B --> C["Criterion-Guided Autoregressive Perception<br/>Dynamic Tool Loop: OmniParser / Point / ZoomIn"]
    C --> D["Evidence-Based Supervision<br/>Logic Consistency Scoring Fjudge on Evidence Chain"]
    D --> E["Selected Action Execution<br/>argmax Fjudge Action Selection"]

Key Designs

1. Experience-Injected Criterion Synthesis: Formulating Explicit Domain Rules Against Completion Illusions

Standard PRMs suffer from an evaluative knowledge gap, frequently judging actions by visual superficiality rather than logical necessity. To resolve this, GUI-PRA implements a two-phase synthesis mechanism. In the Offline Principle Distillation phase, the authors establish a Generate-Contrast-Abstract pipeline using contrastive trajectory sampling on out-of-distribution benchmarks (such as AndroidControl), isolating pairs of successful and logically flawed hard-negative episodes. A strong Meta-Judge (GPT-5.1) abstracts transferable adjudicative heuristics (e.g., verifying precondition fulfillment, data persistence, and termination safety) beyond surface visual artifacts. Semantic clustering subsequently consolidates redundant statements into a compact repository \(\mathcal{P}\). In the Online Principle Instantiation phase, given the runtime instruction \(I\), UI screenshot \(S_t\), and candidate action \((u, a)\), a dense retriever pulls the top-\(k\) relevant principles from \(\mathcal{P}\). These abstract guidelines are dynamically instantiated into localized verification criteria \(C_t\), explicitly commanding the system to check concrete elements (e.g., "confirm that mandatory name and phone fields are non-empty before permitting a submit action") before rendering a decision.

2. Criterion-Guided Autoregressive Perception: Active Multi-Scale Investigation Resolving Visual Ambiguity

Textual criteria alone cannot resolve perceptual uncertainty when fine-grained UI details are obscured within a downscaled global screenshot. GUI-PRA replaces single-pass perception with an active, multi-granularity visual toolset \(\mathcal{T}\) executed in an autoregressive loop. The toolset comprises: OmniParser for global OCR text recognition and Set-of-Mark (SoM) bounding box indexing; Point (powered by Molmo-7B) for grounding semantic text/icon targets into precise coordinates with visual red star markers; ZoomInSubfigure (leveraging Qwen3-VL-8B) for native-resolution cropping of ambiguous regions of interest; and Terminate for process convergence. At each iteration \(k\), the agent executes a structured Reason-Act-Update cycle: it performs Gap Identification by comparing \(C_t\) against current evidence, executes Action Generation by invoking the tool that best resolves the missing visual fact (e.g., zooming into an unverified checkbox or pointing to an unindexed icon), and performs State Update by appending tool feedback \(O_k\) into the evolving history \(H^{\text{inv}}_{k+1} = H^{\text{inv}}_k \cup \{A_k, O_k\}\). The investigation iterates until the evidence definitively resolves all target criteria or reaches a 4-step execution limit, establishing a verifiable and traceable evidentiary foundation.

3. Evidence-Based Supervision: Formal Adjudication Alignment Replacing Black-Box Heuristics

Standard PRMs compute uninterpretable token probabilities or scalar scores via direct likelihood estimation, making them vulnerable to visual overconfidence. GUI-PRA formalizes action selection through an explicit Evidence-Aware Adjudication Protocol:

\[(u_t, a_t) = \arg\max_{(u,a) \in A_t} F_{\text{judge}}(I, C_t, H^{\text{inv}}_{\text{final}}, u, a)\]

where \(F_{\text{judge}}\) scores how rigorously the collected evidence chain \(H^{\text{inv}}_{\text{final}}\) supports candidate action \((u, a)\) under criteria \(C_t\). The judge enforces a discriminative 0–10 scale governed by prioritized constraints: actions violating explicit task constraints (such as ordering requirements or "do not save" negative conditions) receive a fatal score of \(\le 3\) regardless of superficial appeal; suboptimal progress receives intermediate scores (4–8); and near-perfect scores (9–10) are reserved exclusively for actions whose preconditions, grounding accuracy, and logical validity are affirmatively validated by the grounded evidence.

Loss & Training

GUI-PRA is an inference-time agentic supervisory framework that requires zero parameter fine-tuning on the underlying vision-language models, eliminating the extensive resource overhead of training specialized critic checkpoints. Offline principle synthesis employs GPT-5.1 over 50 positive and 50 negative episodes from AndroidControl, condensing into 64 deduplicated core rules for online evaluation (and 143 principles for the expanded offline OS-Critic setup). During online deployment, candidate actions are generated by the execution actor using Best-of-N (\(N=8\)) sampling at temperature \(\tau = 1.0\) across fixed seeds, whereas GUI-PRA's criterion instantiation, tool routing, and adjudication judge operate deterministically at temperature \(\tau = 0.0\). Criterion selection is bounded by 5 candidate retrievals, and tool routing is strictly capped at 4 iterations per candidate action, bounding computational overhead.

Quantitative Experimental Data

Main Results

Online execution performance on AndroidWorld (M3A setting) and Mobile-MiniWoB++ (Pure Vision setting) across diverse vision-language model backbones, reporting Success Rate (SR) and AndroidWorld Difficulty-stratified Success Rate (DSR).

Model / Configuration AndroidWorld Easy DSR (%) AndroidWorld Med DSR (%) AndroidWorld Hard DSR (%) AndroidWorld SR (%) Mobile-MiniWoB++ SR (%)
InternVL3-8B (M3A) Base 5.17 0.00 0.86 6.03 –
+ Standard PRM 6.90 0.00 1.72 8.62 –
+ GUI-PRA 8.62 0.00 0.86 9.48 –
InternVL3.5-8B (M3A) Base 6.04 0.00 0.86 6.90 –
+ Standard PRM 9.91 0.43 0.86 11.20 –
+ GUI-PRA 10.34 1.72 0.86 12.93 –
Qwen2.5-VL-7B Base 15.52 3.88 0.86 20.26 51.09
+ Standard PRM 17.24 1.72 0.86 19.83 45.65
+ GUI-PRA 21.55 2.16 0.86 24.57 53.26
Qwen3-VL-8B-Instruct Base – – – 47.60 59.78
+ Standard PRM 30.60 14.66 4.31 49.57 59.78
+ GUI-PRA 36.21 14.66 3.88 54.74 65.22

Ablation & Controlled Studies

Ablation analysis on AndroidWorld with Qwen3-VL-8B-Instruct, evaluating component contributions, architectural decoupling, and compute-controlled baselines.

Configuration Easy DSR (%) Medium DSR (%) Hard DSR (%) Overall SR (%) Impact vs. GUI-PRA (\(\Delta\))
Standard PRM (Zero-shot) 30.60 14.66 4.31 49.57 -5.17
GUI-PRA (Full Framework) 36.21 14.66 3.88 54.74 –
w/o Principle Repository 36.21 12.93 2.59 51.72 -3.02
w/o Active Perception Tools 32.76 12.93 4.31 50.00 -4.74
Actor Integration (Principle & Tool in Actor) – – – 32.76 -14.84 (vs Base 47.60)
Compute Control: PRM-CoT – – – 49.57 -5.17
Compute Control: PRM-SC3 (3x Sampling Avg) – – – 48.28 -6.46

Key Findings

  • Tool-based evidence gathering drives the primary gain: Removing visual perception tools incurs a substantial drop of 4.74 SR points (falling to 50.00%), confirming that explicit criteria cannot succeed when the verifier remains blind to local UI evidence. Omitting principles causes a 3.02 SR point reduction, showing that tools and principles serve complementary "how" and "what" roles.
  • Test-time compute scaling is ineffective without active grounding: Allocating equivalent test-time compute to the Standard PRM via self-consistency (PRM-SC3, 48.28% SR) fails to beat standard single-pass CoT (49.57%) and lags more than 6 points behind GUI-PRA (54.74%). Simply sampling repetitive reasoning passes cannot substitute for grounded perceptual exploration.
  • Critic and Actor roles must remain decoupled: Merging principles and tools directly into the execution actor severely degrades task performance to 32.76% (dropping 14.84 points below the unaugmented 47.60% base model). Interleaving active visual verification into the actor's generative context corrupts policy generation, underscoring the necessity of a standalone supervisor.
  • Smaller supervisors achieve strong weak-to-strong guidance: An 8B GUI-PRA supervisor improves a 32B Qwen3-VL agent from 55.60% to 57.76% (+2.16 SR points), whereas an 8B Standard PRM achieves only a negligible +0.43 gain, indicating that active evidence collection enables smaller models to reliably supervise larger executors.

Highlights & Insights

  • From Passive Scoring to Active Investigation: GUI-PRA fundamentally shifts process supervision from opaque scalar estimation over static screenshots into an inspectable, tool-driven evidentiary audit, providing a viable blueprint for trustworthy multi-turn agent guidance.
  • Selective and Parsimonious Tool Utilization: Across 1,475 evaluation steps, the system invoked an average of only 1.17 active visual tools per step (1.48 including Terminate). OmniParser and Point handled standard structural parsing, while ZoomInSubfigure was invoked sparingly (237 times) only when ambiguity demanded high-resolution inspection.
  • Resilience to Explicit Tool Exceptions: When Point or OmniParser returned null or missing coordinates, the autoregressive router preserved the identified evidence gap and successfully recovered by triggering zoom inspection or exploratory scrolling (e.g., in Stopwatch and Quick Settings tasks).

Limitations & Future Work

  • Vulnerability to Silent Semantic Misgrounding: While resilient to explicit tool crashes, GUI-PRA remains susceptible to syntactically valid but semantically erroneous outputs. For example, when Point consistently mapped the Google Lens shortcut icon to Camera coordinates, the supervisor accepted the false evidence, inducing an irrecoverable Home-Lens oscillation.
  • Offline Principle Synthesis Lacks Continual Adaptation: The verification principle repository is distilled offline from static contrastive sets. Although it demonstrates cross-domain robustness between mobile and web environments, extending it into an online, continuously evolving memory that synthesizes principles on-the-fly for unseen desktop and OS paradigms remains open work.
  • vs Trained GUI Critics (OS-Oracle, GUI-Critic-R1): OS-Oracle and GUI-Critic-R1 rely on synthesizing extensive negative interaction data and running compute-heavy SFT/GRPO post-training. In contrast, GUI-PRA achieves comparable or superior discrimination zero-shot by augmenting general-purpose MLLMs with active investigative agency.
  • vs Autonomous MLLM-as-a-Judge: Prior proactive evaluation systems rely on unstructured free-form reasoning, which leaves verification criteria implicit and evidence gathering uncontrolled. GUI-PRA provides explicit, state-grounded criteria that tightly constrain downstream tool selection and reward adjudication.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ (Pioneering shift from passive PRM scoring to an active, tool-augmented process reward agent architecture)
  • Experimental Rigor: ⭐⭐⭐⭐⭐ (Thorough evaluation across AndroidWorld, Mobile-MiniWoB++, and OS-Critic Bench, featuring compute-controlled baselines, weak-to-strong validation, latency profiling, and tool reliability audits)
  • Paper Quality: ⭐⭐⭐⭐⭐ (Exemplary problem formulation, clear mathematical and procedural grounding, and insightful failure-mode analysis)
  • Practical Value: ⭐⭐⭐⭐⭐ (Inference-time plug-and-play capability without model fine-tuning; provides a crucial mechanism to prevent irreversible errors in autonomous GUI workflows)