Skip to content

GUI-PRA: Process Reward Agents for GUI Tasks

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/YuanDaoze/GUI-PRA/
Area: Reasoning
Keywords: GUI Agent, Process Reward Model, Experience Injection, Autoregressive Active Perception, Step-Level Verification

TL;DR

GUI-PRA reformulates GUI process-level reward modeling from passive single-pass scoring into an active investigation agent, decoupling "what to verify" via experience-injected criterion synthesis from "how to verify" via criterion-guided autoregressive visual tool invocation, substantially boosting online execution success rates on AndroidWorld and Mobile-MiniWoB++.

Background & Motivation

Multimodal Large Language Model (MLLM) powered Graphical User Interface (GUI) agents hold tremendous promise for automating multi-step user requests across mobile devices and computer operating systems. However, long-horizon GUI interaction remains notoriously fragile: interaction trajectories involve extended sequences of state transitions where environment feedback is sparse and small intermediate action errors compound rapidly, rendering entire subsequent trajectories invalid. To mitigate error accumulation, Process Reward Models (PRMs) have gained popularity as a mechanism to deliver dense, step-level supervision during inference, enabling Best-of-N rejection sampling or search tree pruning.

Directly porting conventional PRMs to GUI automation yields severely degraded supervision due to two fundamental domain bottlenecks. First, standard PRMs suffer from an Evaluative Knowledge Gap: unlike human domain experts who evaluate actions based on functional state-transition rules and procedural preconditions, standard PRMs rely heavily on superficial visual alignment. Consequently, they fall prey to the "completion illusion" (e.g., scoring an agent highly because form fields look visually populated even when the essential "Save" action was omitted) or miss subtle violations of explicit negative constraints. Second, standard PRMs are crippled by Visual Ambiguity: their evaluation is strictly passive and single-pass over static, compressed screenshots, making it impossible to actively locate, inspect, or disambiguate tiny visual UI evidence such as radio button toggles, compact text suffixes, or dense list rows, ultimately prompting hallucinatory scoring.

To overcome both limitations, this paper argues for an architectural decoupling of "what to verify" from "how to verify", transforming process verification from a passive scoring model into an active, evidence-seeking investigator. Core idea: develop GUI-PRA, an active Process Reward Agent that distills generalized adjudication heuristics into a state-specific criterion synthesis mechanism, guided by which the agent autoregressively invokes multi-granularity visual tools to eliminate evidence gaps and render verifiably grounded step-level rewards.

Method

Overall Architecture

GUI-PRA redefines process supervision as a three-phase active investigation pipeline: (1) Experience-Injected Criterion Synthesis, which derives explicit domain verification logic for the current UI state; (2) Criterion-Guided Autoregressive Perception, which orchestrates visual tool use to actively acquire grounding facts; and (3) Evidence-Based Supervision, which computes the final reward score based on logical consistency between the collected evidence and the synthesized criteria.

The framework is anchored by an offline-distilled Synthesized Verification Principle Repository \(\mathcal{P}\). At runtime, given user instruction \(I\), current screen state \(S_t\), and a candidate action pool \(\mathcal{A}_t = \{(u_{t,j}, a_{t,j})\}_{j=1}^k\), a dense retriever fetches relevant principles from \(\mathcal{P}\) to instantiate concrete state-specific criteria \(C_t\). Subsequently, the perception module operates a Reason-Act-Update loop over a multi-granularity toolset \(\mathcal{T}\) (comprising OmniParser, Point, ZoomInSubfigure, and Terminate) to actively gather grounded visual observations until an evidence chain \(H^{\mathrm{final}}_{\mathrm{inv}}\) is established. Finally, the Judge Agent adjudicates candidate actions against this factual chain.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input<br/>Task Instruction I + State S_t + Candidates (u, a)"] --> B["Experience-Injected Criterion Synthesis<br/>Retrieve from Repo P & instantiate concrete criteria C_t"]
    B --> C["Multi-Granularity Visual Toolset<br/>OmniParser global layout / Point grounding / ZoomIn local crop"]
    C --> D["Autoregressive Investigation Loop<br/>Gap identification → tool action → state update until Terminate"]
    D --> E["Evidence-Based Supervision<br/>Evaluate logical alignment between H_inv and C_t for scoring"]
    E --> F["Output<br/>Optimal candidate action & process reward r_t"]

Key Designs

1. Experience-Injected Criterion Synthesis: Offline Principle Distillation & Online State-Specific Instantiation

This design tackles the Evaluative Knowledge Gap where passive PRMs lack domain adjudication logic and misjudge procedural dependencies. GUI-PRA introduces an automated Generate-Contrast-Abstract pipeline that operates offline without manual rule engineering: it first samples balanced successful and failed trajectories from out-of-distribution GUI interaction benchmarks, emphasizing hard negatives where failed actions are visually plausible yet functionally flawed. A powerful Meta-Judge (e.g., GPT-5.1) abstracts these contrastive episodes into reusable verification logic (e.g., State Dependency Validation, system feedback persistence), which are then semantically clustered into a compact repository \(\mathcal{P}\) (comprising 64 core principles for online execution).

During online inference, GUI-PRA executes dynamic principle instantiation: given the natural language instruction \(I\), history \(H_{t-1}\), and candidate action \((u, a)\), a dense retriever extracts the Top-\(k\) most relevant principles from \(\mathcal{P}\). The model instantiates these principles into concrete, state-specific criteria \(C_t\) for the current interface (e.g., translating a generic state-dependency principle into a specific requirement: "verify that the Name and Phone text boxes are populated before accepting a Submit action at coordinate \((x, y)\)"), setting an unambiguous target standard for active perception.

2. Multi-Granularity Visual Toolset: Global Parsing, Spatial Grounding, and Lossless Regional Inspection

This design resolves the Visual Ambiguity dilemma where fine-grained cues are indecipherable in raw, full-screen downscaled images. GUI-PRA engineers an orthogonal, four-component visual toolset \(\mathcal{T} = \{\text{OmniParser}, \text{Point}, \text{ZoomInSubfigure}, \text{Terminate}\}\): - OmniParser (Global UI Perception): Executes a two-stage parsing routine combining Optical Character Recognition (OCR) for all textual content with icon detection to overlay Set-of-Mark (SoM) bounding boxes and index tags, returning a structured layout hierarchy alongside visual anchors. - Point (Spatial Grounding): Powered by Molmo-7B-D-0924, this tool accepts a natural language element description (e.g., "The search icon in the top right corner") and predicts normalized coordinates \((x, y)\). Crucially, the system post-processes the image by stamping an explicit red pentagram marker at \((x, y)\), giving the downstream judge an interpretable, verifiable artifact. - ZoomInSubfigure (Local Detail Disambiguation): Employs Qwen3-VL-8B-Instruct to crop native-resolution sub-images corresponding to a specified Region of Interest (RoI) \([x_{\min}, y_{\min}, x_{\max}, y_{\max}]\). By preserving native pixel density, it isolates subtle status indicators (e.g., toggle ON/OFF states, dense spreadsheet values, or appended filename suffixes) from distracting interface clutter. - Terminate (Investigation Convergence): Serves as a non-visual control signal invoked when the accumulated evidence is sufficient to conclusively adjudicate all targets in \(C_t\).

3. Criterion-Guided Autoregressive Investigation Loop: Evidence-Gap-Driven Dynamic Perception

This design addresses the limitation of single-pass passive visual scoring that fails to collect discriminative facts. Within the perception module, verification proceeds as an iterative Reason-Act-Update inference trajectory. At each step \(k\), conditioned on instruction \(I\), criteria \(C_t\), and current evidence history \(H^{\mathrm{inv}}_k\), the agent executes: 1. Gap Identification: Maps each criterion in \(C_t\) to concrete UI targets and pinpoints missing visual proof. If a criterion mandates checking a toggle state but the global view is ambiguous, that target is flagged as an active gap. 2. Action Generation: Autoregressively generates the optimal tool action \(A_k \in \mathcal{T}\). Existence checks trigger Point, visual ambiguities trigger ZoomInSubfigure, and layout inventories trigger OmniParser. 3. State Update: The returned tool observation \(O_k\) (e.g., cropped high-resolution patch or coordinates) is appended to form \(H^{\mathrm{inv}}_{k+1} = H^{\mathrm{inv}}_k \cup \{A_k, O_k\}\).

The loop terminates upon calling Terminate or reaching a strict cap of 4 routing attempts, ensuring that the verdict stems from a transparent, grounded evidence chain rather than an ungrounded hallucination.

4. Evidence-Based Supervision: Causal Alignment Between Visual Facts and Adjudication Criteria

This design replaces opaque scalar score prediction with an explicit, evidence-aware adjudication protocol. For each candidate thought-action pair \((u, a) \in \mathcal{A}_t\), the Judge Agent formally evaluates the alignment between the collected evidence chain \(H^{\mathrm{final}}_{\mathrm{inv}}\) and criteria \(C_t\):

\[ (u_t, a_t) = \arg\max_{(u,a) \in \mathcal{A}_t} \mathcal{F}_{\mathrm{judge}}(I, C_t, H^{\mathrm{final}}_{\mathrm{inv}}, u, a) \]

The scoring function \(\mathcal{F}_{\mathrm{judge}}\) prioritizes constraint compliance: any violation of explicit task instructions (e.g., ordering constraints or negative prohibitions like "do not save") automatically caps the score at \(\le 3\), regardless of superficial plausibility. High scores (9–10) are reserved exclusively for actions whose preconditions, operational parameters, and progress towards the goal are affirmatively proven by the grounded visual facts.

A Worked Example

Consider the ContactsNewContactDraft task where the user instructs: "Enter First Name: Grace, Last Name: Taylor, Phone: 799-802-1530, Phone Label: Work. Do NOT hit save." 1. Criterion Instantiation: GUI-PRA retrieves relevant principles and instantiates criteria \(C_t\): (i) verify that Name, Phone, and Label Work are populated in their respective fields; (ii) enforce the negative constraint that the top-right "Save" button must not be clicked; (iii) terminate upon completing entry. 2. Investigation & Monitoring: At step 1, Actor types Grace; GUI-PRA calls Point to ground the First Name field and confirms string alignment. In subsequent steps, phone entry and dropdown selection are similarly verified. 3. Terminal Decision Scoring: The Actor produces two candidates: Branch 1 clicks the top-right Save button; Branch 2 emits Terminate. A standard PRM assigns Branch 1 a high score due to typical contact-creation routines. In contrast, GUI-PRA uses ZoomInSubfigure to inspect the top-right button, recognizes it as the prohibited "Save" control, penalizes Branch 1 with a fatal score of 0, and awards Branch 2 a score of 10, preventing an irreversible failure.

Benchmark & Evaluation Protocol

  • AndroidWorld: Dynamic benchmark running on a live Android OS across diverse real-world mobile apps (116 tasks under the standard M3A protocol).
  • Mobile-MiniWoB++: Dense single-page web/mobile tasks evaluated under the Pure Vision setting.
  • OS-Critic Bench (Mobile Subset): Offline benchmark isolating the critic's capacity to discriminate valid from flawed transitions.
  • Evaluation Settings: Online evaluation employs Best-of-N rejection sampling (\(N=8\), temperature \(\tau=1.0\) with fixed distinct random seeds). The criterion synthesis limit is set to 5 attempts, and visual tool routing is capped at 4 iterations. Primary metrics are Success Rate (SR) and Difficulty-stratified Success Rate (DSR: Easy, Medium, Hard).

Key Experimental Results

Main Results

Online execution results across AndroidWorld and Mobile-MiniWoB++ show that GUI-PRA consistently outperforms autonomous Base Models and standard PRM baselines across multiple vision backbones.

Benchmark Backbone & Setup Supervision Role SR (%) Gain over Base (\(\Delta@1\)) Gain over PRM (\(\Delta@2\))
AndroidWorld InternVL3-8B-Instruct (M3A) Base Model 6.03 – –
AndroidWorld InternVL3-8B-Instruct (M3A) w/ PRM 8.62 +2.59 –
AndroidWorld InternVL3-8B-Instruct (M3A) w/ GUI-PRA 9.48 +3.45 +0.86
AndroidWorld InternVL3.5-8B-Instruct (M3A) Base Model 6.90 – –
AndroidWorld InternVL3.5-8B-Instruct (M3A) w/ PRM 11.20 +4.30 –
AndroidWorld InternVL3.5-8B-Instruct (M3A) w/ GUI-PRA 12.93 +6.03 +1.73
AndroidWorld Qwen2.5-VL-7B-Instruct Base Model 20.26 – –
AndroidWorld Qwen2.5-VL-7B-Instruct w/ PRM 19.83 -0.43 –
AndroidWorld Qwen2.5-VL-7B-Instruct w/ GUI-PRA 24.57 +4.31 +4.74
AndroidWorld Qwen3-VL-8B-Instruct Base Model 47.60 – –
AndroidWorld Qwen3-VL-8B-Instruct w/ PRM 49.57 +1.97 –
AndroidWorld Qwen3-VL-8B-Instruct w/ GUI-PRA 54.74 +7.14 +5.17
Mobile-MiniWoB++ Qwen2.5-VL-7B-Instruct Base Model 51.09 – –
Mobile-MiniWoB++ Qwen2.5-VL-7B-Instruct w/ PRM 45.65 -5.44 –
Mobile-MiniWoB++ Qwen2.5-VL-7B-Instruct w/ GUI-PRA 53.26 +2.17 +7.61
Mobile-MiniWoB++ Qwen3-VL-8B-Instruct Base Model 59.78 – –
Mobile-MiniWoB++ Qwen3-VL-8B-Instruct w/ PRM 59.78 0.00 –
Mobile-MiniWoB++ Qwen3-VL-8B-Instruct w/ GUI-PRA 65.22 +5.44 +5.44

On the offline OS-Critic Bench, GUI-PRA boosts Qwen2.5-VL-7B Accuracy from 58.45% to 59.13% and F1 from 65.79% to 66.16%, matching the fully supervised critic reference GUI-Critic-R1 (59.13% Acc, 66.54% F1). On Qwen3-VL-8B, Accuracy improves from 63.24% to 64.30%. On the pre-trained process-supervised Perceval-3B policy, GUI-PRA increases Recall by 18.10 points and F1 by 7.09 points (55.58% \(\to\) 62.67%).

Ablation Study and Empirical Diagnostics

Ablations and diagnostic investigations dissect module contributions, supervision paradigms, and compute budgets.

Diagnostic Category Configuration & Setup Key Metric (SR / Acc / F1) Impact Note / Observation
Component Ablation (AndroidWorld Qwen3-VL-8B) GUI-PRA (Full Framework) SR: 54.74% (Easy 36.21 / Med 14.66 / Hard 3.88) Baseline Complete active investigation architecture
Component Ablation w/o Principle Repository SR: 51.72% (Easy 36.21 / Med 12.93 / Hard 2.59) -3.02% Lacks domain adjudication logic to guide tool selection
Component Ablation w/o Active Perception Tools SR: 50.00% (Easy 32.76 / Med 12.93 / Hard 4.31) -4.74% Drops nearly to standard PRM (49.57%); active tools are critical
Weak-to-Strong Supervision Qwen3-VL-32B-Instruct SR: 55.60% Baseline Unsupervised 32B executor
Weak-to-Strong Supervision 32B Agent w/ 8B Standard PRM SR: 56.03% +0.43% Smaller passive PRM fails to guide larger agent
Weak-to-Strong Supervision 32B Agent w/ 8B GUI-PRA SR: 57.76% +2.16% Grounded active investigation enables small models to guide large models
Critic vs. Actor Role Decoupling Qwen3-VL-8B-Instruct SR: 47.60% Baseline Standalone executor
Critic vs. Actor Role Decoupling 8B Agent integrated w/ Tools & Principles SR: 32.76% -14.84% Conflating investigation into generation context impairs execution
Compute-Budget Control PRM-CoT (1 CoT reasoning pass) SR: 49.57% Baseline Matched thinking token budget
Compute-Budget Control PRM-SC3 (Self-Consistency ×3, \(\tau=0.7\)) SR: 48.28% -1.29% Repeated passive sampling degrades performance
Compute-Budget Control GUI-PRA (Evidence-Seeking) SR: 54.74% +5.17% Gains stem from active visual evidence, not raw test-time compute

Key Findings

  • Active tools provide the foundational performance pillar: Removing perception tools induces the steepest drop (-4.74 SR points), demonstrating that synthesized rules are ineffective if the verifier cannot gather fine-grained visual evidence.
  • Difficulty stratification reveals the verifier boundary: On AndroidWorld, performance gains are heavily concentrated in Easy tasks (jumping from 30.60% to 36.21%) and select Medium tasks, while Hard tasks remain flat (4.31% to 3.88%). Hindsight task-level pass@8 coverage indicates that the underlying actor produces viable paths on only 5 out of 116 Hard tasks (4.31%). A verifier can reliably prune incorrect paths, but cannot invent missing solutions.
  • Effective weak-to-strong guidance: An 8B GUI-PRA supervisor improves a 32B agent by 2.16 SR points, whereas an 8B standard PRM yields a negligible 0.43 point gain, confirming that grounded active verification enables efficient weak-to-strong alignment.
  • Latency profile and tool utilization: Over a 1,475-step evaluation trace, GUI-PRA adds an average of 6.70 seconds per step, invoking 1.17 active visual tool calls (1.48 total including Terminate). Point (790) and OmniParser (702) handle routine grounding, while ZoomInSubfigure (237) is deployed selectively when ambiguity arises.

Highlights & Insights

  • Active investigation over passive scoring: GUI-PRA departs from viewing PRMs as black-box probability estimators, granting them agency to formulate hypotheses, zoom into ambiguous UI regions, and cross-examine visual elements before issuing rewards.
  • Decoupling what to verify from how to verify: Distilling offline principles into an explicit repository provides high-level domain heuristics, while autoregressive perception gathers low-level visual facts, avoiding both manual rule scripting and expensive full-model critic retraining.
  • Strict enforcement of negative constraints: By checking explicit requirements before assigning scores, GUI-PRA decisively eliminates deceptive failures in tasks with "do not save" or strict ordering clauses.

Limitations & Future Work

  • Bottlenecked by visual tool precision: Adjudication quality remains bounded by upstream tool performance (e.g., Molmo point grounding). While the autoregressive loop recovers from explicit missing detections, silent semantic misgrounding (such as repeatedly mistaking the Google Lens icon for Camera) can cause cyclic failure.
  • Offline principle repository scope: Verification principles are currently distilled offline from contrastive trajectories on static benchmarks. Extending this to an online, continuously evolving memory that adapts to new UI paradigms represents an important open avenue.
  • Desktop operating system extension: The current framework is validated on mobile interfaces (AndroidWorld, MiniWoB++); testing on complex multi-window, dynamic desktop workflows remains future work.
  • vs GUI-Critic-R1 / OS-Oracle: Supervised GUI critics require massive amounts of annotated interaction data and compute-heavy fine-tuning. GUI-PRA achieves competitive discrimination zero-shot by coupling distilled principles with dynamic tool use, offering superior cross-platform adaptability.
  • vs ProRe / Agentic Reward Modeling: While recent works explore proactive reward modeling, they rely on unconstrained, free-form text reasoning without explicit verification rubrics or structured visual toolchains. GUI-PRA provides explicit, multi-granularity evidence chains that ground reward decisions in reproducible facts.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ [Conceptualizing PRMs as active, tool-using investigative agents is highly innovative]
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Exemplary evaluation spanning online/offline benchmarks, ablations, weak-to-strong setups, latency audits, and compute-controlled baselines]
  • Writing Quality: ⭐⭐⭐⭐⭐ [Crisply formulated methodology, transparent distinction between actor and verifier boundaries, and rigorous case analyses]
  • Value: ⭐⭐⭐⭐⭐ [Provides a practical, training-free paradigm to resolve error accumulation in long-horizon multimodal agents]