GUI-PRA: Process Reward Agents for GUI Tasks¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/YuanDaoze/GUI-PRA/
Area: Reasoning
Keywords: GUI Agent, Process Reward Model, Experience Injection, Autoregressive Active Perception, Step-Level Verification
TL;DR¶
GUI-PRA reformulates GUI process-level reward modeling from passive single-pass scoring into an active investigation agent, decoupling "what to verify" via experience-injected criterion synthesis from "how to verify" via criterion-guided autoregressive visual tool invocation, substantially boosting online execution success rates on AndroidWorld and Mobile-MiniWoB++.
Background & Motivation¶
Multimodal Large Language Model (MLLM) powered Graphical User Interface (GUI) agents hold tremendous promise for automating multi-step user requests across mobile devices and computer operating systems. However, long-horizon GUI interaction remains notoriously fragile: interaction trajectories involve extended sequences of state transitions where environment feedback is sparse and small intermediate action errors compound rapidly, rendering entire subsequent trajectories invalid. To mitigate error accumulation, Process Reward Models (PRMs) have gained popularity as a mechanism to deliver dense, step-level supervision during inference, enabling Best-of-N rejection sampling or search tree pruning.
Directly porting conventional PRMs to GUI automation yields severely degraded supervision due to two fundamental domain bottlenecks. First, standard PRMs suffer from an Evaluative Knowledge Gap: unlike human domain experts who evaluate actions based on functional state-transition rules and procedural preconditions, standard PRMs rely heavily on superficial visual alignment. Consequently, they fall prey to the "completion illusion" (e.g., scoring an agent highly because form fields look visually populated even when the essential "Save" action was omitted) or miss subtle violations of explicit negative constraints. Second, standard PRMs are crippled by Visual Ambiguity: their evaluation is strictly passive and single-pass over static, compressed screenshots, making it impossible to actively locate, inspect, or disambiguate tiny visual UI evidence such as radio button toggles, compact text suffixes, or dense list rows, ultimately prompting hallucinatory scoring.
To overcome both limitations, this paper argues for an architectural decoupling of "what to verify" from "how to verify", transforming process verification from a passive scoring model into an active, evidence-seeking investigator. Core idea: develop GUI-PRA, an active Process Reward Agent that distills generalized adjudication heuristics into a state-specific criterion synthesis mechanism, guided by which the agent autoregressively invokes multi-granularity visual tools to eliminate evidence gaps and render verifiably grounded step-level rewards.
Method¶
Overall Architecture¶
GUI-PRA redefines process supervision as a three-phase active investigation pipeline: (1) Experience-Injected Criterion Synthesis, which derives explicit domain verification logic for the current UI state; (2) Criterion-Guided Autoregressive Perception, which orchestrates visual tool use to actively acquire grounding facts; and (3) Evidence-Based Supervision, which computes the final reward score based on logical consistency between the collected evidence and the synthesized criteria.
The framework is anchored by an offline-distilled Synthesized Verification Principle Repository \(\mathcal{P}\). At runtime, given user instruction \(I\), current screen state \(S_t\), and a candidate action pool \(\mathcal{A}_t = \{(u_{t,j}, a_{t,j})\}_{j=1}^k\), a dense retriever fetches relevant principles from \(\mathcal{P}\) to instantiate concrete state-specific criteria \(C_t\). Subsequently, the perception module operates a Reason-Act-Update loop over a multi-granularity toolset \(\mathcal{T}\) (comprising OmniParser, Point, ZoomInSubfigure, and Terminate) to actively gather grounded visual observations until an evidence chain \(H^{\mathrm{final}}_{\mathrm{inv}}\) is established. Finally, the Judge Agent adjudicates candidate actions against this factual chain.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input<br/>Task Instruction I + State S_t + Candidates (u, a)"] --> B["Experience-Injected Criterion Synthesis<br/>Retrieve from Repo P & instantiate concrete criteria C_t"]
B --> C["Multi-Granularity Visual Toolset<br/>OmniParser global layout / Point grounding / ZoomIn local crop"]
C --> D["Autoregressive Investigation Loop<br/>Gap identification → tool action → state update until Terminate"]
D --> E["Evidence-Based Supervision<br/>Evaluate logical alignment between H_inv and C_t for scoring"]
E --> F["Output<br/>Optimal candidate action & process reward r_t"]
Key Designs¶
1. Experience-Injected Criterion Synthesis: Offline Principle Distillation & Online State-Specific Instantiation
This design tackles the Evaluative Knowledge Gap where passive PRMs lack domain adjudication logic and misjudge procedural dependencies. GUI-PRA introduces an automated Generate-Contrast-Abstract pipeline that operates offline without manual rule engineering: it first samples balanced successful and failed trajectories from out-of-distribution GUI interaction benchmarks, emphasizing hard negatives where failed actions are visually plausible yet functionally flawed. A powerful Meta-Judge (e.g., GPT-5.1) abstracts these contrastive episodes into reusable verification logic (e.g., State Dependency Validation, system feedback persistence), which are then semantically clustered into a compact repository \(\mathcal{P}\) (comprising 64 core principles for online execution).
During online inference, GUI-PRA executes dynamic principle instantiation: given the natural language instruction \(I\), history \(H_{t-1}\), and candidate action \((u, a)\), a dense retriever extracts the Top-\(k\) most relevant principles from \(\mathcal{P}\). The model instantiates these principles into concrete, state-specific criteria \(C_t\) for the current interface (e.g., translating a generic state-dependency principle into a specific requirement: "verify that the Name and Phone text boxes are populated before accepting a Submit action at coordinate \((x, y)\)"), setting an unambiguous target standard for active perception.
2. Multi-Granularity Visual Toolset: Global Parsing, Spatial Grounding, and Lossless Regional Inspection
This design resolves the Visual Ambiguity dilemma where fine-grained cues are indecipherable in raw, full-screen downscaled images. GUI-PRA engineers an orthogonal, four-component visual toolset \(\mathcal{T} = \{\text{OmniParser}, \text{Point}, \text{ZoomInSubfigure}, \text{Terminate}\}\): - OmniParser (Global UI Perception): Executes a two-stage parsing routine combining Optical Character Recognition (OCR) for all textual content with icon detection to overlay Set-of-Mark (SoM) bounding boxes and index tags, returning a structured layout hierarchy alongside visual anchors. - Point (Spatial Grounding): Powered by Molmo-7B-D-0924, this tool accepts a natural language element description (e.g., "The search icon in the top right corner") and predicts normalized coordinates \((x, y)\). Crucially, the system post-processes the image by stamping an explicit red pentagram marker at \((x, y)\), giving the downstream judge an interpretable, verifiable artifact. - ZoomInSubfigure (Local Detail Disambiguation): Employs Qwen3-VL-8B-Instruct to crop native-resolution sub-images corresponding to a specified Region of Interest (RoI) \([x_{\min}, y_{\min}, x_{\max}, y_{\max}]\). By preserving native pixel density, it isolates subtle status indicators (e.g., toggle ON/OFF states, dense spreadsheet values, or appended filename suffixes) from distracting interface clutter. - Terminate (Investigation Convergence): Serves as a non-visual control signal invoked when the accumulated evidence is sufficient to conclusively adjudicate all targets in \(C_t\).
3. Criterion-Guided Autoregressive Investigation Loop: Evidence-Gap-Driven Dynamic Perception
This design addresses the limitation of single-pass passive visual scoring that fails to collect discriminative facts. Within the perception module, verification proceeds as an iterative Reason-Act-Update inference trajectory. At each step \(k\), conditioned on instruction \(I\), criteria \(C_t\), and current evidence history \(H^{\mathrm{inv}}_k\), the agent executes: 1. Gap Identification: Maps each criterion in \(C_t\) to concrete UI targets and pinpoints missing visual proof. If a criterion mandates checking a toggle state but the global view is ambiguous, that target is flagged as an active gap. 2. Action Generation: Autoregressively generates the optimal tool action \(A_k \in \mathcal{T}\). Existence checks trigger Point, visual ambiguities trigger ZoomInSubfigure, and layout inventories trigger OmniParser. 3. State Update: The returned tool observation \(O_k\) (e.g., cropped high-resolution patch or coordinates) is appended to form \(H^{\mathrm{inv}}_{k+1} = H^{\mathrm{inv}}_k \cup \{A_k, O_k\}\).
The loop terminates upon calling Terminate or reaching a strict cap of 4 routing attempts, ensuring that the verdict stems from a transparent, grounded evidence chain rather than an ungrounded hallucination.
4. Evidence-Based Supervision: Causal Alignment Between Visual Facts and Adjudication Criteria
This design replaces opaque scalar score prediction with an explicit, evidence-aware adjudication protocol. For each candidate thought-action pair \((u, a) \in \mathcal{A}_t\), the Judge Agent formally evaluates the alignment between the collected evidence chain \(H^{\mathrm{final}}_{\mathrm{inv}}\) and criteria \(C_t\):
The scoring function \(\mathcal{F}_{\mathrm{judge}}\) prioritizes constraint compliance: any violation of explicit task instructions (e.g., ordering constraints or negative prohibitions like "do not save") automatically caps the score at \(\le 3\), regardless of superficial plausibility. High scores (9–10) are reserved exclusively for actions whose preconditions, operational parameters, and progress towards the goal are affirmatively proven by the grounded visual facts.
A Worked Example¶
Consider the ContactsNewContactDraft task where the user instructs: "Enter First Name: Grace, Last Name: Taylor, Phone: 799-802-1530, Phone Label: Work. Do NOT hit save."
1. Criterion Instantiation: GUI-PRA retrieves relevant principles and instantiates criteria \(C_t\): (i) verify that Name, Phone, and Label Work are populated in their respective fields; (ii) enforce the negative constraint that the top-right "Save" button must not be clicked; (iii) terminate upon completing entry.
2. Investigation & Monitoring: At step 1, Actor types Grace; GUI-PRA calls Point to ground the First Name field and confirms string alignment. In subsequent steps, phone entry and dropdown selection are similarly verified.
3. Terminal Decision Scoring: The Actor produces two candidates: Branch 1 clicks the top-right Save button; Branch 2 emits Terminate. A standard PRM assigns Branch 1 a high score due to typical contact-creation routines. In contrast, GUI-PRA uses ZoomInSubfigure to inspect the top-right button, recognizes it as the prohibited "Save" control, penalizes Branch 1 with a fatal score of 0, and awards Branch 2 a score of 10, preventing an irreversible failure.
Benchmark & Evaluation Protocol¶
- AndroidWorld: Dynamic benchmark running on a live Android OS across diverse real-world mobile apps (116 tasks under the standard M3A protocol).
- Mobile-MiniWoB++: Dense single-page web/mobile tasks evaluated under the Pure Vision setting.
- OS-Critic Bench (Mobile Subset): Offline benchmark isolating the critic's capacity to discriminate valid from flawed transitions.
- Evaluation Settings: Online evaluation employs Best-of-N rejection sampling (\(N=8\), temperature \(\tau=1.0\) with fixed distinct random seeds). The criterion synthesis limit is set to 5 attempts, and visual tool routing is capped at 4 iterations. Primary metrics are Success Rate (SR) and Difficulty-stratified Success Rate (DSR: Easy, Medium, Hard).
Key Experimental Results¶
Main Results¶
Online execution results across AndroidWorld and Mobile-MiniWoB++ show that GUI-PRA consistently outperforms autonomous Base Models and standard PRM baselines across multiple vision backbones.
| Benchmark | Backbone & Setup | Supervision Role | SR (%) | Gain over Base (\(\Delta@1\)) | Gain over PRM (\(\Delta@2\)) |
|---|---|---|---|---|---|
| AndroidWorld | InternVL3-8B-Instruct (M3A) | Base Model | 6.03 | – | – |
| AndroidWorld | InternVL3-8B-Instruct (M3A) | w/ PRM | 8.62 | +2.59 | – |
| AndroidWorld | InternVL3-8B-Instruct (M3A) | w/ GUI-PRA | 9.48 | +3.45 | +0.86 |
| AndroidWorld | InternVL3.5-8B-Instruct (M3A) | Base Model | 6.90 | – | – |
| AndroidWorld | InternVL3.5-8B-Instruct (M3A) | w/ PRM | 11.20 | +4.30 | – |
| AndroidWorld | InternVL3.5-8B-Instruct (M3A) | w/ GUI-PRA | 12.93 | +6.03 | +1.73 |
| AndroidWorld | Qwen2.5-VL-7B-Instruct | Base Model | 20.26 | – | – |
| AndroidWorld | Qwen2.5-VL-7B-Instruct | w/ PRM | 19.83 | -0.43 | – |
| AndroidWorld | Qwen2.5-VL-7B-Instruct | w/ GUI-PRA | 24.57 | +4.31 | +4.74 |
| AndroidWorld | Qwen3-VL-8B-Instruct | Base Model | 47.60 | – | – |
| AndroidWorld | Qwen3-VL-8B-Instruct | w/ PRM | 49.57 | +1.97 | – |
| AndroidWorld | Qwen3-VL-8B-Instruct | w/ GUI-PRA | 54.74 | +7.14 | +5.17 |
| Mobile-MiniWoB++ | Qwen2.5-VL-7B-Instruct | Base Model | 51.09 | – | – |
| Mobile-MiniWoB++ | Qwen2.5-VL-7B-Instruct | w/ PRM | 45.65 | -5.44 | – |
| Mobile-MiniWoB++ | Qwen2.5-VL-7B-Instruct | w/ GUI-PRA | 53.26 | +2.17 | +7.61 |
| Mobile-MiniWoB++ | Qwen3-VL-8B-Instruct | Base Model | 59.78 | – | – |
| Mobile-MiniWoB++ | Qwen3-VL-8B-Instruct | w/ PRM | 59.78 | 0.00 | – |
| Mobile-MiniWoB++ | Qwen3-VL-8B-Instruct | w/ GUI-PRA | 65.22 | +5.44 | +5.44 |
On the offline OS-Critic Bench, GUI-PRA boosts Qwen2.5-VL-7B Accuracy from 58.45% to 59.13% and F1 from 65.79% to 66.16%, matching the fully supervised critic reference GUI-Critic-R1 (59.13% Acc, 66.54% F1). On Qwen3-VL-8B, Accuracy improves from 63.24% to 64.30%. On the pre-trained process-supervised Perceval-3B policy, GUI-PRA increases Recall by 18.10 points and F1 by 7.09 points (55.58% \(\to\) 62.67%).
Ablation Study and Empirical Diagnostics¶
Ablations and diagnostic investigations dissect module contributions, supervision paradigms, and compute budgets.
| Diagnostic Category | Configuration & Setup | Key Metric (SR / Acc / F1) | Impact | Note / Observation |
|---|---|---|---|---|
| Component Ablation (AndroidWorld Qwen3-VL-8B) | GUI-PRA (Full Framework) | SR: 54.74% (Easy 36.21 / Med 14.66 / Hard 3.88) | Baseline | Complete active investigation architecture |
| Component Ablation | w/o Principle Repository | SR: 51.72% (Easy 36.21 / Med 12.93 / Hard 2.59) | -3.02% | Lacks domain adjudication logic to guide tool selection |
| Component Ablation | w/o Active Perception Tools | SR: 50.00% (Easy 32.76 / Med 12.93 / Hard 4.31) | -4.74% | Drops nearly to standard PRM (49.57%); active tools are critical |
| Weak-to-Strong Supervision | Qwen3-VL-32B-Instruct | SR: 55.60% | Baseline | Unsupervised 32B executor |
| Weak-to-Strong Supervision | 32B Agent w/ 8B Standard PRM | SR: 56.03% | +0.43% | Smaller passive PRM fails to guide larger agent |
| Weak-to-Strong Supervision | 32B Agent w/ 8B GUI-PRA | SR: 57.76% | +2.16% | Grounded active investigation enables small models to guide large models |
| Critic vs. Actor Role Decoupling | Qwen3-VL-8B-Instruct | SR: 47.60% | Baseline | Standalone executor |
| Critic vs. Actor Role Decoupling | 8B Agent integrated w/ Tools & Principles | SR: 32.76% | -14.84% | Conflating investigation into generation context impairs execution |
| Compute-Budget Control | PRM-CoT (1 CoT reasoning pass) | SR: 49.57% | Baseline | Matched thinking token budget |
| Compute-Budget Control | PRM-SC3 (Self-Consistency ×3, \(\tau=0.7\)) | SR: 48.28% | -1.29% | Repeated passive sampling degrades performance |
| Compute-Budget Control | GUI-PRA (Evidence-Seeking) | SR: 54.74% | +5.17% | Gains stem from active visual evidence, not raw test-time compute |
Key Findings¶
- Active tools provide the foundational performance pillar: Removing perception tools induces the steepest drop (-4.74 SR points), demonstrating that synthesized rules are ineffective if the verifier cannot gather fine-grained visual evidence.
- Difficulty stratification reveals the verifier boundary: On AndroidWorld, performance gains are heavily concentrated in Easy tasks (jumping from 30.60% to 36.21%) and select Medium tasks, while Hard tasks remain flat (4.31% to 3.88%). Hindsight task-level pass@8 coverage indicates that the underlying actor produces viable paths on only 5 out of 116 Hard tasks (4.31%). A verifier can reliably prune incorrect paths, but cannot invent missing solutions.
- Effective weak-to-strong guidance: An 8B GUI-PRA supervisor improves a 32B agent by 2.16 SR points, whereas an 8B standard PRM yields a negligible 0.43 point gain, confirming that grounded active verification enables efficient weak-to-strong alignment.
- Latency profile and tool utilization: Over a 1,475-step evaluation trace, GUI-PRA adds an average of 6.70 seconds per step, invoking 1.17 active visual tool calls (1.48 total including Terminate). Point (790) and OmniParser (702) handle routine grounding, while ZoomInSubfigure (237) is deployed selectively when ambiguity arises.
Highlights & Insights¶
- Active investigation over passive scoring: GUI-PRA departs from viewing PRMs as black-box probability estimators, granting them agency to formulate hypotheses, zoom into ambiguous UI regions, and cross-examine visual elements before issuing rewards.
- Decoupling what to verify from how to verify: Distilling offline principles into an explicit repository provides high-level domain heuristics, while autoregressive perception gathers low-level visual facts, avoiding both manual rule scripting and expensive full-model critic retraining.
- Strict enforcement of negative constraints: By checking explicit requirements before assigning scores, GUI-PRA decisively eliminates deceptive failures in tasks with "do not save" or strict ordering clauses.
Limitations & Future Work¶
- Bottlenecked by visual tool precision: Adjudication quality remains bounded by upstream tool performance (e.g., Molmo point grounding). While the autoregressive loop recovers from explicit missing detections, silent semantic misgrounding (such as repeatedly mistaking the Google Lens icon for Camera) can cause cyclic failure.
- Offline principle repository scope: Verification principles are currently distilled offline from contrastive trajectories on static benchmarks. Extending this to an online, continuously evolving memory that adapts to new UI paradigms represents an important open avenue.
- Desktop operating system extension: The current framework is validated on mobile interfaces (AndroidWorld, MiniWoB++); testing on complex multi-window, dynamic desktop workflows remains future work.
Related Work & Insights¶
- vs GUI-Critic-R1 / OS-Oracle: Supervised GUI critics require massive amounts of annotated interaction data and compute-heavy fine-tuning. GUI-PRA achieves competitive discrimination zero-shot by coupling distilled principles with dynamic tool use, offering superior cross-platform adaptability.
- vs ProRe / Agentic Reward Modeling: While recent works explore proactive reward modeling, they rely on unconstrained, free-form text reasoning without explicit verification rubrics or structured visual toolchains. GUI-PRA provides explicit, multi-granularity evidence chains that ground reward decisions in reproducible facts.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ [Conceptualizing PRMs as active, tool-using investigative agents is highly innovative]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Exemplary evaluation spanning online/offline benchmarks, ablations, weak-to-strong setups, latency audits, and compute-controlled baselines]
- Writing Quality: ⭐⭐⭐⭐⭐ [Crisply formulated methodology, transparent distinction between actor and verifier boundaries, and rigorous case analyses]
- Value: ⭐⭐⭐⭐⭐ [Provides a practical, training-free paradigm to resolve error accumulation in long-horizon multimodal agents]