Skip to content

CaptchaMind: Training CAPTCHA Solvers via Reinforcement Learning with Explicit Reasoning Supervision

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/AlibabaResearch/captcha-mind
Area: Multimodal VLM
Keywords: Vision-Language Models, Reinforcement Learning, CAPTCHA Solving, Explicit Reasoning Supervision, Fine-Grained Visual Grounding

TL;DR

Addressing the vulnerability of vision-language models to fine-grained visual reasoning failures under modern CAPTCHAs, this paper presents CaptchaBench (16,000 programmatically generated samples with process-level annotations) and CaptchaMind, an RL-based solver that explicitly supervises intermediate reasoning by rewarding correct bounding-box identification of task-relevant visual regions, achieving an 82.9% average success rate across eight tasks and outperforming leading closed-source models.

Background & Motivation

Multimodal intelligent agents driven by large language models and vision-language models (VLMs) are increasingly deployed to execute complex end-to-end tasks in real-world web environments, such as autonomous web navigation, complex graphic interface operations, and interactive online service completion. However, CAPTCHAs (Completely Automated Public Turing test to tell Computers and Humans Apart), designed to distinguish human users from automated scripts, remain a critical bottleneck that blocks automated agent workflows. Contemporary CAPTCHAs have evolved far beyond classic optical character recognition challenges into multi-step interactive puzzles demanding spatial reasoning, cross-image matching, fine-grained object counting, and sequential coordination. Recent evaluations on benchmarks like OpenCaptchaWorld reveal that even premier proprietary multimodal configurations achieve only around 40% success, trailing far behind human competency and exposing severe reasoning deficiencies.

Existing CAPTCHA-solving methodologies rely almost exclusively on prompting expensive closed-source proprietary VLMs (e.g., Oedipus and Halligan), which incur prohibitive API latency and costs while lacking internal controllability. Concurrently, developing open-source training-based alternatives has historically floundered because supervised fine-tuning yields poor generalization. This barrier stems from two fundamental gaps: the community lacks large-scale, interactive training corpora reflecting commercial CAPTCHA complexity, and existing evaluation-only datasets offer no process-level annotations to supervise intermediate visual thinking. Consequently, models trained purely on end-to-end outcome signals collapse into black-box heuristics, relying on coarse holistic image impressions rather than attending to critical local visual features.

The fundamental tension resides in the misalignment between the need for precise intermediate visual localization and the opacity of end-to-end outcome-driven training. Solving a visual CAPTCHA requires pinpointing subtle clues (such as tiny connecting lines between paired icons, faint dice pip counts, or rotation angles) before executing physical clicks or drags; yet standard training offers only a binary win-or-lose reward at the very end of an episode, leaving the visual reasoning trajectory uninspected and unrewarded. To bridge this divide, this paper posits that intermediate visual attention must be externalized into verifiable tool actions that receive direct, process-level reinforcement. Core idea: Introduce CaptchaBench, a 16,000-sample benchmark with ground-truth process annotations, and develop CaptchaMind, an RL solver that explicitly rewards the model for identifying task-relevant visual regions via an intermediate bounding-box tool under a multi-stage reward scheme, effectively eliminating visual grounding failures in fine-grained reasoning.

Method

Overall Architecture

CaptchaMind formalizes CAPTCHA solving as a partially observable Markov decision process (POMDP). At time step \(t\), the agent receives observation \(h_t\) (current screenshot, textual instruction, and interaction history) and selects an action \(a_t\) from an action space comprising clicking (click), dragging (drag), numerical typing (enter_number/type), and drawing bounding boxes (bounding_box). The training framework consists of two sequential stages: a supervised fine-tuning (SFT) warm-up on synthetic trajectories with explicit chain-of-thought (CoT) reasoning to instantiate baseline tool execution and the think-then-act paradigm on Qwen2.5-VL-7B; followed by reinforcement learning using the Group Relative Policy Optimization (GRPO) algorithm, where intermediate region identification is directly scored against programmatic ground truth alongside interaction progress and final task completion.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Observation<br/>CAPTCHA Screenshot + History + Prompt"] --> B["Stage 1: Programmatic Data Generation & SFT Warm-Up<br/>16K synthetic episodes + multi-step trajectories + CoT"]
    B --> C["Stage 2: Explicit Visual Region Grounding<br/>Invoke bounding_box to expose task-relevant regions"]
    C --> D["Stage 3: Multi-Level RL Reward Formulation<br/>R = R_process + R_feedback + R_outcome"]
    D --> E["Stage 4: GRPO Policy Optimization with Adaptive Supervision<br/>8 rollouts/prompt joint updates, difficulty-gated R_process"]
    E --> F["Action Execution<br/>Accurate Click / Drag / Image Switch / Final Submission"]

Key Designs

1. Programmatic Data Generation & SFT Warm-Up: Bootstrapping Dense Exploration Trajectories

Directly applying reinforcement learning from a cold start fails catastrophically due to extreme reward sparsity, yielding only 25.6% average success. To establish reliable exploration priors, the authors develop an automated Python-based simulator generating 16,000 diverse CAPTCHA instances across eight tasks (categorized into image-switching, multi-step interaction, and single-step decision tasks). The simulator programmatically logs exact ground-truth bounding boxes, valid intermediate state transitions, and step-by-step chain-of-thought (CoT) reasoning traces without human annotators. In the SFT phase, 8,000 samples are utilized to train the base model (Qwen2.5-VL-7B) via behavioral cloning over 3 epochs, establishing stable tool invocation syntax and elevating baseline performance to 68.1%, providing sufficiently dense initial trajectories for subsequent policy optimization.

2. Explicit Visual Region Grounding: Converting Latent Attention into Supervisable Tool Invocations

To overcome the tendency of standard VLMs to hallucinate based on holistic visual impressions, CaptchaMind repurposes the bounding_box tool into an externalization mechanism for intermediate visual reasoning. Before executing state-altering actions (e.g., clicking candidate icons or dragging sliders), the model is instructed to mark the image regions it deems critical to solving the puzzle. Crucially, drawing bounding boxes produces no side effects on the environment state or underlying CAPTCHA logic; rather, it renders the model's internal visual reasoning observable and measurable. By comparing predicted bounding boxes against the ground-truth key regions recorded during data generation, the training framework quantitatively inspects whether the model is attending to the correct visual evidence.

3. Multi-Level RL Reward Formulation: Integrating Process Grounding, Interaction Progress, and Outcome

To provide dense, informative guidance across extended reasoning and action sequences, CaptchaMind introduces a multi-tier composite reward: $\(R = R_{\text{process}} + R_{\text{feedback}} + R_{\text{outcome}}\)$ The reasoning process reward \(R_{\text{process}}\) directly scores visual perception fidelity: a predicted bounding box is marked correct if its intersection-over-union (IoU) with any ground-truth target region exceeds 0.8. A positive process reward is awarded when the model identifies at least 50% of all annotated ground-truth regions, encouraging comprehensive evidence gathering while tolerating minor omissions. The interaction feedback reward \(R_{\text{feedback}}\) provides positive reinforcement whenever intermediate actions produce valid forward transitions according to task state logic in multi-step interaction tasks (such as image_select and click_order). Finally, the outcome reward \(R_{\text{outcome}}\) grants a terminal reward of 1 upon successful puzzle verification and 0 otherwise.

4. GRPO Policy Optimization with Adaptive Supervision: Difficulty-Gated Process Guidance

The agent is optimized using Group Relative Policy Optimization (GRPO) on 16 NVIDIA A100 GPUs, executing joint multi-task training with 8 rollout episodes per prompt. Recognizing that enforcing dense process rewards across all tasks indiscriminately can over-constrain the policy and disrupt established heuristics, the authors devise an adaptive supervision strategy: the process reward \(R_{\text{process}}\) is selectively activated only for tasks where the SFT baseline accuracy falls below 50% (specifically connect_icon and image_select), which are precisely the tasks characterized by severe fine-grained visual bottlenecks. For tasks where SFT already achieves satisfactory performance, \(R_{\text{process}}\) is set to 0, allowing the model to refine its policy using interaction feedback and outcome rewards alone.

Loss & Training

The training pipeline utilizes Qwen2.5-VL-7B as the foundation vision-language model. Stage 1 executes supervised fine-tuning across 8,000 samples for 3 epochs with a learning rate of \(5\times 10^{-6}\). Stage 2 conducts reinforcement learning via GRPO across the remaining 8,000 training samples for 1 epoch, utilizing a batch size of 64, 8 rollouts per prompt, and a learning rate of \(5\times 10^{-6}\) over 20 hours on 16 NVIDIA A100 (80GB) GPUs.

Key Experimental Results

Main Results

Performance on the 1,600-sample CaptchaBench test set across all eight task categories (reported as task success rate in %):

Method Category Model / Method connect_icon coordinates dart_count dice_count rotation_match image_select click_order slide_puzzle Avg. Success Rate (%)
Closed-source VLM Claude-4-Sonnet 7.5 93.5 68.5 51.0 20.0 43.0 71.5 24.5 47.4
Closed-source VLM Claude-3.7-Sonnet 19.0 83.5 51.0 75.5 10.0 38.5 30.0 44.0 43.9
Closed-source VLM Gemini-3-Pro 47.5 62.0 88.5 63.0 50.0 52.0 0.0 3.0 45.8
Closed-source VLM OpenAI-o3 37.0 69.0 51.0 72.0 4.0 46.0 62.0 34.0 46.9
Closed-source VLM GPT-5 15.0 35.5 37.0 42.0 3.5 32.5 67.0 36.0 33.6
Closed-source VLM Qwen-VL-Max 14.5 55.5 40.0 42.5 5.5 39.5 72.5 39.0 38.6
Agentic Architecture Oedipus 35.0 68.0 76.0 44.5 31.5 53.0 68.5 38.5 51.9
Agentic Architecture Halligan 34.0 76.5 69.5 43.0 36.0 66.0 71.0 41.5 54.7
GUI Agent Baseline GUI_R1_7b 0.0 13.5 14.0 0.5 3.5 18.0 2.0 0.0 6.5
Training-based SFT SFT-only (Qwen2.5-VL-7B) 49.0 79.5 77.0 76.0 88.0 14.0 70.5 91.0 68.1
Proposed Method CaptchaMind (Ours) 71.0 91.0 93.0 72.5 86.0 71.0 87.5 91.5 82.9

Ablation Study and OOD Generalization

Ablation analysis on the test set alongside cross-dataset generalization on OpenCaptchaWorld (151 samples across overlapping task types):

Configuration / Benchmark Setting Avg. Success (%) connect_icon image_select click_order slide_puzzle Remarks & Diagnostics
CaptchaMind (Full) 82.9 71.0 71.0 87.5 91.5 Full framework with adaptive process rewards
w/o Reasoning Process Sup. (w/o RPS) 78.9 56.0 (-15.0) 54.5 (-16.5) 90.0 92.0 Severe collapse on fine-grained detail tasks
w/o Feedback Reward (w/o FBR) 80.0 70.5 58.0 (-13.0) 79.0 (-8.5) 91.0 Multi-step sequential interactions degrade
RL-only (w/o SFT warm-up) 25.6 31.0 11.5 37.0 6.0 Reward sparsity prevents effective policy learning
OpenCaptchaWorld (OOD Transfer)
CaptchaMind (Ours) 66.2 (100/151) 6/10 10/20 15/20 29/31 SOTA out-of-distribution transfer without APIs
SFT-only 29.8 (45/151) 1/10 2/20 8/20 15/31 SFT memorization collapses under distribution shift
Claude-4-Sonnet (Closed-source ref.) 51.0 (77/151) 4/10 13/20 15/20 18/31 Strongest proprietary VLM baseline on OOD

Key Findings

  • Process supervision is decisive for fine-grained perceptual bottlenecks: Removing \(R_{\text{process}}\) causes the success rate to plummet by 15.0% on connect_icon and 16.5% on image_select, whereas simpler or single-step tasks exhibit negligible degradation. This confirms that process-level supervision targets exactly where standard outcome rewards fail.
  • Grounding coverage correlates strongly with task success: Diagnostic evaluation over 1,600 episodes shows that episodes with region grounding coverage under 0.2 achieve only a 16% success rate; coverage between 0.4 and 0.6 yields 82% success, and coverage exceeding 0.6 reaches 90%โ€“91%, demonstrating an empirical monotonic relationship between correct intermediate visual grounding and puzzle resolution.
  • Generalist GUI agents fail on CAPTCHA logic: GUI-specialized agents like GUI_R1_7b (achieving only 6.5% overall) fail completely due to formatting mismatches (e.g., outputting unparseable HTML tables) or shallow reasoning patterns (e.g., reflexively clicking prominent navigation arrows without cross-referencing candidate validity).
  • High ecological validity and human perceptual parity: In a double-blind discrimination study with 50 participants, human classification accuracy between synthetic CaptchaBench instances and real commercial CAPTCHAs was 51.04% (\(p = 0.178\), Cohen's \(\kappa = 0.02\)). Furthermore, CaptchaMind achieves a 71.0% zero-shot success rate on 100 live commercial CAPTCHAs from services like GeeTest and hCaptcha.

Highlights & Insights

  • Dual utility of the bounding-box tool: Rather than serving merely as an image-cropping operator, tool invocation is structured as a bidirectional cognitive bridgeโ€”acting simultaneously as an attentional anchor for the model and an inspection interface for programmatic process supervision.
  • Selective difficulty-gated reward scheduling: Enabling dense process rewards only on tasks where initial SFT capability is under 50% successfully avoids reward hacking and gradient interference on well-behaved tasks, simplifying reward engineering.
  • Surpassing commercial APIs with an open-source 7B backbone: Demonstrates that structured reinforcement learning with intermediate process supervision enables a lightweight 7B open-source model to comfortably outperform leading frontier models (Claude-4, GPT-5, OpenAI-o3) on intricate visual verification tasks.

Limitations & Future Work

  • Static screenshot evaluation vs. dynamic bot-detection defenses: The authors acknowledge that due to legal and ethical boundaries, real-world testing was restricted to static screenshots. True deployment scenarios involve complex browser fingerprints, JavaScript dynamic obfuscation, and mouse trajectory dynamics that lie beyond static visual solvers.
  • Exploration across alternative model scales and families: The core RL framework was validated primarily on Qwen2.5-VL-7B. Investigating how explicit reasoning supervision transfers to other architectures (e.g., InternVL, LLaVA) or smaller edge-deployable models remains an open question.
  • Future directions: Integrating reinforcement-trained visual reasoning solvers with dynamic browser automation runtimes, while concurrently designing next-generation cognitive defense mechanisms capable of resisting AI solvers.
  • vs. Oedipus (Deng et al., 2025) & Halligan (Teoh et al., 2025): Prior solvers rely on external prompt engineering and closed-source APIs, which lack internal process control and exhibit high latency; CaptchaMind establishes an end-to-end open-source trainable paradigm with explicit perceptual supervision.
  • vs. OpenCaptchaWorld (Luo et al., 2025b): OpenCaptchaWorld was designed solely as an evaluation benchmark without process annotations; CaptchaBench expands sample size by two orders of magnitude (16,000 instances) and supplies dense key-region ground truth for training.
  • vs. DeepEyes (Zheng et al., 2025) & Ground-R1 (Cao et al., 2025): While the "thinking with images" line of research uses visual tools primarily for zooming or perception enhancement, CaptchaMind directly optimizes intermediate tool-based region coverage with verifiable ground-truth rewards via RL.

Rating

  • Novelty: โญโญโญโญโญ Pioneering training-based CAPTCHA framework that operationalizes bounding boxes as explicit process supervision interfaces under RL.
  • Experimental Thoroughness: โญโญโญโญโญ Comprehensive 8-task benchmark, human indistinguishability study, ablation across all reward terms, and real-world commercial validation.
  • Writing Quality: โญโญโญโญโญ Crisp, well-motivated narrative with transparent technical specifications and diagnostic analyses.
  • Value: โญโญโญโญโญ Crucial milestone for both web automation security research and fine-grained multimodal reasoning training.