Improving Reasoning in Vision-Language Models via Perception Verified Self-Training¶
Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/srbhcs/perception-verified-self-training
Area: Multimodal VLM
Keywords: vision-language models, self-training, perception verification, chain-of-thought reasoning, hallucination mitigation
TL;DR¶
This paper proposes an unsupervised perception-verified self-training framework that disentangles the caption-reasoning-conclusion pipeline, evaluates visual fidelity using FG-CLIP and PaddleOCR via PerceptEval, and employs a two-stage curriculum with caption-guided reasoning enhancement to boost VLM reasoning performance by up to 16% on M3CoT benchmarks.
Background & Motivation¶
Developing human-like multi-step reasoning capabilities in vision-language models (VLMs) remains a fundamental challenge in multimodal artificial intelligence. To instill Chain-of-Thought (CoT) capabilities, conventional approaches heavily rely on manual step-by-step annotations or expensive synthetic rationales queried from proprietary frontier models such as GPT-4V. However, curating multimodal CoT data via proprietary APIs is notoriously prohibitive in cost and difficult to scale across diverse domains. Self-training has emerged as an attractive alternative by using a model's own predictions as self-supervision, achieving prominent success in text-only language models such as STaR.
Nevertheless, directly adapting self-training to multimodal models introduces severe vulnerabilities. While text reasoning operates over symbolic language spaces, multimodal reasoning inherently demands accurate visual perception as its foundational premise. Existing VLM self-training frameworks, such as R3V, filter synthesized rationales solely based on final answer correctness. This outcome-only filtering allows contaminated pseudo-supervision to enter the training loop: models can leverage language shortcuts that bypass visual grounding (e.g., guessing a job interview solely from a camera mention while ignoring the true background) or fabricate visual hallucinations (e.g., hallucinating purple particles or white jars yet stumbling onto the right choice by chance). Training on such ungrounded rationales repeatedly compounds visual hallucinations and entrenches text-shortcut biases across iterative rounds.
The core tension is that visual perception and logical reasoning are deeply entangled inside standard generation prompts, preventing verification of whether intermediate reasoning stems from authentic visual perception or deceptive shortcuts, especially when ground-truth image captions are completely absent during self-training. Core idea: disentangle multimodal generation into an explicit [Caption–Reasoning–Conclusion] trajectory, introduce an unsupervised dual-signal perception verification module (PerceptEval) combining FG-CLIP visual alignment and OCR text agreement, and apply a two-stage curriculum with caption-guided reasoning enhancement to train exclusively on perceptually grounded reasoning.
Method¶
Overall Architecture¶
The framework transforms raw multimodal VQA triplets without intermediate rationales into high-fidelity, visually grounded reasoning data for supervised fine-tuning. The process first prompts the VLM with a structured template to generate an explicit caption, a reasoning chain, and a conclusion. A dual-filtering module subsequently evaluates answer correctness alongside unsupervised caption fidelity via PerceptEval, partitioning the candidate samples into easy, medium, and hard subsets. Finally, a two-stage curriculum fine-tunes the model on easy samples before progressively incorporating medium samples repaired by caption-guided reasoning enhancement.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input Image & Question<br/>(Image, Question)"] --> B["Disentangled Structured Generation<br/>Produces (Caption, Reasoning, Conclusion)"]
B --> C["Dual-Verification Filtering<br/>AnswerEval + PerceptEval Module"]
C -->|Correct Caption & Correct Answer| D["Easy Subset (Deasy)<br/>Stage-1 Baseline Self-Training"]
C -->|Correct Caption & Wrong Answer| E["Caption-Guided Reasoning Enhancement<br/>Regenerate Conditioned on Caption Context"]
C -->|Incorrect Caption / Hallucination| F["Hard Subset Discarded<br/>Block Hallucination Propagation"]
E -->|Regenerated Answer Correct| G["Medium Subset (Dmed)<br/>Stage-2 Mixed Self-Training"]
D --> H["Two-Stage Curriculum SFT<br/>Iterative Fine-Tuning on Original Base M"]
G --> H
H --> I["Converged Reasoned VLM Checkpoint"]
Key Designs¶
1. Disentangled perception-reasoning generation: isolating perception to eliminate shortcut reliance
Standard multimodal CoT prompting encourages the model to generate reasoning and answers in an intertwined format, blurring visual evidence with textual priors and preventing independent verification of perceptual grounding. To resolve this, the framework enforces a structured [CAP-REAS-CONCL] template. The model is mandated to first output an explicit, comprehensive description of the input image (Caption), subsequently derive deductive steps grounded on that perception (Reasoning), and finally emit the answer choice (Conclusion). This structural separation makes the visual perception phase transparent, providing a concrete interface for subsequent automated perception audits.
2. PerceptEval unsupervised perception verification: dual visual-textual consistency with adaptive coverage Without ground-truth reference captions in self-training datasets, automatically verifying whether self-generated captions suffer from visual hallucination presents a major hurdle. The proposed PerceptEval module addresses this via two complementary unsupervised signals: - OCR text agreement: For scientific, chart, and document reasoning where embedded text is crucial, PaddleOCR extracts all visible textual tokens to construct an auxiliary text description \(d_{\text{ocr}}\). Sentence-Transformers embeddings compute the cosine similarity \(s_{\text{ocr}}\) between the generated caption \(\hat{d}_i\) and \(d_{\text{ocr}}\), verifying that textual entities in the image are accurately reflected. - FG-CLIP fine-grained visual agreement: For natural scenes and geometric objects, FG-CLIP computes the vision-language alignment score \(s_{\text{vis}}\) between image \(I_i\) and caption \(\hat{d}_i\), scoring spatial and semantic fidelity. - Domain-aware dynamic thresholding: Fixed thresholds struggle when switching between text-heavy diagrams and purely natural images. The framework computes the text area coverage ratio: $\(r_t = \frac{\text{Total OCR area}}{\text{Image area}}\)$ Dynamic thresholds for \(s_{\text{ocr}}\) and \(s_{\text{vis}}\) are adjusted based on \(r_t\). A generated caption is accepted (\(\text{validcap}_i = \text{True}\)) only if both scores exceed their respective adaptive thresholds, successfully avoiding single-modality blind spots where text is misinterpreted as objects or fine-grained text is overlooked.
3. Curriculum learning with caption-guided reasoning enhancement: progressive expansion of supervision boundaries
Coupling perception fidelity with answer correctness partitions generated samples into three difficulty tiers: easy samples \(D_{\text{easy}}\) (valid caption and correct answer), medium samples \(D_{\text{medium}}\) (valid caption but incorrect answer), and hard samples \(D_{\text{hard}}\) (invalid/hallucinated caption).
- Stage-1 (Easy sample self-training): The base model is initially fine-tuned solely on high-purity \(D_{\text{easy}}\) data. To prevent error drift and representational collapse across multiple iterations, fine-tuning in each round is always initialized from the original base model \(M\) rather than the intermediate checkpoint, iterating until validation performance plateaus.
- Caption-guided reasoning enhancement and Stage-2 (Mixed self-training): Medium samples demonstrate that the model accurately perceives visual elements but fails to properly exploit them during deduction. To salvage these samples, the verified caption \(\hat{d}_i\) is injected into the prompt as explicit contextual grounding (CapEmb-Prompt), instructing the model to regenerate the reasoning and conclusion conditioned on confirmed visual facts. If the regenerated conclusion matches the ground truth, the sample is packaged as refined instance \((\hat{d}_i, \tilde{r}_i, a_i)\) and added to \(D_{\text{med}}\). In Stage-2, the model is fine-tuned on the combined pool \(D_{\text{easy}} \cup D_{\text{med}}\), allowing it to master complex reasoning challenges on top of stabilized perceptual foundations.
A Worked Example¶
Consider a scientific inquiry comparing particle kinetic energy and temperature:
- Question: Compare the average kinetic energies of particles in two samples. Which sample has the higher temperature?
- Baseline behavior: STaR hallucinates visual attributes ("sample A has purple and green particles"), while R3V falls prey to language shortcuts ("particles in the white jar have higher temperature because they are smaller"), both surviving standard self-training filters solely due to lucky multiple-choice guesses.
- Perception-verified pipeline:
1. Under [CAP-REAS-CONCL], the model outputs: "Caption: The image shows two jars, sample A with particle mass 32 u and sample B with particle mass 46 u. Both jars have equal particle counts and average particle speed of 1400 m/s."
2. PerceptEval validates the caption: PaddleOCR confirms text tokens 32u, 46u, 1400 m/s (\(s_{\text{ocr}} > \text{threshold}\)), while FG-CLIP confirms container alignment (\(s_{\text{vis}} > \text{threshold}\)).
3. Reasoning follows physical laws: "Kinetic energy depends on mass and velocity. With equal velocities, particles with greater mass possess higher kinetic energy, corresponding to higher temperature. Conclusion: (b) sample B."
4. The verified sample enters \(D_{\text{easy}}\), providing solid, hallucination-free reasoning supervision.
Loss & Training¶
The fine-tuning phase employs standard autoregressive cross-entropy loss over response tokens. Training is conducted with AdamW for 2 epochs at a learning rate of \(3 \times 10^{-5}\) with an effective batch size of 16. Parameter-efficient fine-tuning utilizes LoRA: rank 64, alpha 16 for Qwen2-VL-7B-Instruct; rank 128, alpha 256 for LLaVA-1.5-7B/13B, with a dropout rate of 0.05. All experiments run on a single NVIDIA A6000 GPU (48 GB VRAM), highlighting high computational efficiency.
Key Experimental Results¶
Main Results¶
The method was evaluated across four diverse domains of the M3CoT benchmark (Language Science, Commonsense, Social Science, Natural Science) and benchmarked against zero-shot prompting variants, direct SFT, and leading self-training methods STaR and R3V.
Table 1: VQA accuracy (%) across reasoning methods and domains on LLaVA-v1.5-7B
| Paradigm | Method | Language Science | Commonsense | Social Science | Natural Science | Gain / Note |
|---|---|---|---|---|---|---|
| Zero-Shot | Direct VQA | 45.50 | 57.58 | 29.62 | 36.40 | Baseline reference |
| Zero-Shot | CoT (Let's think step by step) | 38.86 | 59.34 | 25.48 | 33.59 | Suffers from ungrounded exploration |
| Zero-Shot | Desp-CoT | 34.12 | 54.73 | 25.32 | 32.18 | Lower accuracy |
| Zero-Shot | CCoT | 26.54 | 54.08 | 28.66 | 35.50 | Lower accuracy |
| Zero-Shot | [CAP-REAS-CONCL] | 40.28 | 53.40 | 27.55 | 33.20 | Structured zero-shot baseline |
| Direct SFT | VQA Finetune (direct answer) | 46.45 | 60.22 | 34.24 | 46.10 | Modest improvement |
| Self-Training | STaR | 48.82 | 64.98 | 41.88 | 53.90 | Hint exposure induces shortcuts |
| Self-Training | R3V | 50.24 | 65.49 | 42.20 | 51.34 | Outcome-only filtering saturates early |
| Self-Training | Ours | 64.93 | 72.74 | 46.56 | 58.24 | Outperforms R3V by up to +14.69% |
On the stronger Qwen2-VL-7B-Instruct backbone, the framework maintains consistent advantages across all four domains: Language Science (78.67% vs R3V 76.78%), Commonsense (82.42% vs R3V 78.68%), Social Science (49.04% vs R3V 44.90%), and Natural Science (68.45% vs R3V 63.86%). Furthermore, in zero-shot Out-of-Distribution transfer on MMMU Science validation, the proposed approach achieves 36.48% accuracy, surpassing both STaR (32.43%) and R3V (33.78%).
Ablation Study¶
Systematic ablations evaluate the roles of perception-based filtering, curriculum scheduling, and individual PerceptEval components.
Table 2: Ablation of caption filtering and curriculum learning strategy (Accuracy %)
| Configuration | Description | Language Science | Commonsense | Findings |
|---|---|---|---|---|
| Ours-Stage1 (Ans Filter) | Filters samples solely by answer correctness | 49.76 | 62.42 | Hallucinations pollute pseudo-supervision |
| Ours-Stage1 (Dual Filter) | Introduces PerceptEval, trained on easy samples only | 60.66 | 69.45 | Significant jump (+10.9% / +7.03%) from data purity |
| Ours (W/O CL) | Blends easy and enhanced medium samples from iteration 1 | 57.34 | 65.05 | Premature difficulty mixing causes optimization confusion |
| Ours (Full Model) | Stage-1 easy convergence followed by Stage-2 medium expansion | 64.93 | 72.74 | Curriculum stages unlock peak reasoning performance |
Table 3: Ablation of PerceptEval verification components (Accuracy %)
| Evaluator Setting | Language Science | Commonsense | Modality Mechanism |
|---|---|---|---|
| Only OCR Similarity | 61.13 | 60.44 | Blind to non-textual visual scenes in commonsense |
| Only FG-CLIP Similarity | 56.87 | 69.89 | Misses alphanumeric details in text-heavy tasks |
| Adaptive Dual-Signal (Ours) | 64.93 | 72.74 | Harmonizes visual layout and symbolic text tokens |
Key Findings¶
- The negative CoT paradox in zero-shot VLMs: In zero-shot LLaVA-1.5, CoT prompting routinely underperforms direct answering (e.g., dropping from 45.50% to 38.86% in Language Science). Without grounded guidance, intermediate unverified text tokens accumulate hallucinations and mislead the final choice.
- Perception filtering is the primary performance driver: Moving from answer-only filtering (49.76%) to dual filtering (60.66%) yields an immediate +10.9% leap in Language Science, proving that ungrounded rationales constitute the fundamental bottleneck of multimodal self-training.
- Curriculum inflection point: Iteration curves reveal that introducing medium samples from iteration 1 causes early plateauing. In contrast, introducing caption-guided medium samples at iteration 8 (after Stage-1 plateaus) sparks a sharp secondary surge in validation accuracy.
- Hallucination suppression and concise rationales: On the HallusionBench OCR benchmark, the fine-tuned model scores 60.14%, substantially surpassing base LLaVA (37.76%) and STaR (54.54%). Moreover, output token lengths remain moderate (69.2–93.9 tokens), proving that the model achieves robust reasoning without verbose padding.
Highlights & Insights¶
- Unsupervised multimodal data gatekeeper: Integrating PaddleOCR and FG-CLIP with text coverage adaptation creates an effective reference-free filter. This eliminates dependence on costly closed-source API supervision for data sanitation.
- Salvaging near-miss failures: By isolating medium samples where perception was sound but reasoning faltered, the framework feeds verified perception back into context. This bridges the gap between raw perception and logical deduction.
- Budget-friendly self-improvement: Operating within an SFT-only regime on a single A6000 GPU without reinforcement learning instability (e.g., PPO/GRPO reward hacking), the framework demonstrates an accessible, high-yield blueprint for training performant open-source vision-language models.
Limitations & Future Work¶
- Author-acknowledged limitations: PerceptEval depends on external OCR and CLIP tools; OCR failures on severe affine distortions or handwriting can incorrectly discard valid samples. Additionally, the empirical validation primarily targets single-image multiple-choice benchmarks.
- Observed limitations: Discarding hard samples entirely may starve self-training on niche long-tail visual distributions where base vision encoders struggle. Furthermore, multi-stage offline generation increases upfront forward-pass computation linearly with iteration count.
- Future directions: Incorporating soft perceptual prompts (e.g., attention-driven region bounding) to salvage hard samples, and casting the offline dual-filter checks into dense process rewards (PRM) within online reinforcement learning pipelines.
Related Work & Insights¶
- vs STaR (Zelikman et al., 2022): STaR bootstraps reasoning using answer hints upon failure, which encourages shortcut learning and text-only cheating in multimodal settings. The proposed framework avoids answer leakage by conditioning solely on verified visual descriptions.
- vs R3V (Cheng et al., 2025): R3V performs reflective self-training but filters strictly by final answer correctness, allowing visual hallucinations to propagate into training checkpoints. This paper enforces perception-reasoning disentanglement and dual-signal gating to guarantee grounded reasoning.
- vs Vision-R1 / Visionary-R1 (2025–2026): While recent RL-based VLM approaches rely on complex rollout sampling and reward modeling, this work shows that supervised self-training paired with rigorous perceptual verification provides a highly competitive, compute-efficient alternative.
Rating¶
- Novelty: ⭐⭐⭐⭐☆ [Pioneers perception-verified multimodal self-training and unsupervised dual-signal PerceptEval]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive cross-model validation on LLaVA-1.5 7B/13B and Qwen2-VL across M3CoT, MMMU, and HallusionBench]
- Writing Quality: ⭐⭐⭐⭐⭐ [Clear motivation, well-structured arguments, and illustrative qualitative comparisons]
- Value: ⭐⭐⭐⭐☆ [Offers a practical, low-cost recipe for grounded VLM self-improvement without expensive proprietary APIs or RL policy optimization]