OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration

Conference: ICML 2026
arXiv: 2605.28805
Code: Paper does not provide a repository link (None)
Area: Multimodal VLM / Visual Verification / Reinforcement Learning
Keywords: Multimodal meta-verifier, symbolic rewards, decoupled RL, visual self-correction, RLVR

TL;DR

To address coarse binary (True/False) signals in multimodal visual verifiers and the susceptibility of textual explanations to reward-hacking, this paper proposes OmniVerifier-M1. It replaces textual explanations with symbolic outputs (e.g., bounding boxes) as meta-verification rationales to support rule-based rewards like IoU. Furthermore, it theoretically and experimentally demonstrates that decoupling binary judgment and meta-verification into two independent reward streams (rather than a multiplicative joint reward) significantly improves SNR, transforming the verifier into an agentic system (M1-TTS) capable of driving region-level self-correction.

Background & Motivation

Background: Multimodal Large Language Models (MLLMs) are becoming increasingly powerful in generation and reasoning, necessitating reliable verifiers as reward or reflection signal sources. Existing works fall into two categories: (i) traditional image reward models (e.g., RewardDance, UnifiedReward) focused on text-to-image scoring; (ii) general visual verifiers (e.g., OmniVerifier) that use RLVR (Reinforcement Learning with Verifier Rewards) with binary judgment (True/False) as the reward.

Limitations of Prior Work: Pure binary judgment signals suffer from two issues: first, supervision is at the decision level rather than the rationale level, allowing models to receive full rewards by "guessing correctly" or exploiting surface patterns without fine-grained reasoning; second, using textual explanations as rationales requires an LLM judge for scoring, which is slow and prone to reward hacking.

Key Challenge: Fine-grained feedback requires rationale supervision, but textual rationales are either expensive/vulnerable (model-based judgment) or difficult to standardize (rule-based judgment). Furthermore, binary judgment and meta-verification tasks have naturally different output spaces—the former being discrete and low-entropy, while the latter is continuous and high-dimensional/fine-grained—leading to severe optimization conflicts in a joint reward setting.

Goal: (i) Identify a rationale form that is strictly rule-based and accurately represents visual errors; (ii) address the "gating" of meta-verification gradients by binary accuracy in joint reward settings; (iii) upgrade the verifier into an agent capable of driving region-level self-correction within a generation loop.

Key Insight: Images are highly structured spatial representations, where errors can be naturally localized using symbols like bounding boxes or keypoints. This allows for rule-based rewards (e.g., IoU) instead of model-based judges, eliminating reward hacking. Theoretically, decoupling the meta gradient from the binary accuracy \(p_{acc}\) gating in a joint reward can restore the SNR.

Core Idea: Symbolic rationale (bbox) + Decoupled RL reward. Use bboxes as meta-verification rationales to enable IoU as a rule-based reward; split binary judgment and meta-verification into independent reward streams via mixed-data training.

Method

OmniVerifier-M1 follows the RLVR framework to train a pointwise multimodal verifier \(\pi_\theta(I, P) \to (o, \hat y, e)\), outputting a thought process \(o\), binary judgment \(\hat y\), and an error localization \(e\) (bbox) when \(\hat y = \text{False}\). The methodology centers on "what rationale to use" and "how to combine rewards."

Overall Architecture

Input (image, prompt, ground-truth label, optional ground-truth bbox); output \((o, \hat y, e)\). The reward consists of three parts: format reward \(\mathcal{R}_f\) (requiring <think> tags), accuracy reward \(\mathcal{R}_{acc} \in \{0,1\}\), and meta-verification reward \(\mathcal{R}_{meta}\) (IoU in the symbolic case). Training uses DAPO on OmniVerifier-7B and Qwen3-VL-8B for 80 steps using 16 A800-80G GPUs. M1-TTS utilizes the verifier as an agent tool: identifying error regions to drive region-level editing and iterative replanning until convergence.

Key Designs

1. Symbolic Rationale — Bbox instead of Textual Explanation:

   • Function: Transition the verifier's rationale from "free text" to "structured geometric objects" (bbox/point/line), allowing hard-rule rewards like IoU.
   • Mechanism: For each training sample, besides binary labels, ground-truth bboxes and textual explanations are provided. The symbolic route uses \(\mathcal{R}_{meta} = \text{IoU}(\hat{b}, b^*)\) to evaluate the predicted bbox \(\hat b\), while the textual route uses Qwen3-4B as a judge for semantic equivalence. The model still outputs a final verdict and a list of bboxes after <think>.
   • Design Motivation: Visual errors are essentially spatial ("where is it wrong"). Symbolic bboxes are closer to this essence than text. Rule-based rewards eliminate reward hacking and save computational costs—empirical results show reward calculation at 0.021 ms (symbolic) vs. 20.2 ms (textual) (~1000x speedup). Training time per step improved from 10.27 min to 8.13 min (~20% acceleration), and VRAM usage dropped from 56.9 GB to 48.6 GB. The equivalent performance on ViVerBench (0.661 vs 0.662) proves symbolic is a "lossless but cheaper" alternative.

2. Decoupled RL Reward — Splitting Data/Reward Streams:

   • Function: Separates the tasks of "judging correctness" and "locating errors" into independent data/reward streams rather than a multiplicative joint structure.
   • Mechanism: In the joint objective \(\mathcal{R}_f + \mathcal{R}_{acc} \cdot (\mathbb{I}[y=\text{True}] + \mathbb{I}[y=\text{False}] \cdot \mathcal{R}_{meta})\), the meta gradient is only active when \(y=\hat y=\text{False}\). The decoupled scheme uses a 1:1 balanced dataset to supervise \(\mathcal{R}_{acc}\) and a "grounding-only" subset (replicating \(y=\text{False}\) samples) to supervise \(\mathcal{R}_{meta}\). These streams are mixed during RL rollouts.
   • Design Motivation: Authors prove (Lemma 5.1 / Theorem 5.2) that the meta gradient norm in joint training is multiplicatively gated by \(p_{acc}(\theta)\); in early RL stages where \(p_{acc} \ll 1\), the meta task learns almost nothing. Theorem 5.3 and Corollary 5.4 show \(\text{Var}(\mathcal{G}_{joint}) = p_{acc} \text{Var}(\mathcal{G}_{dec}) + p_{acc}(1-p_{acc})\|\mathbb{E}[\mathcal{G}_{dec}]\|^2\) and \(\text{SNR}(\mathcal{G}_{joint}) \le p_{acc}(\theta) \cdot \text{SNR}(\mathcal{G}_{dec})\), implying joint training is strictly suboptimal. Decoupling removes the Bernoulli gate, restoring pure grounding gradients.

3. M1-TTS — Verifier-Driven Region-Level Agentic Self-Correction:

   • Function: Treats OmniVerifier-M1 as a "fine-grained optimizer" for scheduling agents, converting judgment/localization into tool-level actions (symbolic localization + structured text edit) to drive multi-round self-correction.
   • Mechanism: Per round: base model generates image -> verifier predicts True/False -> if False, verifier provides error bboxes -> planner translates bboxes into "region-aware editing prompts" -> localized inpainting/editing occurs -> next round enters. Replanning is monitored by the verifier until all regions pass.
   • Design Motivation: Traditional multi-turn editing is global, failing on small semantic errors. With symbolic feedback, the agent can focus the generative resource on error regions. This extends the fine-grained advantage of meta-verification from training to inference.

Loss & Training

RL uses DAPO (a GRPO variant). Reward = format (indicator) + accuracy (0/1) + meta (IoU or model-based judge). Decoupled training mixes two streams: balanced data for \(\mathcal{R}_f + \mathcal{R}_{acc}\), and False-only data for \(\mathcal{R}_f + \mathcal{R}_{meta}\). Advantages are estimated within rollout groups per dataset, using standard PPO clipping and KL regularization.

Key Experimental Results

Main Results

ViVerBench covers 6 categories and 16 sub-tasks including Concept Existence, Object Relation, World Dynamics, etc., combined with RefCOCO for localization evaluation. (Table 2 excerpt):

Model	Obj.	Attr.	Spat.	BBox	Point	Count	GUI	Chart	Overall
OmniVerifier-7B (baseline)	0.701	0.703	0.808	0.770	0.659	0.527	0.634	0.600	0.650
OmniVerifier-7B (Joint)	0.723	0.733	0.833	0.827	0.716	0.640	0.694	0.623	0.661
OmniVerifier-7B (Decoupled)	0.741	0.754	0.846	0.854	0.741	0.710	0.722	0.639	0.668

On the stronger Qwen3-VL-8B backbone, the same pattern holds: Decoupled > Joint > baseline. Grounding-heavy sub-tasks like BBox, Point, and Count show the largest gains (+8–18%), confirming that meta-verification supervision strengthens grounding capabilities.

Ablation Study

Config	ViVerBench	GPU VRAM (GB)	Reward Calc (ms/sample)	Training Time (min/step)	Response Length (token)
OmniVerifier-7B baseline	0.650	—	—	—	—
+ Bbox (symbolic)	0.661	48.6	0.021	8.13	384
+ Exp (textual)	0.662	56.9	20.2	10.27	340
Qwen3-VL-8B baseline	0.654	—	—	—	—
+ Bbox	0.671	49.9	0.021	8.74	516
+ Exp	0.670	58.3	20.2	11.08	488

Key Findings

• Symbolic ≈ Textual in performance but significantly cheaper: The performance gap is < 0.001, but symbolic reward calculation is ~1000x faster, VRAM is ~15% lower, and training is ~20% faster.
• Decoupled gains are not just from "more data": The replicated False samples only provide \(\mathcal{R}_{meta}\), not binary supervision. Gains stem from the meta gradient escaping the \(p_{acc}\) gating.
• Grounding sub-tasks benefit the most: BBox improves 0.770→0.854, Count 0.527→0.710, and Point 0.659→0.741, confirming meta-verification signals flow into the model's grounding capacity.
• M1-TTS outperforms global multi-turn editing: In region-level self-correction, the agentic system driven by OmniVerifier-M1 is more efficient and has lower error rates than traditional global regeneration.

Highlights & Insights

• "Substituting structured image essence for free text": Transitioning verifier output from language to symbolic states (bbox/point) solves reward hacking, training efficiency, and localization precision simultaneously. This is a prime example of leveraging task-specific geometric structures.
• Theoretical breakdown of Joint vs Decoupled: Lemmas and Theorems quantify why joint training fails using SNR inequalities, providing more rigor than typical empirical RL papers. This framework is extensible to any dual "discrimination + explanation" RL task.
• Evolution from verifier to agentic optimizer: M1-TTS shifts the verifier from a "score provider" to an "action provider"—producing schedulable region signals. This represents a new paradigm for integrating RL-trained judge models into agent loops, which is valuable for controllable generation and safety alignment.

Limitations & Future Work

• Experiments were validated on OmniVerifier-7B and Qwen3-VL-8B; whether the joint vs. decoupled gap narrows on larger models (30B+) where \(p_{acc}\) approaches 1 remains to be discussed.
• Bboxes are suitable for spatial "where" errors but do not naturally apply to non-spatial errors like "style mismatch" or "inconsistent lighting." Future work needs more symbolic forms (masks, histograms).
• M1-TTS relies on unified multimodal models for region-level edits; for text-to-image diffusion pipelines, region editing still requires extra inpainting modules.
• Training was limited to 80 steps and evaluated on a single benchmark. Long-term scaling might still lead to reward degradation or adaptive hacking by generators.

Related Work & Insights

• vs OmniVerifier (Zhang et al. 2025): Ours is a direct upgrade—replacing "binary only" with "bbox + decoupled meta," and pushing the verifier into an agentic loop.
• vs DeepSeekMath-V2 / Wang et al. 2026: While they introduce meta-verification using textual rationales in language/math domains, this paper proves symbolic is superior for the visual domain.
• vs RewardDance / UnifiedReward: Traditional reward models output scalars; Ours outputs "judgment + localization + explanation," significantly improving supervisability.
• vs ReflectionFlow / OmniVerifier-TTS: These perform global multi-turn edits, whereas Ours uses region-level feedback, increasing precision for complex compositional generation.
• Transferable Insight: Any "judgment + explanation" RL task can benefit from decoupled training. Any domain allowing structured output (OCR, UI, Layout) should consider symbolic rationales over textual ones for efficiency and robustness against reward hacking.

Related Papers

• [NeurIPS 2025] DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning
• [CVPR 2026] Mitigating Memorization in Text-to-Image Diffusion via Region-Aware Prompt Augmentation and Multimodal Copy Detection
• [ICCV 2025] LMM-Det: Make Large Multimodal Models Excel in Object Detection
• [CVPR 2026] Bidirectional Multimodal Prompt Learning with Scale-Aware Training for Few-Shot Multi-Class Anomaly Detection
• [ICML 2026] Smoothing Slot Attention Iterations and Recurrences