RAISE: Requirement-Adaptive Evolutionary Refinement for Training-Free Text-to-Image Alignment

Conference: CVPR 2026 arXiv: 2603.00483 Code: https://github.com/LiyaoJiang1998/RAISE Area: Image Generation Keywords: Inference-time compute scaling, text-to-image alignment, evolutionary optimization, requirement-driven, multi-agent

TL;DR

This paper proposes RAISE, a framework that models T2I generation as a requirement-driven adaptive evolutionary process. A requirement analyzer decomposes prompts into structured checklists; multi-action mutations (prompt rewriting + noise resampling + instruction-based editing) evolve candidate populations in parallel; tool-augmented visual verification eliminates non-compliant candidates each round. The result is adaptive inference-time scaling that achieves 0.94 SOTA on GenEval while reducing generated samples by 30–40% and VLM calls by 80% compared to reflection fine-tuning baselines.

Background & Motivation

Background: Although T2I diffusion models can generate photorealistic images, their faithfulness to complex prompts (multi-object, spatial relations, attribute binding) remains insufficient. Inference-time scaling—allocating additional computation at inference to improve alignment—has emerged as a promising direction, including noise-level scaling (e.g., searching for optimal initial noise) and prompt-level scaling (rewriting prompts with VLMs).

Limitations of Prior Work: - Training-free methods (TIR, T2I-Copilot): rely on fixed iteration budgets or thresholds, failing to adapt to prompt difficulty; multi-round refinement often stagnates or degrades; T2I-Copilot selects only a single action per round, limiting exploration. - Training-based methods (Reflect-DiT, ReflectionFlow): require large-scale reflection datasets and joint fine-tuning of diffusion models and VLMs, incurring high cost, overfitting to reflection trajectories, and poor transferability to new base models. - All existing methods lack the ability to analyze which specific requirements in a prompt remain unmet.

Key Challenge: Existing approaches either allocate computation uniformly (wasteful for simple prompts, insufficient for complex ones) or depend on training (model-bound and costly), and none use requirement satisfaction as a signal to drive compute allocation.

Key Insight: T2I generation is analogized to the software engineering workflow of "requirements analysis → implementation → verification"—decomposing user prompts into a verifiable requirement checklist, identifying unmet items each round, targeting computation accordingly, and stopping once all requirements are satisfied.

Core Idea: A requirement-driven adaptive evolutionary framework in which multiple agents (analyzer, rewriter, verifier) collaborate, multi-action mutations generate candidate populations in parallel, tool-augmented structured verification provides fine-grained feedback, and compute scales adaptively to semantic complexity.

Method

Overall Architecture

RAISE is a three-agent system sharing a single VLM backbone (Mistral-Small-3.2): - Analyzer: Parses the user prompt and generates a requirement checklist \(\mathcal{R}_i\) (comprising satisfied requirements \(\mathcal{R}_i^+\) and unsatisfied requirements \(\mathcal{R}_i^-\)), corresponding binary verification questions \(Q_i\), and a continuation decision \(d_i^{analyzer}\). - Rewriter: Generates improved prompts or editing instructions based on \(\mathcal{R}_i^-\). - Verifier: Extracts evidence using visual tools and answers each verification question per candidate, outputting satisfaction status.

Key Designs

1. Requirement-Driven Adaptive Scaling

   • Function: Dynamically determines whether to continue iteration based on the degree of requirement satisfaction.
   • Mechanism: At the start of each round, the analyzer receives the user prompt, the best candidate image from the previous round, and its verification feedback, then outputs an updated requirement checklist. Iteration terminates when any of the following conditions is met: (a) the analyzer judges that the primary requirements are satisfied; (b) the verifier confirms all requirements are met; or (c) the maximum number of rounds \(K_{max}=4\) is reached.
   • Design Motivation: Simple prompts converge within 1–2 rounds, while complex prompts automatically receive a larger compute budget, avoiding the waste or insufficiency of fixed budgets.

2. Multi-Action Mutation Refinement

   • Function: Concurrently executes multiple complementary improvement strategies each round.
   • Three mutation actions:
     • (1) Resampling: Retains the original prompt \(x_{user}\) and replaces only the random noise \(\epsilon \sim \mathcal{N}(0,I)\), exploring diverse spatial layouts.
     • (2) Prompt rewriting: The rewriter modifies prompt semantics based on \(\mathcal{R}_i^-\), paired with multiple new noise samples to generate candidates.
     • (3) Instruction editing: Starting from the best image of the previous round, three editing instructions are generated—top edit (most important unmet requirement), random edit (a randomly selected unmet requirement), and comprehensive edit (all unmet requirements)—executed via Flux Kontext.
   • Strategy scheduling: Early rounds (\(i \leq K_{min}\)) apply generative mutations (resampling + rewriting) for broad exploration; later rounds (\(i > K_{min}\)) apply rewriting + editing for targeted refinement.
   • Each round produces a fixed pool of \(n_i = 8\) candidates.

3. Structured Tool-Augmented Verification

   • Function: Bridges the gap between visual perception and textual reasoning.
   • Mechanism: Visual tools (Grounded SAM 2 + Florence-2 for detection/captioning, MiDaS for depth estimation) extract structured evidence \(G_{i,j} = (caption, \{(label_k, bbox_k, depth_k)\}, image\_size)\) from each candidate image. The verifier feeds evidence together with verification questions into the VLM, producing a triplet (question, yes/no answer, explanation) for each requirement.
   • Fitness scoring: NVILA-Lite-2B-Verifier computes alignment scores between the user prompt and each image to select the globally optimal candidate.
   • Design Motivation: Direct VLM-based visual judgment is prone to hallucination; object detection and depth information provided by tools supply reliable anchors for reasoning.

Implementation Details

• Generator: FLUX.1-dev (28-step diffusion); Editor: FLUX.1-Kontext-dev
• VLM backbone: Mistral-Small-3.2-24B, orchestrated by LangGraph with Ollama local inference
• Fitness function: NVILA-Lite-2B-Verifier
• \(K_{max}=4\), \(K_{min}=2\)

Key Experimental Results

Main Results (GenEval)

Method	Type	Samples	VLM Calls	Overall	Two Obj	Counting	Colors	Position	Attr Bind
FLUX.1-dev	Baseline	1	0	0.67	0.81	0.75	0.80	0.21	0.48
ReflectionFlow	Train	32	64	0.91	0.98	0.89	0.95	0.89	0.75
Qwen-Image-RL	UMM	1	1	0.91	0.95	0.93	0.92	0.87	0.83
T2I-Copilot	Free	11.3	22.6	0.74	0.91	0.68	0.86	0.55	0.46
RAISE	Free	18.6	7.3	0.94	1.00	0.95	0.98	0.83	0.87

DrawBench Comparison

Method	Samples	VLM Calls	VQAScore↑	ImageReward↑	HPSv2↑
FLUX.1-dev	1	0	0.778	1.06	0.298
ReflectionFlow (32)	32	64	0.844	1.10	0.302
T2I-Copilot	11.2	22.3	0.820	0.94	0.298
RAISE (≤4 rounds)	21.2	8.6	0.885	1.15	0.305

Key Findings

• GenEval overall score of 0.94 surpasses all methods, including unified multimodal models requiring large-scale pretraining such as Qwen-Image-RL (0.91) and GPT Image 1 (0.84).
• Significant efficiency advantage: reduces samples by 41.9% (18.6 vs. 32) and VLM calls by 88.6% (7.3 vs. 64) compared to ReflectionFlow.
• Two Object and Colors categories reach 100% and 98%, demonstrating the strong alignment guarantees provided by requirement-based verification.
• Adaptive behavior: GenEval averages 18.6 samples vs. DrawBench's 21.2—more reasoning-intensive prompts automatically receive greater compute.
• The large gap versus T2I-Copilot (0.94 vs. 0.74) demonstrates the combined advantage of multi-action mutation and structured verification.
• Continuous Pareto frontier improvement: RAISE continues to improve as the sample budget increases, whereas baseline methods plateau rapidly.

Highlights & Insights

• Requirement-analysis-driven adaptive compute allocation is the core innovation—elevating prompt understanding from "holistic scoring" to "itemized checklists," making feedback actionable.
• Concurrent multi-action mutation greatly expands the search space—resampling explores layout, rewriting corrects semantics, and editing refines details; the three dimensions are complementary and executed in parallel.
• Tool-augmented verification addresses hallucination in direct VLM-based visual judgment by anchoring reasoning with "hard evidence" from detection and depth tools.
• Generality of the evolutionary framework: the framework is not tied to a specific generative model; FLUX can be replaced by any T2I model, demonstrating broad applicability.

Limitations & Future Work

• The upper bound of 8 candidates per round × 4 rounds = 32 images may still be insufficient for extremely complex prompts.
• The framework relies on the reasoning capability of the VLM (Mistral-Small-3.2); reasoning errors propagate to subsequent rounds.
• Constraints beyond text descriptions (e.g., sketch guidance, reference image style) are currently unsupported.
• Instruction editing depends on the capability of Flux Kontext and may be insufficient for large-scale modifications (e.g., completely changing composition).
• Computational overhead remains relatively high (~20 images + ~8 VLM calls on average), making real-time deployment challenging.
• A fair efficiency comparison with the latest unified multimodal models (e.g., Qwen-Image-RL at 0.91) has not been conducted.

Related Work & Insights

• vs. T2I-Copilot: Both are training-free; T2I-Copilot uses single-action-per-round and fixed-threshold stopping (0.74), whereas RAISE uses concurrent multi-action mutation and requirement-adaptive stopping (0.94)—a substantial gap.
• vs. ReflectionFlow: Requires constructing million-scale reflection datasets and joint fine-tuning; RAISE requires no training, achieves superior performance (0.94 vs. 0.91), and is several times more efficient.
• vs. Noise Scaling: The ceiling of pure noise search (0.85) is surpassed by RAISE's semantic-level refinement.

Rating

• Novelty: ⭐⭐⭐⭐⭐ — Modeling T2I alignment as a requirement-driven evolutionary process is highly novel; the system design integrating multi-agent, multi-action mutation, and structured verification is comprehensive.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ — Dual benchmarks (GenEval + DrawBench), efficiency analysis, Pareto frontier, and complete ablation studies.
• Writing Quality: ⭐⭐⭐⭐ — Framework diagrams are clear and formalization is rigorous, though notation is dense.
• Value: ⭐⭐⭐⭐⭐ — SOTA results + training-free + model-agnostic design confer high practical value for inference-time T2I optimization.

Related Papers

• [CVPR 2026] TAP: A Token-Adaptive Predictor Framework for Training-Free Diffusion Acceleration
• [CVPR 2026] WISER: Wider Search, Deeper Thinking, and Adaptive Fusion for Training-Free Zero-Shot Composed Image Retrieval
• [AAAI 2026] Infinite-Story: A Training-Free Consistent Text-to-Image Generation
• [CVPR 2026] TINA: Text-Free Inversion Attack for Unlearned Text-to-Image Diffusion Models
• [CVPR 2026] PixelRush: Ultra-Fast, Training-Free High-Resolution Image Generation via One-step Diffusion