GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning¶
Conference: ECCV2026
Paper: ECCV Paper
Area: Image Generation
Keywords: text-to-image, multimodal reasoning, tool invocation, iterative reflection, reinforcement learning
TL;DR¶
GenAgent treats the generator as a frozen tool and trains one multimodal model through cold-start training and agentic reinforcement learning to rewrite prompts, inspect images, and reflect, raising FLUX's GenEval++ score from 0.325 to 0.561 with at most two generation rounds in the main experiments.
Background & Motivation¶
Text-to-image models can produce visually convincing images, but visual plausibility does not guarantee satisfaction of every user constraint. When a prompt specifies object counts, relative positions, colors, and commonsense relationships together, a generator may satisfy several requirements while silently missing another. Stronger language understanding can turn implicit requirements into drawable descriptions, but a single rewrite never observes the generated image and cannot identify which constraint the generator failed to satisfy. Integrating understanding and generation into a unified model is one option, but it requires expensive interleaved image-text training data and couples improvements in generation with the maintenance of understanding capabilities.
Decoupled systems preserve existing generators and use multimodal models to analyze their outputs; however, earlier systems often connect multiple roles through handcrafted prompts or rely on many candidate images and separate verifiers. This increases deployment complexity without directly solving the learning problem of changing the next generation attempt in response to an observed error. GenAgent therefore does not introduce another diffusion backbone: it trains a controller that coordinates understanding, judgment, reflection, and stopping decisions. These behaviors must develop together: unreliable tool formatting prevents useful feedback, while final success alone reveals little about whether intermediate reflection helped.
The paper's diagnostic results show why these skills cannot simply be assumed: Qwen2.5-VL-7B has a tool invocation error rate of 13.36%, 49.28% of rewrites differ from the original prompt in word count by no more than 5, and reflection contributes only a 0.36% improvement (Table 1). Word-count difference is only a coarse proxy for rewriting magnitude, not semantic quality; the central issue is insufficient experience executing reliable interaction trajectories. The authors consequently teach basic behavior using high-quality trajectories before optimizing long-horizon decisions through actual generation feedback. Core Idea: freeze the image generator, concentrate learning in a single multimodal agent, and jointly supervise final-image success, consistent improvement through reflection, and balanced training across interaction lengths.
Method¶
Overall Architecture¶
The input is the user's original text request, and the output is the final image in an interaction trajectory rather than merely an optimized prompt. At deployment, one Qwen2.5-VL-7B policy handles reasoning, tool invocation, image judgment, and reflection; the generation model turns the tool-call prompt into an image. Training begins with hint-guided cold-start training, followed by one reinforcement learning stage combining hybrid rewards and round-aware resampling to obtain a policy that controls its own interactions. The two RL designs determine which trajectories deserve rewards and which trajectories enter an update, respectively; they are not additional inference-time roles.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}}%%
flowchart TD
DATA["Teacher trajectories<br/>and hints"] --> SFT["Hint-Guided<br/>Cold-Start Training"]
subgraph RL["Reinforcement Learning Training"]
REWARD["Hybrid Rewards"] --> SAMPLE["Round-Aware Resampling"]
end
SFT --> REWARD
SAMPLE -.->|Policy update| AGENT["Single-Model<br/>Generation Loop"]
QUERY["User requirements"] --> AGENT
AGENT -->|Rewritten prompt| TOOL["Frozen generation tool"]
TOOL -->|Returned image| AGENT
AGENT -->|Satisfied or round limit reached| OUT["Final image"]
The intermediate state contains the original request, reasoning text, tool prompts, returned images, and judgments from previous rounds, allowing subsequent rewrites to address observed failures rather than start from scratch. The external reward model supplies training supervision and is not a verifier that must remain in the deployed system. The dashed edge denotes parameter updates, while the solid feedback loop denotes inference-time data flow; reward evaluation should not be mistaken for a mandatory third deployed model. The default generator is the 8-step distilled version of FLUX.1-dev, and subsequent references to FLUX.1-dev in the paper use this version.
Key Designs¶
1. Hint-Guided Cold-Start Training: teach invocation, image inspection, and grounded rewriting first
The authors construct a prompt pool from open-source data and synthetic requests requiring complex reasoning, generate images 3 times per candidate with FLUX.1-dev, and retain only hard prompts for which every attempt fails. This focuses training on combinations that direct generation cannot easily satisfy rather than repeatedly solving simple requests that already succeed immediately. Qwen3-VL-235B-A22B-Thinking supplies first-round reasoning and rewritten prompts; after filtering incorrect tool formats, the generator produces actual images. The judging model then receives explicit evaluation rules and explains whether the images satisfy the requirements; these rules assist trajectory synthesis and are not explicitly supplied at inference time. Successful trajectories can be retained, whereas failed trajectories proceed to second-round reflection synthesis.
For the second round, Gemini 2.5 Pro serves as a stronger teacher, and reference images associated with the seed examples help it identify effective rewriting directions. These reference images are distillation-time hints, not required user inputs at test time; generated prompts that explicitly depend on a hint image are filtered out. The authors compare consecutive images to remove regressions, then retain high-quality examples through final quality checks and trajectory-consistency filtering. Balanced sampling by data type and whether the second round terminates yields 32K cold-start examples. Deliberately avoiding termination in every example prevents a fixed one-reflection limit from becoming entrenched and leaves room for RL to explore longer trajectories.
2. Hybrid Rewards: final success and improvement along the way require different supervision
A final-image success reward alone cannot distinguish useful correction from a lucky sample and gives insufficient feedback to trajectories that improve without fully succeeding. GenAgent uses Qwen3-VL-30B-A3B as a generative reward model, first analyzing the image against evaluation criteria and condition-specific hints, then checking whether all requirements in the original request are met. The pointwise reward \(r_{\mathrm{point}}\) is 0 or 0.7: the agent receives 0.7 only if every condition is satisfied, rather than accumulating a separate score for each correct attribute. Thus, quality here primarily means prompt-constraint satisfaction and should not simply be equated with visual aesthetics. The format reward \(r_{\mathrm{format}}\) is 0 or -0.2 and constrains the executable structure of the trajectory.
The pairwise reward \(r_{\mathrm{pair}}\) checks whether consecutive images consistently improve; a multi-round trajectory receives 0.3 only when every subsequent image is better than its predecessor, and otherwise receives 0. Image presentation order is randomized during comparison to reduce the judge's positional bias. The pairwise weight \(\lambda\) is 0.5 when the final image still fails and 1 when it succeeds, distinguishing partial improvement from improvement that ultimately completes the task. This creates an interpretable training preference: encourage genuine correction without allowing local progress to fully replace final instruction compliance. Equations (4), (6), and (7) contain extraction damage in the supplied text, particularly missing operators in the total reward expression; this note retains only reward values and conditions confirmed by the prose rather than guessing the exact combined formula.
3. Round-Aware Resampling: prevent rewards from pushing the policy toward excessive reflection
Trajectory length represents more than compute consumption here: single-round trajectories mainly exercise comprehension and initial rewriting, while multi-round trajectories also exercise visual judgment and correction. Using naturally sampled trajectories without adjustment can favor early stopping or encourage reliance on later reflection for pairwise rewards at the expense of the initial prompt. For each request, the authors first oversample \(G'=12\) trajectories, group them by tool invocation count, and uniformly sample across round-count groups to obtain the final \(G=8\) trajectories for GRPO updates. This changes the composition of the training batch rather than forcing every test request to use the same number of rounds. Hybrid rewards still evaluate trajectory quality, whereas round-aware resampling preserves opportunities to learn different skills; they provide complementary controls within the same RL process.
The ablation gives a concrete explanation: without resampling, the reflection ratio rises to 67% while first-round rewriting performance is only 39.3%; the full model reaches 46% and 48.2%, respectively (Figure 4 and Section 4.4). The former has a larger reflection gain but a lower final score, showing that more reflection is not itself a quality guarantee. A useful interpretation is management of the training distribution: optimizing corrected outputs should not displace the ability to produce a strong initial image. This interpretation concerns the paper's trajectory sampling mechanism and does not imply a separate round-count predictor or a new generation network.
4. Single-Model Generation Loop: let actual outputs determine whether to continue or stop
In the first round, the policy interprets the user request, produces reasoning text and a tool-formatted prompt, and invokes the frozen generator to obtain an image. The policy then inspects the request, prior prompt, and image together: it terminates if the requirements are satisfied, or identifies deficiencies and produces another rewritten prompt. Subsequent rounds carry the interleaved image-text history until the policy is satisfied or reaches the maximum round count \(n_{\max}\), after which the last generated image is returned. This is not external reranking of all candidate images; the stopping decision is itself part of the learned policy. Training permits up to 3 rounds, whereas the main results use at most 2 rounds to match multi-round baselines; actual interaction counts can be lower through early stopping.
The paper's editing-like behavior means targeted prompt revision followed by image regeneration, not an explicit local image-editing interface. Replacing the generator with Qwen-Image does not require retraining the agent, suggesting that understanding and correction policies can transfer beyond the training generator to some extent. Nevertheless, transfer remains limited by the new tool's prompt understanding and generation ceiling and does not establish universal generalization across arbitrary tools or tasks. The single model unifies control and judgment responsibilities, not the image generator's parameters.
A Worked Example¶
Figure 1 requests a photograph containing 1 wine glass, 2 television remotes, and 3 suitcases, illustrating why observing outputs matters more than merely lengthening a prompt. The first rewrite emphasizes composition and photographic detail, but the returned image contains only 1 remote and 2 suitcases, missing explicit count requirements. The policy's judgment locates the failure in missing objects instead of describing image quality as generally poor. The next rewrite distinguishes objects more concretely, specifies suitcase sizes and colors, and removes irrelevant photographic terminology. When the returned image meets the counts, the policy emits a completion marker and returns that image; this is the successful trajectory illustrated in the paper, not a guarantee of correcting every counting error.
Loss & Training¶
SFT uses LLaMAFactory and AdamW for 3 epochs, with batch size 256 and learning rate \(1\times10^{-5}\). The SFT loss covers only model responses and masks environment feedback so that generator observations are not treated as outputs to imitate. RL uses GRPO implemented in verl for 1 epoch, with batch size 240, learning rate \(1\times10^{-6}\), and KL coefficient 0.0. GRPO computes advantages by normalizing rewards with the mean and standard deviation within each request's trajectory group, then updates policy-generated response tokens; image observations do not contribute to the response loss. The maximum response length is 16,384 tokens, and generation and reward calculation run as independent services during training. The authors use 40 NVIDIA A100 GPUs, allocating 24, 8, and 8 to the policy, generator, and reward model; inference requires 2 GPUs for the policy and generator, respectively. These resource counts do not establish measured per-image latency; detailed inference-time analysis is deferred to supplementary material not supplied for this note.
Key Experimental Results¶
Main Results¶
The table below selects Overall scores from Table 2 on page 9 and Tables 3 and 4 on page 10; all are higher-is-better, but the benchmark scales differ and should not be compared directly across columns. GenEval++ evaluates compositional instruction following, WISE evaluates world knowledge and semantic reasoning, and Imagine evaluates surreal creative generation while preserving object identity. Multi-round main experiments use \(n_{\max}=2\); the default generator is the 8-step distilled FLUX version except in the Qwen-Image rows.
| Method / Generation Tool | GenEval++ Overall | WISE Overall | Imagine Overall |
|---|---|---|---|
| FLUX.1-dev direct generation | 0.325 | 0.55 | 6.072 |
| PromptEnhancer / FLUX | 0.382 | 0.56 | 6.281 |
| T2I-Copilot / FLUX | 0.496 | 0.67 | 6.740 |
| GenAgent single rewrite only / FLUX | 0.482 | 0.67 | 6.408 |
| GenAgent +RL / FLUX | 0.561 | 0.69 | 6.825 |
| Qwen-Image direct generation | 0.668 | 0.62 | 7.329 |
| GenAgent / Qwen-Image | 0.725 | 0.72 | 7.794 |
| GPT4o | 0.739 | 0.80 | 8.560 |
Relative to direct FLUX generation, GenAgent's absolute gains on GenEval++ and WISE are 0.236 and 0.14, respectively. The abstract expresses these as 23.6% and 14.0%, which should be read as percentage-point-style score increases rather than relative growth rates; Section 4.3 explicitly reports +0.236 and +0.14. Tool transfer raises Qwen-Image's GenEval++ score from 0.668 to 0.725, still below GPT4o's 0.739, so it does not support a claim of universally outperforming closed-source models.
Ablation Study¶
Table 5 on page 11 compares training stages and RL components using the same default generation tool; Base denotes the agent before the paper's SFT/RL, not direct FLUX generation.
| Config | WISE | GenEval++ | Imagine |
|---|---|---|---|
| Base | 0.63 | 0.357 | 6.258 |
| SFT | 0.64 | 0.507 | 6.770 |
| RL without pairwise reward | 0.68 | 0.543 | 6.814 |
| RL without round-aware resampling | 0.68 | 0.546 | 6.821 |
| Full RL | 0.69 | 0.561 | 6.825 |
SFT improves GenEval++ from 0.357 to 0.507 but WISE only from 0.63 to 0.64; cold-start training more clearly improves executable behavior and handling of compositional requirements than it automatically supplies all missing knowledge reasoning. Removing pairwise rewards and removing resampling lower GenEval++ by 0.018 and 0.015 relative to the full model, respectively; both contribute, but these differences are not independently additive causal shares. Table 1 on page 6 also shows that SFT reduces tool invocation errors from 13.36% to 0.35%, providing a more reliable interaction starting point for RL.
Key Findings¶
Table 6 on page 13 fixes the image generation budget: BoN generates candidates from the initial rewritten prompt and uses the training reward model for selection, whereas GenAgent changes the next prompt using feedback.
| Method | Image Budget | GenEval++ (%) |
|---|---|---|
| BoN, N = 1 | 1 image | 48.20 |
| BoN, N = 2 | 2 images | 51.07 |
| GenAgent, 2 rounds | 2 images | 56.10 |
At a budget of 2 images, GenAgent exceeds BoN by 5.03 percentage points, supporting the value of rewriting from failure information rather than resampling under an unchanged prompt. This matches image counts, not necessarily total GPU time, reasoning tokens, or verifier-call costs. Section 4.4 reports negligible marginal gains by the third round and over-reflection under ambiguous prompts, so test-time scaling does not imply that additional rounds always provide substantial benefits.
Highlights & Insights¶
- Decision-making responsibilities are unified, not every model parameter. A single policy handles judgment and correction while external generation tools remain upgradeable, providing a more flexible improvement path.
- Reflection can be evaluated through image changes rather than the plausibility of its verbal explanation. Pairwise rewards tie supervision to generated outcomes, though their reliability still depends on the judge's visual accuracy.
- The distribution of interaction lengths is also a distribution of skills. Figure 4's combination of more reflection and weaker first-round rewriting cautions against substituting trajectory length for final-performance analysis.
Limitations & Future Work¶
- The authors acknowledge that generative reward models remain vulnerable to reward hacking and propose finer-grained, verifiable attribute checklists; improved BoN selection alone does not prove that the reward model is fully reliable.
- The tool set currently covers image generation only, so local corrections still require rewritten prompts and regeneration; explicit editing tools are a proposed extension, not an existing capability.
- Generator limits on fine-grained attributes and ambiguous prompts constrain reflection gains, and the final round can be worse than an earlier one; this matters because the system returns the final image.
- The main tables do not report confidence intervals, and end-to-end latency fairness cannot be determined from the main text alone; this note does not treat small score differences as statistically significant.
- The available source is extracted main-paper text without supplementary material, and some equations and figure text are corrupted; this note neither reconstructs exact damaged equations nor estimates additional values from unreadable plots.
Related Work & Insights¶
- Versus PromptEnhancer: a single rewrite uses input semantics, whereas GenAgent also exploits errors revealed by generation; single-pass rewriting may remain a simpler option under tight latency constraints.
- Versus T2I-Copilot and ReflectionFlow: the former relies on a handcrafted multi-agent workflow, and the latter involves separate functional models and candidate sampling; GenAgent concentrates control in one trainable policy at the cost of dedicated trajectory distillation and RL.
- Versus Bagel and T2I-R1: unified architectures integrate understanding and generation inside the model, while GenAgent decouples them at the system level; tool-transfer results support flexibility but do not prove superiority over unified models in every setting.
- Research direction: constraint checklists could help distinguish generator limitations from inadequate prompt expression before deciding to regenerate, request clarification, or invoke an editing tool; this is a reader-proposed extension, not a mechanism validated in the paper.
Rating¶
- Novelty: 4/5. The contribution is the combination of a single-policy generation loop, hybrid rewards, and round-aware resampling rather than a new generation backbone.
- Experimental Thoroughness: 4/5. The study covers 3 task types, component ablations, tool transfer, and image-budget comparisons, but statistical and complete cost evidence remains limited.
- Writing Quality: 4/5. The problem and ablation logic are clear, but abstract percentages, absolute score gains, and editing-like behavior require careful distinction.
- Value: 4/5. The framework offers a practical direction for adding learned feedback control to existing generators, subject to training cost and reward robustness.