Flash-BoN: Instant Drafts for Inference-Time Scaling in Diffusion Models¶

Conference: ECCV2026
Paper: ECCV Paper
Project: https://flash-bon.github.io
Area: Image Generation
Keywords: inference-time scaling, diffusion models, inexpensive drafts, multi-stage verification, candidate diversity

TL;DR¶

Flash-BoN spends generation compute on many inexpensive drafts, selects one through multi-stage comparisons, and resumes it with full computation, outperforming full-generation Best-of-N under wall-clock evaluation, with Wan 14B improving from 0.60 to 0.68 normalized AUC on GenAI-Bench.

Background & Motivation¶

Inference-time scaling for text-to-image generation does not necessarily require changing model weights: trying different noise seeds for the same prompt and selecting the most suitable image can improve results. The simplest Best-of-N (BoN) approach fully generates every candidate before scoring them. Trajectory searches such as BFS and DFS instead inspect candidates repeatedly during denoising to decide when to branch, prune, or backtrack, moving more decisions into the generation process. The difficulty is that verification with a vision-language model (VLM) is not free and can cost more than a single denoising forward pass. Counting only denoising network function evaluations (NFEs) can therefore favor methods with frequent verification without establishing that they are faster.

The paper instead constrains end-to-end wall-clock time, including generation, verification, subsequent refinement, and data transfer. With this accounting, BoN already matches or exceeds several guided searches, suggesting that time spent on intermediate judgments could instead fund exploration of different seeds. However, standard BoN still pays full rendering costs for candidates that will eventually be discarded. If the immediate question is whether composition and semantics look promising, every image need not reach final detail quality before selection. The central difficulty becomes deciding how much computation can be removed while preserving reliable draft selection and meaningful preferences after refinement.

Flash-BoN combines timestep truncation, layer skipping, and activation proxies, determining a configuration once per model rather than searching again for each prompt. It also avoids treating the highest absolute score as a reliable answer, since integer scoring creates many ties; relative comparisons are concentrated on progressively smaller candidate sets. Core Idea: split full image generation into inexpensive drafts suitable for selection and full-compute continuation for the winner alone, allowing a limited runtime to cover more of the noise space.

Method¶

Overall Architecture¶

Inputs are a text prompt, a pretrained generator, and a wall-clock budget; the output is an image produced through candidate selection and refinement. An offline Joint Draft Configuration stage first identifies a fast configuration subject to sufficient perceptual similarity; online generation then reuses it to produce drafts from different seeds. Multi-Stage Elo Selection follows, starting with independent scores and progressively adding relative comparisons, before Selective Continuation completes the winner. A draft is not merely a preview shown to the verifier: it also corresponds to a resumable denoising latent, so discarded candidates never need their remaining computation. The verifier directs the search, while an independent evaluator reports results; these roles should not be conflated.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    INPUT["Pretrained model and<br/>calibration prompts"] --> CONFIG["Joint Draft Configuration"]
    CONFIG -->|"Reuse configuration online"| DRAFT["Prompt and different seeds<br/>generate drafts and cache states"]
    DRAFT --> SELECT["Multi-Stage Elo Selection"]
    SELECT --> REFINE["Selective Continuation"]
    REFINE --> OUTPUT["Final image<br/>sent to independent evaluator"]

Key Designs¶

1. Joint Draft Configuration: distribute acceleration across three approximations

Timestep truncation executes only the initial portion of the denoising trajectory, leaving an unfinished state whose inexpensive preview supports candidate assessment. Layer skipping reduces work within each timestep: when skipping is enabled, the preceding activation passes onward without executing a designated interval of network blocks. This neither deletes model parameters nor learns a new network dynamically for each input; a fixed configuration controls the execution path. Activation proxies exploit a different redundancy: the features of the same layer often change smoothly across nearby denoising timesteps. The paper uses second-order Taylor forecasting from cached features to predict the current layer output instead of performing some actual forward computations. The configuration also controls the frequency of full computation, avoiding indefinite reliance on approximations; the temporal derivative expressions are damaged in the text extraction, so incomplete expressions are not reconstructed as exact formulas here. These techniques reduce trajectory length, network depth, and repeated computation across timesteps, respectively, rather than acting as interchangeable versions of simply sampling fewer steps.

The joint configuration specifies draft step count, the number of initial steps before skipping begins, the start and end of a contiguous skipped-layer interval, and full-computation frequency. Overusing any single acceleration technique can destroy structure and prevent the verifier from distinguishing genuinely promising candidates. On 120 held-out calibration prompts, the authors run dual annealing for 200 iterations, considering wall-clock speedup and LPIPS perceptual differences from full-compute outputs. They select the fastest configuration satisfying a minimum similarity requirement from the Pareto frontier, then reuse it across prompts and benchmarks. LPIPS calibrates draft fidelity here; it is not the online semantic reward and does not directly prove that draft rankings remain unchanged. The main text does not give the final per-model configurations or similarity threshold; the exact objective and optimizer comparisons are deferred to Appendix C, which is absent from the cache. The search mechanism can therefore be explained, but its optimal configurations cannot be fully reproduced from this cache alone.

2. Multi-Stage Elo Selection: prune cheaply before resolving ties among strong candidates

Each draft first receives an independent integer score from 1 to 10, a parallelizable operation with limited resolution near the top of the range. Page 8 reports tied highest scores for over 80% of prompts, with approximately 30% having at least 20 candidates sharing the highest score. The method converts scores into initial Elo ratings, preserving their coarse prior information, and retains a top-ranked fraction of candidates. For draft index \(j\) and pointwise score \(s_j\), the paper initializes ratings as follows:

\[ r_j^{(0)} = 1500 + (s_j - 5)\cdot 100. \]

Candidates are then sorted by current Elo and compared sparsely with adjacent entries, focusing limited calls on nearby ranks. Each outcome updates both ratings; high, medium, and low verifier confidence multiply the Elo K-factor by 1.2, 1.0, and 0.7, respectively. After another retention step, the method runs a complete pairwise tournament within the small surviving set and chooses the highest-rated draft. Both default retention ratios are 0.5, and the sparse stage averages 1.5 comparisons per survivor. This reduces dense comparisons over the large original pool rather than eliminating comparisons; strong drafts mistakenly removed in the first stage cannot be recovered later. The approximate total-call expression is unclear in the extraction, and exhaustive comparisons after fixed-ratio retention still contain a quadratic term, so it does not establish strictly linear complexity. The online verifier can be a general VLM or reward model, but quality should be reported with a different evaluator to avoid merely selecting that verifier's preferred images.

3. Selective Continuation: resume the winning state instead of redrawing every candidate

Each draft stores its latent at truncation and its random seed; the main text also mentions optional layer-activation caches. During draft generation, cached tensors can be offloaded to the CPU to limit the candidate pool's GPU memory footprint. After selection, only the winning state returns to the GPU, where the remaining denoising steps run with layer skipping and activation proxies disabled. This differs from using the draft image to launch a fresh text-to-image generation, because continuation inherits the candidate's intermediate state. Discarded candidates consume no remaining denoising budget, allowing the saved compute to support exploration of additional independent seeds. Offloading and restoration must also count toward runtime, so the benefit is not inferred solely from theoretical forward-pass counts.

The authors invoke the Markov property of denoising to justify continuation from the current latent: future evolution depends on the current state rather than requiring the entire history. However, this establishes that continuation is possible, not that an approximate trajectory becomes identical to a full-compute trajectory with the same seed. Earlier approximation errors are already embedded in the latent, and the remaining full computation may not repair incorrect composition or missing entities. Draft configuration therefore needs a fidelity constraint, and the verifier must be able to see structure that matters for the final task. Refinement concentrates resources on the selected candidate rather than offering a theoretical guarantee of lossless acceleration.

A Worked Example¶

The following 16-draft example illustrates the default retention rules; it is a procedural example, not a measured prompt instance from the paper. For a prompt requiring several objects to satisfy spatial relations, generate drafts from 16 different seeds and save the latent at each truncation point. Score the 16 previews, initialize Elo from their scores, and retain the top 8. The sparse stage schedules approximately 12 adjacent-candidate comparisons under the default ratio, then keeps the top 4 by Elo. These 4 require 6 exhaustive pairwise comparisons, after which only 1 winning draft resumes full computation for the remaining denoising steps. This accounting gives approximately 34 verification calls across comparisons and pointwise scoring; it is a call count for the example, not a latency estimate. The other 15 candidates do not finish their trajectories, but their draft-generation and verification costs have already been incurred and remain part of the budget. Selection may still fail if the correct spatial relations are not visible in the drafts, illustrating the joint limits imposed by preview fidelity and verification ability.

Loss & Training¶

The main method does not train the generator or add an end-to-end loss; offline calibration searches execution configurations rather than model weights. Inference does not perform gradient optimization for each prompt: improvements arise from seed exploration and selection. The paper separately introduces Flash-Flow-GRPO to apply draft selection to reinforcement learning post-training; this is an extension, not a required training stage for the main method. For each prompt, this extension generates \(2G=16\) inexpensive drafts and retains the highest-scoring 6 and lowest-scoring 2, yielding \(G=8\) training trajectories. Full forward passes and gradient computation operate only on these selected trajectories; retaining low-scoring samples provides negative examples instead of collecting only high-reward candidates. Compared with randomly drawing a small group, this increases the chance of obtaining informative positive and negative examples for difficult prompts. Besides trajectory selection, the authors increase the KL penalty to account for the changed selection distribution, so the training comparison does not change only one factor. The main text reports approximately 10 times fewer gradient steps to match the baseline's final performance, which does not directly establish a 10-fold end-to-end training speedup. Full training hyperparameters and the KL coefficient are deferred to the missing Appendix A.4, preventing independent reproduction of this training result from the current cache.

Key Experimental Results¶

Main Results¶

The following table extracts BoN and Flash-BoN from the paper's Table 1 on page 10, retaining all 9 model-benchmark combinations; differences are calculated from the listed values. The metric is the area under the score-versus-wall-clock curve divided by total time, or normalized AUC; higher is better, and it is not single-timepoint accuracy. Experiments use NVIDIA H200 GPUs, include generation, verification, refinement, and data transfer, and warm up the system. GenAI-Bench and UniGenBench each use 300 sampled prompts, while GenEval uses the full set; evaluation includes budgets up to 300 seconds per prompt.

Benchmark	Model	BoN	Flash-BoN	Absolute AUC Gain
GenAI-Bench	Wan 1.3B	0.65	0.66	0.01
GenAI-Bench	Wan 14B	0.60	0.68	0.08
GenAI-Bench	FLUX.1-dev	0.60	0.68	0.08
GenEval	Wan 1.3B	0.43	0.45	0.02
GenEval	Wan 14B	0.42	0.49	0.07
GenEval	FLUX.1-dev	0.42	0.47	0.05
UniGenBench	Wan 1.3B	0.54	0.57	0.03
UniGenBench	Wan 14B	0.53	0.60	0.07
UniGenBench	FLUX.1-dev	0.54	0.61	0.07

The original table also includes ZOS, DFS, and BFS; Flash-BoN is highest in every column, and BoN is the strongest comparison baseline. The 0.08 difference is an absolute AUC gain, not automatically an 8% relative improvement; differences between models also cannot be explained by parameter count alone.

Ablation Study¶

The following is a strategy-combination analysis rather than a strict ablation removing the three acceleration knobs individually; sources are Figure 6 on page 13 and its discussion on page 11. Conditions are Wan 1.3B, GenAI-Bench, and wall-clock-normalized AUC, with the same metric for every row.

Config	Original AUC	AUC with Flash Drafts	Absolute AUC Gain
Reflection-Tuning	0.46	0.62	0.16
BFS	0.49	0.55	0.06

Reflection-Tuning replaces full images in its feedback loop with drafts, allowing more prompt revisions within the same budget; BFS accelerates generation between verification points and restores full computation after the final pruning decision. Their different gains show that how saved compute is reinvested also matters: local trajectory search need not benefit as much as broader exploration. The BFS baseline is 0.49 in Figure 6 but 0.50 in the corresponding column of Table 1; the cache does not explain the discrepancy, so both source values are preserved rather than silently reconciled. Per-knob ablations, random-configuration controls, ranking-strategy comparisons, and few-step distilled-model experiments are deferred to Appendix B, absent from the cache, so their numbers are not supplied here.

Key Findings¶

Larger models benefit more: GenAI-Bench gains are 0.01 for Wan 1.3B and 0.08 for both Wan 14B and FLUX.1-dev, consistent with cheap exploration becoming more useful as full candidates become expensive.
The category analysis on page 11 identifies stronger gains for spatial relations, comparisons, and part relations; narrower advantages for attributes and counting suggest that fine details are harder to judge from drafts.
Figure 7 on page 13 reports a Pearson correlation of \(r=0.75\) between candidate diversity and final quality, supporting an association rather than proving that diversity causes the improvement.
Diversity uses the Vendi Score of a DINOv2 feature-similarity matrix, defined as the exponential entropy of its normalized eigenvalues; identical images score 1, while \(n\) perfectly distinct images ideally score \(n\).
Figure 10 and page 14 report final scores of 0.699 for Flash-Flow-GRPO and 0.692 for the baseline, with the former reaching the baseline's final level at step 60; the metric is GenAI-Bench VQAScore.

Highlights & Insights¶

The acceleration objective shifts from completing the same image faster to finding a better image in the same time. Execution approximations that tolerate some error thereby become tools for expanding candidate coverage.
Multi-stage verification combines the low cost of absolute scoring with the resolution of relative comparisons. Its benefit depends on early-pruning recall, not merely the precision of the final Elo ranking.
Resumable draft states matter more than merely retaining low-quality images. Selection can save the latter portion of discarded trajectories instead of requiring all generation to restart afterward.
Separating the verifier from the evaluator is central to the experimental design. Figure 8 shows errors when one model plays both roles, reminding readers that higher scores need not imply better semantic quality.

Limitations & Future Work¶

The authors propose future text-to-video extensions, but the main experiments concern text-to-image generation; using Wan does not establish video temporal consistency.
Unstable draft details directly affect selection, especially counting and fine-grained attributes. Training comparators specifically for draft-quality inputs is a proposed direction, not an implemented component here.
Wall-clock results depend on the H200, generator, verifier, and transfer implementation; identifying shortcomings in NFE rankings does not guarantee identical gains in every deployment.
Calibration depends on the relationship between LPIPS fidelity and semantic rankability; examining when these diverge is a reader-suggested investigation, not an established finding of this paper.
The cache contains pages 1 through 19 of the main paper and references but no appendices, leaving gaps in exact configurations, per-module ablations, detailed verifier settings, and training hyperparameters.
Multi-stage filtering reduces pairwise costs, but the final stage remains exhaustive; overhead must be considered against pool size, and the damaged approximate call-count expression is not a complexity proof.

Compared with BoN: BoN completes every candidate trajectory, whereas Flash-BoN completes only the selected draft. Both explore independent seeds broadly, but the latter can enlarge the pool under the same runtime.
Compared with BFS, DFS, and ZOS: these baselines allocate compute to local trajectory exploration or noise optimization, whereas Flash-BoN prioritizes more inexpensive independent candidates. The results do not imply that every guided search is ineffective at every budget.
Compared with TaylorSeers and diffusion acceleration methods: activation forecasting, layer skipping, and reduced timesteps are not individually new inventions; the contribution is joint calibration and repurposing acceleration for candidate search rather than faithful single-sample generation.
Compared with Reflection-Tuning: reflection changes the text condition, while drafts change the cost of each image-generation cycle, making the two complementary. Semantic revision and approximate execution can therefore contribute together.
Research direction: calibration could directly penalize ranking reversals between drafts and refined outputs. This would target selection more directly than perceptual similarity alone, but it is a reader-proposed direction rather than a reported result.

Rating¶

Novelty: 4/5. The main contribution is wall-clock-aware budget allocation and a complete system combining joint draft acceleration with multi-stage selection.
Experimental Thoroughness: 4/5. It covers 3 benchmarks and 3 model settings, verification bias, and a training extension, but the missing appendices prevent checking fine-grained ablations from the current cache.
Writing Quality: 4/5. The problem and pipeline are clear; some cost expressions and cross-figure numerical differences require further checking against the appendices.
Value: 4/5. Relevant to latency-constrained image generation, with the main lesson being explicit verification-cost accounting and less full-generation waste on discarded candidates.