Training-Free Refinement of Flow Matching with Divergence-based Sampling¶

Conference: ECCV2026
Paper: ECCV Original
Project: https://yeonwoo378.github.io/official_fds
Area: Image Generation
Keywords: flow matching, velocity-field divergence, zeroth-order search, inference-time refinement, numerical solvers

TL;DR¶

Flow Divergence Sampler (FDS) uses velocity-field divergence to select a less locally ambiguous state from the current state and random neighbors before selected solver steps, requiring no retraining and reducing ImageNet JiT-L/16 + Heun FID from 2.713 to 2.496, versus 2.886 for a wall-clock-matched baseline with more steps.

Background & Motivation¶

Flow matching supervises a network with interpolation velocities between noise and data, but one intermediate location can be consistent with multiple noise-target pairs. Under squared-error training, the optimal prediction is the conditional mean of their sample-wise velocities. When those velocities point toward different modes, their mean may lie between modes rather than toward any particular target. The paper connects this local ambiguity to blur and missing details in finite-step sampling.

Methods such as HRF and VRFM modify training or architecture to model multimodal velocities explicitly. Another approach increases integration steps or uses higher-order solvers to reduce temporal discretization error. The former cannot directly reuse existing large models, while the latter still advances from the current state along the existing field. This motivates a different intervention: keep the model unchanged, but move the intermediate state at the same time before integrating onward from a locally less ambiguous location.

The difficulty is deciding which neighbor is more reliable without access to all training samples and velocities compatible with that location. The paper derives a proxy from marginal velocity-field divergence. Core Idea: estimate local velocity disagreement through the frozen model's spatial derivatives, then refine states with a small random search, spending additional inference compute on where integration starts rather than only on how many steps it takes.

Method¶

Overall Architecture¶

FDS receives the current state, time, and frozen velocity network. It evaluates local reliability through a divergence proxy, selects a better state among random perturbations and the original state, and then advances time with the existing Euler, Heun, or other solver. It wraps the base solver with state refinement, without introducing a training objective or requiring an external reward model.

Two movements must be distinguished: candidate search changes the spatial location at a fixed time, whereas the solver subsequently integrates from the selected state to the next time. The resulting trajectory includes additional state interventions; it is not merely a more accurate numerical solution of the original ODE.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Current state and time"] --> Signal["Divergence reliability proxy"]
    Signal --> Search["Local search with<br/>the original state retained"]
    Search --> Budget["Truncated refinement<br/>and solver integration"]
    Budget --> Output["Next-time state"]
    Output -->|Not finished| Input

Key Designs¶

1. Divergence reliability proxy: make unobservable velocity disagreement measurable

Let \(v_t\) denote the sample-wise interpolation velocity and \(u_t(x_t)\) the true marginal velocity. The optimal conditional flow-matching predictor is the mean sample-wise velocity given an intermediate state, so irreducible dispersion around that mean can remain even with a well-optimized network. Equation (6) measures this conditional mean-squared discrepancy, not the network's fitting error relative to the optimal velocity field.

\[ \mathcal L^*_{\mathrm{CFM}}(x_t,t) =\mathbb E\!\left[\|u_t(x_t)-v_t\|^2\mid x_t\right], \qquad \widehat\delta_t(x)=\nabla_x\cdot u_\theta(x,t). \]

Under a Gaussian prior and the paper's interpolation schedules, Theorem 1 expresses the true discrepancy through true velocity-field divergence and known time-dependent coefficients. At a fixed time, divergence can therefore compare discrepancy at nearby states. Replacing the true field with the pretrained network yields the practical proxy on the right. The claim is not that lower divergence guarantees a better image; the ideal-field relationship motivates a computable ranking signal for neighborhood search.

Divergence is the trace of the velocity field's input Jacobian. Explicitly constructing this high-dimensional matrix would be expensive, so the authors use Hutchinson's trace estimator with random probes rather than storing the full Jacobian. This still requires input derivatives: training-free does not mean ordinary forward passes only. The cache does not include Appendix D's further reliability analysis or implementation details such as the probe count, so these estimator settings cannot be filled in from the available material.

A further conceptual boundary matters: the marginal velocity of an ideal continuous flow can already transport the target distribution. Nonzero conditional velocity variance alone does not establish that its endpoint distribution is incorrect. The paper uses this quantity to improve empirical quality for learned models and finite-step sampling; proving that every low-divergence intervention preserves the target distribution would be a different result.

2. Local search with the original state retained: avoid differentiating divergence again

Directly optimizing the divergence proxy by gradient descent would require second-order derivatives because divergence already contains first-order spatial derivatives of the network. FDS instead performs zeroth-order candidate search: add small Gaussian perturbations to the current state, evaluate the original and perturbed states, and retain the one with the lowest proxy. This avoids second-order derivatives for proxy optimization, not the first-order derivatives used to evaluate divergence.

\[ x^{(0)}=x_t,\qquad x^{(m)}=x_t+\sigma_t\xi^{(m)},\quad \xi^{(m)}\sim\mathcal N(0,I),\quad m=1,\ldots,M, \qquad \widetilde x_t=x^{(m^*)},\quad m^*=\arg\min_{m\in\{0,\ldots,M\}}\widehat\delta_t(x^{(m)}). \]

The original state always remains eligible, allowing the algorithm to stay in place if a perturbation does not lower the estimated proxy. Here \(M\) counts perturbed candidates, so the comparison actually includes \(M+1\) states. The default \(M=1\) compares one random neighbor with the original state; it does not eliminate selection. With \(N\) refinement rounds, the selected state becomes the center for the next round.

This retention rule only selects a smaller currently estimated proxy. Random trace-estimation error prevents interpreting it as guaranteed monotonic improvement in true discrepancy or final FID. Perturbation magnitude also needs to decrease during generation to avoid disrupting established details late in sampling. The paper decays the perturbation scale from its maximum toward zero and compares alternative schedules in ablations.

3. Truncated refinement and solver integration: allocate the budget to useful stages

After local search, the base solver receives the refined state and advances with the existing velocity field. Because this interface is not specific to Euler's update, the authors also attach FDS to Heun, UniPC, RK4, CFG-Zero*, and the inverse-problem guidance framework TFG. Composability comes from intervening on the state rather than replacing each solver's internal numerical approximation.

The default candidate and iteration counts are \(N=M=1\); Section 6.2 gives a default maximum perturbation magnitude of \(\sigma_{\max}=0.01\). Figure 6 uses cosine decay and a truncation threshold of \(T_{\mathrm{trunc}}=0.5\), corresponding to the early-stage refinement discussed in the ablation section. Increasing candidate or iteration counts can help, but returns quickly diminish.

The source contains an inconsistency that must be retained: Section 3.2 says FDS is disabled for \(t<T_{\mathrm{trunc}}\), whereas Section 6.2 explicitly restricts refinement to \(t\leq T_{\mathrm{trunc}}\) and uses Figure 6 to support early intervention. This note interprets the ablations according to Section 6.2 rather than presenting these opposite conditions as a settled implementation specification. Reproduction requires checking the authors' code or supplementary material.

A Worked Example¶

Consider the two-dimensional checkerboard in Figure 2. An intermediate state lies near crossing interpolant paths toward different target cells. Following their average velocity with a finite-step solver may place it between valid cells. At the same time, FDS proposes a neighbor and compares its divergence with that of the original state; if the neighbor scores lower, integration continues from it, otherwise the original state is retained.

Figure 2b illustrates refinement at \(t_k=0.6\) followed by integration to \(t_{k+1}=1.0\). This mechanism illustration should not be conflated with the early-truncation configuration used in Figure 6's main ablations. It explains why spatial selection can alter the result, but a two-dimensional visualization does not replace proxy-reliability testing in high-dimensional image models.

Loss & Training¶

FDS does not update pretrained parameters, add a fine-tuning loss, or use training samples or external rewards for candidate selection. The conditional flow-matching objective motivates the proxy theoretically; the actual additions are inference-time divergence evaluation and candidate search.

Section 5.2 reports approximately 1.5 times the baseline wall-clock time at the same base-solver NFE. Table 1 therefore includes additional-step baselines matched for wall-clock time. Its NFE should not be interpreted as the total cost including all derivative computations. The detailed timing procedure is placed in Appendix C and Table 6, which are absent from the current cache.

Key Experimental Results¶

Main Results¶

The following entries come from the paper's Table 1. CIFAR-10 uses EDM; ImageNet uses JiT at \(256\times256\) resolution. Lower FID is better. The dagger denotes additional base-solver evaluations to match FDS wall-clock time, not equal NFE with FDS.

Configuration	NFE	CIFAR-10 EDM conditional FID	CIFAR-10 EDM unconditional FID	ImageNet JiT-B/16 FID	ImageNet JiT-L/16 FID
Euler	50	3.003	3.034	4.151	3.859
Euler†	77	2.515	2.550	4.061	3.857
Euler + FDS	50	2.319	2.440	3.799	3.519
Heun	99	1.904	2.021	3.637	2.713
Heun†	153	1.910	2.034	3.815	2.886
Heun + FDS	99	1.786	1.953	3.394	2.496

The DrawBench results below come from Table 2. All four scores are interpreted as higher-is-better; IR denotes ImageReward and Aes. the Aesthetic Predictor score. Unlike Table 1, this table does not identify wall-clock-matched baselines, so it does not establish an equal-compute advantage for text-to-image generation.

Model	IR	HPSv2	Aes.	CLIP
SD3-M	82.36	27.72	5.70	28.47
SD3-M + FDS	89.33	27.76	5.72	28.76
FLUX.1	92.95	29.37	6.18	27.33
FLUX.1 + FDS	94.63	29.39	6.14	27.46

Ablation Study¶

These entries come from the tabular panels in Figures 6c and 7b, not another numbered table. All use ImageNet \(256\times256\), JiT-B/16, Euler, and 50 NFE.

Analysis	Configuration	FID
No-refinement baseline	FDS disabled	4.151
Perturbation schedule	Cosine	3.799
Perturbation schedule	Linear	3.823
Perturbation schedule	Concave	3.893
Perturbed candidates	M=1	3.799
Perturbed candidates	M=2	3.795
Perturbed candidates	M=8	3.785
Perturbed candidates	M=20	3.765

Key Findings¶

On JiT-L/16, increasing Heun from 99 to 153 NFE worsens FID from 2.713 to 2.886, while FDS achieves 2.496. This supports the importance of where additional compute is spent, not a universal claim that higher-order solvers have no remaining potential.
Increasing candidates from 1 to 20 only improves FID from 3.799 to 3.765, supporting the lightweight default. Cosine scheduling also outperforms the linear and concave alternatives in this setting.
Metrics do not improve uniformly: FLUX.1 Aes. falls from 6.18 to 6.14; Table 1's JiT-L/16 + Heun IS falls from 330.23 to 329.70. The most consistent evidence concerns FID, not simultaneous improvement across every quality dimension.

Highlights & Insights¶

Using the model's derivatives to evaluate candidates avoids training a reward model for every task. The theoretical relationship explains the signal rather than relying solely on a visually motivated heuristic.
Retaining the original state is an important implementation detail: perturbations are not compulsory, and worse neighbors can be rejected. It also makes the distinction between perturbed candidates and total compared states essential.
Wall-clock-matched comparisons are more informative than equal NFE alone. Divergence is not free, so its derivative cost must enter the budget before deciding whether spatial refinement is worthwhile.

Limitations & Future Work¶

Section 6.2 acknowledges rapid saturation as candidate and iteration counts increase, while the default still incurs approximately 1.5 times the baseline runtime. Whether parallel candidates add almost no latency depends on memory, batching, and hardware, rather than being a general guarantee.
Moving from the true-field theorem to a neural proxy requires reliable spatial derivatives; accurate velocity predictions do not automatically imply accurate derivatives. Note assessment: high-dimensional tests should measure derivative error, random trace-estimation variance, and candidate-ranking stability.
Reducing local velocity disagreement can alter the generated distribution. FID and preference metrics alone cannot exclude reduced mode coverage. Note assessment: distribution preservation, diversity, and recall deserve separate evaluation.
The cache contains the main method, experiments, and analyses but not the referenced Appendices A–F. The truncation direction is inconsistent as described above, and Equation (7)'s text extraction is damaged. This note therefore does not reconstruct its exact coefficients or invent sample counts, hardware, seeds, text-to-image step counts, or significance claims.

vs HRF / VRFM: These methods address multimodal velocities through training, whereas FDS selects states using an existing marginal field at inference time. Table 5 involves different models and training procedures, so the gap cannot be attributed entirely to training-based approaches being inherently inferior.
vs UniPC / RK4: Higher-order solvers improve temporal integration; FDS first changes the starting state. Table 4 supports complementarity, although other metrics such as IS should also be monitored.
vs external reward search / TFG: FDS obtains its ranking signal from the velocity field itself, while TFG supplies task constraints and can be combined with it. A testable extension would jointly constrain task consistency and divergence reduction to prevent low-divergence candidates from drifting away from the condition.

Rating¶

Novelty: 4/5. A clear connection between irreducible velocity discrepancy and divergence supports training-free state search.
Experimental Thoroughness: 4/5. Multiple backbones, solvers, and tasks include wall-clock-matched controls, but supplementary reproduction details are absent from the current cache.
Writing Quality: 3/5. The mechanism is intuitive, but contradictory truncation directions impede reproduction, and the boundary between ideal flow and practical degradation needs clarification.
Value: 4/5. A useful inference enhancement for pretrained generators, provided derivative costs and changes in distribution coverage are evaluated together.