Continuous Adversarial Flow Models¶

Conference: ECCV 2026
Paper: ECCV Official Page
Area: Image Generation
Keywords: continuous-time flows, adversarial post-training, directional derivatives, flow matching, guidance-free generation

TL;DR¶

CAFM compares real and predicted velocities in the discriminator's directional-derivative space to adversarially post-train continuous-time flow models, reducing guidance-free ImageNet 256px FID from 8.26 / 7.17 to 3.63 / 3.57 for SiT / JiT while retaining their original multi-step sampling procedures.

Background & Motivation¶

Flow matching supervises the velocity field between noise and data with mean squared error, offering a simple and scalable training procedure. However, finite-capacity models do not necessarily generate samples close to the real image distribution. Classifier-free guidance (CFG) often improves image quality, but changes the sampling distribution. This paper asks whether the underlying model can generate better samples without guidance, rather than whether sampling can be compressed into a few steps.

The authors attribute the issue to the inductive bias of the training criterion: different criteria can target the same ground-truth velocity field under ideal conditions yet generalize differently with finite model capacity. Pointwise Euclidean error does not explicitly distinguish perceptual differences such as texture and contours, while a fixed perceptual network can be exploited by the generator. A learned discriminator offers an alternative, but existing Adversarial Flow Models (AFMs) compare discrete-time state transitions. Their objective degenerates as the time interval approaches zero, preventing a direct, stable extension to continuous time.

Core Idea: preserve the state interpolation and velocity-prediction interface of flow matching, move discrimination from positions to the discriminator's derivatives along velocity directions, and use a jointly learned perceptual criterion to change finite-capacity generalization without turning the task into few-step distillation.

Method¶

Overall Architecture¶

During training, a real image and Gaussian noise are interpolated at a random time to construct a noisy state, and the generator predicts its velocity. The discriminator still takes a state and time and outputs a scalar, but real/fake scores come from its Jacobian-vector product (JVP): directional derivatives are evaluated at the same state along the real conditional velocity and the generator's predicted velocity, followed by alternating network updates.

The objective does not require integrating a complete generated image during training, so it remains simulation-free. Sampling only requires the trained generator; the discriminator is discarded. The continuous flow can generate samples through ODE integration, while the experiments preserve each baseline's ODE or SDE sampling configuration. Continuous time does not mean single-pass generation.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    INPUT["Real image, noise<br/>time and condition"] --> FLOW["Same-State<br/>Velocity Comparison"]
    FLOW --> JVP["Derivative-Space<br/>Discrimination"]
    JVP --> TRAIN["Stable Adversarial<br/>Post-Training"]
    TRAIN -.->|Training updates| FLOW
    TRAIN -->|Keep generator| SAMPLE["Original multi-step sampler<br/>Output image"]

Key Designs¶

1. Same-State Velocity Comparison: preserve the probability path instead of predicting a discrete target position

With linear interpolation, data lies at \(t=0\) and noise at \(t=1\). For a particular image-noise pair, the conditional velocity is their difference, while the generator only observes the interpolated state, time, and class or text condition. This real conditional velocity is supervision supplied by one training pair, not the marginal velocity the model ultimately seeks to learn. Standard flow matching learns the expectation of conditional velocities given the state.

\[ x_t=(1-t)x+tz,\qquad \bar v_t=z-x,\qquad u_t=G(x_t,t). \]

CAFM preserves this input-output interface, avoiding the need to convert a pretrained generator into a discrete transition network with separate source and target time inputs. SiT predicts velocity directly; JiT predicts clean images, which the authors convert to velocities before passing them to the discriminator. This preserves compatibility with the original generator: the supervision criterion changes, not the data space or a manually specified manifold.

2. Derivative-Space Discrimination: evaluate flow directions with a learned scalar potential

Asking a discriminator to distinguish two nearly identical positions makes useful discrimination harder as the interval shrinks. Instead of constructing a tiny discrete step, CAFM computes continuous derivatives with automatic differentiation. At the same \((x_t,t)\), it uses either the real conditional velocity or predicted velocity as the spatial tangent and \(T=1\) as the temporal tangent, yielding two scalar scores. The discriminator learns a scalar potential over states, and each score measures how this potential changes along a space-time direction; it does not simply classify a velocity vector as an ordinary input.

The JVP primals remain the noisy state and time, while real and predicted velocities are tangents. The discriminator must therefore distinguish directions through its local geometry. The generator influences the JVP through its predicted velocity and receives gradients propagated through the JVP. Since sampling runs backward from \(t=1\) to \(t=0\), Figure 2 visualizes the negative discriminator potential for intuition. Interpreting potential ascent or descent without accounting for the time direction would be misleading.

3. Stable Adversarial Post-Training: anchor scalar drift and let the discriminator adapt sufficiently

Constraining derivatives alone does not fix the absolute offset of the discriminator output, so the authors add an output-squared centering penalty. Another ambiguity arises when high-dimensional velocities are mapped to a scalar: different directions can yield the same JVP score, allowing the generator to exploit the current discriminator's null space. A squared-velocity-norm regularizer can favor minimum-norm solutions early in training and reduce such behavior. This is the continuous-time form of optimal transport regularization, not MSE pulling predicted velocities back toward real conditional velocities.

The authors gradually reduce this regularizer when training from scratch, but set it to zero when post-training an existing FM model to avoid reintroducing Euclidean norm bias. The discriminator receives multiple updates per generator update, and the experiments omit the gradient penalties used by discrete AFMs. Implementation uses forward-mode automatic differentiation for JVPs, vmap for different tangents at the same state, and RMSNorm instead of LayerNorm in the discriminator to improve stability. The generator architecture remains unchanged.

A Worked Example¶

Consider an ImageNet post-training update for SiT. Take a real image encoding with its class label and Gaussian noise of the same shape, sample a time, and interpolate them into a noisy latent. Pretrained SiT predicts its velocity; the corresponding real conditional velocity is the noise minus the real latent.

For a discriminator update, use the two velocities as separate JVP tangents and push the real-direction score toward +1 and the predicted-direction score toward -1, without updating the generator. For a generator update, freeze discriminator parameters but retain gradients through the predicted tangent, pushing the predicted-direction score toward +1. The actual schedule uses \(N=16\) discriminator updates per generator update, rather than immediately updating each network once on every minibatch.

After training, generate images from Gaussian noise with the original SiT sampling code. Neither the discriminator nor a training example's real velocity is needed. The reported SiT results still use a 250-step Euler-Maruyama SDE sampler, demonstrating a post-training quality improvement rather than a reduction in sampling steps.

Loss & Training¶

Several typeset equations are corrupted in the supplied text extraction. The following uses the fully readable computation in Algorithm 1 instead of guessing the damaged contrastive-function equation. Let \(d_v\) and \(d_u\) denote the JVP scores for real and predicted velocities, respectively. The implemented optimization terms are:

\[ \mathcal L_D=\mathbb E[(d_v-1)^2+(d_u+1)^2]+\lambda_{\mathrm{cp}}\mathbb E[D(x_t,t)^2]. \]

\[ \mathcal L_G=\mathbb E[(d_u-1)^2]+\lambda_{\mathrm{ot}}\mathbb E[\|u_t\|_2^2/n]. \]

Here, \(n\) is the velocity dimension and \(\lambda_{\mathrm{cp}}=0.001\); post-training uses \(\lambda_{\mathrm{ot}}=0\). Generator parameters are frozen during discriminator updates, and discriminator parameters are frozen during generator updates while gradients with respect to generator outputs remain necessary. The algorithm's mean averages over samples and the relevant tensor dimensions. The norm regularizer must not be mistaken for conditional velocity matching.

For ImageNet post-training, both networks use a learning rate of \(10^{-5}\), Adam with \(\beta=(0,0.95)\), generator EMA decay of 0.99, and \(N=16\). SiT uses a batch size of 256 and warms up only the discriminator for the first 2 epochs; JiT uses a batch size of 1024 and a 4-epoch warmup. Each model receives 10 additional epochs in total. The paper counts epochs using images seen by both networks combined, not 10 extra complete generator training passes.

The text-to-image experiment uses Z-Image, first training with FM on the authors' data for 10K iterations, then switching to CAFM for 20K iterations while continuing the FM control to match iteration counts. CAFM uses generator / discriminator learning rates of \(5\times10^{-5}\) / \(3\times10^{-5}\), \(N=16\), and EMA decay of 0.99. Matching iterations does not imply matched wall-clock time or total FLOPs.

Key Experimental Results¶

Main Results¶

ImageNet results come from Tables 1 and 2 at 256px resolution; lower FID and higher IS are better. SiT is latent-space SiT-XL/2, and JiT is pixel-space JiT-H/16. Guided rows show each method's best FID from the CFG sweep, so their optimal CFG values differ; these are not paired results at the same CFG.

Model	Training epochs	CFG	FID	IS
SiT	1400	None	8.26	131.65
SiT + CAFM	1400 + 10	None	3.63	178.08
SiT	1400	1.5	2.06	277.50
SiT + CAFM	1400 + 10	1.3	1.53	263.52
JiT	600	None	7.17	151.54
JiT + CAFM	600 + 10	None	3.57	198.08
JiT	600	2.2	1.86	303.40
JiT + CAFM	600 + 10	1.8	1.80	290.71

Text-to-image results come from Tables 5 and 6 at 512px resolution. All values below are Overall scores, where higher is better. PE denotes prompt expansion; GenEval settings with different PE or CFG conditions must not be conflated. The DPG table does not separately specify a PE condition.

Benchmark	Setting	FM	CAFM
GenEval	No PE, no CFG	0.33	0.44
GenEval	With PE, no CFG	0.60	0.71
GenEval	With PE, with CFG	0.81	0.85
DPG-Bench	No CFG	72.25	77.21
DPG-Bench	With CFG	83.67	85.21

Ablation Study¶

The following summarizes continued-training controls and CFG sensitivity from Tables 1 and 2, rather than claiming to reproduce component-removal ablations in the appendix. Each row compares the same model at the same CFG setting. Continued FM training also adds 10 epochs.

Config	Original FID	Control or post-training FID	Note
SiT, no CFG, continued FM	8.26	8.64	Longer training alone does not improve results
JiT, no CFG, continued FM	7.17	9.30	Authors report difficulty reproducing the official checkpoint's performance
SiT, CFG 1.6, CAFM	2.25	2.37	Degradation under strong guidance
JiT, CFG 2.2, CAFM	1.86	1.88	Not every CFG setting improves

Key Findings¶

The clearest gains are guidance-free: SiT FID decreases by 4.63 and JiT by 3.60. Best guided FID also improves, but JiT's change from 1.86 to 1.80 is modest.
CAFM prefers a lower optimal CFG, consistent with reduced dependence on guidance. Degradation at high CFG indicates that guidance should be retuned after post-training.
Higher DPG Overall scores do not mean every submetric improves. With CFG, Global decreases from 90.34 to 89.55 and Entity from 90.56 to 89.83, while Relation increases from 88.17 to 91.20.
From-scratch SiT-B/2 training converges more slowly than FM at the same epoch budget. Section 4.3 supports starting with \(N=4\) and \(\lambda_{\mathrm{ot}}=4\), reducing the regularizer to 1 at 160 epochs, and increasing \(N\) to 8 at 700 epochs. These are not the fixed post-training settings.

Highlights & Insights¶

Instead of directly assessing whether a velocity looks real, the discriminator evaluates it through a directional derivative in state space. This incorporates the derivative structure of continuous time into adversarial training and avoids degeneration from shrinking discrete intervals.
Sharing an ideal velocity field does not imply sharing a finite-capacity solution. Treating a learned loss as a source of generalization bias allows the method to retain the network, data space, and sampler.
Zero norm regularization during post-training and nonzero regularization from scratch serve different purposes. The former adjusts generation behavior from a capable starting model, while the latter initially suppresses directional ambiguity; the two stages should not blindly share hyperparameters.

Limitations & Future Work¶

The authors explicitly acknowledge that better guidance-free metrics do not guarantee recovery of the true data distribution, particularly in low-density regions and for outliers. FID and text-alignment scores are not proofs of distribution recovery.
The additional discriminator, forward and backward JVP computation, and multiple discriminator updates increase training cost. Matching epochs or iterations does not establish compute parity, and the main text lacks a complete wall-clock comparison that directly quantifies training cost for deployment planning.
JiT's FM control is substantially worse than its official checkpoint, making attribution to the objective alone less clean than an ideal paired experiment. Repeated seeds, confidence intervals, and compute-matched comparisons would strengthen the evidence.
The method is not uniformly superior to all alternatives. Table 4 reports guidance-free FID of 2.77 for SiD versus 3.57 for JiT + CAFM, with different model sizes and settings. Methods using DINOv2 in Table 3 also require separate consideration.
The supplied cache contains only the main text and references, not Appendices A, D, E, or F. Their additional ablations, proofs, and implementation details were not independently checked for this note, and damaged equations were not reconstructed by guesswork.

vs Flow Matching: FM supervises conditional velocities with MSE; CAFM supervises the same probability path through learned directional-derivative scores. The main benefit is altered finite-capacity generalization, not a redefinition of the theoretical target trajectory.
vs Adversarial Flow Models: AFM discriminates discrete target states, while CAFM discriminates directions through discriminator derivatives. The latter's experiments omit gradient penalties, but this does not establish stability for every dataset and architecture.
vs Adversarial Distillation: Common adversarial post-training targets one- or few-step generation, whereas CAFM primarily improves continuous-flow quality. SiT / JiT retain 250-step SDE / 50-step Heun ODE sampling, respectively.
vs Perceptual Losses and Manifold Flow Matching: A generator can exploit fixed feature losses, while explicit manifold methods require known geometry. CAFM jointly learns the evaluation criterion without explicitly recovering or parameterizing the true data manifold.

Rating¶

Novelty: 4/5. Placing adversarial objectives in discriminator JVPs provides a concrete mechanism for extending discrete adversarial flows to continuous time.
Experimental Thoroughness: 4/5. Covers latent and pixel spaces and text-to-image generation, but compute fairness and statistical uncertainty need stronger treatment.
Writing Quality: 4/5. The main argument and algorithm are clear; corrupted equation extraction and missing appendices limit detailed verification from the local text.
Value: 4/5. Provides a quality-oriented post-training path for existing FM models without changing generator architecture, but does not directly reduce sampling steps.