Skip to content

title: >- [Paper Note] DuoFlow: JVP-Free Finite-Difference Mean Flows for One-Step Image Generation description: >- [ECCV 2026][image_generation][DuoFlow] An error-driven JVP-free finite-difference framework for MeanFlow, mitigating truncation and trajectory errors via stochastic signed differencing and joint velocity estimation to achieve one-step generation SOTA from scratch. tags: - ECCV 2026 - image_generation - MeanFlow - One-Step Generation - JVP-Free date: 2026-09-19 content_hash: 42cbea1ca9ccd0ee

DuoFlow: JVP-Free Finite-Difference Mean Flows for One-Step Image Generation

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/stepfun-ai/DuoFlow
Area: Image Generation
Keywords: one-step generation, MeanFlow, finite difference, JVP-free, trajectory consistency

TL;DR

Addressing the computational bottleneck where MeanFlow relies heavily on expensive Jacobian-vector products (JVP), DuoFlow establishes an error-driven JVP-free differential framework that suppresses step-truncation and trajectory-velocity errors, cutting 1-NFE FID by ~28% on ImageNet \(256 \times 256\) from scratch with a \(2.54\times\) end-to-end training speedup.

Background & Motivation

One-step image generation is of vital importance for deploying diffusion and flow-based generative models in latency-critical scenarios. In contrast to traditional multi-step ODE/SDE numerical solvers requiring dozens to hundreds of evaluation steps, single-step models drastically slash latency and compute cost. Among recent advances, MeanFlow provides an elegant mathematical foundation by enforcing an identity that couples interval-wise mean velocities with instantaneous velocities, thereby enabling one-step transport learning trained completely from scratch without external teacher distillation. However, in standard implementations, MeanFlow enforces this identity by evaluating directional time derivatives along model-predicted trajectory velocities via automatic differentiation's Jacobian-vector products (JVP). In modern deep learning systems, JVP introduces heavy computational graph overhead and disrupts compiler optimizations such as fused attention kernels, creating severe scalability bottlenecks.

Replacing JVP with numerical finite differences appears straightforward at first glance, but a naive operator swap often leads to training divergence or degraded generation quality. The root issue is that the directional derivative in MeanFlow is evaluated along a dynamic trajectory velocity predicted by the model itself, rather than an exogenous ground-truth field. Consequently, supervision quality is jointly dictated by two intertwined sources of inaccuracy: the numerical truncation error from finite differencing and the trajectory-velocity error from model prediction drift. Deterministic forward differences introduce a persistent first-order bias that accumulates across training iterations, steering model parameters into ill-conditioned, high-curvature loss regions.

Rather than treating differencing as an ad-hoc drop-in replacement, this paper approaches JVP-free MeanFlow through an explicit error decomposition. By breaking down the supervision error into step-truncation and trajectory-velocity error channels, the authors engineer targeted counter-mechanisms for both. The core idea is to achieve expected second-order truncation accuracy with only one additional forward pass via stochastic signed one-sided differencing, while jointly estimating mean and instantaneous velocities at the same sampled state alongside progressive self-bootstrapping to enforce trajectory consistency.

Method

Overall Architecture

DuoFlow preserves the core MeanFlow identity while replacing JVP with an error-controlled differential estimator. The pipeline operates entirely within standard forward/backward execution graphs, maintaining full compatibility with operator fusion and compiler acceleration. In each training step, the network takes a latent state \(z_t\) and jointly predicts the instantaneous velocity \(v_\theta\) and interval mean velocity \(u_\theta\). A single additional forward pass on a stochastically perturbed state computes the finite-difference directional derivative. Finally, an alignment-gated progressive self-bootstrapping mechanism dynamically blends external CFG targets with internal velocity predictions to update model parameters.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Latent State z_t and Time (r, t)"] --> B["Same-State Joint Velocity Estimation<br/>Shared backbone outputs u_θ and v_θ"]
    B --> C["Stochastic Signed One-Sided Differencing<br/>Perturbed state forward pass for directional derivative"]
    C --> D["Progressive Self-Bootstrapping<br/>Cosine-similarity gating weight κ blends target velocity"]
    D --> E["Detached-Loss Normalization<br/>Balanced gradient update for u and v parameters"]

Key Designs

1. Error-Driven Orthogonal Decomposition: Pinpointing Supervision Drift In ideal MeanFlow supervision, the directional derivative of mean velocity \(u\) should be evaluated along the optimal trajectory velocity \(v^\star\), denoted \(\mathcal{D}^{v^\star}u\). In a numerical differential setup without JVP, two successive approximation errors are introduced: $$ \hat{\mathcal{D}} u - \mathcal{D}^{v^\star} u = \underbrace{(\hat{\mathcal{D}} u - \mathcal{D}^{v_\theta} u)}{E}}} + \underbrace{(\mathcal{D}^{v_\theta} u - \mathcal{D}^{v^\star} u){E $$ where }}\(E_{\text{trunc}}\) is the step-truncation error arising from finite step approximation, and \(E_{\text{traj}}\) is the trajectory-velocity error stemming from the model's imperfect velocity field \(v_\theta\). This analytical decomposition reveals that reducing the step size \(h\) alone is futile if \(v_\theta\) drifts, and conversely, any systematic bias in differencing destabilizes the trajectory velocity.

2. Stochastic Signed One-Sided Differencing: Second-Order Truncation in Expectation via One Forward Pass Standard second-order central differencing requires two extra forward passes, which imposes substantial latency. Conversely, deterministic forward differencing requires only one extra evaluation but suffers from an uncancelled first-order truncation bias \(\frac{h}{2}(\mathcal{D}^{v_\theta})^2 u_0\). To resolve this dilemma, DuoFlow samples a random sign \(\sigma \sim \text{Unif}\{+1, -1\}\) and uses machine-precision-aware step sizes \(h_{\text{base}}\) (e.g. \(\sqrt{\varepsilon_{\text{mach}}}\) for BF16) along with state-velocity RMS normalization to construct the perturbed state \(z_{t+\sigma h} = z_t + \sigma h v_\theta(z_t, t)\). The differential estimator is: $$ \hat{\mathcal{D}}h^\sigma u\theta = \frac{u_\theta(z_{t+\sigma h}, r, t+\sigma h) - u_\theta(z_t, r, t)}{\sigma h} = \mathcal{D}^{v_\theta}u_0 + \frac{\sigma h}{2}(\mathcal{D}^{v_\theta})^2 u_0 + \mathcal{O}(h^2) $$ Taking the expectation over \(\sigma \in \{+1, -1\}\) eliminates the \(\mathcal{O}(h)\) term, yielding second-order truncation behavior in expectation: $$ \mathbb{E}\sigma \left[ \hat{\mathcal{D}}_h^\sigma u\theta \right] = \mathcal{D}^{v_\theta} u_\theta + \mathcal{O}(h^2) $$ Because zero-mean variance is naturally averaged out across mini-batches during SGD, coherent bias accumulation across long horizons is eliminated without adding multi-step evaluation overhead.

3. Same-State Joint Velocity Estimation: Harmonizing Trajectory Consistency To suppress \(E_{\text{traj}}\), DuoFlow discards separate or decoupled modeling of the instantaneous velocity \(v\) and mean velocity \(u\). Instead, both are predicted by a single shared backbone conditioned on different temporal intervals (\(r=t\) yields \(v_\theta\), while \(r<t\) yields \(u_\theta\)) evaluated at the identical state \(z_t\). Constraining both velocity fields under shared local feature representations ensures that improvements in instantaneous velocity immediately translate into more accurate trajectory directions for differential evaluation.

4. Progressive Self-Bootstrapping: Driving Trajectory Dynamics to a Fixed-Point Tendency During early training, self-predicted velocities are noisy, making full self-targeting dangerous; conversely, persistently relying on external targets (\(\dot{z}_t = \epsilon - x_0\)) prevents the model from achieving internal dynamical consistency. DuoFlow introduces a batch-level cosine alignment gating mechanism: computing the similarity between model velocity \(v_\theta\) and external CFG velocity \(v_{\text{ext}}\), it sets \(\kappa = 0.5 + 0.5 \cdot \frac{\bar{d}}{\sqrt{\max(\bar{q}_v \bar{q}_t, \epsilon)}}\) and constructs \(v_{\text{target}} = \kappa \, \text{sg}[v_\theta] + (1-\kappa) v_{\text{ext}}\). As training matures, \(\kappa\) smoothly increases toward saturation, guiding trajectory dynamics toward a fixed-point-like self-consistent state.

Loss & Training

The supervision targets for mean and instantaneous velocities are defined with stop-gradient (\(\text{sg}\)): $$ u_{\text{target}} = v_{\text{target}} - \text{sg}[\Delta_\theta], \quad \Delta_\theta = (t-r)\hat{\mathcal{D}}h^\sigma u\theta $$ $$ \mathcal{L}u = |u\theta - u_{\text{target}}|2^2, \quad \mathcal{L}_v = |v\theta - v_{\text{target}}|_2^2 $$ To maintain gradient balance as \(\kappa\) evolves, detached-loss normalization is applied: \(\bar{\mathcal{L}}_u = \frac{\mathcal{L}_u}{\text{sg}[\mathcal{L}_u] + \epsilon}\) and \(\bar{\mathcal{L}}_v = \frac{\mathcal{L}_v}{\text{sg}[\mathcal{L}_v] + \epsilon}\). The final objective is: $$ \mathcal{L} = \bar{\mathcal{L}}_u + w_v \bar{\mathcal{L}}_v, \quad w_v = \frac{1}{(1 - \bar{\kappa})^2} $$ The network is optimized using Adam with a learning rate of \(2 \times 10^{-4}\) and an EMA decay rate of 0.9999.

Key Experimental Results

Main Results

On class-conditional ImageNet \(256 \times 256\), DuoFlow is compared against from-scratch diffusion and flow models using the standard pre-trained VAE latent space (240 epochs unless marked with \(\dagger\) for 640 epochs):

Model / Method Params NFE FID (↓) Relative Gain vs. MeanFlow
iCT-XL/2 (Consistency Training) 675M 1 34.24 -
Shortcut-XL/2 675M 1 10.60 -
MeanFlow-B/2 (JVP Baseline) 131M 1 6.17 Baseline
DuoFlow-B/2 (Ours) 131M 1 5.19 -15.9%
MeanFlow-M/2 (JVP Baseline) 308M 1 5.01 Baseline
DuoFlow-M/2 (Ours) 308M 1 4.72 -5.8%
MeanFlow-L/2 (JVP Baseline) 459M 1 3.84 Baseline
DuoFlow-L/2 (Ours) 459M 1 3.14 -18.2%
MeanFlow-XL/2 (JVP Baseline) 676M 1 3.43 Baseline
DuoFlow-XL/2 (Ours) 676M 1 2.48 -27.7%
DuoFlow-XL/2 (640 epochs) 676M 1 1.99 -
MeanFlow-XL/2 (2-NFE) 676M 2 2.93 Baseline
DuoFlow-XL/2 (2-NFE) 676M 2 1.96 -33.1%

Note: Sourced from Table 1 of the original paper. In the 1-NFE XL/2 setting, DuoFlow improves FID from 3.43 to 2.48 (a ~28% relative gain); at 2-NFE it achieves 1.96 FID.

Ablation Study

On a matched 80-epoch ImageNet \(256 \times 256\) setup (Table 3 in the paper), the components are isolated:

Configuration / Variant Signed Diff. Joint Velocity Prog. Bootstr. FID (↓) Relative Change
MeanFlow (JVP Baseline Reproduced) - - - 14.13 Original JVP supervision
DuoFlow w/ Deterministic One-Sided ✗ ✗ ✗ 19.06 Severely degraded due to \(\mathcal{O}(h)\) bias (-34.9%)
DuoFlow w/ Signed One-Sided Diff. ✓ ✗ ✗ 14.11 Recovers JVP accuracy (+0.0%)
DuoFlow + Joint Velocity ✓ ✓ ✗ 11.30 Suppresses trajectory drift (+20.0%)
DuoFlow (Full Model) ✓ ✓ ✓ 9.17 Self-consistent dynamic (+35.1%)

Note: In micro-benchmarks (Table 2), directional derivative evaluation with DuoFlow is \(7.17\times\) to \(34.83\times\) faster than torch.func.jvp across batch sizes 1 to 64, cutting peak memory by \(1.71\times\) to \(2.68\times\). In 8-GPU H800 full training, wall-clock time drops from 0.89s/iter to 0.35s/iter (\(2.54\times\) speedup), reducing memory from 97GB to 73GB.

Key Findings

  • Stochastic sign randomization prevents divergence: Deterministic one-sided differencing accumulates coherent \(\mathcal{O}(h)\) bias, pushing training into unstable loss regions (best FID 19.06 before divergence). Random sign flipping achieves zero-mean \(\mathcal{O}(h^2)\) error in expectation, stabilized by batch averaging.
  • Early variance-dominated vs. late bias-dominated regimes: In early iterations, deterministic differencing displays slightly higher cosine similarity to JVP due to lower variance; however, as velocity estimates improve, the systematic bias of deterministic differencing dominates, and stochastic signed differencing decisively surpasses it.
  • Progressive gating prevents collapse: Fixing \(\kappa=0.9\) prematurely collapses training (FID degrades to 35.33), whereas alignment-gated progressive self-bootstrapping dynamically ramps up self-consistency to achieve 9.17 FID.

Highlights & Insights

  • Error-driven rather than black-box approximation: Instead of treating finite differencing as a simple derivative proxy, DuoFlow exposes the coupled \(E_{\text{trunc}} + E_{\text{traj}}\) error structure to steer algorithmic design.
  • Zero-mean stochastic cancellation: By leveraging mini-batch averaging in deep learning, a single extra forward evaluation combined with randomized sign sampling captures second-order accuracy properties typically requiring multiple evaluations.
  • Compiler-friendly execution: Eliminating JVP restores standard forward-backward passes, unlocking full compiler compatibility and fused-kernel acceleration for large-scale flow generative models.

Limitations & Future Work

  • Domain scope: Experiments are currently validated on class-conditional ImageNet \(256 \times 256\); scaling to higher resolutions (\(512\times 512\)+), text-to-image synthesis, and video generation remains future work.
  • Smoothness assumptions: The mathematical cancellation of the first-order truncation term assumes local velocity field smoothness, which might be challenged across sharp multi-modal semantic boundaries.
  • Future synergies: Combining DuoFlow with representation alignment methods (e.g., REPA) and improved transformer architectures (LightningDiT).
  • vs. MeanFlow [Geng et al., 2025]: MeanFlow introduced the mean-velocity identity but relied on JVP, which incurs heavy memory footprints and breaks fused kernels. DuoFlow removes JVP via error-driven differencing, achieving superior generation quality (FID 2.48 vs. 3.43) with a \(2.54\times\) end-to-end training speedup.
  • vs. Shortcut Models / Consistency Training: Compared to earlier from-scratch one-step models where FID remained above 10.0, DuoFlow delivers an FID of 2.48 (and 1.99 with extended training), significantly closing the gap with multi-step ODE samplers.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ [Pioneering error-driven formulation for JVP-free MeanFlow with expected second-order stochastic differencing]
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive micro-benchmarks, end-to-end training speedups, ablation studies, and step-size sweeps]
  • Writing Quality: ⭐⭐⭐⭐⭐ [Clear mathematical exposition, rigorous error analysis, and structured experimental narrative]
  • Value: ⭐⭐⭐⭐⭐ [Removes a major systems hurdle for scaling one-step flow models in modern training frameworks]