Exposure Bias Can Alleviate Itself via Directional and Frequency Rectification in Flow Matching

Conference: ECCV 2026
Paper: ECCV Paper
Area: Image Generation
Keywords: Flow matching, exposure bias, directional rectification, frequency compensation, self-feedback training

TL;DR

DEFAR simulates single-step inference during flow matching training and turns the model's own drift into directional rectification and frequency compensation signals, reducing SiT-B/4's ImageNet-256 FID with CFG from 31.64 to 29.81 at a comparable training time without changing the inference model architecture.

Background & Motivation

Flow matching training typically interpolates between a real image and Gaussian noise over time, asking a network to predict the velocity along this path. Every training input is an intermediate state constructed from real data, whereas generation must use the model's previous prediction to produce the next input. A velocity error changes both the current position and the state on which the next prediction is evaluated. This training-inference input mismatch is the exposure bias studied here. It is not simply the error against the supervised velocity at one step, but the additional mismatch arising when an error changes subsequent inputs.

Input perturbation (IP) simulates drift with additional noise, while SDSS/MDSS simulates single-step or multi-step inference during training to expose the network to biased states. However, retaining the original path's velocity as a fixed target after a state has drifted mainly encourages stability under perturbations; it does not directly specify how to correct the trajectory from its new position. The authors also observe that high-noise velocity predictions lack low-frequency structure, while the difference between predictions on biased and ideal inputs is rich in this information. Bias may therefore indicate not only the amount of error, but also which information remains poorly learned.

The paper connects geometric direction with spatial loss weighting: a known ground-truth endpoint provides a new corrective direction from a drifted state, and the prediction difference caused by drift identifies regions to emphasize in the original objective. Core Idea: use the model's own single-step drift both to define a corrective direction and to reweight the training loss, allowing exposure bias to supply signals for its own mitigation.

Method

Overall Architecture

DEFAR, short for DirEctional-Frequency Adaptive Rectification, is a training framework rather than an additional inference network. Each iteration constructs ideal states at two timesteps from the same real image and noise sample, then simulates one inference step from the earlier state to obtain a biased state. Anti-Drift Rectification (ADR) constrains the predicted direction at this state, while Frequency Compensation (FC) reweights the standard flow matching loss at the earlier timestep.

Time runs from noise at 0 to data at 1. Under the default linear interpolant, the supervised velocity is the real image minus the initial noise. Ideal states serve only as training references; actual generation neither knows the real endpoint nor reconstructs these supervision branches.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    INPUT["Training image, noise<br/>Two ordered timesteps"] --> SIM["Single-Step Bias Construction"]
    SIM --> ADR["Anti-Drift Rectification ADR"]
    SIM --> FC["Frequency Compensation FC"]
    ADR --> LOSS["Joint loss updates the network"]
    FC --> LOSS
    LOSS --> MODEL["Trained velocity network"]
    NOISE["Inference noise and condition"] --> MODEL
    MODEL --> SAMPLE["Conventional sampling produces images"]

ADR and FC are two training branches sharing a simulated state, not two modules executed sequentially during generation. They modify directional supervision and the spatial allocation of the original loss, respectively. Conventional ODE/SDE sampling is retained after training.

Key Designs

1. Single-Step Bias Construction: separate velocity prediction error from error caused by input drift

For two timesteps satisfying \(t_1>t_0\), construct the ideal states and use the network's prediction at \(t_0\) to take one step, producing a biased state at \(t_1\). Query the network on both the biased and ideal states at the same timestep \(t_1\). Their velocity difference defines a proxy for exposure bias. Because the network and timestep are shared, this difference measures the response to an input changed by a previous prediction, rather than directly subtracting the ground-truth velocity from a prediction.

The following transcribes only the readable single-step update and difference relations in Algorithm 1:

\[ \hat{\mathbf{x}}_{t_0,t_1}=\mathbf{x}_{t_0}+(t_1-t_0)\mathbf{v}_{\theta}(\mathbf{x}_{t_0},t_0),\qquad \boldsymbol{\delta}_{t_0,t_1}=\mathbf{v}_{\theta}(\hat{\mathbf{x}}_{t_0,t_1},t_1)-\operatorname{StopGradient}\!\left(\mathbf{v}_{\theta}(\mathbf{x}_{t_1},t_1)\right). \]

The algorithm explicitly stops gradients through the ideal branch, making it a reference for the biased branch. This is not a teacher model: it is the same network evaluated on an ideal input, and its prediction is not necessarily correct. A training iteration queries the network on the earlier ideal state, the later ideal state, and the later biased state, making each iteration more expensive than standard FM. This avoids long trajectory unrolling but does not fully reproduce real multi-step error accumulation.

2. Anti-Drift Rectification ADR: point from the drifted position toward the real endpoint

Suppose a network should follow a straight path to a real image but drifts sideways at one step. Continuing in the original parallel direction does not remove that lateral displacement. ADR instead defines the correction from the current biased state to the real endpoint, making supervision depend on the drift. For the default linear path, this direction is \(\mathbf{x}_*-\hat{\mathbf{x}}_{t_0,t_1}\), rather than the unchanged constant velocity target from the original path.

ADR compares normalized predicted velocity with the normalized target direction. It primarily constrains where the trajectory should go, without requiring matching magnitudes. On a linear path without drift, the endpoint minus the current ideal state is a positive scalar multiple of the original supervised velocity, so directional supervision agrees with standard FM. This explanation applies to valid nonterminal states; handling of zero norms and endpoints cannot be fully verified from the cached text.

Writing \(\hat{\mathbf{v}}=\mathbf{v}_{\theta}(\hat{\mathbf{x}}_{t_0,t_1},t_1)\) for the prediction at the biased state, the directional loss in Algorithm 1 is:

\[ \mathcal{L}_{\mathrm{ADR}}=\mathbb{E}\left[\left\|\frac{\hat{\mathbf{v}}}{\|\hat{\mathbf{v}}\|_2}-\frac{\mathbf{v}_{\mathrm{ADR}}}{\|\mathbf{v}_{\mathrm{ADR}}\|_2}\right\|_2^2\right],\qquad \mathbf{v}_{\mathrm{ADR}}=\mathbf{x}_*-\hat{\mathbf{x}}_{t_0,t_1}\quad(a_t=t,\ b_t=1-t). \]

Normalization prevents the target's absolute scale from dominating this regularizer, but also means that it does not calibrate sampling speed by itself. Standard FM supervision remains necessary. The authors call this state-dependent directional supervision active rectification; it is a training mechanism, not access to a real reference image for correction at inference time.

3. Frequency Compensation FC: use biased prediction differences to locate regions needing stronger supervision

A correct direction alone does not guarantee reliable global structure early in generation. The authors apply a 2D FFT to predicted velocities, split low and high frequencies using a cutoff radius, and define the Predicted Frequency Ratio (PFR) as total low-frequency spectral energy divided by total high-frequency spectral energy. Their ImageNet-256 analysis finds low PFR in high-noise model predictions but high PFR in exposure bias, suggesting that the bias highlights low-frequency structure missing from the predictions. This complementarity weakens at low noise, motivating an alternative to a fixed low-pass preference throughout generation.

FC normalizes per-location bias magnitude information into nonnegative weights, increasing the contribution of corresponding spatial locations to the original FM error at the earlier timestep. Algorithm 1 includes a base weight of 1, a scaling coefficient \(\alpha\), and a denominator stability term \(\xi\), preserving basic supervision outside emphasized regions. FFT is used to analyze frequency behavior, not as a filtering operation required at every training iteration. FC reweights the loss in the spatial domain; it does not directly replace the generated image's spectrum.

The authors also define Frequency Emphasis of Loss (FEL) to test whether the loss focuses on these structures. Low- and high-frequency components of the target velocity are separately inverse-transformed into the spatial domain, where percentile thresholds define salient-region masks. FEL is the ratio of normalized loss summed over low-frequency salient regions to that summed over high-frequency salient regions. It measures regional loss allocation, not spectral energy obtained by applying an FFT to the loss map. The analysis uses cutoff frequency \(\min(H,W)//8\) and thresholds of 20% and 25% for low- and high-frequency regions, respectively.

The cached rendering of Eq. (15) does not fully agree with Algorithm 1, and some plus signs and norm superscripts are not reliably readable. This note therefore does not claim to provide an exact, directly reproducible weight formula. The supported mechanism is that bias generates spatial weights for the original FM objective; it should not be described as replacing the supervised velocity with bias or frequency-filtering the ADR loss.

A Worked Example

Consider a training image with \(t_0=0.2\) and \(t_1=0.6\), chosen only to illustrate the process. Construct two ideal states using the same image and noise, predict the velocity at 0.2, and advance by 0.4 time units. The resulting state generally differs from the ideal 0.6 state constructed directly from the real image.

ADR points from this biased state to the known endpoint and constrains the subsequent prediction's direction. FC compares predictions at the two 0.6 states, identifies spatial locations with stronger bias, and uses them to redistribute the FM loss at 0.2. Problems exposed at a later timestep thus feed back into earlier learning. This does not forcibly replace a predicted image with a real one or invoke an additional corrector at test time.

Loss & Training

The joint objective is a weighted sum of ADR and FC:

\[ \mathcal{L}=\beta_1\mathcal{L}_{\mathrm{ADR}}+\beta_2\mathcal{L}_{\mathrm{FC}},\qquad \beta_1=10,\quad\beta_2=1,\quad\alpha=1. \]

The FC term already contains reweighted standard FM supervision and should not be treated as a third additional term separate from FM. Ordered timestep pairs are sampled each iteration, and the standard linear path uses \(a_t=t\) and \(b_t=1-t\). SiT-B/4 is the default ablation backbone. Low-resolution datasets are processed in pixel space, while ImageNet-256 uses the latent space of a pretrained VAE-ft-EMA tokenizer.

The main experiments train from scratch on Ascend 910B devices. REPA, DDT, and OT-CFM experiments use A100 GPUs, and the ImageNet-512 REPA/DDT results fine-tune official ImageNet-256 checkpoints. The main text provides shared hyperparameters, but the detailed grid search is in an appendix absent from this cache, so parameter sensitivity has not been verified here.

Key Experimental Results

Main Results

The table below selects results from Table 1: class-conditional generation with SiT-B/4, 50 NFE, and 50K generated evaluation samples. Paired values are without/with classifier-free guidance (CFG); lower FID and sFID are better. Hour denotes training time on the corresponding hardware, not sampling time.

Dataset	Method	Iterations	Hour	FID, without/with CFG	sFID, without/with CFG
ImageNet-256	SiT	500k	166	61.64 / 31.64	12.20 / 8.44
ImageNet-256	IP	500k	166	60.45 / 29.84	12.49 / 8.80
ImageNet-256	SDSS	500k	228	61.25 / 30.29	13.23 / 8.95
ImageNet-256	MDSS, 4 steps	500k	228	63.66 / 31.90	14.38 / 10.05
ImageNet-256	DEFAR, comparable time	250k	144	60.40 / 29.81	9.05 / 7.35
ImageNet-256	DEFAR, same iterations	500k	289	56.39 / 28.34	9.50 / 7.84
CIFAR-10	SiT	250k	11	14.09 / 9.36	6.28 / 5.73
CIFAR-10	DEFAR, comparable time	120k	10	12.01 / 7.73	5.46 / 5.13

The comparable-time setting reduces FID with CFG by 1.83 on ImageNet-256 and 1.63 on CIFAR-10. Longer ImageNet training reaches 28.34, but its 289 hours must not be presented as equal-cost to SiT's 166 hours. Halving the iteration count also does not imply halving training time.

Ablation Study

The following also comes from Table 1 and restricts comparison to ImageNet-256, 500k iterations, SiT-B/4, and 50 NFE, avoiding the different training lengths in the CIFAR-10 ablations. ADR alone retains standard FM loss, while FC alone uses reweighted FM loss.

Config	Hour	FID, without/with CFG	sFID, without/with CFG	Note
SiT	166	61.64 / 31.64	12.20 / 8.44	Standard FM
DEFAR w/o FC	228	59.77 / 29.69	12.77 / 9.00	ADR alone
DEFAR w/o ADR	289	57.17 / 28.71	10.65 / 8.25	FC alone
DEFAR	289	56.39 / 28.34	9.50 / 7.84	ADR and FC jointly

The full model reduces FID with CFG by 1.35 relative to ADR alone and 0.37 relative to FC alone. This supports complementarity, although the first comparison differs in training time and cannot attribute the entire gain to FC's mechanism. ADR alone worsens sFID relative to the baseline, showing that directional regularization does not monotonically improve every generation metric.

Key Findings

• In Table 3, SiT-XL/2 FID with CFG improves from 5.88 to 5.74 at comparable training times of 2260 and 2300 hours. Extending training to 4640 hours reaches 4.14, which is not an equal-cost gain.
• Table 4 evaluates ImageNet-512 fine-tuning at 250 NFE with CFG: REPA improves from 3.99 to 3.46 FID and DDT from 1.90 to 1.82. These cannot be ranked directly against the 50-NFE results in Table 1.
• The frequency plots aggregate 50,000 ImageNet-256 samples and 50 uniformly discretized timesteps, providing empirical support for early low-frequency deficiency. The paper does not establish that this behavior must occur in every flow matching model.

Highlights & Insights

• The most useful shift is redefining the goal of perturbation training. Exposure to drifted inputs does not itself teach correction; ADR uses the known endpoint to address this distinction.
• FC does not require an external teacher to label low-frequency structure. Differences in the same network's responses to ideal and biased inputs provide feedback, although feedback quality remains limited by the network's capabilities.
• Frequency analysis is separated from the training implementation. FFT helps explain why spatial reweighting may improve structure, while the weights themselves come from model bias rather than fixed frequency-band enhancement.

Limitations & Future Work

• Training overhead is substantial: at the same 500k ImageNet-256 iterations, training time increases from 166 to 289 hours. Avoiding an additional deployment-time correction network does not eliminate learning cost.
• The single-step difference is only a proxy for exposure bias; the paper does not prove general equivalence to real drift over long trajectories. Longer rollouts or adaptive timestep intervals require separate evaluation.
• Better FID does not imply simultaneous improvement in quality and coverage. For example, comparable-time ImageNet-256 Precision with CFG falls from 0.542 to 0.531 while Recall rises from 0.524 to 0.532.
• Evidence for the low-frequency mechanism mainly concerns image tasks. Extending the approach to video, audio, or large text-conditioned models requires reexamining the frequency structure and bias proxy rather than directly transferring the conclusion.
• The cache contains damaged equation extraction and lacks the cited appendix. Exact weight implementation, normalization stability handling, and grid-search details still require the original typeset source or official code. These are reproducibility limits, not details to fill in with guessed formulas.

Related Work & Insights

• vs IP: IP uses additional input noise to improve tolerance. DEFAR generates drift using the current network's predictions, relating training signals to the model's behavior rather than only a fixed perturbation rule.
• vs SDSS/MDSS: Both expose training to inference states, but DEFAR adds a directional target from the biased state toward the endpoint instead of relying solely on the original fixed velocity supervision. The compared baselines adapt DDPM ideas to FM.
• vs DG and OT-CFM: DG adds external discriminator guidance at inference time, while OT-CFM changes noise-data coupling. DEFAR changes corrective supervision during training. In Table 2's five-seed mini-batch OT experiment, OT-CFM achieves Avg. FID 4.095±0.022 and adding DEFAR yields 3.960±0.034, supporting complementarity rather than guaranteeing gains with every sampler.

Rating

• Novelty: 4/5. Combines state-dependent directional supervision with bias-driven spatial weights, going beyond input perturbation alone.
• Experimental Thoroughness: 4/5. Covers datasets, model scales, sampling steps, and compatibility, but most main tables lack multi-seed intervals and costs require separate interpretation.
• Writing Quality: 4/5. The main narrative maps clearly to the algorithm. Damaged cached equations limit implementation verification; extraction quality should not be conflated with the paper's typesetting quality.
• Value: 4/5. Relevant to image generation with available training budget and a preference for unchanged deployment architecture, rather than a training-free sampling patch.