Salt: Self-Consistent Distribution Matching with Cache-Aware Training for Fast Video Generation¶

Conference: ECCV 2026
arXiv: 2604.03118
Code: To be confirmed
Area: Video Generation
Keywords: Video Generation, Diffusion Model Distillation, Distribution Matching Distillation, Semigroup Regularization, KV Cache-Aware Training

TL;DR¶

Salt proposes the SC-DMD framework, which fixes the compositionality deficit of Distribution Matching Distillation (DMD) in multi-step inference via a semigroup-defect regularizer. Combined with cache-aware mixed-step training, it further improves the stability of autoregressive video generation, significantly boosting generation quality under an extremely low inference budget of 2-4 NFEs.

Background & Motivation¶

Distilling video generation models to extremely low inference budgets (2-4 denoising steps) is key to achieving real-time deployment. Current mainstream approaches are divided into two paradigms: trajectory-style consistency distillation (e.g., PCM/LCM) and Distribution Matching Distillation (DMD). Trajectory-style methods force the student model to mimic the teacher's denoising trajectory through regression supervision, which performs well in the image domain. However, in video generation, a single condition can correspond to multiple plausible ways of motion continuation (e.g., the same motion can vary in speed, amplitude, or trajectory). Regression losses tend to collapse to conditional averages under such multi-modal uncertainty, leading to overly smooth frames and conservative motion. DMD, on the other hand, bypasses trajectory regression and uses distribution matching gradients to force the student's output distribution to approach the various modes of the teacher's reference distribution (mode-seeking). This often produces sharper and more expressive samples under extremely low step counts, making it a core design in current video distillation and real-time autoregressive generation systems.

However, DMD suffers from a fundamental structural flaw: its training objective is purely local—each noise level is supervised independently as a "single-step generator," without enforcing how these step-by-step behaviors interact when chained sequentially. The essence of few-step inference is precisely the composed rollout of learned denoising operators. When the operators at each step lack compositionality constraints, errors accumulate sequentially along the inference path. Consequently, increasing the number of denoising steps actually decreases generation quality—for example, leading to overexposure, loss of fine-grained textures, or mixed semantics of multiple objects. This paper refers to this issue as the "compositionality deficit" of DMD. In autoregressive generation scenarios, this problem is further amplified: each newly generated block depends on the KV cache of previous blocks, and errors from early blocks are encoded into the cache and drift step-by-step during the generation process, creating a vicious cycle.

The key insight of this work is to retain the distribution alignment capability of DMD as the primary driving force to maintain single-step sharpness, while introducing a lightweight regularizer to explicitly couple adjacent denoising operations, so that the operators at each step approximately satisfy the semigroup law of flow maps. Core Idea: The difference between the direct endpoint (single-step \(t_s \to t_e\)) and the composed endpoint (two-step \(t_s \to t_m \to t_e\)) of the student denoising operator \(\Psi_\theta\) is added to the DMD objective as a regularization term, fixing the multi-step compositionality deficit without sacrificing single-step sharpness. To address the issue of KV cache quality varying with inference steps in autoregressive generation, mixed-step training and cache-conditioned feature alignment are further proposed, allowing the generator to systematically expose itself to various cache quality conditions during training.

Method¶

Overall Architecture¶

At its core, Salt adds two conditionally activated regularization paths to the DMD framework, targeting different bottlenecks in non-autoregressive and autoregressive scenarios. In non-autoregressive distillation (using Wan 2.1 I2V 14B / T2V 1.3B as flow backbones), the paper adds a shortcut self-consistency loss (SC-DMD) over the standard DMD training grid to align the direct endpoint with the two-step composed endpoint. In autoregressive distillation (using Self Forcing / LongLive / Causal Forcing as backbones), the paper further introduces a mixed-step mechanism: in each iteration, a step count \(K\) is randomly sampled from \(\{2, 4, 8\}\). For \(K=8\), the SC-DMD regularization is activated to constrain the longest composition chain. For low-step rollouts where \(K \in \{2, 4\}\), cache-conditioned feature alignment is performed using the next denser scheduled step as a reference. The three losses (DMD, SC, Align) are jointly optimized on the same student generator conditioned on the step counts, without changing the inference pipeline or model architecture.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Noise + Text + KV Cache"] --> B["Mixed-Step Training<br/>K∈{2,4,8}"]
    B --> C["Multi-Step Denoising by K-Step Schedule"]
    C --> D["SC-DMD Regularization<br/>Only active when K=8"]
    C --> E["Cache-Conditioned Alignment<br/>Only active when K∈{2,4}"]
    C --> F["DMD Loss<br/>Active for all K"]
    D --> G["Joint Loss<br/>Updates Generator θ"]
    E --> G
    F --> G
    G --> H["Output Video Chunk"]

Key Designs¶

1. SC-DMD: Fixing Compositionality Deficit via Semigroup-Defect Regularization

The compositionality deficit of DMD stems from its purely local training signal—distribution matching at each noise level is performed independently, with no mechanism ensuring that direct denoising from \(t_s\) to \(t_e\) is equivalent to the composed denoising from \(t_s\) to \(t_m\) and then to \(t_e\). The SC-DMD regularization idea directly derives from the semigroup law of flow maps in probability flow ODEs: for an exact ODE flow map \(\Phi\), for any \(t > u > s\), we have \(\Phi^{t \to s} = \Phi^{u \to s} \circ \Phi^{t \to u}\). The paper defines the deviation of the student's single-step Euler operator \(\Psi_\theta\) from this law directly as a penalty term. Specifically, for triplets \((t_s, t_m, t_e)\) sampled such that \(t_s > t_m > t_e\), it computes the \(L_2\) difference between the direct endpoint \(x_{t_e}^{(1)} = \Psi_\theta^{t_s \to t_e}(x_{t_s})\) and the composed endpoint \(x_{t_e}^{(2)} = \Psi_\theta^{t_m \to t_e}(\Psi_\theta^{t_s \to t_m}(x_{t_s}))\), i.e., \(\mathcal{L}_{\text{SC}} = \mathbb{E}[\|x_{t_e}^{(1)} - x_{t_e}^{(2)}\|_2^2]\). Here, \(t_e\) is anchored to the inference grid (ensuring the regularization acts on endpoints actually cared about during inference), while \(t_m\) is randomly sampled from a denser training grid—since the time points where \(t_m\) resides already have sufficient DMD distribution matching supervision, the operators are relatively well-trained, making the composed endpoint a reliable reference. This carefully designed grid alignment avoids the issue in shortcut-style supervision where training grid mismatches cause "the composed branch to pass through under-optimized intermediate operators, regressing the direct endpoint towards an unreliable target." Experiments demonstrate that merely increasing the training grid density without adding SC regularization (DMD-4 vs. DMD-8) does not improve generation quality and even degrades minor metrics, whereas SC-DMD comprehensively outperforms the DMD baseline under 4-step inference, confirming that the gains come from explicit compositionality constraints.

2. Mixed-Step Autoregressive Training: Covering KV Cache Quality Distribution

In autoregressive video generation, the conditional inputs for each new chunk include the KV cache from previous chunks. Unlike static conditions such as text, the KV cache itself is dynamically generated by the model, and its quality varies with the step count of previous rollouts: high-step rollouts produce richer and more accurate caches, while low-step rollouts result in degraded, noisier caches. If training always uses a fixed step count \(K\), the model only experiences a single range of cache quality, leading to training-inference distribution mismatch when encountering different cache qualities during inference. More severely, the accumulation of autoregressive errors across chunks causes the KV cache to gradually drift away from the training conditional distribution, causing semantic degradation. Salt's solution is to systematically cover various cache qualities encountered during inference: in each iteration, a step count \(K\) is sampled from \(\{2, 4, 8\}\) with probabilities \(\{0.2, 0.4, 0.4\}\), and chunk-level autoregressive rollouts are executed with this schedule. This not only exposes both the generator and discriminator to various fidelity modes of the KV cache but also provides the necessary training baseline for applying SC-DMD to autoregressive scenarios—SC regularization is only activated on the longest composition chain with \(K=8\), where multi-step inconsistency is most prominent and the regularization benefit is maximized. Ablation studies clearly confirm the necessity of this design: simply overlaying the SC loss on the standard baseline degrades performance (Total 84.62 \(\to\) 83.83), and SC only becomes effective when combined with mixed-step rollouts (Total 83.83 \(\to\) 85.02), indicating that mixed steps expose the distribution shift under low-quality caches, giving SC-DMD the proper context to take effect.

3. Cache-Conditioned Feature Alignment: Transferring Weak Cache Outputs to Strong References

Mixed-step training exposes the model to various KV cache conditions, but generation quality under low step counts is still limited by poor KV caches. The paper introduces a cache-conditioned reference alignment loss to further bridge this gap: for low-step rollouts with \(K \in \{2, 4\}\), the next denser schedule (\(2 \to 4\) or \(4 \to 8\)) is used to construct a high-quality reference rollout (sharing the same input noise and text), and the low-step output is aligned with this high-step reference. The alignment adopts TRD-style relational feature space (relation alignment): frame-level spatial token features are extracted from intermediate feature layers, and an \(S \times S\) token relation matrix is computed for each frame (features are first \(L_2\)-normalized along the channel dimension before computing the inner product). Then, the \(L_1\) difference with a margin is used as the loss. This design with a margin \(\delta\) encourages the model to ignore small, uninformative relation discrepancies and only penalize components that significantly deviate from the reference relation pattern. The ablation results highly align with the design intent: the alignment loss slightly improves performance under the 4-step setup (about \(+0.1\) in Total with Semantic 80.65 \(\to\) 81.49) but provides significant help in the more extreme 2-step inference (Semantic improves from 80.65 to 81.49), as it precisely guides the most fragile low-step cache outputs to match high-quality references.

Loss & Training¶

The complete autoregressive training objective is \(\min_\theta [ \mathcal{L}_{\text{DMD}}(\theta; \psi) + \lambda_{\text{SC}} \cdot \mathcal{L}_{\text{SC}}(\theta) \cdot \mathbb{1}_{[K=8]} + \lambda_{\text{align}} \cdot \mathcal{L}_{\text{align}}(\theta) \cdot \mathbb{1}_{[K \in \{2,4\}]} ]\), where \(\lambda_{\text{SC}}=0.2\). DMD loss is active for all steps, SC loss is active only when \(K=8\), and the alignment loss is active only when \(K \in \{2, 4\}\). The generator and discriminator use the AdamW optimizer with learning rates of \(2 \times 10^{-6}\) and \(4 \times 10^{-7}\), respectively; the alignment loss is enabled after approximately 600 warm-up epochs. In the non-autoregressive setup, mixed-step training is not required, and \(\mathcal{L}_{\text{DMD}} + \lambda_{\text{SC}} \cdot \mathcal{L}_{\text{SC}}\) is optimized directly on an 8-point training grid.

Key Experimental Results¶

Main Results¶

Non-Autoregressive I2V (Wan 2.1 14B, 4 NFE, VBench-I2V)

Metric	PCM (8 steps)	DMD (4 steps)	LightX2V (4 steps)	Ours SC-DMD (4 steps)
I2V Score	93.63	93.09	93.50	93.90
Quality Score	78.52	78.89	80.92	80.86
Background Consistency	97.34	92.79	95.87	95.97
Motion Smoothness	98.24	97.99	97.89	98.37
Temporal Flickering	97.67	95.21	96.30	97.41
Imaging Quality	70.42	70.35	71.67	72.16

Autoregressive T2V (Wan 2.1 14B, 4 NFE, 5s, VBench-extend)

Model	Total Score	Quality Score	Semantic Score
Self Forcing	84.20	84.74	82.05
Ours - Self Forcing	84.47	85.27	81.28
LongLive	84.40	85.12	81.53
Ours - LongLive	84.93	85.41	83.00
Causal Forcing	84.62	85.41	81.47
Ours - Causal Forcing	85.08	85.96	81.59

Ablation Study¶

SC-DMD Core Ablation (Wan 2.1 1.3B T2V, 4-Step Inference)

Configuration	Quality	Semantic	Total	Spatial Relation	Description
DMD-8	84.05	76.50	82.54	67.13	Dense 8-point grid, no SC
DMD-4	84.39	76.36	82.78	69.49	Sparse 4-point grid baseline
SC-DMD	84.76	77.77	83.36	71.91	8-point grid + SC regularization

Autoregressive Components Ablation (Causal Forcing Backbone, 4-Step Inference)

Configuration	Total	Quality	Semantic	Description
Causal Forcing Baseline	84.62	85.41	81.47	Official baseline
+ Naive \(\mathcal{L}_{\text{SC}}\)	83.83	84.35	81.77	SC is harmful without mixed-step
+ Mixed-Step + \(\mathcal{L}_{\text{SC}}\)	85.02	85.91	81.48	Mixed-step is a prerequisite
+ Full Config	85.08	85.96	81.59	Alignment loss added

Key Findings¶

SC-DMD regularization is the primary source of improvement: in non-autoregressive ablations, DMD-4 performs on par with or even better than DMD-8 under 4-step inference, signifying that the gains stem from the explicit compositionality constraints rather than a denser grid.
In autoregressive scenarios, mixed-step rollout is a necessary prerequisite for SC-DMD to take effect—adding the SC loss alone degrades performance due to the training-inference distribution shift.
Cache-conditioned feature alignment offers the most significant gains under the extremely low 2-step inference budget (Semantic \(+0.84\)), which perfectly aligns with the design intent.
The paper directly measures the semigroup defect as a diagnostic metric; SC-DMD's defect value (\(0.0111\)) is lower than that of DMD (\(0.0135\)), providing quantitative evidence of improvement at the mechanism level.
In 30-second long video generation, Salt's gains extend to long sequences, showing clear improvements in semantic consistency (Semantic of Causal Forcing increases from 60.25 to 62.77).

Highlights & Insights¶

First to explicitly discover and name the compositionality deficit of DMD: Although DMD has become the de facto standard in video distillation, no prior work explicitly identified that its training objective is purely local and does not constrain multi-step composition behavior. The paper quantitatively characterizes this issue through counter-intuitive experiments where "increasing steps degrades quality" and establishes the semigroup defect as a direct diagnostic metric.
Simplicity and complementarity of semigroup-defect regularization: Merely adding an \(L_2\) endpoint alignment constraint on the Euler operator naturally complements DMD—the regularizer fixes multi-step compositionality while DMD retains single-step sharpness, with computational overhead of only one extra forward pass.
Treating KV cache as a step-varying conditional variable: This perspective is highly original. Mixed-step training allows the model to systematically cover the cache quality distribution, a strategy far better aligned with the inference distribution than "always training with high-quality cache."
Conditional activation logic of the training mechanisms: The two regularizers have specific activation windows (SC only for \(K=8\), Align only for \(K \in \{2,4\}\)), playing different regularization roles in different cache quality intervals without redundant overhead.
In-depth analysis of grid mismatch: The paper analyzes the causes of degradation in naive DMD + SCFM (grid mismatch during training causes the composed branch to pass through under-optimized operators), providing strong guiding value for future DMD improvement works.

Limitations & Future Work¶

SC-DMD relies on the availability of sampleable intermediate timesteps \(t_m\) on the training grid. In extremely low-step scenarios (e.g., 1-2 steps), the available intermediate steps decrease, which limits the coverage of the regularization.
The reference rollout in the alignment loss is also generated by the model itself, meaning it may still carry biases. When the reference cache itself is of poor quality, alignment might guide the model in the wrong direction.
The method focuses on video distillation. Although the design philosophy of semigroup regularization is theoretically transferable to image distillation or other few-step generation paradigms, cross-domain validation was not conducted in the paper.
Mixed-step training increases the computational overhead of each iteration (due to rollouts of different step counts), and although the paper states that the total number of training epochs is not high, the actual cost in large-scale model scenarios still needs evaluation.

vs. DMD (Distribution Matching Distillation): DMD performs independent distribution matching at each noise level without constraining multi-step composition. SC-DMD adds a semigroup-defect regularizer on top of it, preserving DMD's mode-seeking sharpness while fixing the compositionality.
vs. Trajectory-style consistency distillation (PCM / LCM / rCM): Trajectory-style methods force the student to mimic the teacher's trajectory using regression loss, which easily leads to conditional averaging under multi-modal video distributions. SC-DMD introduces the semigroup law as a regularizer rather than a primary objective, avoiding mode-averaging.
vs. Shortcut models (SCFM): Shortcut methods use compositional consistency as the primary self-distillation objective, but direct hybridization with DMD leads to severe degradation due to grid mismatch (Total 75.06). SC-DMD avoids this trap: SC and DMD operate on the same aligned grid.
vs. Self Forcing / Causal Forcing / LongLive: Salt brings consistent improvements without modifying the inference pipeline or model architecture, demonstrating the generalizability and practicality of the method.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ [Precisely identifies the long-ignored compositionality deficit in the DMD framework and proposes semigroup-defect regularization—a physically meaningful and clean solution]
Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Covers 2 non-autoregressive setups + 3 autoregressive backbones, with comprehensive evaluations across steps/lengths, direct diagnostic measurement of semigroup defect, and thorough ablation analyses]
Writing Quality: ⭐⭐⭐⭐⭐ [Clear problem definition, progressively driven motivation, well-structured method explanations, and in-depth analysis of key design decisions (grid design, differences between SC and shortcut) in the appendix]
Value: ⭐⭐⭐⭐⭐ [As DMD is the current de facto standard for video distillation, fixing its fundamental defect has broad practical significance; cache-aware training also provides an important reference for autoregressive video generation pipelines]