MoGAN: Improving Motion Quality in Video Diffusion via Few-Step Motion Adversarial Post-Training¶

Conference: ECCV2026
Paper: ECCV Paper
Area: Video Generation
Keywords: video diffusion, optical-flow discriminator, adversarial post-training, distribution matching distillation, motion quality

TL;DR¶

MoGAN combines optical-flow discrimination with distribution matching on a three-step distilled Wan2.1 model, raising VBench Motion Score from DMD's 0.859 to 0.973 while retaining the three-step inference path.

Background & Motivation¶

Video diffusion models can generate clear individual frames, but frame fidelity does not guarantee believable motion throughout a clip. Limbs can deform between frames, backgrounds can flicker, and moving objects can leave ghosting artifacts. Standard denoising or flow-matching objectives do not directly require realistic motion statistics, so lower reconstruction error need not imply better temporal behavior. Figure 1 illustrates this mismatch with different random seeds from the same model: a sample with a loss of approximately 0.36 has worse motion than one with a loss of approximately 0.39. This example demonstrates that training error and motion quality are not equivalent, rather than establishing a universal causal result for every diffusion model.

Few-step distillation amplifies another problem: reducing motion can produce a high smoothness score. In the main experiment, DMD is smoother than its teacher on VBench but reduces dynamics degree from 0.83 to 0.73. Simply removing flicker can therefore yield an almost static image sequence without solving natural action generation. Trajectory control and physical simulation offer external constraints but typically require additional inputs; preference- or reward-based post-training depends on evaluators that reliably judge continuous motion. The authors particularly question sparse-frame vision-language rewards for fine-grained temporal changes, rather than claiming that all reinforcement learning methods fail to improve motion.

MoGAN shifts the supervision space from appearance to continuous optical flow, asking a discriminator to learn how displacement fields evolve in real videos. However, optimizing flow realism alone can encourage jitter or noise, making a teacher-distribution constraint necessary. The three-step distilled model supplies not only faster inference but also sufficiently clean predictions for optical-flow estimation. Core Idea: recover motion dynamics through adversarial learning over optical-flow sequences, while distribution matching and discriminator regularization prevent “more motion” from becoming “more noise.”

Method¶

Overall Architecture¶

Inputs comprise text prompts, noisy latent variables, and real motion videos used only during training. The generator is initialized from DMD-distilled Wan2.1-T2V-1.3B and produces video-latent predictions along a three-step path. During training, windowed decoding produces contiguous pixel frames, and frozen RAFT extracts optical flow between adjacent frames. Real videos follow the same motion-representation path; the discriminator distinguishes real from generated motion, while the generator receives differentiable adversarial feedback. Meanwhile, a frozen teacher and a trainable score estimator for the generated distribution provide DMD constraints against excessive appearance and text-conditioning drift. The diagram shows training data flow and feedback; inference retains only three-step generator sampling and ordinary video decoding, without RAFT or the discriminator.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    INPUT["Text and noise"] --> WARM["Few-Step Warm-Up"]
    WARM --> WINDOW["Windowed Decoding"]
    WINDOW --> FLOW["Frozen RAFT<br/>flow extraction"]
    REAL["Real motion videos"] --> FLOW
    FLOW --> DISC["Optical-Flow Discriminator"]
    DISC --> JOINT["Joint Regularized Training"]
    TEACHER["Frozen teacher and<br/>generated-distribution score estimator"] --> JOINT
    JOINT -.->|Training feedback| WARM
    WARM -->|Inference: three-step sampling and ordinary decoding| VIDEO["Generated video"]

Key Designs¶

1. Few-Step Warm-Up: give optical-flow estimation a credible image signal

Optical-flow estimation relies on recognizable correspondences between adjacent frames; feeding heavily corrupted diffusion states directly to RAFT can produce displacement fields unrelated to object motion. MoGAN therefore starts with 5K steps of DMD warm-up so that intermediate predictions from the three-step generator are already sufficiently clear. Post-training obtains clean predictions at the distilled discrete timesteps instead of unrolling every teacher diffusion step to compute a motion loss. The algorithm also uses backward simulation to construct samples from the current generator, then adds noise to create states required for updates. Few-step generation is thus part of what makes flow supervision usable, not merely a reduction in the original model's sampling count.

The generator still receives the user's text condition, preserving the text-to-video task. The adversarial branch uses a separate fixed discriminator prompt, which should not be confused with the generator's user prompt. Inference remains three-step, and the additional motion-training components are not deployed inside the sampling loop. This design also limits the experimental claim: the paper studies post-training an existing few-step generator, not training a video GAN from scratch.

2. Windowed Decoding: make pixel-space motion supervision differentiable back to the generator

The generator operates in latent space, whereas RAFT requires pixel frames, so motion gradients must pass through the video decoder. Wan's decoder maintains recurrent state across chunks, making full decoding with the entire backward graph memory-intensive. The authors combine contiguous chunk subsampling, truncated back-propagation through time, and gradient checkpointing, computing flow and adversarial losses only within a contiguous window. They detach the recurrent state at the window boundary and terminate decoding once the target window is covered, avoiding computation for irrelevant frames. The contiguous window retains adjacent-frame relationships instead of reducing motion judgment to independent still-image scores.

The main configuration uses 12 latent chunks corresponding to 49 pixel-space frames. The decoder and RAFT parameters are frozen, but freezing parameters does not detach gradients with respect to their inputs. Generator updates still propagate through pixel decoding and optical-flow estimation, with truncation applied only at the specified recurrent boundary. This explains why the motion-generator training branch remains expensive: fewer inference steps do not remove the cost of pixel-space back-propagation. Long-range dependencies outside the window are not fully covered by this local supervision, which is a practical boundary of the method.

3. Optical-Flow Discriminator: judge motion sequences rather than image style

RAFT produces horizontal and vertical displacement for adjacent frames, so a video containing \(T\) frames normally yields \(T-1\) two-dimensional flow fields. The authors append a final flow map to match the video's temporal length and add per-pixel displacement magnitude as a third channel. The discriminator consequently receives a motion representation containing displacement direction and strength rather than raw RGB content. After encoding, this representation enters a discriminator adapted from Wan2.1 DiT, exposing temporal changes in flicker, local distortions, and action amplitude more directly. Optical flow reduces appearance interference, but it is not a physical ground truth independent of image texture or estimator bias.

The discriminator uses the first 16 layers of the pretrained DiT and attaches lightweight prediction branches at layers 7, 13, and 15. Each branch aggregates features through auxiliary-token cross-attention and a small MLP; branch outputs are concatenated and passed through another MLP to produce a real/fake logit. Its diffusion timestep is fixed at \(t^*=0\), and its text condition is fixed to “a video with good motion.” Fixing these inputs focuses discrimination on motion realism rather than changing the criterion with user text or random diffusion noise. Real and generated videos share the same representation, so supervision concerns the motion distribution rather than reproduction of a specified trajectory.

4. Joint Regularized Training: constrain the teacher distribution to prevent adversarial shortcuts

If the motion discriminator becomes too strong or can be fooled by high-frequency flow, the generator may obtain better scores through micro-jitter rather than natural action. MoGAN retains DMD's teacher-distribution constraint and continually updates the score estimator for the current generated distribution. The teacher stays frozen, while the generated-distribution score estimator is trained with flow matching and updated more frequently than the generator to reduce estimation lag. The generator receives distribution-matching and motion-adversarial feedback separately; DMD is not a pixelwise reconstruction loss paired with a particular real video. The algorithm distinguishes DMD generator updates, generated-distribution score updates, motion-discriminator updates, and motion-generator updates, rather than minimizing one scalar with every network simultaneously.

The adversarial objective uses logistic GAN losses; the following restates Equations (4)–(5) by network role to avoid the prose's confusing parameter-subscript descriptions. Here \(D_\phi\) is the motion discriminator, \(o_r\) and \(o_g\) denote real and generated flow, and \(g\) is softplus.

\[ g(a)=\log(1+e^a),\qquad \mathcal{L}_D=\mathbb{E}[g(-D_\phi(o_r))+g(D_\phi(o_g))],\qquad \mathcal{L}_G=\mathbb{E}[g(-D_\phi(o_g))]. \]

The discriminator also receives regularizers called R1/R2 in the paper, applied to real and generated flow inputs respectively. Equations (6)–(7) are written as squared differences between discriminator outputs on original and Gaussian-perturbed inputs, so they are not silently replaced here with standard explicit input-gradient penalty formulas. These constraints limit excessive sensitivity to small perturbations and serve a different purpose from DMD's constraint against departing from the teacher distribution. Removing either type worsens results in the ablations, with removing DMD particularly damaging both motion and appearance.

Loss & Training¶

Post-training uses AdamW with a learning rate of \(1\times10^{-5}\) for 800 steps, excluding the preceding 5K warm-up steps. Motion-generator and discriminator loss weights are both 0.5, R1/R2 weights are both 0.3, and the perturbation parameter is \(\sigma=0.01\). Generator batch size is 16 and discriminator batch size is 64; the latter does not retain the same expensive generator backward path. The training prompt set contains 5K diverse texts, and real motion data comprise 15K videos curated for rich dynamics. The main text does not provide a complete public inventory sufficient to reconstruct this curated dataset; these counts are not a reproducible data recipe. The paper reports post-training within 16 hours on 8 H200 GPUs with FSDP and activation checkpointing enabled. In Table 3, the motion-generator branch takes 15.6 seconds and peaks at 82 GB; branch batch sizes differ, so their timings should not simply be summed to infer end-to-end throughput.

Key Experimental Results¶

Main Results¶

The following selects results from Table 1 on page 11; both prompt sets use VBench metrics, with 5 seeds per prompt and matching seeds across models. Smoothness is shown as a percentage; dynamics degree, Motion Score, aesthetics, and image quality retain their reported scales, with higher values better. The authors import FlowMo results from its paper rather than rerunning it with the matched seeds used for the two core baselines, so it is not included here.

Dataset	Model	Steps	Smoothness (%)	Dynamics Degree	Motion Score	Aesthetic	Image Quality
VBench	Wan2.1	50	98.0	0.83	0.905	0.57	0.66
VBench	DMD-only	3	98.8	0.73	0.859	0.57	0.69
VBench	MoGAN	3	98.6	0.96	0.973	0.59	0.68
VideoJAM-Bench	Wan2.1	50	97.9	0.85	0.915	0.55	0.63
VideoJAM-Bench	DMD-only	3	98.5	0.81	0.898	0.57	0.65
VideoJAM-Bench	MoGAN	3	98.5	0.98	0.983	0.57	0.66

Section 5.2 defines Motion Score as the arithmetic mean of normalized motion smoothness and dynamics degree:

\[ M=\frac{S/100+D}{2}. \]

Here \(S\) is smoothness reported as a percentage and \(D\) is dynamics degree; smoothness uses frame-interpolation evaluation, while dynamics uses optical-flow evaluation. This penalizes smooth-but-static videos without directly establishing physical correctness or semantic accuracy of the requested action.

Ablation Study¶

The following reproduces Table 2 on page 13 with the same automatic metrics as the main experiment; its full-model values match the VideoJAM-Bench row, but the caption does not explicitly restate the prompt set, so that dataset attribution is not independently confirmed. “Without optical flow” replaces adversarial discrimination with video-latent-space discrimination rather than disabling all adversarial training.

Config	Smoothness (%)	Dynamics Degree	Motion Score	Aesthetic	Image Quality
Full model	98.5	0.98	0.983	0.57	0.66
Without DMD	99.1	0.35	0.674	0.50	0.55
Without R1/R2	95.1	0.86	0.905	0.55	0.64
Without optical flow	98.0	0.85	0.915	0.56	0.67

Key Findings¶

Direct subtraction in Table 1 gives a VBench Motion Score gain of 0.114 over DMD; smoothness decreases slightly while dynamics degree rises from 0.73 to 0.96, so the benefit is not further smoothing.
Removing DMD in Table 2 reduces dynamics degree from 0.98 to 0.35 despite 99.1% smoothness; the authors attribute this to failed generation confounding the metrics, not genuinely superior smooth video.
The human study on pages 12–13 uses 148 videos, 5 independent annotations per video, and 3 criteria, yielding 2,220 responses per comparison setting; the abstract reports motion preferences of 52% versus 38% against the teacher and 56% versus 29% against DMD.
Numerical caveat: prose on pages 11–12 gives DMD's VideoJAM-Bench Motion Score as 0.830, whereas Table 1 gives 0.898; this note uses the table. The no-DMD score of 0.674 in Table 2 also differs from 0.6705 computed from its displayed components, so the reported value is retained without correction.
The paper labels some Motion Score gains as dynamics-degree gains; those conflicting percentages are not repeated here. Complete numerical labels in the human-study bar chart were not reliably extracted into the text cache, so remaining preference rates are not invented.

Highlights & Insights¶

The discrimination space itself is an inductive bias: optical-flow inputs bring learning closer to continuous motion rather than relying on a generic RGB discriminator to allocate capacity to time.
Few-step distillation and motion post-training support one another: the former supplies clean, inexpensive predictions, while the latter repairs the motion distribution's tendency toward static output.
DMD does more than preserve image quality: its ablation identifies it as a constraint against flow-noise shortcuts, suggesting that motion supervision should not be designed independently of distributional stability.
Reader insight: other video post-training tasks could seek differentiable representations that expose their target failure modes and combine them with teacher constraints; this is a transfer hypothesis, not a result demonstrated in the paper.

Limitations & Future Work¶

The authors explicitly acknowledge that two-dimensional flow can misinterpret occlusion, out-of-plane motion, and fast articulation, and is unreliable for very small motion or complex depth changes.
Pixel decoding and RAFT back-propagation still cost computation; latent motion surrogates, geometrically consistent motion fields, and physical priors are proposed future directions rather than current components.
Main experiments center on Wan2.1-T2V-1.3B; the discussion of overhead for larger models is not evidence of demonstrated cross-scale results.
Human evaluation finds slightly weaker text alignment than the 50-step teacher; retaining three-step speed does not imply exceeding the teacher on every dimension.
The cache contains the complete main-text method, results, and references but not the appendix mentioned in the paper; finer architectural details, data curation, and equation-layout ambiguities cannot be further verified from it.

vs DMD / DMD-v2: distribution matching primarily preserves the few-step generation distribution; MoGAN retains it as a constraint while adding sequential flow discrimination to address motion defects.
vs VideoJAM: VideoJAM strengthens generation through joint appearance-motion representations; MoGAN post-trains an existing few-step model without adding its motion-discrimination branch to inference.
vs APT and video-latent-space GANs: general adversarial post-training targets perceptual realism, whereas MoGAN discriminates flow sequences; Table 2 shows that simply reverting to video latent space does not deliver equivalent motion gains.
vs optical-flow-guided prompt optimization: the cited approach optimizes prompt embeddings using a single flow image, whereas MoGAN updates the generator using flow sequences, changing both the supervision scope and the optimized component.

Rating¶

Novelty: 4/5. Sequential flow discrimination, few-step generation, and distribution constraints form a targeted combination, although the underlying components have established precedents.
Experimental Thoroughness: 4/5. Two benchmarks, human preferences, and key ablations provide complementary evidence, while one backbone and incomplete data disclosure limit generalization.
Writing Quality: 3/5. The main procedure is understandable, but numerical table–prose discrepancies, cross-references, and some equation descriptions need revision.
Value: 4/5. Repairing the static-output tendency of few-step video generation without additional inference components offers a useful post-training direction.