EchoStyle: Unlocking High-Fidelity Video Stylization with Reverse Data Synthesis¶

Conference: ECCV 2026
Paper: ECCV Official
Project: https://echostyle2026.github.io/
Area: Video Generation
Keywords: Video Stylization / Diffusion Transformer / Reverse Data Synthesis / Long Video Generation / Video-to-Video

TL;DR¶

EchoStyle addresses content leakage from style reference images, paired data scarcity, and temporal drift in extended sequences by formulating text-driven video stylization as a unified video-to-video generation task on Wan2.2-I2V, curating the 20k-pair V-Style20k dataset via a reverse data synthesis pipeline, and enabling minute-level temporally consistent stylization through an init-follow-mode mechanism with sliding-window inference.

Background & Motivation¶

Video stylization aims to transform input raw video sequences into designated artistic styles and holds tremendous potential across creative media, animated film production, and digital advertising. Early paradigms heavily relied on CNNs or ViTs prioritizing real-time latency at the expense of expressiveness and visual fidelity. Recent advances in diffusion models have motivated two dominant directions: adapting pre-trained text-to-video models via post-training (such as StyleCrafter and StyleMaster) and training-free keyframe editing combined with cross-frame feature injection (such as AnyV2V and FreeVis). However, the vast majority of existing methods adopt reference images as the style prior. Because a reference image inevitably encapsulates style-irrelevant content, textures, and geometry, it triggers severe content leakage into stylized videos, while feature-injection pipelines frequently disintegrate under strong motion dynamics, causing severe flickering and structural distortion.

The core tension underlying these shortcomings is the acute scarcity of high-quality paired video training data and the fundamental distribution contamination that plagues conventional forward data synthesis. Learning a robust video stylization model requires strict pairs sharing identical geometric motion but rendered in contrasting styles. When leveraging current generative models to synthesize stylized videos from real-world inputs (forward synthesis), pre-trained models struggle with stylistic diversity and introduce severe temporal artifacts, flickering, and style drift. Using such flawed synthetic videos as target ground truth establishes a contaminated training objective and firmly caps the model's performance. Furthermore, existing research remains largely confined to short clips ($\le 5\text{s}$) due to GPU memory constraints and lacks a robust recurrent mechanism for long-form generation, where per-segment errors rapidly compound into dramatic drift.

To overcome these structural bottlenecks, this work takes an inverted angle of attack: replacing reference image conditioning with text prompts to steer clean concept-level styles, and completely reversing the synthesis trajectory by using authentic human-created artistic videos as targets while synthesizing their realistic counterparts as source inputs. The core idea is: treat authentic stylized videos as the pure target distribution, leverage the intrinsic realism bias of pre-trained models to reverse-synthesize realistic reference videos for building the 20k-pair V-Style20k dataset, and perform unified channel-concatenated latent visual alignment on Wan2.2-I2V coupled with an init-follow-mode sliding-window strategy to unlock temporally coherent, arbitrary-length video stylization.

Method¶

Overall Architecture¶

EchoStyle models text-driven video stylization as an end-to-end, scalable video-to-video (V2V) generation pipeline. The network receives a text prompt $p$ encompassing both target style and scene content, an input reference video $\mathbf{R}$ providing motion and structural content, and a frame modification mask $\mathbf{M}$. All video conditions and the noisy target video latents are mapped via a pre-trained VAE into a shared latent space and concatenated along the channel dimension before entering a Diffusion Transformer (DiT) backbone initialized from Wan2.2-I2V-14B. To scale seamlessly to extended durations, the framework introduces an init-follow-mode dual training scheme that seamlessly transitions into sliding-window autoregressive inference at test time.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Conditions<br/>Reference Video R + Prompt p + Mask M"] --> B["Reverse Data Synthesis Pipeline<br/>Stylized Video→Endpoint Realization→Motion V2V→VLM QA"]
    B --> C["Multi-Channel Latent Visual Alignment<br/>VAE encodes R, V, M into channel-concatenated DiT input"]
    C --> D["Init-Follow-Mode Mechanism<br/>Short clip init-mode + 16-frame overlap follow-mode training"]
    D --> E["Sliding-Window Autoregressive Inference<br/>Recurrent clip inference with overlap concatenation"]
    E --> F["High-Fidelity Temporally Coherent Stylized Video"]

Key Designs¶

1. Reverse Data Synthesis Pipeline: Inverting synthesis direction to preserve target distribution purity To resolve the fundamental limitation where forward synthesis contaminates the target learning distribution with severe artifacts and style drift, EchoStyle inverts the data creation paradigm. It collects authentic, human-crafted stylized video clips as target ground truth $\mathbf{V}$, and reverse-synthesizes corresponding realistic videos as source inputs $\mathbf{R}$. This design strategically capitalizes on the strong realism bias of pre-trained models, converting an architectural bias into a data generation asset. The automated pipeline operates in three sequential stages: first, endpoint image realization extracts the initial and final frames from the stylized video and transforms them into photorealistic endpoint anchors using Qwen-Image equipped with an Anything-to-Real LoRA; second, motion-coherent V2V generation extracts depth priors via Depth Anything from the stylized clip, feeding depth maps and the two realistic endpoint frames into VACE to synthesize a realistic video that precisely mirrors the scene's dynamic motion and camera trajectory; third, automated VLM quality assessment and filtering employs Qwen-VL to evaluate candidates across visual realism, motion plausibility, and fine-grained cross-modal correspondence, discarding any distorted or drifting samples. Through this pipeline, the authors establish V-Style20k, the first large-scale paired dataset comprising over 20,000 video pairs at $480 \times 832$ resolution spanning 14 prevalent artistic genres (such as Japanese anime, watercolor, ink wash, and Pixar 3D), each paired with detailed descriptions.

2. Multi-Channel Latent Visual Alignment: Compact channel concatenation for content-style re-fusion Rather than adopting cumbersome dual-branch ControlNet architectures or slow-converging In-context frame concatenation, EchoStyle introduces a streamlined channel-wise alignment scheme. The reference video $\mathbf{R}$, target stylized video $\mathbf{V}$, and mask $\mathbf{M}$ are compressed into latent representations $\mathbf{z}_R$, $\mathbf{z}_V$, and $\mathbf{z}_M$ via a shared pre-trained VAE. During the diffusion process at forward timestep $t \in [1, T]$, Gaussian noise is added to $\mathbf{z}_V$ to obtain noisy latent $\mathbf{z}_V^{(t)}$. These tensors are concatenated along the channel dimension to form the unified model input: $$\mathbf{z}_{\text{in}} = \text{Concat}(\mathbf{z}_V^{(t)}, \mathbf{z}_R, \mathbf{z}_M)$$ This composite latent is fed into the Wan2.2-I2V-14B Diffusion Transformer backbone. Conditioning signals, including timestep $t$ and text prompt $p$, are modulated via adaptive layer normalization (adaLN) and injected via cross-attention layers. This visual alignment avoids rigid pixel-level feature fusion artifacts found in ControlNet and prevents token sequence explosion inherent in In-context conditioning, enabling efficient and robust re-fusion of source geometry with prompt-driven artistic style.

3. Init-Follow-Mode Mechanism: Dual-mode training and sliding-window inference for long-sequence stability To overcome sequence length barriers and eliminate transition discontinuities in extended generation, EchoStyle establishes an init-follow-mode paradigm. During training, the framework probabilistically alternates between two modes. In init mode (sampled with 80% probability), the model conditions on $\mathbf{z}_R$ with an all-zero mask $\mathbf{M}=0$ to master standalone short-clip stylization. In follow mode (sampled with 20% probability), the framework constructs a temporal reference video $\mathbf{T}$ by replacing the initial $x$ frames (set to $x=16$) of $\mathbf{R}$ with the corresponding stylized frames from ground truth $\mathbf{V}$, while setting the first $x$ frames of mask $\mathbf{M}_x$ to 1: $$\mathbf{z}_{\text{in}} = \text{Concat}(\mathbf{z}_V^{(t)}, \mathbf{z}_T, \mathbf{z}_{M_x})$$ This dual-mode strategy trains the network to utilize pre-existing stylized segments as reliable temporal priors. During inference, a long video is partitioned into overlapping segments $\{\mathbf{R}_0, \mathbf{R}_1, \dots, \mathbf{R}_n\}$ sharing an overlap of $x$ frames. The starting segment $\mathbf{R}_0$ is stylized via init mode, whereas subsequent segments $\mathbf{R}_i$ ($i > 0$) take the trailing $x$ stylized frames $\mathbf{O}_{i-1}[-x:]$ from the preceding output as visual guidance under follow mode: $$\mathbf{O}_i = \Phi(\mathbf{R}_i, \mathbf{M}_x, p_i, \mathbf{O}_{i-1}[-x:])$$ The final extended video is generated by concatenating non-overlapping segments along the temporal axis: $\mathbf{O}_{\text{long}} = \mathbf{O}_0 \mathbin{\Vert} \sum_{i=1}^{n} \mathbf{O}_i[x:]$. This clip-guided mechanism significantly reduces single-frame error accumulation and ensures seamless temporal coherence across minute-long videos.

Loss & Training¶

The framework freezes the VAE and text encoders, fine-tuning the Wan2.2-I2V-14B DiT backbone with low-rank adaptation (LoRA, rank $r=64$, scaling $\alpha=32$ across all linear layers). Training minimizes the standard flow-matching loss between the predicted velocity field and ground-truth vectors: $$\mathcal{L}_{\text{LoRA}} = \mathbb{E}_{t, \mathbf{z}_V, \boldsymbol{\epsilon}} \left[ \left\| \boldsymbol{v}_\theta(\mathbf{z}_{\text{in}}, t, p) - (\mathbf{z}_V^{(0)} - \boldsymbol{\epsilon}) \right\|^2 \right]$$ Experiments are conducted on 16 NVIDIA A100 GPUs using bfloat16 mixed-precision and the AdamW optimizer ($\beta=(0.9, 0.999)$, weight decay 0.01, constant learning rate $2 \times 10^{-5}$). Batches are organized with spatial resolution $480 \times 480$ and temporal bucket lengths from 33 to 69 frames at 16 FPS. Optimization runs for 20,000 steps (~80 GPU hours). Text prompts contain full captions (style instructions plus detailed content) with 80% probability and pure style instructions with 20% probability, boosting general instruction adherence.

Key Experimental Results¶

Main Results¶

Quantitative benchmarking is performed on a 50-sample evaluation set spanning nine distinct artistic genres, compared against premier commercial proprietary models (Runway, Kling-O1, Seedance 2.0) and open-source baselines (VACE-Depth, VACE-Flow, AnyV2V). Style similarity win rate is evaluated via Gemini-3-pro pairwise comparison, style consistency via CSD score, and video quality through VBench metrics.

Table 1: Quantitative comparison between EchoStyle and SOTA methods (bold denotes best, underline denotes second-best)

Method	Style Sim.	Style Consis.	Motion Smooth.	Imaging Qual.	Static Qual.	Dynamic Qual.	Aesthetic Qual.
AnyV2V	0.351	0.877	0.957	0.654	0.733	0.611	0.815
VACE-Depth	0.280	0.805	0.972	0.687	0.845	0.700	0.837
VACE-Flow	0.394	0.819	0.960	0.681	0.856	0.732	0.852
Runway	0.239	0.860	0.982	0.704	0.721	0.637	0.747
Kling-O1	0.809	0.883	0.984	0.746	0.879	0.775	0.806
Seedance 2.0	0.628	0.831	0.980	0.725	0.860	0.741	0.885
EchoStyle (Ours)	0.802	0.895	0.966	0.696	0.850	0.783	0.878

Table 2: Comparison of Human Preference (bold denotes best, underline denotes second-best)

Method	Style Similarity	Style Consistency	Content Preservation
AnyV2V	0.490	0.768	0.790
VACE-Depth	0.489	0.779	0.850
VACE-Flow	0.472	0.765	0.855
Runway	0.386	0.849	0.844
Kling-O1	0.815	0.853	0.833
Seedance 2.0	0.792	0.821	0.889
EchoStyle (Ours)	0.826	0.870	0.860

Ablation Study¶

Ablations systematically assess the visual alignment architecture and the init-follow-mode mechanism:

Table 3: Ablation study on model architecture and temporal conditioning

Configuration	Key Observations & Failure Modes	Assessment
ControlNet Branch	Rigid pixel-level feature fusion creates localized geometric artifacts and structural stiffness	Suboptimal generation quality
In-context Conditioning	Concatenating reference frames along the token axis doubles sequence length, slowing convergence severely	Excessive compute overhead
Visual Latent Alignment (Ours)	Direct multi-channel concatenation in VAE space balances parameter efficiency, fidelity, and convergence	Optimal architecture
w/o Follow-Mode	Lacks temporal prior supervision, triggering severe boundary flickers and color shifts across recurrent segments	Boundary discontinuities
Full Init-Follow Mechanism (Ours)	16-frame overlap conditioning bridges recurring segments, ensuring uninterrupted temporal stability	Smooth long-sequence generation

Key Findings¶

Superior dynamic and aesthetic consistency: EchoStyle achieves top ranking in objective dynamic quality (0.783) and style consistency (0.895), outperforming leading commercial models Kling-O1 and Seedance 2.0. In human blind tests, EchoStyle achieves the highest style similarity (0.826) and consistency (0.870).
Style decay in keyframe-guided baselines: First-frame guided models like VACE suffer from severe "style loss" over time, progressively converging back to non-stylized raw footage as temporal distance increases, while AnyV2V is architecturally constrained to clips under 16 frames.
Artifacts in closed-source models: Kling-O1 exhibits simplified textures and flatter lighting in complex environments alongside progressive semantic drift, whereas Seedance 2.0 struggles to reproduce high-frequency brush strokes (such as ink wash and impasto oil painting) and exhibits style decoupling between subjects and backgrounds.

Highlights & Insights¶

Inverted data synthesis philosophy: Rather than fighting synthetic artifacts in generated artistic styles, EchoStyle anchors real human art as ground truth and exploits the natural realism bias of foundational video models to generate realistic reference inputs, achieving clean distribution alignment.
Streamlined channel-wise latent fusion: Avoiding cumbersome auxiliary networks and token-doubling contexts, EchoStyle relies on clean channel concatenation in the VAE latent space, fully unlocking the generative prior of Wan2.2-I2V with lightweight LoRA tuning.
Init-follow sliding-window paradigm: Introducing a simple 16-frame temporal condition in 20% of training iterations equips short-video diffusion models with autoregressive long-form capabilities, preventing boundary flicker and visual drift across minute-long clips.

Limitations & Future Work¶

Dependence on depth and motion quality in reverse synthesis: If the source stylized video contains severe lens distortion, non-rigid occlusions, or ambiguous depth maps, reverse V2V generation may yield minor motion misalignments, degrading training pair correspondence.
Inference latency for extended videos: Generating multi-minute videos requires successive autoregressive DiT sampling on a 14B model; integrating few-step flow distillation or consistency models could substantially lower computational latency for real-time production.
Localized and multi-style compositional editing: Current text prompts steer global visual styles; generalizing the mask and condition mechanism toward regional multi-style composition or interactive brush control represents a promising next step.

vs AnyV2V / FreeVis: Training-free methods rely on image editors and cross-frame attention injection, limiting generation to $<16$ frames and suffering from structural collapse under dynamic motion. EchoStyle trains end-to-end on paired video data with an init-follow mechanism, providing robust temporal stability.
vs StyleCrafter / StyleMaster: Earlier diffusion post-training relied on image datasets or reference images that leak redundant content. EchoStyle operates via pure text prompts for concept-level style transfer and trains on genuine video-to-video pairs from V-Style20k.
vs Kling-O1 / Seedance 2.0: Proprietary commercial solutions exhibit simplified textures, style-content decoupling, and long-range semantic drift. EchoStyle rivals commercial systems in aesthetic quality while outperforming them in style consistency and dynamic fidelity.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ Inverting data synthesis turns model bias into a major asset; the init-follow long-video mechanism is elegant and practical.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive benchmarking against top proprietary engines and open-source models across objective VBench/CSD metrics and human preference studies.
Writing Quality: ⭐⭐⭐⭐⭐ Clear motivation, rigorous methodological formulation, and well-structured comparative insights.
Value: ⭐⭐⭐⭐⭐ Provides an open-source high-fidelity text-driven video stylization paradigm along with a valuable 20k paired video benchmark.