StreamEdit: Training-Free Video Editing via Few-Step Streaming Video Generation¶

Conference: ECCV 2026
arXiv: 2605.21466
Code: https://dsl-lab.github.io/StreamEdit/
Area: Video Generation / Video Editing
Keywords: Video Editing, Few-Step Sampling, Streaming Generation, Attention Manipulation, Training-Free

TL;DR¶

StreamEdit reformulates training-free video editing from the dominant "data-to-data" paradigm to a "noise-to-target under source conditions" generation problem. Directly integrated into pre-trained streaming video generators (Self Forcing / LongLive), it uses dual-branch few-step sampling + a self-attention bridge + cross-attention grounding/enhancement + source-oriented guidance to achieve faster, structure-preserving, and highly reliable editing of arbitrary-length videos within 5-15 steps.

Background & Motivation¶

In recent years, training-free visual editing has achieved impressive results on images, 3D, and videos. The mainstream approaches follow a "data-to-data" (\(data\rightarrow data\)) pipeline: either inversion-based, which routes through a circuitous source -> noise -> target path by first inverting source samples back to noise before denoising under target conditions; or inversion-free, which uses noisy auxiliary samples to approximate the source -> target transition trajectory. However, both classes of methods struggle with complex, high-dimensional video data. Inversion-based methods introduce extra iterations and cumulative errors, often distorting edit-irrelevant regions in complex videos (e.g., causing backgrounds to become blurry). Inversion-free methods slowly reveal visible edits over many rounds because early samples remain closely bonded to the source video. Furthermore, their irregular sampling trajectories require overly cautious step sizes, making them slow and unstable for difficult cases. Ultimately, video editing remains bottlenecked by high computational costs and limited editing quality.

Meanwhile, the recent acceleration dividends of modern generative models (Consistency Models, rectified flow, Distribution Matching Distillation [DMD]) have almost exclusively benefited the "noise-to-data" (\(noise\rightarrow data\)) generation paradigm, enabling high-quality, few-step inference. Streaming video generation has further extended this few-step capability to autoregressive generation of arbitrary lengths. However, these speedups have rarely been transferred to editing. This is because the "data-to-data" pipeline is tightly coupled with its own iterative process, making it impossible to directly leverage the speed of few-step sampling—one cannot simply cram a pipeline designed for source -> noise -> target into a 5-step generator.

The authors' key insight is that video editing is essentially a problem of "video-conditioned video generation," which should be addressed as a generation task directly. Core Idea: Reformulate training-free video editing as "noise-to-target under source conditions" generation. By spawning a parallel source branch sharing noise with the target branch on a pre-trained streaming generator, dual-branch few-step sampling is executed. A self-attention bridge is then utilized to inject the structure, motion, and background priors of the source video into the target generation. Combined with cross-attention grounding/enhancement and source-oriented guidance, this satisfies condition injection requirements while retaining the high speed of few-step sampling.

Method¶

Overall Architecture¶

StreamEdit is built upon two representative streaming (chunk-level autoregressive) video generators, Self Forcing and LongLive, requiring no training. It accomplishes editing purely through sampling strategies and attention manipulation. The inputs consist of a source video, source prompt, target prompt, and user-provided trigger words (optionally including an edited first frame as a visual prompt). The output is the target edited video generated chunk-by-chunk.

The overall framework is governed by two main processes: sampling strategies and attention manipulation. On the sampling side, the randomized few-step sampling is extended to a dual-branch setup: the source branch is fixed to constant endpoint sampling ("constant clean sample + noise"), while the target branch undergoes standard randomized few-step generation. The two branches run in parallel, sharing the same noise at each step, making the intermediate states of the source branch readily available for synchronous feature injection. Source-oriented guidance is also stacked on the sampling side to suppress jitter induced by randomness. On the attention side, the self-attention bridge is responsible for injecting the structure, motion, and background of the source into the target. Cross-attention grounding determines the mask of the editing area (used by the bridge and guidance), and cross-attention enhancement serves as an adjustable "edit strength valve." For long videos, since the base model is inherently streaming and autoregressive, the method naturally supports arbitrary lengths by rolling chunk-by-chunk (Self Forcing uses clip rolling, while LongLive directly scales via short-window attention and sink tokens).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Source Video + Source/Target Prompts<br/>+ Trigger Words (Optional First Frame)"] --> B["Dual-Branch Few-Step Sampling<br/>Source Fixed Endpoint + Target Randomized, Sharing Noise"]
    B --> C["Cross-Attention Grounding<br/>Foreground-Background Attention Diff → Adaptive Edit Mask"]
    C --> D["Self-Attention Bridge<br/>Q/K Blending + Delayed Source KV Injection"]
    D --> E["Cross-Attention Enhancement<br/>Additive Enhancement for Trigger Words, Edit Strength Valve"]
    E --> F["Source-Oriented Guidance<br/>Correct Edit-Irrelevant Jitter via Source Velocity Error"]
    F -->|Chunk-by-Chunk Autoregressive Rolling| G["Target Edited Video<br/>Arbitrary Length"]

Key Designs¶

1. Dual-Branch Few-Step Sampling: Propagating Source Information and Target Generation in Parallel via Shared Noise

Inversion-based methods are computationally expensive and introduce cumulative errors from imperfect inversion, losing background details. Inversion-free methods require too many steps, and are slow and unstable. StreamEdit bypasses these traditional paths by treating editing as conditional generation—exploiting the inherent few-step capability of distilled models to run two parallel sampling branches. The target branch is standard randomized few-step generation, predicting clean samples \(z^{\text{tgt}}_{t_i}\) step-by-step from pure noise. The source branch utilizes "fixed-endpoint" sampling, which always interpolates around the same clean source sample \(x_0^{\text{src}}\), thereby carrying drift-free source information in its intermediate states. Crucially, both branches share the exact same random noise \(\epsilon_{t_i}\) at each step, allowing source intermediate features to be synchronously injected into the target generation. This also aligns the subsequent guidance error projections between the two branches. The target velocity is the "conditional velocity" \(v^*_\theta(x^{\text{tgt}}_{t_i}, t_i \mid x^{\text{src}}_{t_i})\) integrated with the self-attention bridge and cross-attention enhancement. This step establishes the structural backbone for injecting source conditions, while how much to inject and how to do it is handled by the subsequent bridge and enhancement.

\[x^{\text{src}}_{t_i}=(1-t_i)x_0^{\text{src}}+t_i\epsilon_{t_i}, \quad x^{\text{tgt}}_{t_i}=(1-t_i)z^{\text{tgt}}_{t_i}+t_i\epsilon_{t_i}, \quad \epsilon_{t_i}\sim\mathcal{N}(0,I)\]

2. Self-Attention Bridge: Transferring Structure, Motion, and Background to the Target via Q/K Blending and Delayed Source KV Injection

Parallel sampling branches alone are insufficient; source features must actively guide target generation. The authors construct a "bridge" between the self-attention layers of the source and target branches, handling three distinct tasks. First, query blending is employed to preserve structure and motion: many attribute edits modify object appearance while maintaining overall motion trajectories. Since queries encode spatiotemporal structure and motion cues, the source and target self-attention queries are blended using a time-varying ratio, \(\tilde{Q}^{\text{tgt}}_{\text{SA}}=r_{t_i}Q^{\text{tgt}}_{\text{SA}}+(1-r_{t_i})Q^{\text{src}}_{\text{SA}}\), where \(r_{t_i}=1-t_{i-1}^{\rho}\) (early denoising steps establish semantics and are dominated by the source prior to guide structural formation, while later steps gradually release control to target adaptation, with \(\rho\) controlling the retention strength). Second, key blending ensures editing effectiveness and consistency: blending only queries can be unstable when query-key matching is weak. Thus, keys are blended together (with a similar temporal blending applied to key matrices of previous frames to strengthen historical matching). Since the background of the source and target are naturally similar, key blending for previous frames is restricted to the foreground region (using \(M_{\text{prev}}\) as a mask), while the background retains the target's own keys. This allows the edited region to aggregate relevant information without interference from irrelevant regions. Third, source KV injection preserves the background: even with Q/K blending, fine background details may still dissipate. To counter this, key-value pairs (K, V) from the source are explicitly concatenated into the target attention. Because continuous injection would suppress the formation of target concepts (failing the edit when \(t_{\text{inj}}=1\)), injection is restricted to low-noise steps (where denoising refines finer details), utilizing the Iverson bracket \([t<t^{\text{inj}}]\) as a switch. In experiments, \(t_{\text{inj}}\) is fixed to 0.5.

3. Cross-Attention Grounding and Enhancement: Generating Edit Masks and Providing an Adjustable Edit Strength Valve

Where does the edit-region mask required by the self-attention bridge come from? Instead of introducing external segmentation models, the authors reuse cross-attention maps. Traditional methods binarize trigger-word attention maps using fixed thresholds, which requires per-model or per-sample tuning. StreamEdit introduces foreground-background attention difference for adaptive grounding: it averages the attention maps of trigger words and non-trigger words separately, subtracts them, and applies a Heaviside step function to produce the mask \(M^{\phi}\). The source mask is extracted at \(t=0\) from the clean source latent (as distilled models preserve spatial structures across time steps), while the target mask is extracted at \(t=t_{\text{inj}}\) once target semantics have matured. The union of these two masks, \(M=\cup_\phi M^\phi\), covers both "regions to be modified in the source" and "edited regions in the target." For enhancement, a controllable valve for edit strength is introduced: multiplicative enhancement tends to over-polarize token representations, and negative weights might accidentally suppress trigger concepts. Consequently, additive enhancement is applied to trigger words within the edit mask. This is equivalent to adding \(\ln w_{p,q}\) to the attention scores before the softmax operation, where \(w=\omega\) if the token is a trigger word and inside the mask, and \(1\) otherwise. Since FlashAttention does not expose explicit score maps, the authors use a mathematical trick ("running FlashAttention twice for numerator/denominator separately and dividing") to implement this without violating computational complexity. This allows adjustable edit strength with minimal impact on the background.

\[A^{\text{tgt}}_{\text{CA}}(p,q)=\frac{\exp\!\left(S(p,q)+\ln w_{p,q}\right)}{\sum_{j}\exp\!\left(S(j,q)+\ln w_{j,q}\right)}, \quad w_{p,q}=\begin{cases}\omega, & p\in\mathcal{T}\ \text{and}\ M(q)=1\\ 1, & \text{otherwise}\end{cases}\]

4. Source-Oriented Guidance: Projecting Source Denoising Error onto the Target to Prevent Jitter in Edit-Irrelevant Regions

While random sampling provides editing flexibility, it turns the source branch into a noisy trajectory. The resulting noisy source conditions can cause jitter in edit-irrelevant regions (e.g., backgrounds), even with the self-attention bridge. Crucially, this error is observable—it is the difference between the predicted source velocity and the ground-truth velocity of its linear interpolation: \(g_{t_i}=v^{\text{gt}}_{t_i}-v^{\text{src}}_{t_i}\), where \(v^{\text{gt}}_{t_i}=\epsilon_{t_i}-x_0^{\text{src}}\). Since both branches share the same noise, this error from the source branch can be projected onto the target branch for correction. To preserve the stochasticity necessary for effective edits within edit-relevant regions, correction is primarily applied to edit-irrelevant areas using a soft mask weight (AMN, calculated by taking the absolute value of the channel-wise mean velocity difference, followed by min-max normalization). This step resembles classifier-free guidance but guides toward the "source" rather than a condition, specifically bringing the background and other irrelevant areas back into alignment with the source video.

\[\tilde{v}^{\text{tgt}}_{t_i}=v^{\text{tgt}}_{t_i}+\text{AMN}(v^{\text{tgt}}_{t_i}-v^{\text{src}}_{t_i})\odot g_{t_i}, \quad z^{\text{tgt}}_{t_{i-1}}=x^{\text{tgt}}_{t_i}-t_i\tilde{v}^{\text{tgt}}_{t_i}\]

5. Visual Prompts: Treating the User-Provided First Frame as the "Previous Chunk" to Supplement Text Editing with Fine-Grained Control

Text-only editing often lacks fine-grained visual control, leaving outcomes highly dependent on hard-to-tune prompts and model comprehension, lowering the success rate. Exploiting the streaming generation mechanism's inherent conditioning on prior frames, the authors treat the user-provided target first frame as the previous chunk of the video sequence. This seamlessly propagates the fine-grained edit of the first frame into subsequent target generation. This design also has an extra benefit: it can leverage off-the-shelf image editing models (such as Qwen-Image-Edit) to first edit the initial frame, which acts as a visual prompt, making difficult editing tasks (e.g., transforming a person into a porcelain doll) much more reliable. Programmatically, this only requires one additional forward pass of the video generation model to cache the visual prompt: Self Forcing inserts the conditional KV directly at the beginning of the KV cache, while LongLive duplicates the visual condition three times into the attention sinks.

A Complete Example¶

Take "converting a golden retriever into a black-and-white border collie" as a single-chunk execution pipeline: The source clean data is first processed to obtain the source mask \(M^{\text{src}}_{\text{curr}}\) via cross-attention grounding, and its source KV is cached as the historical condition for the next chunk. Entering the denoising loop, a shared noise \(\epsilon_{t_i}\) is sampled at each step from \(t_N\) to \(t_1\). The source branch follows fixed-endpoint interpolation while the target branch follows target prediction interpolation. Q/K blending is executed via the self-attention bridge (where the source prior dominates early steps to preserve structure). At \(t = t_{\text{inj}} = 0.5\), when target semantics have matured, the target mask \(M^{\text{tgt}}_{\text{curr}}\) is extracted and unioned with the source mask to form \(M_{\text{curr}}\). From this step onward, source KVs are injected to refine background details. At each step, cross-attention enhancement scales up the activation of the trigger word "border collie," and source-oriented guidance pulls back any background jitter towards the source video. The loop terminates to yield \(z^{\text{tgt}}_{0}\). Finally, a forward pass on the target clean data is run to cache target KVs and masks for the subsequent chunk. Throughout this process, the attention masks transition through three updates (\(M^{\text{src}}_{\text{curr}}\to M_{\text{curr}}\to M_{\text{prev}}\)) with zero additional NFEs.

Loss & Training¶

StreamEdit is entirely training-free, involving no loss functions or parameter updates, operating purely via inference-time sampling and attention manipulation. Key inference settings: CFG is turned off, Number of Function Evaluations (NFE) is \(2\times(\text{steps}+1)\) (to accommodate dual-sample generation); 15 steps are used for standard comparisons, and 5 steps are used for streaming long video editing; \(\rho=2\) and \(t_{\text{inj}}=0.5\) are fixed throughout; \(\omega\) is set to 4 under Self Forcing, and 2 for other settings; the self-attention bridge and cross-attention enhancement are applied to all attention layers, while grounding is applied to the first 20 of the 30 cross-attention layers. All experiments were conducted on a single A100 GPU.

Key Experimental Results¶

Main Results¶

Evaluations on FiVE-Bench (100 videos, 420 edit prompt pairs, 6 editing categories) compare background/structure preservation, text alignment, image quality, temporal consistency, and VLM editing success rate (FiVE-Acc), as well as per-frame runtime.

Method	Struc.Dist×10³↓	PSNR↑	LPIPS×10³↓	SSIM×10²↑	CLIPS.edit↑	FiVE-Acc↑	Time/Frame (s)↓
TokenFlow	35.62	19.06	263.61	72.51	21.15	27.43	8.04
Wan-Edit	12.53	25.57	94.61	82.55	21.23	46.97	3.07
UniEdit-Flow	18.31	24.43	223.07	78.60	20.92	50.10	1.11
Ours (SF)	10.27	28.47	49.84	87.52	21.73	51.94	0.60
Ours (LL)	15.77	25.43	65.35	84.15	22.13	55.04	0.69
Ours (LL)§	15.69	25.74	67.04	84.29	22.11	61.19	0.76

(§ indicates using a pre-edited first frame as a visual prompt; SF=Self Forcing, LL=LongLive). StreamEdit significantly dominates metrics for background/structure preservation (Structure Distance, PSNR, LPIPS, SSIM), keeps text alignment and edit success rate among the top tier, and exhibits the lowest per-frame generation time (0.6s, dropping below 0.32s/frame under 5 steps). The Self Forcing backbone favors structure and context preservation, while the LongLive backbone exhibits stronger editing capability, offering flexible choices based on user preference.

Ablation Study¶

Ablation of main components (conducted on Self Forcing): Source-Oriented Guidance (S.O.G.) and Self-Attention Bridge (S.A.B.).

Configuration	PSNR↑	LPIPS×10³↓	SSIM×10²↑	FiVE-Acc↑	Description
Full (SF)	28.47	49.84	87.52	51.94	Full Model
w/o S.O.G.	21.86	122.43	72.40	53.58	Without guidance: Edit flexibility slightly increases, but background alignment with the source Logo deteriorates significantly
w/o S.A.B.	13.42	381.19	47.70	67.74	Without bridge: Source constraints completely fail; outputs drift entirely to the target prompt, losing relevance to the source video

Key Findings¶

The self-attention bridge is the core mechanism for injecting source conditions: removing it causes LPIPS to spike from 49.84 to 381.19 and PSNR to plummet from 28.47 to 13.42, which results in generation largely unconstrained by the source. Although the FiVE-Acc appears "higher" (67.74), this is because it freely drifts to the target, neglecting the source video entirely—a classic misalignment between edit success rate and fidelity.
Source-oriented guidance is a key remedy for background jitter caused by random sampling: removing it leads to a degradation across all background-preservation metrics (PSNR/LPIPS/SSIM), validating its effectiveness in binding the background back to the source.
Two hyperparameters act as precise control knobs: Smaller \(\rho\) values preserve structure better but weaken editing capabilities (\(\rho=1\) fails to modify a jeep), while larger values support flexible edits (successful at \(\rho=3\)). Larger \(\omega\) values amplify editing intensity (e.g., turning an elephant increasingly blue). Sweeping \(\rho\) and \(\omega\) reveals a clear trade-off curve, demonstrating excellent controllability and robustness.
Cross-attention enhancement (Appendix B.1) acts as an "optional yet highly effective valve": omitting it still yields successful edits and preserves high background fidelity, but adding it significantly boosts FiVE-Acc with negligible impact on background preservation, showing that it improves editing success at a minimal cost to irrelevant regions.

Highlights & Insights¶

The paradigm shift is the primary highlight: reformulating editing from \(data\rightarrow data\) to "noise-to-data under source conditions" bridges the gap between few-step generation acceleration and editing tasks, enabling successful edits in just 5-15 steps. This is a classic example of "finding a new path by changing perspectives."
Dual-branch shared noise is an elegant coupling mechanism: sharing the identical \(\epsilon\) between the source and target branches enables step-by-step source feature injection and allows the projection of velocity prediction errors from the source branch to correct the target branch dynamically. A single design cleverly accomplishes both "condition injection" and "guidance correction."
Training-free adaptive grounding: leveraging the difference in foreground-background cross-attentions combined with a Heaviside step function avoids the need to tune hardcoded thresholds across models, avoids external segmentation networks, and updates masks three times at zero additional NFE cost.
Additive cross-attention enhancement is implemented using two forward passes (numerator/denominator) of FlashAttention, avoiding falling back to standard matrix multiplication that slows down inference. This mathematical workaround is highly useful for any work attempting score-level manipulation in FlashAttention.
Visual prompts treat the "first frame as the previous chunk," hooking up off-the-shelf image editors almost cost-free and significantly improving the success rates of difficult edits (e.g., transforming humans to porcelain dolls), showing clever utilization of the streaming autoregressive architecture.

Limitations & Future Work¶

The authors acknowledge the need to scale up capabilities, improve real-time performance, and extend the framework to broader video editing scenarios.
The method is heavily reliant on the quality and native 832x480 resolution of the pre-trained streaming generators (Self Forcing / LongLive). Input videos must be resized to this resolution and back, potentially sacrificing high-resolution details.
Manual input of trigger words (source/target) is required, tying grounding quality to trigger word selection. Pure text-based editing on difficult cases still requires visual prompt fallback, highlighting the limitations of text-only controllability.
While default values are provided for the hyperparameters (\(\rho\), \(\omega\), \(t_{\text{inj}}\)), some configurations are model-dependent (SF uses \(\omega=4\), while LL uses \(\omega=2\)). Transferring across models still requires minor parameter tuning.
Evaluations are primarily restricted to the six categories in FiVE-Bench. Long-video comparisons rely on a 20-participant user study for ranking rather than quantitative parameters, meaning quantitative evidence for long videos remains relatively weak.

vs UniEdit-Flow / FlowEdit (inversion-free stream editing): These methods still operate under the \(data\rightarrow data\) framework using noisy samples to approximate the source -> target transition. Consequently, early samples remain close to the source, visible edits emerge slowly, and irregular trajectories demand small step sizes. StreamEdit switches to noise\(\rightarrow\)target generation, establishing target concepts rapidly within few steps, which is stabler and faster for complex cases.
vs Inversion-based methods (Null-text / RF-Inversion, etc.): These route through source -> noise -> target, requiring extra iterations and generating cumulative errors from imperfect inversions, which compromises the background. StreamEdit uses a fixed-endpoint source branch and source KV injection to preserve backgrounds directly, yielding substantially better PSNR/LPIPS.
vs MasaCtrl / FateZero (attention manipulation editing): While also manipulating attention, they do so on traditional multi-step diffusion. StreamEdit adapts attention bridging, grounding, and enhancement systematically to few-step streaming generators, introducing time-varying Q/K blending ratios and delayed source KV injection tailored for the few-step paradigm.
vs Self Forcing / LongLive (streaming video generation backbones): These are originally pure generative models. StreamEdit utilizes them for editing without changing their architectures or training them, effectively unlocking a new application for streaming generators and validating the feasibility of treating editing as conditional generation.
vs StreamV2V / AdaFlow (long video editing): In a user study of long videos with over 470 frames, StreamEdit (using LongLive in 5 steps) ranked first across three metrics: overall quality, background preservation, and editing fidelity.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ Reformulating editing as noise-to-data generation conditioned on the source and grafting it onto streaming few-step generators is a paradigm-level reformulation, rather than minor modular tuning.
Experimental Thoroughness: ⭐⭐⭐⭐ The use of FiVE-Bench with multiple metrics, dual backbones, and detailed ablation/hyperparameter sweeps is robust, though quantitative proof for long videos relies solely on user study rankings.
Writing Quality: ⭐⭐⭐⭐ The logical flow is clear, figures and text align well, and complete algorithms are provided in the appendix, though a few heavily formatted equations require referencing Figure 2 for ease of reading.
Value: ⭐⭐⭐⭐⭐ This is the first work to successfully translate few-step generation dividends into video editing. Clocking in at 0.6s/frame and supporting arbitrary lengths, it is highly practical, balancing both efficiency and quality.