ECCV2026 Video Generation AI paper notes paper summaries Diffusion Models Alignment/RLHF Compression Speech & Audio Personalized Generation

🎬 Video Generation¶

🎞️ ECCV2026 · 169 paper notes

📌 Same area in other venues: 📷 CVPR2026 (182) · 🔬 ICLR2026 (97) · 💬 ACL2026 (4) · 🧪 ICML2026 (32) · 🤖 AAAI2026 (11) · 🧠 NeurIPS2025 (23)

🔥 Top topics: Video Generation ×75 · Diffusion Models ×36 · Alignment/RLHF ×10 · Compression ×8 · Speech & Audio ×6

End-to-End Training for Autoregressive Video Diffusion via Self-Resampling: Resampling Forcing trains a causal diffusion predictor on histories containing its own resampled model errors while retaining clean video targets, removing auxiliary teachers and discriminators and improving long-video temporal quality through native 15-second training.
ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytelling: ShotStream reformulates multi-shot video generation as next-shot prediction conditioned on historical visuals, combining sparse conditioning, boundary-aware dual caches, and two-stage self-forcing distillation to achieve 15.95 FPS on one H200 and a transition-control score of 0.978.
MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens: Without updating generator weights, MemRoPE continuously folds evicted context into long- and short-term EMA memory and applies RoPE to unrotated keys only at attention time, enabling hour-scale generation with a fixed cache and a VBench-Long average of 87.99 versus 87.01 for the comparison method on one-hour LongLive videos.
AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation: AnyFlow replaces endpoint-only video distillation with transitions across arbitrary time intervals, then corrects student rollouts using three-segment shortcut simulation and distribution matching, reaching 84.05 VBench at 4 NFEs and 84.41 at 32 NFEs with its 14B causal model.
PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generation: PhyGDPO proposes a group-wise direct preference optimization framework for physically consistent video generation. It utilizes real-world videos as winning samples and a Plackett-Luce group-wise probability model instead of pairwise comparisons to capture global preferences. Co-designed with VLM-guided physical reward reweighting and a LoRA-switching reference mechanism, it efficiently achieves superior physical consistency over Sora2 and Veo3.1 at the 14B parameter scale.
VGGRPO: Towards World-Consistent Video Generation with 4D Latent Reward: VGGRPO connects video latents directly to a geometry foundation model and performs GRPO post-training with camera smoothness and reprojection consistency rewards, achieving a 66.00% motion-quality win rate on dynamic scenes with Wan2.2-5B while reducing reward computation time from 54.73 s to 41.33 s.
Taming Text-to-Sounding Video Generation via Advanced Modality Condition and Interaction: For the text-to-sounding video generation (T2SV) task that simultaneously generates visual frames and synchronized audio, this paper proposes a dual-agent caption rewriting framework (CRR) to provide separate, interference-free, modality-pure captions for the video tower and the audio tower. It then employs a dual-tower diffusion model, BridgeDiT, using dual cross-attention (DCA) to bridge and exchange features between the two towers, achieving state-of-the-art (SOTA) performance on most metrics across three benchmarks.
Anchor Forcing: Anchor Memory and Tri-Region RoPE for Interactive Streaming Video Diffusion: Anchor Forcing addresses evidence loss at prompt switches with anchor-guided re-caching and positional distribution shift during long rollouts with tri-region RoPE, improving the LongLive baseline from 26.37 to 73.00 in dynamic degree and from 78.92 to 82.25 in quality on interactive 60-second videos.
HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising: HiAR conditions each video block on earlier blocks at the current denoising step's output level, rather than fully clean predictions, and uses bidirectional teacher-trajectory regularization to prevent motion collapse, achieving a VBench Total of 0.821, Drift of 0.257, and reported throughput of 30 fps on 20-second videos.
PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference: PackForcing divides video history into intact initial anchors, compressed and dynamically retrieved intermediate history, and intact recent context, combining these with temporal RoPE correction to generate 120-second videos from a Wan2.1-based model trained with roughly 5-second windows; it achieves the highest 60-second and 120-second VBench Dynamic Degree among the evaluated methods, although a bounded active context must be distinguished from constant storage for all history.
Pathwise Test-Time Correction for Autoregressive Long Video Generation: The paper introduces training-free pathwise Test-Time Correction (TTC), which anchors appearance to the first frame, re-noises the corrected state, and restores historical context during few-step distilled sampling, increasing Self-Forcing's subject consistency from 92.5 to 94.0 on 30-second videos while reducing throughput from 15.79 to 10.53 fps.
PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models: PhysRVG evaluates video continuation through collision-weighted deviations from reference trajectories and adds Flow Matching supervision when reinforcement learning exploration fails, reaching 0.64 IoU and 15.03 trajectory offset on PhysRVGBench, although these results primarily establish trajectory-fitting performance in restricted rigid-body scenarios.
AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories: Instead of fusing historical views into one global point cloud, AnchorWeave retrieves complementary local point clouds, renders multiple anchor videos, and learns to reconcile them, improving its CogVideoX variant from 19.01 to 20.96 dB PSNR and from 0.6145 to 0.6727 SSIM over single-anchor conditioning under partial-revisit evaluation.
Delving into Latent Spectral Biasing of Video VAEs for Superior Diffusability: SSVAE uses latent masked reconstruction and local correlation regularization to make video VAE channel spectra few-mode dominated and spatio-temporal spectra low-frequency dominated, achieving about 3x faster UR convergence in a specified 17-frame 512p text-to-video experiment without claiming that better generation implies universally superior reconstruction.
LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing: LiveEdit transfers the editing capabilities of a bidirectional diffusion Transformer to a causal streaming editor (4-step inference) through a three-stage progressive distillation pipeline. Leveraging the high temporal redundancy of self-attention layers in background regions, it designs an AR mask cache to achieve real-time inference at 12.66 FPS while preserving the editing quality of the bidirectional model.
OmniForcing: Unleashing Real-time Joint Audio-Visual Generation: OmniForcing distills the 19B bidirectional audio-visual model LTX-2 into a causal model that jointly generates one-second blocks, stabilizes training with audio sinks, and mitigates error accumulation through joint Self-Forcing, achieving approximately 0.7-second first-chunk latency and 25 FPS at 480p on one H200 without preserving every aspect of teacher quality losslessly.
DualCamCtrl: Dual-Branch Diffusion Model for Geometry-Aware Camera-Controlled Video Generation: DualCamCtrl treats RGB and depth as separately modeled but jointly evolving generative modalities, improving camera adherence through semantic-first bidirectional interaction and 3D fusion; on RealEstate10K text-to-video generation, it reduces AC3D's rotation error from 2.67 to 1.23 while lowering FVD from 415.6 to 408.1.
Grounding World Simulation Models in a Real-World Metropolis: Seoul World Model (SWM) trains a retrieval-augmented video diffusion model on cross-temporal street-view data and constrains autoregressive generation with geometric and semantic conditioning plus a continuously updated virtual lookahead sink, producing more faithful real streets in unseen Busan and Ann Arbor rather than merely plausible fictional cities.
PhysChoreo: Physics-Controllable Video Generation with Part-Aware Semantic Grounding: PhysChoreo reconstructs objects in a single image as language-editable, part-level physical fields and drives a pretrained video model with temporally editable simulation, improving physical plausibility and temporal instruction alignment on the paper's 10 video evaluation cases without guaranteeing that the final pixels strictly follow the simulation.
RefAlign: Representation Alignment for Reference-to-Video Generation: RefAlign proposes a Reference Alignment loss (RA loss) to explicitly align the intermediate features of the DiT reference branch with the semantic space of a Visual Foundation Model (VFM) during training. Specifically, the positive term pulls features of the same subject closer to ensure identity consistency, while the negative term pushes features of different subjects apart to enhance semantic discriminability. The alignment module is removed during inference to achieve zero extra overhead. It achieves SOTA performance with a TotalScore of 60.42% on OpenS2V-Eval, effectively alleviating copy-paste artifacts and multi-subject confusion in reference-to-video generation.
WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generation: WorldWander places synchronized egocentric and exocentric videos in Wan2.2-5B's joint attention and shares temporal position encodings between corresponding frames, using LoRA to learn bidirectional translation without camera poses or depth and reducing real-world ego-to-exo FVD on EgoExo-8K from fine-tuned ReCamMaster's 968.1 to 374.7.
Out of Sight, Out of Mind? Evaluating State Evolution in Video World Models: StEvo-Bench turns the question of whether a world keeps evolving during occlusion or camera lookaway into a diagnostic evaluation with 225 natural-process tasks, finding task success below 10% for every evaluated model and showing that recalling an old view is not equivalent to updating world state correctly.
Recurrent Autoregressive Diffusion: Global Memory Meets Local Attention: RAD adds recurrent memory after local attention in a video DiT, using overlapping frame-wise windows for detail transfer and clean hidden-state prefetching for efficient training; on Minecraft, LSTM-f improves PSNR from 14.24 for LSTM-c to 16.59, although gains do not extend to every dataset and metric.
TinyHistory: Lightweight Video History Embeddings via Two-Stage Context Learning: TinyHistory first trains a lightweight history encoder to query frames at arbitrary temporal positions, then jointly trains it with a video diffusion model for continuation, reducing history context to 195 tokens/s in Wan 2.1 evaluations at 480p while outperforming a sliding window in character consistency and user preference.
Unison: Harmonizing Motion, Speech, and Sound for Human-Centric Audio-Video Generation: Unison generates speech and sound effects through separate streams, coordinates their acoustic information with semantic gating, and learns mutual audio-video guidance through progressive cross-modal forcing, improving audio quality and overall temporal alignment on a curated held-out test set without consistently surpassing LTX-2 in lip synchronization or visual aesthetics.
Video Generation Models are General-Purpose Vision Learners: GenCeption converts WAN text-to-video generation into text-controlled, single-pass perception through RGB-formatted targets, sparse query tokens, and unified L2 post-training, approaching or exceeding specialists on several benchmarks while still encountering substantial interference between sparse keypoint and dense tasks during joint training.
Taming Camera-Controlled Video Generation with Verifiable Geometry Reward: CamVerse converts the reconstructed 3D camera trajectories of generated and reference videos into segment-level relative pose rewards for online GRPO; Table 1 reduces T2V translation error from the SFT baseline's 0.0395 to 0.0293, but I2V perceptual quality does not improve alongside it.
VOID: Video Object and Interaction Deletion: VOID extends video object removal from spatial inpainting to downstream interaction rewriting through counterfactual paired training, VLM-guided quadmasks, and an optional flow-warped noise second pass, receiving 64.8% of selections in a seven-way human evaluation on real videos.
DeRA: Decoupled Representation Alignment for Video Tokenization: DeRA separately learns appearance and motion representations in a one-dimensional video tokenizer, supervises them with image and video foundation models, and uses symmetric gradient projection to mitigate objective conflicts in the shared encoder, reducing UCF-101 reconstruction rFVD from LARP's 20 to 15 with the same 1024-token budget while improving downstream autoregressive generation.
Egocentric World Model for Photorealistic Hand Object Interaction Synthesis: EgoHOI encodes hand kinematics, metric head motion, and first-frame object appearance as three conditioning streams for the video latent space of Wan 2.1 14B, generating egocentric hand-object interactions without future object states and achieving PSNR 21.05 and a hand missing ratio of 5.84% on HOT3D.
FreeSwim: Revisiting Sliding-Window Attention Mechanisms for Training-Free Ultra-High-Resolution Video Generation: FreeSwim divides high-resolution refinement in a pretrained video DiT between an inward-window branch for detail and a full-attention branch for global semantics, using cross-attention override and cross-step caching to achieve a 73.7% 1080P VBench overall score with Wan2.1-1.3B without training new parameters.
LatSearch: Latent Reward-Guided Search for Faster Inference-Time Scaling in Video Diffusion: LatSearch trains a latent reward model that can directly score the latent vectors of intermediate states in the denoising trajectory (rather than decoded videos), and utilizes it to drive "Reward-Guided Resampling and Pruning (RGRP)" in the latent space during inference. This achieves video generation quality comparable to or even better than the SOTA, while reducing runtime by up to 79%.
MemoBench: Benchmarking World Modeling in Dynamically Changing Environments: MemoBench tests whether video generators remember and update objects outside the field of view using 360 visible-disappear-reappear clips; evaluation of 10 models yields a best Object Reappearance Score (ORS) of only 0.582 and shows that avoiding camera motion can inflate consistency scores.
PAI-Studio: Cinematic Video Background Replacement with Camera-Aware Motion: PAI-Studio jointly conditions video generation on a foreground video and 1–3 background images, combining real cinematographic data, temporal positional encoding cloning, and bidirectional attention to learn background motion, foreground preservation, and relighting, achieving MSE 0.00645, SSIM 0.789, and LPIPS 0.197 under multi-reference Cine-Restore evaluation.
InteractiveAvatar: Real-Time Streaming Video Generation for Consistent and Intent-Aware Avatars: InteractiveAvatar integrates long-short visual memory and intent-driven action control into an autoregressive diffusion avatar model, achieving 26.68 FPS under a 576p experimental setting with improved object persistence and text alignment, although the complete interaction pipeline still takes approximately 2.6 seconds to deliver its first frame.
MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation: MMControl proposes the first unified multi-modal control framework for joint audio-video generation. Through MMCU, heterogeneous control signals (including reference image, reference audio, depth maps, and pose sequences) are uniformly encoded. They are then non-intrusively injected into a frozen Joint DiT backbone via a dual-stream bypass architecture. During inference, modality-specific guidance scale factors are utilized to independently adjust the visual and acoustic control strengths. This approach comprehensively outperforms existing uni-modal control methods in identity consistency, timbre fidelity, and structural alignment.
MoGAN: Improving Motion Quality in Video Diffusion via Few-Step Motion Adversarial Post-Training: MoGAN combines optical-flow discrimination with distribution matching on a three-step distilled Wan2.1 model, raising VBench Motion Score from DMD's 0.859 to 0.973 while retaining the three-step inference path.
OmniDance: Multimodal Driven Dance Video Generation with Large-scale Internet Data: OmniDance uses approximately 300k Internet dance clips and coordinated music injection, curriculum training, and modality-specific sampling to extend Wan2.2-TI2V-5B into a unified text/music/jointly driven dance video model, improving overall quality and controllability on the CIPE-Dance test set without making joint conditioning best on every metric.
Rethinking Reward Signals in Video GRPO: When Scores Become Targets: TaRoS dynamically combines video GRPO rewards using their means and sparsity within each prompt group; with the same Qwen2.5-VL-72B evaluator, it raises the Wan2.1-T2V-1.3B VBench total score from DanceGRPO's 79.72 to 80.56.
SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing: SVG-EAR replaces score-only block selection with centroid compensation for skipped blocks and exact computation where compensation is most inaccurate, achieving 31.043 dB PSNR against full-attention outputs and a 1.93-fold end-to-end speedup on HunyuanVideo 720p text-to-video generation without training.
Benchmarking Scientific Understanding and Reasoning for Video Generation using VideoScience-Bench: VideoScience-Bench places video generators in experimental scenarios requiring combinations of scientific concepts, evaluates their outputs with expert ratings and an evidence-grounded VideoScience-Judge, and shows that visual coherence is not scientific correctness, while checklist-based judging reaches a Spearman correlation of 0.96 with expert rankings.
Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Joints: This paper proposes using sparse 3D hand joints as explicit control signals to drive egocentric video generation. Underpinned by three key designs—occlusion-removed context aggregation, 3D depth-weighted occlusion-aware propagation, and 3D geometric embedding—the proposed lightweight module (~20k parameters) built on WAN 2.1 achieves precise video control for fine-grained hand-object interactions, significantly outperforming existing 2D trajectory and implicit pose-based methods.
IC-World: In-Context Generation for Shared World Modeling: IC-World combines multi-view images of the same world into a grid for joint video generation, then applies GRPO fine-tuning with 3D geometry and motion consistency rewards, reducing the Wan2.1 baseline's M-FID from 185.2354 to 157.6099 on the two-view dynamic-scene benchmark.
VIGOR: VIdeo Geometry-Oriented Reward for Temporal Generative Alignment: VIGOR selects reliable geometric regions using shallow VGGT attention and evaluates cross-frame pointwise reprojection error, improving video geometry through SFT/DPO or inference-time search and raising PSNR from 19.68 to 22.66 dB in CausVid Best-of-10 experiments.
ID-LoRA: Identity-Driven Audio-Video Personalization with In-Context LoRA: ID-LoRA uses a shared audio-video backbone conditioned on a first frame and reference speech, separates reference from target through negative temporal positions, and strengthens vocal identity through identity guidance, reaching 0.477 speaker similarity on cross-video evaluation while allowing text to change the scene and speaking style.
In-Context Sync-LoRA for Portrait Video Editing: Sync-LoRA trains an in-context LoRA for LTX-Video on highly synchronized synthetic video pairs, letting an edited first frame specify appearance and a clean source video specify motion, achieving a useful balance between edit fidelity, frame-level synchronization, and identity preservation rather than leading every synchronization metric.
Learning to Generate Rigid Body Interactions with Video Diffusion Models: KineMask combines velocity-mask control, two-stage training that removes future motion conditions, and event descriptions to generate subsequent rigid-body interactions from an image and initial velocity, improving motion and interaction quality in synthetic tests and perceptual evaluation of real-world scenes.
Learning Zero-Shot Subject-Driven Video Generation Using 1% Compute: This paper decomposes subject-driven video generation (SDV-Gen) into two alternately optimized sub-tasks — identity injection learned from subject-image pairs and motion-awareness preservation maintained on 4,000 arbitrary videos — and fine-tunes CogVideoX-5B on 200K subject-image pairs plus 4,000 videos in 288 A100 hours (about 1% of the training compute of prior zero-shot baselines), yielding identity-faithful, motion-rich videos at inference with no per-subject tuning at all.
Tuning-free Visual Effect Transfer across Videos: RefVFX uses cross-content, shared-effect training triplets and joint video conditioning to transfer dynamic effects from a reference video onto another video or image without tuning for each new effect, achieving a 65.7% overall-match preference win rate against Lucy Edit on Neural V2V.
AHOY! Animatable Humans under Occlusion from YouTube Videos with Gaussian Splatting and Video Diffusion Priors: AHOY reconstructs complete, animatable 3DGS avatars from heavily occluded monocular YouTube videos. It first leverages DensePose + FLUX + multi-view diffusion to build a coarse canonical avatar, uses a LoRA-fine-tuned Wan 2.2 video diffusion model via RF-Inversion to generate multi-view "hallucinated" supervisory videos, and finally employs map-pose/LBS-pose decoupling to absorb inconsistencies in the generated data, training a high-fidelity Gaussian avatar with pose-dependent deformations.
AR-CoPO: Align Autoregressive Video Generation with Contrastive Policy Optimization: AR-CoPO proposes a chunk-level contrastive policy optimization framework to align few-step autoregressive video generators: by constructing initial noise neighborhood candidates through a forking mechanism at a randomly selected chunk, and utilizing local GRPO updates for credit assignment, it sidesteps the fundamental mismatch between SDE-style GRPO and the near-deterministic dynamics of consistency models; meanwhile, a semi-on-policy training paradigm is designed to leverage fixed reference replay and a ratio-clipping trust-region constraint to improve text-semantic alignment quality and prevent motion quality collapse caused by pure exploration, achieving joint improvements on both VBench and VideoAlign benchmarks based on Self-Forcing.
CameraAnything: Refilming Videos with Arbitrary Camera Control: CameraAnything injects per-pixel camera rays into the attention positional encoding of a video diffusion model and trains with independently varied synthetic camera supervision to jointly change viewpoint, focal length, output format, and shot transitions in one generation process, reducing DAVIS retake rotation error from ReCamMaster's 5.11982 degrees to 2.7600 degrees.
ControlHair: Synergizing Physics Simulator and Video Diffusion for Controllable Dynamic Hair Rendering: ControlHair delegates hair motion to a physics simulator and uses per-frame strand maps and human poses to condition video diffusion, reaching 17.15 dB PSNR on reconstruction of ten VEO 2 videos; this measures control-signal adherence rather than end-to-end physical accuracy.
DiffHDR: Re-Exposing LDR Videos with Video Diffusion Models: DiffHDR treats clipped highlights and noise-obscured shadows in a single 8-bit LDR video as radiance-inpainting regions, uses Log-Gamma mapping to generate HDR in the latent space of a frozen LDR video VAE, reaches 6.89 FOVVDP versus SingleHDR's 6.56 on Cinematic Video, and supports text- or reference-guided reconstruction.
DiT as Real-Time Rerenderer: Streaming Video Stylization with Autoregressive Diffusion Transformer: RTR-DiT first fine-tunes a bidirectional DiT teacher on a self-built text/reference-paired stylization dataset, then distills it into a 2-step causal autoregressive generator via Self Forcing and Distribution Matching Distillation, and pairs it with a reference-preserving KV cache update strategy so that a single H20 GPU can stream-stylize videos of arbitrary length while switching text prompts or reference images on the fly.
EcoVideo: Entropy-Orchestrated Video Generation Paradigm in Cloud-Edge Dynamics: EcoVideo replaces cloud-edge video generation through sequential large/small-model denoising with cloud generation of informative keyframes and edge reconstruction of intermediate frames, organizing computation through attention entropy and resource-aware scheduling to achieve 1.84× end-to-end speedup in the main Wan2.1 experiment and 2.91× under cloud compute contention.
EFlow: Fast Few-Step Video Generator Training from Scratch via Efficient Solution Flow: EFlow pairs a token-droppable Gated Local–Global Attention backbone with a training recipe for solution flow matching — block-skipped "weak-path" guidance plus a Mean-Velocity Additivity regularizer — so that a 4-step video generator can be trained from scratch in a single stage, reaching 2.5× the training throughput of standard solution flow and 23×–234× lower inference latency than 50-step models.
FlexAM: Flexible Appearance-Motion Decomposition for Versatile Video Generation Control: FlexAM specifies appearance with masked video and motion with variable-density 3D point trajectories carrying multi-frequency positional encoding and dynamic depth, allowing one video diffusion model to perform appearance editing, camera control, and object manipulation while achieving text alignment of 32.55 and temporal consistency of 0.976 in the paper's motion-transfer evaluation.
GimbalDiffusion: Gravity-Aware Camera Control for Video Generation: GimbalDiffusion constructs extreme-view training samples from gravity-aligned panoramic videos and captions level-view crops to reduce prompt-angle entanglement, lowering WAN 2.2 5B PitchErr on SpatialVID-extreme from 14.39° without null-pitch conditioning to 8.36°, although not every visual quality metric improves.
Goku: A Million-Scale Universal Dataset and Benchmark for Instruction-Based Video Editing: Goku constructs 2 million video editing pairs through task-specific synthesis and progressive filtering, while its mask-guided Goku-Edit model raises instruction-only IF on Goku-Bench from LucyEdit's 0.549 to 0.627 across appearance, structural, reference-based, and compositional editing.
LibraGen: Playing a Balance Game in Subject-Driven Video Generation: LibraGen adapts a video foundation model with 9,000 high-quality SFT samples, combining pairing-specific LoRA merging, complementary DPO, and dynamic CFG to preserve identity and generation capabilities, reaching a text alignment score of 3.594 in Table 1 on its in-house benchmark.
Making Avatars Interact: Towards Text-Driven Human-Object Interaction for Controllable Talking Avatars: InteractAvatar uses parallel perception-planning and video-synthesis streams to make avatars follow text instructions and interact with objects in the reference scene while speaking to audio, reaching VLM-QA 29.07 and PI 0.850 on GroundInter in its joint text-and-audio generation mode.
MemLearner: Learning to Query Context Memory for Video World Models: MemLearner proposes an adaptive context memory mechanism based on learnable query tokens (Q tokens), enabling video world models to learn end-to-end "what useful information to retrieve from historical frames" when generating long videos, rather than relying on manual rules to retrieve keyframes. Serving as an information bridge between C (context) and P (predicted tokens), Q tokens perform context queries via 3D attention inside a pretrained Video DiT. Combined with a hierarchical strategy of shallow query and deep generation along with attention pruning, this significantly reduces computational overhead. Under occlusion and dynamic object scenarios, it improves PSNR by 1.38 dB and reduces LPIPS by 0.057 compared to CaM.
Next-Frame Decoding for Ultra-Low-Bitrate Image Compression with Video Diffusion Priors: This paper proposes NeFIC, which reinterprets generative decoding in ultra-low-bitrate image compression (ULB-IC) as a virtual temporal evolution process from a compact anchor to the final reconstructed image. By leveraging a pre-trained Video Diffusion Model (VDM) as a temporal prior for next-frame prediction, and employing a two-stage training strategy (interval adaptation + one-step generation bypass), NeFIC achieves high-quality and highly efficient image compression at extremely low bitrates.
OmniHuman: A Large-scale Dataset and Benchmark for Human-Centric Video Generation: OmniHuman organizes 1 million human-centric videos into scene-, interaction-, and individual-level supervision, paired with OHBench for diagnosis at the same levels; fine-tuning LTX-2 on 20% of the data improves audiovisual alignment and most human-centric metrics, but not every metric.
OneVAE: Joint Discrete and Continuous Optimization Helps Discrete Video VAE Train Better: OneVAE alternates continuous and discrete reconstruction within a shared video VAE, combining progressive initialization and architectural improvements to raise PSNR from 27.22 to 28.15 in the unified-training ablation while supporting both latent representations at the same compression configuration.
PhyDetEx: Detecting and Explaining the Physical Plausibility of T2V Models: PhyDetEx fine-tunes Qwen2.5VL 7B with LoRA on 2,588 plausible/implausible video pairs, combining physical judgments and explanations in a language-modeling task and achieving 75.6% implausible-class accuracy, a reported F1 of 83.5, and a Reasoning Score of 4.24 on PID Test.
TryOnCrafter: Unleashing Camera Trajectories for Realistic Video Virtual Try-on via a Renderable 4D Try-on Proxy: TryOnCrafter transfers a target garment into an animatable Gaussian avatar, aligns it with a background point cloud, and renders new camera trajectories to guide a video DiT, achieving an Overall Score of 75.47 on CaM-VVTBench.
Versatile Editing of Video Content, Actions, and Dynamics without Training: DynaEdit builds a noise-free editing trajectory from the source video straight to the edited video on a pretrained image-to-video flow model (WAN2.1 14B), pushes the starting timestep all the way to pure noise so that coarse spatio-temporal structure can actually be rewritten, and then fixes the two resulting failure modes with Similarity Guided Aggregation (SGA), which soft-selects the edit direction least divergent from the source video, and Annealed Noise Correlation (ANC), which suppresses high-frequency jitter -- thereby supporting object insertion/replacement, action rewriting and global dynamic effects with no training at all, matching the only trained commercial model, Runway Aleph.
ViBe: Ultra-High-Resolution Video Synthesis Born from Pure Images: ViBe adapts an existing video DiT using images alone, removing the modality adapter while retaining the spatial adapter at inference, and combines sparse attention with high-frequency reconstruction training to reach a 74.4% six-metric mean in Table 1's 4K setting, 0.8 percentage points above CineScale.
Accelerating Text-to-Video Generation with Calibrated Sparse Attention: CalibAtt compiles input-stable block sparsity and spatial row repetition into two attention execution paths without updating model weights, reducing Wan 2.1 14B's 720p generation latency from 1244 to 785 seconds while retaining comparable VBench performance.
AVTok: 1D Unified Tokenization for Holistic Audio-Video Generation: AVTok proposes the first unified audio-video tokenizer. It employs a two-stream Transformer architecture (shared encoder-decoder + modality-specific learnable queries) to jointly encode audio-video pairs into a single 1D discrete latent space. Coupled with the VFAL progressive training strategy and representation alignment learning, it achieves state-of-the-art performance in both reconstruction quality and downstream autoregressive generation tasks (A2V / V2A / joint generation), while using significantly fewer parameters and computation than dominant dual-branch approaches.
ConfCtrl: Enabling Precise Camera Control in Video Diffusion via Confidence-Aware Interpolation: ConfCtrl uses endpoint-conditioned video interpolation as a generative prior, changes the starting distribution of rectified flow with confidence-weighted projected point-cloud latents, and refines camera conditioning through predict-update modules, achieving 15.54 dB PSNR, 0.143 m translation error, and 0.103 rad rotation error on CO3D-Hydrant.
Cycle-World: Mitigating Error Accumulation in Long-term Video World Models via Reverse-Prediction Cycle Consistency: Cycle-World asks whether a generated video chunk can reconstruct its predecessor, using this constraint for both training and inference-time latent correction to reach a 60-second VBench total of 82.88 and improve physical consistency, without establishing drift-free generation at arbitrary lengths.
DCARL: A Divide-and-Conquer Framework for Autoregressive Long-Trajectory Video Generation: DCARL jointly generates sparse keyframes spanning the target timeline, then constrains segment-wise interpolation with noisy keyframes and clean history, reducing FVD from SEVA's 548.0 to 203.7 and camera trajectory error ATE from 0.294 to 0.237 on 32-second OpenDV-YouTube videos.
EmbodiedHead: Real-Time Listening and Speaking Avatar for Conversational Agents: EmbodiedHead converts single-stream audio into continuous FLAME head motion using explicit listening-speaking states, then jointly optimizes a few-step Rectified-Flow DiT and a differentiable renderer to achieve 59 FPS including rendering on one RTX 3090 while reducing spurious mouth opening during listening.
FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control: FlexComposer casts asset insertion into an existing video as trajectory-guided conditional generation: Grounded-SAM2 plus SpatialTracker v2 first canonicalize a static image or a dynamic clip into a "centered, in-place" foreground representation, the translation equivariance of the VAE latent space then transports that representation onto the latent coordinates of a user-specified 3D trajectory with zero learnable parameters, and a Wan2.1-I2V-14B video diffusion transformer handles lighting, shadows, and occlusion blending — reaching an EPE of 2.15 and an FVD of 82.6 on DAVIS under the V2V setting.
From Synchrony to Sequence: Exo-to-Ego Generation via Interpolation: This paper reformulates exocentric-to-egocentric (Exo2Ego) video generation from a condition-output task into the generation of a single continuous sequence: a frozen frame interpolator synthesizes a transition video between the last exo frame and the first ego frame, camera poses are bridged by Slerp plus linear interpolation, and the exo, transition, and ego segments are concatenated into \(S=(X,I,G)\) for a Diffusion Forcing Transformer to model with per-frame independent noise, surpassing TrajectoryCrafter, Wan-FCtrl, Wan VACE, and Exo2EgoSyn in PSNR, SSIM, and LPIPS on the Health, Bike, and Cooking categories of Ego-Exo4D.
HairWeaver: Few-Shot Photorealistic Hair Motion Synthesis with Sim-to-Real Guided Video Diffusion: HairWeaver uses a physics simulator to generate roughly 1,500 CG human videos with pixel-accurate hair motion conditions (a UVW buffer), then applies a two-stage sim-to-real strategy — first train a temporary Style-Alignment-LoRA to pull a pretrained video DiT into the CG domain, freeze it and train a Motion-Context-LoRA to learn motion control, and discard the Style-Alignment-LoRA entirely at inference — so that fine-grained, controllable hair dynamics are transferred without dragging the CG look along, beating Wan-2.2-Animate and UniAnimate-DiT on both a self-collected CG test set and the real-world NeRSemble test set, and winning 49.9% of preferences in a user study against 16.4% for the runner-up.
Infinite-Homography as Robust Conditioning for Camera-Controlled Video Generation: InfCam supplies depth-independent rotational conditioning through infinite homography, combines frozen Wan2.1 with trainable control layers to learn translation-induced residual parallax, and removes training-pair biases through trajectory and focal-length augmentation, reducing WebVid rotation error from TrajectoryCrafter's 5.007 to 3.162.
Layer-Aware Video Composition via Split-then-Merge: Split-then-Merge (StM) learns reconstruction from automatically decomposed unlabeled videos, using perturbed foreground conditions and region-weighted supervision to integrate existing dynamic subjects into new backgrounds, outperforming comparison methods on foreground identity, background identity, action, and background motion metrics in cross-video DAVIS composition tests.
Learning Transferable Dynamics Priors from Action to World Modeling: This work proposes A2World, which pre-trains an action-conditioned multi-view diffusion world model on large-scale real-world robot manipulation data (2.156 million trajectories, 20+ robotic arm morphologies). The learned action-to-dynamics prior can be adapted either into a long-horizon simulator (A2World-sim) or an instruction-conditioned policy (A2World-policy), achieving state-of-the-art performance on the LIBERO benchmark and real robots.
Memory-V2V: Memory-Augmented Video-to-Video Diffusion for Consistent Multi-Turn Editing: Memory-V2V adds retrievable, compressible visual constraints from previous edits to video diffusion models, reducing average MEt3R from independent ReCam generation's 0.1892 to 0.1357 in three-turn novel-view synthesis while supporting appearance-consistent editing across long-video segments.
MSEditor: Toward Consistent Multi-Shot Video Editing: MSEditor is the first framework built specifically for multi-shot video editing: it repurposes multi-view video datasets into multi-shot supervision with geometrically consistent masks, injects that cross-shot prior into a Wan2.1 diffusion backbone through a zero-initialized Supervisory Adapter, and lets semantically related shots exchange features inside one attention window via Cross-Shot Packing and Sparse Cross Attention, so that an edit made on the first frame propagates consistently across all shots in a single inference pass.
PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generation: PhysRAG introduces Retrieval-Augmented Generation (RAG) to video diffusion models. It first retrieves reference videos most physically relevant to the input prompt from a handcrafted physical video database, encodes their spatiotemporal features using VideoMAE V2, and selectively extracts and injects physical priors into the DiT denoising process via a learnable query bottleneck. It achieves SOTA with an average score of 0.58 on the PhyGenBench physical common sense benchmark, while only increasing inference overhead by 1.24%.
Pretrained Video Models as Differentiable Physics Simulators for Urban Wind Flows: This paper fine-tunes LTX-Video, a 2B-parameter pretrained latent video diffusion model, into WinDiNet on urban CFD data, generating a 112-frame 2D wind velocity sequence in under a second, outperforming purpose-built neural operators on several metrics, and — because the whole forward chain is end-to-end differentiable — back-propagating a comfort loss directly to building coordinates for layout inverse optimization.
Seeing Fast and Slow: Learning the Flow of Time in Videos: The paper learns visual speed perception from audio pitch changes and temporal resampling, annotates the SloMo-44K dataset, and uses it for explicit speed-controlled generation and temporal super-resolution with deblurring; on blurred-input 8-fold reconstruction, FVD decreases from the base model's 436.1 to 134.3.
SHIFT: Motion Alignment in Video Diffusion Models with Adversarial Hybrid Fine-Tuning: SHIFT learns motion rewards from optical-flow residuals and long-term point trajectories, combining advantage-weighted denoising of generated samples with a supervised real-video anchor to reach 86.70 VBench Motion on SVD at 2.46 times the per-epoch cost of SFT, primarily preventing fine-tuning from making videos nearly static.
SIFT: Self-Imagination Fine-Tuning for Physically Plausible Motion in Video Diffusion Models: SIFT proposes a self-imagination fine-tuning paradigm that discards real video inputs and starts entirely from pure Gaussian noise, forcing the diffusion model to "imagine" and generate videos solely based on LLM-generated text prompts. By combining decoupled motion feedback supervised by dual heterogeneous motion classifiers with a progressive hard case replay strategy, SIFT significantly improves the physical plausibility and motion decoupling capabilities of two backbone models, Wan and CogVideoX, without relying on decoupled motion annotation data. VLM and human evaluations consistently show that SIFT outperforms baseline methods such as SFT and VideoREPA.
Surprise Forcing: What to Remember, When to Skip in Long Video Generation: Surprise Forcing adds surprise-based historical frame selection and reduced denoising schedules for easy chunks to an existing LongLive autoregressive video model without additional training, achieving 81.88 Quality and 17.18 FPS on 60-second VBench-Long videos, although it is not faster than LongLive itself.
Target-Bench: Can Video World Models Achieve Mapless Path Planning with Semantic Targets?: This paper introduces Target-Bench, the first benchmark for evaluating whether video world models can perform mapless path planning toward natural-language semantic targets: 450 target-oriented scenarios (47 semantic categories, 72 with implicit targets) are collected with a quadruped robot together with SLAM-based metric trajectories and point-cloud maps, generated futures are decoded into metric-scale trajectories via 3D reconstruction with segment-level scale recovery, and planning ability is scored by tendency-oriented metrics (Soft Endpoint SE, Approach Consistency AC, Weighted Overall WO); the best off-the-shelf model, Wan2.2-I2V-Flash, reaches only 0.368, while fine-tuning an open-source 5B model on 325 real scenarios lifts its score from 0.068 to 0.428.
TriMotion: Modality-Agnostic Camera Control for Video Generation: TriMotion learns a shared camera-motion representation from geometry-grounded video-pose-text triplets and constrains generation with a frozen latent motion predictor, reaching 221.59 FVD and 3.6797 CamMC in pose-conditioned V2V while supporting cross-modal motion composition.
When Distillation Breaks Motion Control: Restoring Generative Trajectories for Fast Video Generators: MotionEcho retains a distilled student's few-step generation backbone and calls a fine-step teacher only when motion errors warrant clean-latent endpoint correction, achieving motion fidelity of 0.931 in 9 seconds with 8 steps on TurboBench, versus 0.876 in 114 seconds for MotionClone on full VideoCrafter2.
WorldCache: Content-Aware Caching for Accelerated Video World Models: Without retraining the video diffusion model, WorldCache improves both cache-hit decisions and feature approximation, reducing latency from 55.04 to 24.48 seconds in a Cosmos-2B image-conditioned generation experiment while retaining a PAI-Bench Overall score of 0.7977 versus 0.8030; the paper reports 2.3× acceleration, although its speedup and quality-retention figures require attention to rounding conventions.
Aligning Human Sense: Calibrated Distributional Reward Learning for Video Generation: The paper connects consensus-based annotation calibration, multidimensional reward learning, and Wasserstein-constrained GRPO, improving action pairwise accuracy excluding ties from 80.30% to 83.33% and the reported aggregate scores of three video generators, although not every individual metric improves.
Better Call CineCrew: Consistent Ultra-Long Narrative-to-Film Generation: CineCrew uses FilmDSL to organize narrative intent, cinematic staging, assets, and repair feedback into an executable production workflow without retraining video models, improving character, prop, and story continuity on 20 MovieBench narratives without leading every visual-quality or physics metric.
Controlling Motion Transfer in Diffusion Transformers via Attention Heads: HALO identifies motion-specific and structure-specialized heads in video DiTs, supervises generation latents with semantically refined displacements, and preserves layout through low-entropy reference value features, improving main-benchmark motion fidelity MF from DiTFlow's 59.6 to 66.2 without updating model parameters while maintaining target-text alignment.
CustomX: Unified Character, Action, and Scene Customization in Video World Models: CustomX unifies 3DGS scenes, multi-view characters, and text instructions as a conditional autoregressive video generation problem. Given any scene and character, users can drive the character to perform hundreds of actions using natural language, outperforming base models and specialized world models across generation quality, character consistency, and action control success rate.
DeforM: Reasoning-Guided Physics-Aware Video Generation via Spatial-Temporal Masking: DeforM converts vision-language reasoning about which objects should deform into spatial-temporal masks, then guides Wan2.2 through either training-free attention modulation or learned dual-attention injection, improving the same-backbone physical score from 3.49 to 4.40 on a deformation test set; this measures evaluator-assessed plausibility, not guaranteed compliance with physical laws.
Distribution-Alignment Bridge for Uncertainty-Aware Text-to-Video Retrieval: Addressing the failure of deterministic point matching to account for cross-modal ambiguity and one-to-many semantic associations, this paper proposes the Distribution-Alignment Bridge (DAB) framework, which models texts and videos as Gaussian distributions and aligns them via a sampling-free deterministic diffusion bridge and directional KL contrastive learning, achieving marked recall improvements and uncertainty-calibrated retrieval.
EchoStyle: Unlocking High-Fidelity Video Stylization with Reverse Data Synthesis: EchoStyle addresses content leakage from style reference images, paired data scarcity, and temporal drift in extended sequences by formulating text-driven video stylization as a unified video-to-video generation task on Wan2.2-I2V, curating the 20k-pair V-Style20k dataset via a reverse data synthesis pipeline, and enabling minute-level temporally consistent stylization through an init-follow-mode mechanism with sliding-window inference.
EraseSAE: Surgical Concept Erasure in Text-to-Video Diffusion Models via Sparse Autoencoders: Addressing concept entanglement and collateral background degradation in text-to-video diffusion models, EraseSAE introduces a decompose-attribute-erase paradigm leveraging a dual-branch Partitioned Convolutional Sparse Autoencoder (PConvSAE) to extract monosemantic features and dynamic spatiotemporal masks for spatially-modulated classifier-free guidance, achieving surgical erasure with minimal side effects.
Event-Driven Video Generation: EVD adds a lightweight token-level event head to pretrained video DiTs. It constrains the latent updates to occur only in actively interacting regions during training and inference using event gating. This reduces dynamic errors in video generation, such as objects moving prematurely, lack of contact, support relation jumps, and post-event drift.
GeCo: Evaluating Geometric Consistency for Video Generation via Motion and Structure: Addressing geometric deformation and occlusion inconsistency in video generation, GeCo introduces a dense, scale-invariant metric fusing optical flow residuals and depth reprojection errors, serving both as an interpretable diagnostic tool and as a training-free guidance loss that suppresses spurious motion.
GeoFlow: Efficient Driving Video Generation via Geometry-Aligned Priors: Addressing the high latency and spatiotemporal inconsistency of diffusion and flow matching models initialized from standard Gaussian noise, GeoFlow leverages multi-view metric geometry and spatially-adaptive noise injection to construct a geometry-aligned prior as the source distribution, straightening and shortening the transport path to surpass 40-step baseline quality in only 8 sampling steps.
GeoNVS: Geometry Grounded Video Diffusion for Novel View Synthesis: GeoNVS proposes a plug-and-play GS-Adapter. It "lifts" the diffusion features of reference views into 3D Gaussians and splats them back to the target view, injecting geometric priors in the feature space instead of at the image input level. Consequently, it significantly outperforms SEVA / CameraCtrl in camera controllability and geometric consistency, reducing translation error by up to 2x and Chamfer distance by up to 7x.
GO-Renderer: Generative Object Rendering with 3D-aware Controllable Video Diffusion Models: GO-Renderer proposes a generative object rendering framework that tightly couples feedforward coarse 3D geometric proxies with a video Diffusion Transformer (DiT), leveraging object-centric 3D coordinate maps and Negative RoPE Shifts to bypass brittle physical inverse rendering and synthesize high-fidelity, multi-view consistent object videos along arbitrary trajectories and novel lighting environments.
GraphVid: Interactive Graph-Controllable Video Generation: GraphVid introduces directed interaction scene graphs to interactively guide multi-object dynamics in single-image-to-video generation. By incorporating relation semantics via an edge-aware graph neural network and modulating a frozen video Diffusion Transformer with lightweight LoRA adapters, GraphVid outperforms trajectory-conditioned baselines while requiring only 0.6B trainable parameters and 27K video clips.
HairOrbit: Multi-view Aware 3D Hair Modeling from Single Portraits: HairOrbit reformulates strand-level 3D hair reconstruction from a single portrait into a calibrated multi-view reconstruction problem using cross-view video diffusion priors, enabling complete, high-fidelity, and view-consistent 3D hair strand synthesis even in previously invisible side and back regions.
ICDepth: Taming Video Diffusion Models for Video Depth Estimation via In-Context Conditioning: ICDepth introduces an In-Context Conditioning (ICC) framework that adapts a pretrained native 3D attention video diffusion transformer (Wan2.1) for video depth estimation, employing SAND-Attention to eliminate noise contamination and SRFM to inject semantic and multi-resolution priors, achieving state-of-the-art results using only 0.8M synthetic frames.
Joint Alignment and Distillation for Video Generation via Sample-Guided Distribution Matching: Proposes DM-Align, a unified single-stage sample-guided distribution matching framework that aggregates few-step distillation gradients and preference alignment gradients within a shared score space, completely circumventing multi-step sampling bottlenecks and post-distillation model collapse.
MagicPrompt: Ultra-Lightweight Prompt Tuning for Video Generation: Addressing the prohibitive memory overhead of fine-tuning billion-scale video diffusion models and the optimization instability of reward training during high-noise stages, MagicPrompt embeds lightweight soft prompts into frozen self- and cross-attention modules with distribution shift calibration and couples dual-space reward feedback, achieving generation quality on par with or surpassing LoRA using less than 1% trainable parameters.
Measuring 3D Spatial Geometric Consistency in Dynamic Video Generation: Addressing the insensitivity of existing video generation metrics to geometric distortions and their over-penalization of valid dynamic motion, this paper introduces SGC, a diagnostic metric that decouples static background from dynamic foreground and measures multi-region camera pose divergence to accurately quantify 3D spatial geometric consistency.
MV-Forcing: Long Multi-View Video Generation via 4D-Grounded Spatio-Temporal Self-Forcing: Addressing geometric inconsistency and compounding exposure bias in long-horizon multi-view video synthesis of dynamic scenes, MV-Forcing unifies temporal and view-wise autoregression in a single few-step diffusion model, leveraging a recurrent 4D reconstruction model as an accumulated geometric bridge alongside spatio-temporal self-forcing distillation to eliminate multi-axis error drift.
NanoVSR: Towards Real-Time Video Super-Resolution on Edge Devices: NanoVSR eliminates expensive optical flow and spatial-temporal attention in favor of a fully convolutional bidirectional recurrent framework with direct additive propagation and structural reparameterization, collapsing into a single stream of standard convolutions at inference to achieve 27.20 FPS on an NVIDIA Jetson Orin NX (25W) with 28.64 dB PSNR on REDS4.
Natural Language Camera Movement Understanding: Addressing the core weaknesses of vision-language models (VLMs) in confusing rotation with translation, swapping directions, and mistaking foreground motion for camera trajectories, this paper introduces a 17-class cinematographic taxonomy, the dual-domain ACaM benchmark, and targeted geometric augmentations to fine-tune specialized VLMs that substantially outperform commercial frontier models.
NoisEasier: Test-Time Noise Optimization for Text-to-Video Generation: NoisEasier introduces a test-time noise optimization framework based on Video Consistency Models (VCMs) that jointly refines initial latents and intermediate perturbations under multi-objective differentiable rewards and negative-aware calibration, substantially boosting fine-grained compositional video alignment without altering model parameters.
OctWorld: Long-Range World-Consistent Video Generation with Octree-based 3D Mapping: Building upon a chunk-wise autoregressive video diffusion model coupled with a dynamic octree-based TSDF spatial memory (OctMap), OctWorld synthesizes high-fidelity, world-consistent RGB-D videos across long camera trajectories and wide viewing angles, maintaining rigorous geometric and photometric coherence when revisiting explored regions.
OmniCamera: A Unified Framework for Multi-task Video Generation with Arbitrary Camera Control: OmniCamera conceptually decouples video generation into two orthogonal axes—scene content and camera pose—unifying three camera modalities (text, 3D trajectory, reference video) across three generation tasks (T2V, I2V, V2V) via a single DiT with the hybrid OmniCAM dataset and dual-level curriculum co-training.
OSVE: One Step Video Editing with One Step Diffusion Models: The first One-to-One framework adapting one-step Text-to-Image diffusion models for text-guided video editing via a single-pass structure-aware inversion encoder and unified latent frame concatenation, achieving visual quality on par with multi-step baselines while running 155–171× faster at over 15.6 FPS.
Physics Question Scene Graph: Fine-grained Evaluation of Physical Plausibility in Text-to-Video Generation: PQSG decomposes the evaluation of physical plausibility in videos into a three-layer hierarchical question graph (Object \(\rightarrow\) Action \(\rightarrow\) Physics). VLMs automatically generate questions with logical dependencies and answer them node-by-node. PQSG achieves a significantly higher correlation with human judgment than existing metrics on the FinePhyEval dataset, while precisely localizing which dimension (Object/Action/Physics) violates physical laws.
QWERTY: Training-Free Motion Control via Query-Warped Video Diffusion Transformers: Addressing the challenge of steering motion in video Diffusion Transformers (DiTs) without expensive retraining, QWERTY introduces a training-free framework that performs Semantic-Temporal Channel Decomposition (STCD) and query-only warping on 3D full attention, combined with self-guided latent optimization to achieve precise object trajectory and camera motion control.
ReCamDriving: LiDAR-Free Camera-Controlled Video Synthesis for Novel Trajectories: Addressing out-of-distribution artifact overfitting in reconstruction-repair methods and geometric sparsity in LiDAR-based camera control, ReCamDriving presents a pure vision-based two-stage progressive diffusion framework guided by dense 3DGS renderings, coupled with a cross-trajectory data curation strategy (ParaDrive), delivering state-of-the-art camera controllability and 3D-consistent novel-trajectory video generation.
Reward Lightning: Fast Video Generation via Homologous Preference Distillation: To resolve gradient conflicts caused by representational mismatch in joint video acceleration and alignment, Reward Lightning proposes a homologous preference distillation framework that shares a latent feature backbone with time-modulated attention heads, achieving high-fidelity, well-aligned video synthesis in merely 1 to 4 sampling steps.
RhymeFlow: Training Free Acceleration for Video Generation with Asynchronous Denoising Flow Scheduling: RhymeFlow breaks the conventional synchronous denoising paradigm of video diffusion models where every frame undergoes identical dense computation, introducing content-aware sequential keyframe selection, progressive asynchronous scheduling, and lightweight latent trajectory projection to achieve 1.53× to 1.78× training-free speedups with negligible perceptual quality degradation.
Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion: Ring Forcing introduces a ring-structured autoregressive video diffusion framework that embeds target video clips into distant history via sequence reversal to enforce explicit long-range retrieval, complemented by timestep-composed spatiotemporal compression and physical sparse RoPE to achieve minutes-long coherence and robust object permanence under a fixed context budget.
RotateAttention : RoPE-Aware Rotation and Range Rectification for INT4 Quantized Attention in Video Generation: Targeting the quadratic attention bottleneck in DiT-based video generation models equipped with 3D RoPE, RotateAttention introduces RoPE-aware orthogonal rotations to equalize Query-Key outliers with zero or negligible overhead, combines range-rectified affine quantization for probability matrix \(P\), and leverages selective FP16 fallback to achieve up to a 1.68× end-to-end speedup with virtually lossless visual quality.
SparseCtrl-HOI: Sparse Temporal Control for Human-Object Interaction Video Generation: Addressing the excessive annotation cost and motion rigidity of dense pose control in Human-Object Interaction (HOI) video generation, this paper proposes SparseCtrl-HOI, a DiT-based sparse temporal control framework that anchors interaction keyframes at specified timestamps via Time-Controlled Rotary Positional Embedding (TiRoPE), injects MLLM motion priors via a Q-Former, and decouples training into two stages to synthesize physically plausible, high-fidelity live-streaming e-commerce videos.
Test-Time Noise Guided Adaptation for Realistic Autoregressive Video Generation: Addressing the failure mode where autoregressive video diffusion models drift into "terminal points" devoid of plausible continuations due to accumulated exposure bias, TANGO formulates test-time noise-guided adaptation by repurposing the denoiser as a critic of its own outputs via one-step look-ahead noise distribution consistency.
TIGER: Taming Identity, Geometry, and Generative Priors for High-Quality Face Video Restoration: Addressing the long-standing trilemma between identity shift, viewpoint-entangled guidance, and perceptual realism in face video restoration, TIGER constructs a cross-source 3D parameter-fused geometry prior and an ArcFace identity prior to condition a one-step rectified flow based on Wan2.1, achieving photorealistic and temporally coherent restoration in a single forward pass under a progressive three-stage optimization.
Unified Panoramic–Gaussian Representation for Monocular 4D Scene Synthesis: Addressing the limitation of monocular 4D reconstruction being confined to observed view interpolation and generative priors suffering severe geometric distortion under large camera shifts, PanoGaussian presents a unified Panoramic-Gaussian representation that couples panoramic trajectory alignment, progressive view expansion with frozen geometry, and masked 4DGS distillation to synthesize physically coherent 4D scenes across unseen regions.
Video Generation Models Are Inherent Lighting Estimators: V-LITE draws inspiration from VFX light probe practices to recast dynamic HDR lighting estimation from single in-the-wild videos into guided video inpainting, unlocking modern video diffusion models' implicit physical lighting priors via a log-domain HDR-aware VAE and LoRA fine-tuning.
What Moves? Localized Motion Representations for Compositional Scene Control: Addressing the fundamental tension where localized motion representations require global context to resolve relative dynamics, this paper introduces a promptable, region-specific motion encoder trained via masked future point-trajectory prediction, enabling precise compositional motion transfer and localized action recognition without motion leakage.
Your Data Manifold is Secretly a Reward Model: Shell-LCC for Text-to-Video Generation: This paper proposes Shell-LCC, which models the patch-level latent manifold of high-quality SFT data as isotropic shell-like high-density regions to extract dense, differentiable, and free geometric reward signals. This significantly mitigates low-level distortions (blur, over-smoothing, motion artifacts, etc.) in T2V generation without introducing external reward models or human annotations.
ART-VSR: Adaptive Rectified Trajectories for One-Step Video Super-Resolution: ART-VSR estimates how far each token should travel along a generative path, then rectifies its latent starting state accordingly, enabling one-step restoration with a Wan2.1 video prior; it achieves 0.197 LPIPS on REDS under the authors' unified re-evaluation protocol, without dominating every fidelity or perceptual metric.
Beyond Inpainting: Unleash 3D Understanding for Stable Camera-Controlled Video Re-rendering: DepthDirector controls novel viewpoints with warped depth and preserves appearance and motion through the source video, avoiding erroneous textures inherited from warped RGB; its full model achieves RE 1.010, TE 0.053, and CamMC 1.434 under the absolute-camera protocol.
Consistent Video-to-Video Translation via Explicit Correspondences: vid2vid-long uses the current input block to retrieve matching outputs generated much earlier, then conditions video translation through per-token correspondence cross-attention, reducing five-step Rolling Forcing LPIPS on VACE-Bench from 0.427 to 0.240 while also reducing throughput from 10.99 to 8.10 FPS.
Control-DINO: Feature Space Conditioning for Controllable Video Diffusion: Control-DINO controls frozen CogVideoX with full-channel dense DINOv3 features, reduces source-appearance copying by pairing original features with multiple augmented appearances, reaches a COLMAP registration rate of 0.9932 on stride=16 T&T appearance transfer, and extends the same feature interface to 3D-guided generation.
Direct Autoregressive Diffusion Distillation via Error-aware Causal Pretraining: To resolve training collapse and prohibitive trajectory-matching overhead in autoregressive video diffusion distillation caused by architectural mismatch, this paper introduces Error Forcing—a parallelizable causal pretraining scheme that injects empirical prediction residuals into the historical context to mitigate exposure bias, enabling the causal student to directly undergo few-step distribution matching distillation without ODE trajectory matching.
DIVER: Disentangling Camera–Object and Active–Passive Motion for Video Generation: A unified flow-matching video generation framework that decouples camera from object motion via a canonical static-view dual-stream architecture and models action-consequence causality through active-passive motion dropout, enabling arbitrary viewpoint exploration and bidirectional causal interaction reasoning.
DreamWorld: Geometry-Grounded Video Diffusion for 3D-Consistent World Modeling: DreamWorld presents a geometry-grounded decoupled world modeling framework that distills rich spatial priors from 3D foundation models into a geometry video diffusion model to predict complete 3D structural representations, which subsequently guide an appearance video diffusion model to synthesize photorealistic, cross-view consistent 3D world exploration videos.
EMOSH: Expressive Motion and Shape Disentanglement for Human Animation: EMOSH utilizes the Expressive Human Model (EHM), a highly expressive 3D parametric human model, as control signals. By explicitly disentangling shape and pose parameters, it fundamentally eliminates the body shape leakage issue. Combined with a coarse-to-fine hybrid motion injection and a spatially aligned conditioning mechanism, it achieves high-fidelity human animation generation while preserving fine-grained control over expressions and gestures.
Fine-Grained Text-to-Video Retrieval for Camera-Trap Data: This paper introduces Prompting-MammAlps, the first fine-grained text-to-video retrieval benchmark for camera-trap wildlife data, and proposes an interpretable framework that decouples offline spatiotemporal action localization (SALMA) from online LLM-guided predicate code generation, avoiding hallucinations while delivering robust set-based retrieval.
GeoV2V: Geometry-Grounded Video Diffusion Model for Driving Scene Generation: Built upon the Wan 2.1 backbone, GeoV2V presents a geometry-grounded video-to-video diffusion framework and a synchronized multi-lane synthetic dataset Para4D, achieving high-fidelity, geometrically consistent, and dynamic lighting-preserving driving video synthesis under large lateral lane shifts without requiring 3D bounding box annotations.
Inference-time Motion Calibration for Video Generation: Addressing motion artifacts, severe temporal flicker, and semantic drift in pretrained video generators, this paper presents a training-free inference-time motion calibration framework that couples external VFM-based velocity guidance (DLC) with online internal attention modulation (DRC), amortized by ATR to incur only 1.3x baseline latency.
Lumos-Nexus: Efficient Frequency Bridging with Homogeneous Latent Space for Video Unified Generation: Lumos-Nexus proposes a training-efficient, two-stage unified video generation framework. During training, only a lightweight generator is fine-tuned to absorb VLM semantics. During inference, the generation responsibility is progressively transferred from the lightweight generator to a large-capacity pretrained generator via Unified Progressive Frequency Bridging (UPFB). This achieves coarse-to-fine video synthesis within a shared homogeneous latent space, balancing reasoning-driven semantic accuracy and high-fidelity visual quality.
MorphGS: Morphology-Adaptive Articulated Motion Transfer from Videos: MorphGS reformulates monocular articulated motion transfer as a target-driven analysis-by-synthesis optimization problem, coupling rig-anchored morphology parameterization with differentiable 3D Gaussian rasterization to recover motion without intermediate source reconstruction or parametric shape templates.
MotionDreamer: Universal Skeletal Motion Generation for 3D Rigged Shapes: Addressing the morphological restrictions of parametric templates and the computational bottleneck of per-instance optimization, MotionDreamer (termed SkelMo in the paper) introduces a curated dynamic dataset of 25,200 diverse rigged animations and a topology-agnostic diffusion framework with skinning-aware semantic injection and bidirectional video-skeleton fusion, synthesizing high-fidelity 3D skeletal motions across arbitrary unseen morphologies from monocular driving videos.
MVI2V: Human Centric Image to Video Generation with Multiview Consistent Appearance: MVI2V addresses the severe clothing and texture hallucination of unseen angles (e.g., garment back) in human-centric image-to-video generation by introducing an independent lightweight LoRA reference stream, cross-stream bidirectional self-attention, an inpainting training subtask, and an angular-filtered data curation pipeline, achieving high multiview appearance fidelity with only ~2.1% extra parameters.
OmniFace: Bridging the Image-to-Video Gap for High-Fidelity Face Swapping via Diffusion Transformer: OmniFace tackles the persistent trade-off between identity fidelity, dynamic attribute preservation, and temporal consistency in video face swapping by introducing SyncID-Pipe—a data pipeline that transfers image face swapping superiority to the video domain via bidirectional quadruplets—and pairing it with a DiT architecture featuring modality-aware conditioning, synthetic-to-real training, and identity-coherence reward weighting to achieve new state-of-the-art performance.
OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-based Video Editing: Presents OpenVE-3M, a large-scale, high-quality dataset comprising 3 million instruction-video editing pairs across 8 categories, along with the human-aligned OpenVE-Bench and the 5B OpenVE-Edit model, which significantly outperforms existing 14B open-source models.
PhysAlign: Learning Physical Priors for Dynamical Event-Driven Video Generation via Representation Alignment: To tackle physical violations and causal failures in video diffusion models during abrupt interactions like collisions and collapses, PhysAlign decouples physical dynamics from appearance distributions via photometric perturbation and a discrete codebook, regulates prior injection via event-aware temporal gating, and distills spatiotemporal relations from V-JEPA2 to achieve state-of-the-art physical commonsense without sacrificing visual fidelity.
PhysPO: Physics-Aware Local Preference Optimization for Physically Consistent Video Diffusion: PhysPO resolves physical commonsense violations in video diffusion by creating a counterfactual pair generation pipeline (CounterPhyPipe) that preserves global scene identity while introducing local physical violations, coupled with a local DPO framework that isolates physically evolving regions via spatiotemporal high-frequency filtering and enforces background neutrality regularization.
Predictive Structure Improves Video Diffusion Dynamics: Addressing the frequent violations of gravity, support, and object permanence in video diffusion generation, this paper proposes Latent Dynamics Optimization (LDO), which uses a frozen V-JEPA 2 predictive world model as a dynamics prior through Gram-matrix relational distillation and rollout-level GRPO policy optimization to significantly improve physical commonsense.
SA-V2V: Training-Free Subject-Aware Video-to-Video Personalization: Addressing the deep entanglement of motion, subject, and background within DiT spatiotemporal attention, SA-V2V establishes a training-free and inversion-free video personalization framework leveraging intrinsic functional attention decomposition, stochastic temporal injection, and alternating target-aware feature guidance to faithfully transfer subject identity while cloning reference dynamics and background.
Salt: Self-Consistent Distribution Matching with Cache-Aware Training for Fast Video Generation: Salt proposes the SC-DMD framework, which fixes the compositionality deficit of Distribution Matching Distillation (DMD) in multi-step inference via a semigroup-defect regularizer. Combined with cache-aware mixed-step training, it further improves the stability of autoregressive video generation, significantly boosting generation quality under an extremely low inference budget of 2-4 NFEs.
SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing: SAMA addresses the fundamental tension between precise semantic modification and temporal motion preservation by factorizing instruction-guided video editing into sparse semantic anchoring and self-supervised motion alignment, achieving strong zero-shot editing capabilities without paired editing data and setting new state-of-the-art benchmarks among open-source models.
Schroedinger’s Cat: Probabilistic Representation and Prediction of Potential Scene Kinematics: GARFIELD models the distribution over plausible future scene kinematics via structured spatio-temporal latents, supporting both coherent multi-trajectory generation and instantaneous sampling-free probability density estimation at 97× faster trajectory sampling than video models.
Spotlight: Identifying and Localizing Video Generation Errors Using VLMs: To tackle nuanced, transient generation failures in modern high-fidelity text-to-video models, this paper introduces Spotlight—a benchmark containing 1,604 human-annotated errors across 600 videos from top generators with timestamps, error categories, and natural language explanations—revealing that current leading VLMs lag behind human-level fault localization and reasoning by nearly 2× under bipartite matching metrics.
SSBP: Stage-Specialized Block Pruning for Video Diffusion Models: Addressing the issue where trajectory-averaged importance in block pruning obscures stage-critical components across coarse-to-fine video denoising, SSBP introduces stage-specialized pruned variants with cross-stage parameter sharing, guided by an inter-stage variance selector and intra-stage quality selector, delivering a 30% per-step speedup on Wan2.1-1.3B while maintaining competitive VBench generation fidelity.
STANCE: Controllable Video Generation for Structured Dynamics via Sparse-To-dense ANChored Encoding: STANCE resolves motion cue dilution and physical incoherence in controllable video diffusion by lifting sparse user hints into pixel-aligned 2.5D Instance Cues, anchoring motion tokens via Dense RoPE, and coupling RGB with a shared-backbone auxiliary structural stream as a geometry witness.
StreamEdit: Training-Free Video Editing via Few-Step Streaming Video Generation: StreamEdit reformulates training-free video editing from the dominant "data-to-data" paradigm to a "noise-to-target under source conditions" generation problem. Directly integrated into pre-trained streaming video generators (Self Forcing / LongLive), it uses dual-branch few-step sampling + a self-attention bridge + cross-attention grounding/enhancement + source-oriented guidance to achieve faster, structure-preserving, and highly reliable editing of arbitrary-length videos within 5-15 steps.
Stylized Video Generation via Decoupled Data Synthesis and Gated Style Token Injection: Addressing motion degradation from static training data and representation bottlenecks from encoder-based style compression, this paper constructs the decoupled MoStyle-5K dataset and introduces Implicit Gated Style Token Injection (IGST) with hybrid optimization and Diffusion-DPO post-training, achieving superior style fidelity and coherent motion dynamics.
SwiftWA: An Efficient Action-Centered World-Action Model: Addressing the inference latency and compounding errors of conventional world–action models that jointly roll out future visual dynamics, SwiftWA proposes an action-centered architecture with blockwise causal masking that leverages future video prediction solely as auxiliary training supervision, enabling low-latency action-only decoding at inference time to achieve a 9× speedup and superior real-world task success.
UniTemp: Unlocking Video Generation in Any Temporal Order via Autoregressive Distillation: Addressing the limitation where autoregressive video diffusion models only support forward generation and suffer from inter-block boundary flickering in backward generation due to causal 3D VAEs, UniTemp introduces blockwise anchor latents and trains a single unified student model via bidirectional distillation to unlock seamless any-direction video generation, extension, and inbetweening.
VC-VAE: Leveraging Video Codecs for Training-Efficient and High-Fidelity Video VAE: VC-VAE incorporates the classical video codec design principle of explicitly decoupling static keyframe anchors from sparse motion residuals into Video VAEs. By combining strong pre-trained image VAE initialization with a parameter-free Temporal Dynamic Difference Convolution (TDC) operator, it halves training compute while setting new SOTA benchmarks in video reconstruction and generation fidelity.
VGEdit: Unlocking Video Generation Priors for Reasoning-Informed Image Editing: Reframes reasoning-informed image editing as a short video generation process where intermediate frames act as an implicit visual reasoning chain, bridging the task mismatch via MLLM instruction transfer and unlocking physical priors with frame-collage reward-driven DiffusionNFT without ground-truth videos.
VoCa: Unified Autoregressive Modeling for Talking Audio-Video Generation: VoCa establishes a unified autoregressive generation framework that leverages an LLM to map text transcripts into shared representations driving dedicated speech and video decoders simultaneously, utilizing a speech refiner and causal rolling-window denoising distillation to achieve strictly synchronized, streaming talking-head audio-video generation.
WildWorld: A Large-Scale Dataset for Action-Conditioned World Modeling with Explicit State Annotations: WildWorld harvests over 108 million frames of video from Monster Hunter: Wilds with per-frame annotations of actions, skeletons, world states, camera poses, and depth, and derives WildBench to separate "visually plausible" from "follows the action and matches state trajectories". The experiments support the value of explicit state information but do not yet show that models reliably simulate long-horizon game dynamics.