PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models

Conference: ECCV2026
Paper: ECCV Paper
Project: https://lucaria-academy.github.io/PhysRVG/
Area: Video Generation
Keywords: video continuation, rigid body motion, physically verifiable rewards, collision detection, reinforcement learning

TL;DR

PhysRVG evaluates video continuation through collision-weighted deviations from reference trajectories and adds Flow Matching supervision when reinforcement learning exploration fails, reaching 0.64 IoU and 15.03 trajectory offset on PhysRVGBench, although these results primarily establish trajectory-fitting performance in restricted rigid-body scenarios.

Background & Motivation

Video generators can produce sharp, smooth footage while predicting incorrect trajectories when objects collide, rebound, or swing. Visual continuity does not imply correct dynamics: a ball passing straight through another ball can still look temporally smooth. More physics-related training videos can improve appearance and motion statistics, while simulator-generated trajectories can constrain outputs, but the former lacks precise motion-error feedback and the latter requires additional motion planning and control inputs.

Reinforcement learning offers another route, provided its reward distinguishes plausible appearance from correct motion. A vision-language model (VLM) or a human rating may overlook a brief collision; raw trajectory error can favor easy straight-line motion. Meanwhile, every candidate in a difficult rollout group may be poor. Relative ranking alone cannot necessarily point toward correct behavior, making pure Group Relative Policy Optimization (GRPO) unstable.

The paper first restricts the task to rigid-body video continuation with multiple observed frames, then extracts reference trajectories from real videos to measure object positions and collision events. Core Idea: use observable trajectory error to guide exploration, and use the absolute quality of a candidate group to decide when to add reference-video supervision, enabling reasonable candidates before relying more heavily on reward-driven improvement.

Method

Overall Architecture

The input consists of a text prompt and the first 5 video frames; the output is the subsequent sequence, rather than an arbitrary future inferred from one image. Training starts with multi-frame conditioning adaptation, generates a candidate group under each condition, scores it with a collision-weighted trajectory reward, and updates the model through MDcycle joint optimization.

Reference videos supply supervision targets and reference trajectories. Their future frames are not provided as the future input to be generated; reference trajectories are primarily used for training rewards and evaluation. At inference time, the model continues from text and initial video frames without requiring a simulator to plan the complete future motion.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Text and first 5 frames"] --> B["Multi-frame conditioning adaptation"]
    B --> C["Candidate videos under one condition"]
    C --> D["Collision-weighted trajectory reward"]
    G["Reference video and initial annotations"] --> D
    D --> E["MDcycle joint optimization"]
    G --> E
    E --> F["Update model and resample"]
    F --> C

Key Designs

1. Multi-frame conditioning adaptation

A single image generally does not determine initial velocity, so one starting position admits multiple plausible futures. Scoring against one reference trajectory can therefore penalize a model simply for choosing a different initial velocity. The authors use multiple observed frames to constrain the initial state, converting image-to-video generation into video-to-video generation. Specifically, they replace the image condition of a pretrained diffusion Transformer with the first 5 frames and perform full-parameter fine-tuning with Flow Matching.

This stage supplies basic conditioned continuation, not completed physics learning. The model can still generate visually coherent futures with random trajectories. Its benefit is to make later reward comparisons more meaningful: candidates share initial motion information, so errors are less dominated by velocity ambiguity inherent to a single image. Multiple two-dimensional observations still do not automatically reveal mass, friction, depth, or external forces; the determinism argument depends on the restricted setting rather than a universal physical guarantee.

2. Collision-weighted trajectory reward

During data preparation, the authors annotate object centers in the first frame, marking both active and passive bodies for collisions and one object for other scenarios. SAM2 tracks objects separately in reference and candidate videos to produce masks. The mean pixel coordinate of each mask becomes the two-dimensional object center. The reward thus compares position sequences over time without asking another large model to subjectively judge whether an entire video obeys physics.

The basic Trajectory Offset (TO) aggregates Euclidean distances between corresponding object centers at corresponding frames. Minimizing this global discrepancy alone can neglect brief but decisive contact events: most frames may look reasonable while objects penetrate or fail to rebound at impact. The authors describe this preference for simple motion and avoidance of complex collisions as reward hacking, motivating event-dependent weighting.

Collision detection first differences positions to obtain velocities and then differences velocities to obtain accelerations, locating impacts through abrupt acceleration changes. Collision frames receive weight \(w^{col}\), adjacent frames receive \(w^{adj}\), and other frames receive \(w\). Weighting neighboring frames also emphasizes motion changes around contact. The following reward relation is transcribed from Eq. (9), retaining its time indices and normalization:

\[ O_c=\frac{1}{NT}\sum_{s=1}^{N}\sum_{t=T_{\mathrm{obs}}}^{T}w_t\left\|p^{\mathrm{gt}}_{t,s}-p^{\mathrm{sample}}_{t,s}\right\|_2,\qquad R=-O_c. \]

Here \(N\) is the number of annotated objects, \(T\) is the sequence length, and \(p_{t,s}\) is a two-dimensional center. Removing temporal weights gives the paper's unweighted offset form. Smaller error produces greater reward. The objective does not explicitly evaluate momentum conservation or solve Newtonian equations: reference videos supply the target motion. Consequently, verifiability means measurable trajectory discrepancy, not verification of every physical law.

Equation extraction in the cache contains misplaced characters; the expression above is reconstructed from the adjacent explanation. The collision threshold, numerical values of the three weights, and complete rules for constructing the weighting timeline are delegated to an Appendix absent from the supplied material. They cannot be recovered here. The training reward's weighted \(O_c\) should also not be conflated with the unweighted TO reported in result tables.

3. MDcycle joint optimization

For candidates generated under one condition, GRPO constructs advantages by subtracting the group-average reward and dividing by the group reward standard deviation, encouraging relatively better samples. When every candidate fails, however, the best sample can remain far from correct motion. This explains why pure reinforcement learning can improve easy scenarios while degrading difficult or out-of-distribution cases: relative advantage does not encode the absolute fact that the entire group is inadequate.

MDcycle additionally computes the group-average weighted offset \(\bar O_c\). If it exceeds a threshold, the method adds a Flow Matching loss on the reference video to the RL loss, supplying finer-grained supervision. Otherwise, it retains only the RL objective for reward-driven exploration. Although the paper describes Mimicry and Discovery phases, Eq. (10) does not switch mutually exclusive losses: difficult groups retain RL and receive additional supervision.

\[ \mathcal L=\mathcal L_D+\alpha\mathcal L_M,\qquad \alpha=\begin{cases}1,&\bar O_c>\mathrm{Threshold},\\0,&\text{otherwise}.\end{cases} \]

Here \(\mathcal L_D\) is the negative GRPO objective and \(\mathcal L_M\) is the Flow Matching loss. As candidate quality improves, fewer groups trigger additional supervision, shifting training from imitation dependence toward more exploration. Out-of-distribution cases are not identified by a separate distribution classifier; that term is the authors' interpretation of difficult high-error cases. The actual gate uses the measurable average offset.

A Worked Example

Consider the two-ball collision setting discussed in the paper: the first 5 frames show one ball approaching another. The model generates multiple futures, and each first-frame ball center prompts SAM2 to extract the reference and candidate trajectories. A candidate in which a ball continues straight through the target incurs both post-contact position error and greater weighting near the collision.

The group-average weighted offset is then compared with threshold 8. Above 8, GRPO is retained and reference-video Flow Matching supervision is added; otherwise, only GRPO is used. The 5 observed frames and threshold 8 come from the paper, while the candidate behavior is an explanatory illustration rather than a newly reported experiment.

Loss & Training

Training follows I2V → V2V → RL: full-parameter fine-tuning first adapts conditioning, and LoRA is then used for MDcycle post-training. GRPO retains a clipped importance-ratio objective and KL regularization without training a separate value model. Flow Matching learns the velocity field from noise toward reference videos.

To reduce exploration cost, the authors use hybrid ODE-SDE sampling, applying stochastic SDE sampling at only two consecutive steps within a selected window and ODE sampling elsewhere. The best window in Table 3 is the 75%–100% time interval, described as the high-noise region. This is a flow-matching time coordinate and should not be mistaken for the final near-completion phase of generation. The corresponding configuration reports NFE 2, versus 16 for the full-range SDE comparison.

Table 3 also reports the best noise intensity as \(\sigma_t=1.0\) and the threshold as 8. The cache lacks the Appendix's full pseudocode, learning rate, LoRA rank, and hardware details, so these settings do not constitute a complete reproduction recipe.

Key Experimental Results

Main Results

The paper uses approximately 10M training videos and separately describes about 700 manually curated high-quality rigid-body videos from real recordings, games, and other sources. Approximately 50 of these form a test set excluded from training. The 700 videos are not all independent test examples, and the method should not be described as trained on only 700 videos.

The following selection comes from Table 1. The VBench total score evaluates visual quality; VideoPhy-2 SA and PC denote semantic adherence and physical commonsense metrics. PhysRVGBench IoU measures spatial overlap of motion masks, while TO measures two-dimensional center-trajectory discrepancy. Full IoU aggregation details are referred to the Appendix, which is absent from the cache.

Model	VBench Total ↑	SA ↑	PC ↑	IoU ↑	TO ↓
Wan2.2 14B	76.83	0.64	0.34	0.12	162.40
Kling2.5	78.77	0.70	0.41	0.23	103.22
HunyuanVideo	76.97	0.60	0.32	0.10	181.62
Magi-1	77.16	0.67	0.38	0.27	113.42
VideoMAR	74.31	0.62	0.32	0.31	109.43
CosmosPredict2	76.91	0.70	0.39	0.33	79.67
PhysRVG	78.89	0.76	0.44	0.64	15.03

Ablation Study

The following selection comes from Table 2, retaining the authors' sequential-addition ordering. MDcycle introduces gated supervision within the RL framework; it does not append a separate video generator.

Config	VBench Total ↑	SA ↑	PC ↑	IoU ↑	TO ↓
Baseline	74.86	0.57	0.21	0.15	162.78
+Fine-tuning (FT)	77.45	0.63	0.34	0.41	48.60
+GRPO	77.91	0.69	0.38	0.52	26.38
+MDcycle	78.53	0.74	0.43	0.62	19.47
+Collision Detect	78.89	0.76	0.44	0.64	15.03

Key Findings

• MDcycle reduces TO from 26.38 to 19.47, and collision detection reduces it further to 15.03. These address unstable exploration and the dilution of important events in average error, respectively.
• The visual total score exceeds Kling2.5 by only 0.12, whereas TO decreases from CosmosPredict2's 79.67 to 15.03. Physical trajectory metrics reveal the intended benefits more clearly than overall visual scores.
• Table 3 compares sampling configurations under the same data and GPU-hours budget: the high-noise-window configuration achieves IoU 0.64 and TO 15.03, versus 0.55 and 27.24 for full-range SDE. This is a budget-controlled configuration comparison, not an inference speedup factor.

Highlights & Insights

• The reward moves from subjective impressions of an entire video to localized spatiotemporal errors. Collision weighting recognizes that brief events can determine physical correctness even when steady motion occupies most frames.
• MDcycle combines relative advantage with absolute quality. The former sets within-group preferences, while the latter determines when external supervision is needed, a combination transferable to generation tasks with measurable errors.
• Task conditioning is part of reward design. Providing initial motion through multiple frames makes comparison against one reference trajectory more defensible than in a single-image setting.

Limitations & Future Work

• The following are reading-based assessments of the main paper. The authors place their formal limitations, future work, and ethical discussion in an unavailable Appendix; those sections have not been verified.
• Approximately 50 test videos, four rigid-body motion categories, and single-seed main comparisons do not establish broad physical generalization. The tables report no confidence intervals.
• SAM2 failures, occlusion, camera motion, and two-dimensional projection can contaminate trajectory rewards. Center trajectories also cannot fully assess rigid-body rotation, shape preservation, genuine contact, or energy conservation.
• I2V and V2V baselines receive different initial motion information. Standardized generation settings and designed prompts do not establish identical conditioning information. The authors also explicitly acknowledge the difficulty of strictly fair comparisons with simulator-based methods.
• Proprietary training videos, missing collision parameters, and incomplete base-model details limit independent reproduction. Controlled simulation data, tracking confidence, and tests across objects and dynamics parameters could help distinguish reference-trajectory fitting from physical generalization.

Related Work & Insights

• vs PhysGen / GPT4Motion: These methods use simulators to provide motion priors, whereas PhysRVG uses reference trajectories as post-training feedback. It reduces dependence on complete motion planning at inference but requires reference training videos.
• vs PhysMaster / PhyT2V: These methods use preferences or MLLM evaluation, whereas PhysRVG measures two-dimensional trajectory discrepancy. Its signal is more specific, but its scope depends on trackable rigid bodies and reference trajectories.
• vs Flow-GRPO / MixGRPO: PhysRVG builds on stochastic sampling and group-relative optimization for flow models, adding physical rewards and supervision gates for difficult groups. The transferable lesson is to diagnose failures in rewards and exploration separately before modifying each.

Rating

• Novelty: 4/5. The combination of measurable trajectory rewards, collision events, and quality-gated supervision targets a clear problem.
• Experimental Thoroughness: 3/5. Main comparisons and sequential ablations are useful, but test scale, single-seed evaluation, and reproduction materials limit the conclusions.
• Writing Quality: 4/5. Motivation and ablations form a coherent argument, although claims of internalizing physical laws exceed the available evidence.
• Value: 4/5. The method provides practical reward design and difficult-sample optimization ideas for rigid-body video post-training.