Skip to content

EgoDyn-Bench: Evaluating Ego-Motion Understanding in Vision-Centric Foundation Models for Autonomous Driving

Conference: ECCV 2026
Paper: ECCV page
Code: https://github.com/TUM-AVS/EgoDyn-Bench
Area: Autonomous Driving
Keywords: ego-motion understanding / physical consistency / vision-language model / evaluation benchmark / video question answering

TL;DR

EgoDyn-Bench uses a deterministic oracle to map continuous ego-vehicle kinematics onto discrete motion concepts, casting ego-motion understanding as a semantic VideoQA benchmark over 3-second clips; a large-scale audit of 20+ vision-centric foundation models (closed-source MLLMs, open-source VLMs, driving VLAs) exposes a Perception Bottleneck โ€” models possess physically coherent reasoning yet cannot ground it in visual observations, frequently losing to classical geometric baselines, while supplying explicit trajectory text restores performance and reveals that ego-motion logic is drawn almost exclusively from the language modality.

Background & Motivation

Classical autonomous driving systems model ego-motion explicitly through estimable physical state variables: velocity, acceleration, yaw rate. These representations are not optional auxiliary signals but the foundation that keeps perception, planning, and control consistent with the vehicle's underlying dynamics. Recent vision-centric foundation models, especially VLMs, propose a different route: high-level reasoning and planning performed directly from visual observations, with explicit ego-state representations typically removed and motion left to be inferred implicitly from image sequences. This raises a fundamental question: do such models form a physically consistent understanding of ego-motion, or is their visual reasoning decoupled from the vehicle's dynamics?

Existing evaluation frameworks cannot answer this, and the gap follows a clear fault line. The classical side โ€” optical flow, visual odometry, displacement-based ADE/FDE metrics โ€” offers rigorous geometric tracking but no semantics at all, with no notion of what a motion conceptually is. The foundation-model side covers high-level reasoning (structured decision outputs in DriveLM and Reason2Drive, spatial relations of external objects in Ego3D-Bench and RADAR, general physics audits in DriveBench, QuantiPhy, and Morpheus) yet never checks those judgments against the ego-vehicle's own kinematic state. Whether a model's semantic interpretation is consistent with the car's own motion is an entire axis with no evaluation โ€” so a model may score well through spurious visual correlations rather than genuine kinematic understanding, a real risk for downstream generalization and safety.

Two enablers make this actionable now. First, real logs with synchronized kinematics (nuScenes) plus a simulation stack (CARLA driven by CommonRoad scenarios) allow targeted filling of the high-dynamics regimes that real-world logs under-sample. Second, a deterministic oracle can derive labels directly from measurable kinematics instead of subjective human annotation, which is what makes it possible to decouple a model's internal physical logic from its visual perception and diagnose the two separately. The core idea is to recast ego-motion understanding from numerical regression into semantic question answering, generate discrete motion-concept labels from withheld kinematic ground truth via a deterministic oracle, and measure the internal coherence of a model's answers with a set of hard Boolean physical constraints โ€” separating perception failure from reasoning failure.

Method

Overall Architecture

EgoDyn-Bench has three parts: a task formulation mapping visual inputs to semantic motion concepts, a dataset of real-world plus augmented driving sequences, and a reproducible labeling pipeline deriving ground truth from physical signals. The input-output contract is simple: given a 3-second visual observation sequence \(\mathcal{O}=\{I_0,\dots,I_N\}\) and a natural language query \(P\) about the vehicle's movement, a model must return a semantic response \(\hat{R}\) chosen from a predefined answer space (binary or multiple-choice); meanwhile a physical state sequence \(\mathcal{S}\) (speed, acceleration, jerk, yaw rate, heading) that is recorded but strictly withheld from the model is mapped by a deterministic oracle to the ground-truth answer \(R^*\) used for scoring.

The fixed 3-second window (\(\tau=3\,\text{s}\)) is a deliberate choice: long enough for a characteristic maneuver to unfold into observable spatio-temporal change, short enough to isolate distinct semantic behaviors and avoid confounding scene transitions, and short enough that sensor drift is negligible. The experiments use 3-second clips, but the formulation itself remains open across temporal horizons. The full flow, from data to scoring, is shown below.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["driving sequences<br/>nuScenes real + CARLA synthetic"] --> B["hybrid dataset construction<br/>4-stage balancing + style transfer"]
    B --> C["semantic abstraction & deterministic oracle<br/>kinematics โ†’ motion concepts"]
    C --> D["input settings & trajectory encoding ablation<br/>vision-only vs. vision + dynamics"]
    D --> E["WPCR physical consistency diagnostic<br/>correctness and coherence scored apart"]

Key Designs

1. Hybrid dataset construction: filling the dynamic blind spots of real driving logs with simulation

Real data is authentic but its dynamics distribution is badly skewed โ€” logs like nuScenes are naturally dominated by low-dynamic routine driving, so exactly the regimes with the most diagnostic value (emergency braking, high lateral acceleration) are rare. Evaluating on real data alone therefore cannot distinguish "the model cannot" from "the data never contained this motion." The paper's four-stage curation pipeline addresses this in order: dynamic characterization first quantifies how strong the low-dynamic bias of the real logs actually is; dynamic mining then extracts the rare but informative maneuvers directly from nuScenes; targeted augmentation covers the remaining under-represented regions by synthesizing dynamically diverse trajectories in CARLA, driven by CommonRoad scenarios together with an adaptable motion planner; finally all sequences undergo human validation of motion signals and labeling rules. The result is 1,000 balanced 3-second scenarios, 500 real and 500 simulated.

Simulation introduces a visual domain gap, and the paper makes a deliberate trade-off: a photometric style transfer model (NVIDIA Cosmos Transfer 2) is applied to the CARLA scenarios, but the priority is explicitly the preservation of geometric and kinematic cues rather than exact visual realism, because the benchmark evaluates motion understanding rather than photometric fidelity. Whether this trade-off holds is verified by geometric baselines across domains โ€” if optical baselines behave consistently on real and style-transferred sequences, synthetic data can legitimately be used to assess real-world ego-motion understanding (Section 5.3). The original Figure 2 shows the effect with two comparisons: CARLA-derived scenarios visibly expand the state-space coverage, and the positive-label fractions are leveled out, correcting the low-dynamic bias of nuScenes.

2. Semantic abstraction and the deterministic oracle: compressing continuous kinematics into scoreable discrete concepts

Asking models to regress speed and acceleration numerically leads to a dilemma: the regression error is hard to interpret and ambiguous (does being off by 0.5 m/sยฒ mean the model understands motion or not?), and numerical precision is a different thing from understanding motion concepts. Instead, the physical state sequence \(\mathcal{S}\) is mapped to a discrete semantic label by a deterministic oracle:

\[\mathcal{F}_\theta(\mathcal{O},P)\rightarrow\hat{R},\qquad \mathcal{G}(\mathcal{S},P)\rightarrow R^*\]

The oracle applies a deterministic thresholding scheme: thresholds are calibrated on the dataset distribution and cross-verified against standard automotive kinematics literature to ensure physical plausibility. Crucially, the thresholds are used exclusively by the oracle and are never disclosed to the evaluated models โ€” a model must infer semantic motion concepts from visual observations alone, which closes the shortcut of simply reading the label. Adjustable thresholds also mean the semantic definitions can be adapted to new domains or specific research requirements, and a sensitivity analysis that perturbs all thresholds uniformly by a factor \(\alpha\in[0.5,1.5]\) confirms that model rankings stay stable at Kendall's \(\tau>0.9\), so the conclusions do not hinge on one particular threshold setting.

The label space covers 14 question categories inside a unified prompt template, spanning two complementary reasoning dimensions: direct dynamics, probing instantaneous or aggregated motion properties such as speed regime, braking intensity, lateral acceleration, and driving smoothness; and temporal comparative queries, requiring reasoning about the ordering or co-occurrence of events across the clip (compound behaviors such as brake-then-turn or stop-and-go). Scoring uses Balanced Accuracy (the mean of class-wise recalls) and Macro-F1 rather than raw accuracy, since even a category-balanced benchmark leaves room for models to exploit linguistic priors; temporal queries that compare event ordering additionally report Temporal Accuracy.

3. Input settings and trajectory encoding ablation: separating "cannot see" from "cannot infer"

A single overall accuracy figure cannot tell whether a model failed to see the motion or saw it and failed to infer it. The paper therefore defines two input settings and three controls that peel the variables apart. Vision-only gives the model visual observations alone, uniformly sampling 10 frames from each 3-second clip (โ‰ˆ3.3 FPS) โ€” enough temporal resolution for macroscopic dynamic reasoning while keeping evaluation over 20+ models computationally feasible. Three ablations surround it: no visual input at all (\(\mathcal{O}=\{\varnothing\}\)), a single static frame (\(\mathcal{O}=\{I_0\}\)), and shuffled frame order.

Vision + Dynamics additionally hands the model explicit ego-motion signals as structured text, and further ablates four textual encodings to isolate the effect of representation format itself: a high-level Summary of 8 scalar statistics covering kinematic means and extrema (e.g. max/mean speed, max lateral acceleration); a dense kinematic Timeseries giving per-channel \(v,a,\omega,j\) values at \(N\) evenly spaced timesteps; spatial Coordinates giving zero-centered \(x,y\) waypoints and heading \(\theta\) at \(N\) timesteps; and a Full combination of timeseries and coordinates. All four carry identical physical information and differ only in form, and every explicitly provided kinematic quantity is temporally aligned with the \(N\) subsampled images.

This is what makes the design diagnostic rather than merely descriptive: if the same physical information changes Balanced Accuracy by more than fifteen points when the encoding changes, the model must genuinely process the dynamic representation instead of forwarding something that already spells out the answer โ€” which is precisely what determines whether the later claim that "representation quality matters more than parameter scale" holds up.

4. WPCR, a physical-consistency diagnostic: scoring coherence separately from correctness

Accuracy has a structural blind spot: one model may answer a few questions correctly while contradicting itself logically, while another may reason physically and consistently but simply fail to align with the visual observation โ€” the two are indistinguishable in Balanced Accuracy. The paper therefore introduces the Weighted Physics Consistency Rate (WPCR), which is not an accuracy against ground truth but a diagnostic of internal physical coherence: whether the full set of answers a model gives for a single clip (\(\tau=3\,\text{s}\)) satisfies the physics of motion.

The constraints are a set of Boolean implication rules \(\mathcal{R}=\{r_m\}\), ten hard rules in four groups: heading / lateral dynamics (e.g. "Heading Change = Yes โ‡’ Turn Direction โ‰  Straight", "Turn Direction = Straight โ‡’ No Significant Heading Change"), speed regime / mean speed (e.g. "Speed Regime = Highway โ‡’ Mean Speed is Not Low", "Speed Regime = Stopped โ‡’ Speed Trend โ‰  Accelerating"), the compound brake-then-turn ("Brake-then-Turn = Yes โ‡’ Braking Intensity โ‰  None"), and stop-and-go ("Stop-and-Go = Yes โ‡’ Speed Regime โ‰  Stopped"). The hard form rather than a soft score is a deliberate choice: ego-motion concepts are physically discrete and mutually exclusive, so there is no meaningful notion of partial correctness, and a soft metric would only hide systematic reasoning failures behind gradual penalty curves โ€” masking exactly the architectural deficits the benchmark exists to expose.

To ensure that high scores come from genuine coherence rather than from evading committed predictions, each clip's consistency contribution is weighted by the fraction of rules it triggers. With \(\mathcal{C}\) the set of evaluated clips, \(T_c\) the number of applicable rules and \(V_c\) the number of violations for clip \(c\):

\[\text{WPCR}=\frac{1}{|\mathcal{C}|}\sum_{c\in\mathcal{C}}\mathbb{1}\big[V_c=0\wedge T_c>0\big]\cdot\frac{T_c}{|\mathcal{R}|}\]

(โš ๏ธ Refer to the original paper for the exact form of the indicator; the intended meaning is that only clips with zero violations and at least one triggered rule count, and each counts in proportion to its triggered-rule fraction.) A model that answers nothing, or hedges, triggers few rules and is therefore scaled down automatically. A companion metric, Physics Coverage (PCov) โ€” the mean fraction of constraints triggered per clip, \(T_c/|\mathcal{R}|\) โ€” says whether WPCR is even meaningful: a low PCov would make a high WPCR uninformative, whereas a high PCov confirms the consistency rules are actively exercised across the benchmark.

A Worked Example

One constructed 3-second clip illustrates the annotate โ†’ answer โ†’ score loop. Numbers are illustrative for exposition, not reported values.

Take a synthetic high-speed car-following clip: the withheld physical state \(\mathcal{S}\) shows a mean speed of 22 m/s, essentially constant heading, almost no braking, and low lateral acceleration. The oracle \(\mathcal{G}(\mathcal{S},P)\) turns this into labels: Speed Regime = Highway, Mean Speed is Not Low, Turn Direction = Straight, Braking Intensity = None, Heading Change = No, Stop-and-Go = No.

The model answers most of these correctly (Speed Regime = Highway, Braking Intensity = None, Turn Direction = Straight, Stop-and-Go = No), but it also answers Speed Trend = Accelerating (wrong; the clip is roughly constant-speed) and, in addition, answers Heading Change = Yes โ€” which directly contradicts its own answer "Turn Direction = Straight." At scoring time this clip triggers six rules (R1 because Heading Change = Yes, R3 and R4 because Turn Direction = Straight, R5 because Speed Regime = Highway, plus two compound rules), and it violates R1: the model claims a heading change while also claiming the turn direction is straight. With \(V_c=1\), the clip's consistency contribution drops to zero and its weight \(T_c/|\mathcal{R}|=6/10\) is voided. Its semantic Balanced Accuracy is not bad at all (only two of fourteen categories wrong), yet WPCR has already flagged it as physically incoherent โ€” exactly the failure mode WPCR targets: the answers are not wildly wrong, they are assembled without a shared, self-consistent concept of motion.

Loss & Training

As a diagnostic benchmark, this work involves no model training and no loss function. The only "strategy" content is the evaluation protocol itself: uniform sampling at โ‰ˆ3.3 FPS, a unified prompt template, a deterministic parser that maps free-text model responses back into the label space, and temporal alignment between the explicit kinematic inputs and the images.

Key Experimental Results

Main Results

The table below is the leaderboard under the vision-only setting, where models see images with no explicit kinematic information. Geometric baselines marked ยน are evaluated only on the geometrically answerable subset of 6 of the 14 questions, so their numbers are not directly comparable to the 14-question VLM numbers โ€” they serve to draw a line against VLMs on that same subset. All values are percentages.

Model Acc โ†‘ BAcc โ†‘ Macro-F1 โ†‘ Temporal Acc โ†‘ WPCR โ†‘ PCov โ†‘
Geometric / learned baselines (6/14 subset)
Flow Heuristic [13,22] / 47.0 42.2 48.5 46.5 79.6
Visual Odometry [11,23,30] / 63.8 63.0 62.7 48.0 97.1
RAFT Flow [35] / 59.6 56.8 53.9 47.5 85.3
TartanVO [39] / 54.4 50.7 65.9 42.4 99.8
Visual-input ablation (Qwen3-VL-8B)
No vision \(\mathcal{O}=\{\varnothing\}\) 42.3 33.5 19.3 33.3 20.2 100.0
Single static frame \(\mathcal{O}=\{I_0\}\) 47.5 37.0 30.0 41.0 97.4 83.9
Shuffled frames 49.1 39.0 31.7 40.4 98.4 84.8
Ordered 10 frames (default) 49.8 38.9 33.0 39.3 97.9 84.6
Closed-source MLLMs
Gemini 3 Pro 53.6 47.0 44.3 47.3 59.3 98.4
GPT-5.1 54.3 45.2 40.1 43.0 96.3 76.9
Claude Sonnet 4.5 49.6 38.0 32.9 36.8 83.4 94.4
gemini-2.0-flash 46.2 36.5 32.4 37.2 54.1 97.5
Open-source VLMs
Cosmos Reason 2-8B 48.8 39.9 35.8 41.0 92.2 80.6
Qwen3-VL-8B 49.8 38.9 33.0 39.3 97.9 84.6
Qwen3-VL-4B 49.7 39.8 34.4 39.7 82.9 90.0
Qwen3-VL-2B 47.1 37.4 31.2 42.3 39.1 99.8
InternVL3.5-38B 46.5 37.6 26.8 44.8 72.6 72.4
InternVL3.5-8B 47.0 38.8 30.0 42.7 48.6 78.0
Camreasoner-8B 45.0 36.9 27.4 37.2 46.3 92.2
Driving VLAs
RoboTron-Drive 48.0 38.6 32.0 41.3 69.8 96.8
ImpromptuVLA 47.3 37.8 28.9 40.2 41.1 72.8

Ablation Study

The first ablation separates explicit dynamics from visual input (all rows on Qwen3-VL-8B; the last four rows include visual frames plus trajectory text):

Setting Acc โ†‘ BAcc โ†‘ Temporal Acc โ†‘ WPCR โ†‘ PCov โ†‘
Trajectory text only, no visual frames (Summary) 65.9 59.6 53.1 33.0 100.0
+ visual frames, Summary encoding 63.6 54.7 43.4 89.7 90.9
+ visual frames, Timeseries encoding 69.7 62.2 76.7 92.6 90.6
+ visual frames, Coordinates encoding 55.3 46.6 49.9 97.9 62.2
+ visual frames, Full encoding 69.0 61.3 77.3 97.8 78.1

The second ablation swaps the visual domain, verifying that the difficulty comes from ego-motion reasoning itself rather than a simulation-to-reality gap:

Model Real (nuScenes) โ†‘ Raw sim โ†‘ Style-transferred โ†‘ ฮ”max โ†“
Qwen3-VL-8B 40.0 39.7 41.0 1.3
Visual Odometry [11,23,30] 63.5 62.4 64.0 1.6
Flow Heuristic [13,22] 49.3 39.0 44.4 10.3
RAFT Flow [35] 65.5 60.5 54.6 10.9
TartanVO [39] 51.3 51.0 56.0 5.0

Key Findings

  • Classical baselines beat VLMs. On the geometrically answerable 6/14 subset, Visual Odometry reaches 63.8% BAcc, clearly above GPT-5.1 (55.1%), Gemini 3 Pro (59.6%), and Qwen3-VL-8B (52.3%). More tellingly, temporally shuffling the input frames leaves performance essentially unchanged (39.0 shuffled vs. 38.9 ordered), so the deficit is not in the visual encoder but in downstream temporal integration.
  • A scale-and-domain paradox. The best closed-source model, Gemini 3 (BAcc 47.0%), exceeds the best open-source 8B model, Cosmos Reason 2-8B (39.9%), by only 7.1 points despite an orders-of-magnitude scale difference; the domain-specific RoboTron-Drive (38.6%) matches or falls below general open-source models of comparable size. Raising temporal resolution to 10 FPS buys only +0.4pp, confirming the bottleneck is structural rather than input-limited.
  • Predictive fallback bias. Raw accuracy sits consistently above balanced accuracy across VLMs (Qwen3-VL-8B: 49.8 vs. 38.9), meaning that when physical reasoning fails, models collapse onto a single dominant answer instead of discriminating across classes. The Visual Odometry baseline nearly eliminates this gap (65.1 vs. 63.8), suggesting explicit geometric representations anchor predictions and mitigate response bias.
  • Explicit dynamics help across the board โ€” but how they are given matters. With identical physical information, Coordinates encoding yields 46.6% BAcc against 62.2% for Timeseries, a 15.6-point spread: models must actually process the dynamic representation rather than unconditionally forwarding a label. Structured kinematic data also outperforms high-level semantic summaries on every model.
  • Motion reasoning almost entirely bypasses vision. Replacing visual frames entirely with trajectory text lifts Qwen3-VL-8B from 38.9% to 59.6% BAcc (+20.7pp); reintroducing visual frames on top of that recovers only a further 2.6pp under the best encoding (to 62.2%), and with suboptimal encodings performance regresses below the text-only baseline (Summary: 54.7% < 59.6%). This asymmetry is strong evidence that visual observations contribute a negligible temporal signal, and can even interfere.
  • Consistency depends only on the presence of visual context. WPCR jumps from 20.0 with no visual input to 97.4 with a single static frame, then barely moves as more frames are added (ordered 10 frames 97.9, shuffled 98.4). Physical consistency is thus driven by the existence of any visual context, not by temporal reasoning over the frame sequence; adding trajectory text further reduces the visual contribution to near zero.
  • Domain invariance holds. Qwen3-VL-8B scores 40.0/39.7/41.0 BAcc across real, raw simulated, and style-transferred domains (ฮ”max 1.3), and Visual Odometry scores 63.5/62.4/64.0 (ฮ”max 1.6); the stability of geometric baselines across domains also supports using synthetic data to assess real-world ego-motion understanding. Separately, 80 of the 500 style-transferred clips (16%) carry spatial artifacts from upstream CARLA rendering; because these are temporally stable within a clip, optical flow is preserved while photometric quality degrades. Per-clip accuracy on these 80 clips differs from the other 420 by โ‰ค3pp across leaderboard models with mixed direction, further confirming that photometric quality is not meaningfully exploited.

Highlights & Insights

  • A deterministic oracle turns annotation from human judgment into derivation from measurable kinematics. Labels come straight from physical signals, so they are reproducible, threshold-adjustable, and amenable to sensitivity analysis (Kendall's \(\tau>0.9\) under uniform threshold perturbation with \(\alpha\in[0.5,1.5]\)), while human annotation noise is removed as a confounder entirely. The "physical signal โ†’ deterministic thresholds โ†’ discrete labels" recipe transfers to any embodied setting with high-precision state logs โ€” robot manipulation, racing, drones.
  • WPCR's two anti-gaming safeguards are worth reusing. Weighting by the triggered-rule fraction closes the "no prediction, no violation" escape hatch, and reporting PCov alongside declares whether the rules were actually exercised, preventing inflated coherence under low coverage. Any internal-consistency metric can adopt this coverage guardrail.
  • "Shuffling frames costs nothing, but swapping vision out boosts performance" is a general-purpose modality probe. To find out which modality a model actually relies on, replace that modality with its minimal equivalent text description and watch whether performance rises or falls; a rise means the modality was never really used. It costs almost no extra annotation and transfers directly to video QA and robotic physical-reasoning evaluation.
  • Hard Boolean constraints rather than soft scores follow from a principle. Motion concepts are discrete and mutually exclusive, so a soft metric would bury systematic reasoning failures inside gradual penalty curves and hide the architectural deficit the benchmark was built to expose โ€” a useful methodological reminder when designing a diagnostic benchmark: metrics should amplify failure signals, not smooth them.
  • "Representation quality over parameter scale" is directly actionable for pretraining. Under Timeseries encoding, smaller open-source models match or exceed closed-source MLLMs on temporal and consistency metrics, implying that the performance driver is the quality of the dynamic representation and its alignment, not the parameter count; the paper accordingly argues for investing in physical-alignment strategies during pretraining rather than continuing to scale.

Limitations & Future Work

  • Limitations the authors acknowledge: EgoDyn-Bench targets short 3-second horizons where individual maneuvers are cleanly separable and attributable, and extending to long-horizon and multi-agent scene-level reasoning is a natural next step. As a grounding diagnostic it isolates whether models align physical concepts with visual observation rather than measuring closed-loop driving performance.
  • Narrow observation setup: visual input comes from a single front-facing stream (nuScenes front camera for the real portion), so multi-view / surround fusion and side-rear motion โ€” for example an approaching vehicle during a lane change โ€” lie outside the evaluation, even though these are where motion understanding most often breaks in real driving.
  • Limited data diversity: the real portion comes only from nuScenes (Boston/Singapore), giving narrow geographic and cultural coverage; the simulated portion depends on CARLA plus CommonRoad scenario libraries, so synthetic high-dynamics regimes still differ from real emergencies in kinematic detail โ€” and it is precisely those regions that are used for balancing, which amplifies the impact.
  • Some evaluated entries lack fully public version details (API snapshots of frontier closed-source models, the exact scale configuration of individual VLAs), so cross-model comparison depends on the snapshot date; the "Parsable" column also shows a few models with unparseable responses, and the effect of those on final scores is not quantified.
  • Improvement directions: the paper itself points to bringing kinematic encoding and explicit alignment strategies into pretraining so that visual perception and dynamic representations align natively; going further, the WPCR constraints are currently used only for evaluation but could be inverted into an auxiliary training regularizer that penalizes "correct but incoherent" answers during training.
  • vs DriveLM / Reason2Drive (logical reasoning and decision interpretability): they represent behavior as discrete linguistic instructions and assess semantic plausibility, but such instructions are inherently discrete and lack continuous temporal constraints, so they cannot guarantee that a sequence of decisions corresponds to a kinematically feasible maneuver. EgoDyn-Bench requires reasoning grounded in temporally continuous motion and tests whether the semantic interpretation matches the kinematics.
  • vs Ego3D-Bench / RADAR (object-centric spatial intelligence): they evaluate spatial relations and volumetric overlap of external objects; ego-motion understanding is fundamentally self-referential and requires anchoring the agent's own motion state within the temporal visual stream. That is the axis EgoDyn-Bench adds.
  • vs nuPlan / ScenePilot-Bench / EgoTraj-Bench (closed-loop and trajectory): they evaluate trajectory generation with displacement-based metrics such as ADE/FDE, yet spatial accuracy guarantees neither kinematic feasibility nor compliance with underlying physical concepts. EgoDyn-Bench does not assess motion generation at all; it provides an isolated diagnostic of the model's intrinsic high-level physical understanding.
  • vs DriveBench / QuantiPhy / Morpheus (general physics audits): DriveBench exposes "text-only resilience," where models rely on language priors rather than visual grounding, while QuantiPhy and Morpheus test general conservation laws and external object collisions. EgoDyn-Bench isolates embodied kinematics โ€” whether a model can infer its own mechanically valid motion states directly from sequential visual streams โ€” and its finding that the visual contribution is nearly zero corroborates DriveBench's text-only resilience while pushing the evidence one step further: replacing vision with text actually helps.

Rating

  • Novelty: โญโญโญโญ โ€” the first benchmark to isolate kinematic ego-motion consistency as its own evaluation axis, and the combination of deterministic-oracle labels with hard WPCR constraints is new for driving benchmarks; the underlying paradigm, however, is still "a VLM diagnostic benchmark along a new dimension," with limited methodological novelty.
  • Experimental Thoroughness: โญโญโญโญโญ โ€” 20+ models, 1,000 three-second scenarios, 14,000 QA pairs, plus geometric baselines, four trajectory-encoding ablations, four visual-input ablations, a three-domain gap analysis, and a threshold sensitivity analysis: broad coverage with well-designed controls.
  • Writing Quality: โญโญโญโญ โ€” the diagnostic chain is clear, with the "Perception Bottleneck" and "functional disentanglement" conclusions building on each other, and the benchmark comparison table (Table 1) positions the work cleanly; the downside is that exact thresholds, prompt templates, and the advanced embedding ablations live only in the supplementary material, leaving readers to chase them down to judge how the 3-second / 10-frame setting affects the conclusions.
  • Value: โญโญโญโญ โ€” the conclusions give direct guidance for embodied and driving pretraining (representation quality beats scale; explicit alignment beats more visual frames), and the diagnostic protocol itself is reusable; it loses a point for covering only 3-second monocular clips and for resting heavily on the extreme control setting of replacing vision with trajectory text.