LangDriveCTRL: Natural Language Controllable Driving Scene Editing with Multi-modal Agents¶

Conference: ECCV 2026
Paper: ECCV 2026 · Project Page
Area: Autonomous Driving
Keywords: driving scene editing, multimodal agents, scene graph, trajectory generation, video diffusion

TL;DR¶

LangDriveCTRL reconstructs each real driving video into a 3DGS scene graph of a static background plus dynamic object nodes with trajectories, then lets an LLM orchestrator dispatch multimodal object grounding, node editing, behavior editing, behavior review, video diffusion and video review agents and tools, turning natural-language instructions into object-level removal/insertion/replacement and multi-object behavior editing with nearly twice the instruction alignment of prior state of the art, while preserving photorealism, scene structure and traffic realism.

Background & Motivation¶

Synthetic data is increasingly adopted to compensate for the limited diversity of real driving logs, especially for safety-critical scenarios whose collection is prohibitively expensive. Traditional simulators such as CARLA and AirSim can generate diverse scenarios, but they rely on hand-crafted 3D assets, engineer-written scenario scripts and human-in-the-loop refinement, which does not scale. Two lines of work have recently attacked this with natural language: LLM-agent pipelines that reconstruct the scene into an explicit 3D representation and orchestrate modular tools (ChatSim, ChatDyn, AutoVFX), and implicit world models that edit the video directly in pixel space from text (Cosmos).

Each line is missing exactly what the other has. The agentic pipelines have three concrete defects: they reason only in the text modality without bringing in multimodal scene context, so they neither localize the object the instruction means nor generate trajectories that fit it; they composite the background and the inserted object naively, which falls apart under large viewpoint or lighting changes; and — most importantly — they verify nothing along the way, so per-step errors accumulate into the final video. World models sit at the opposite corner: the output is strikingly photorealistic and the behavior plausible, but controllability is weak. They expose no object-level editing interface, they may silently alter the background or insert objects the user never asked for, and they are feed-forward, so there is no way to roll back a mistake.

The tension is therefore precise: getting controllability and photorealism at once requires language reasoning that is grounded in multimodal scene evidence and a loop that can send intermediate results back for correction, and existing work splits these two properties across two camps. This paper's angle is to anchor editing on an explicit scene graph, let every agent and tool read and write that graph, and insert a reviewer agent at both behavior generation and video rendering, turning a feed-forward pipeline into a closed loop. Core idea: make the 3DGS scene graph the shared interface between language and the 3D scene, have an orchestrator decompose the instruction into object-level sub-tasks, let a multimodal grounding agent localize target nodes using appearance/behavior/position evidence and a behavior editing agent drive a multi-object diffusion simulator via counterfactual behavior combinations, then correct trajectories and renderings with two reviewer loops — thereby obtaining instruction alignment, structure preservation, photorealism and traffic realism simultaneously.

Method¶

Overall Architecture¶

The system takes a real driving video, a free-form natural-language instruction and the scene map as input (the original object trajectories and the map are assumed to be given), and outputs the same video with its object composition, object behavior and the resulting camera viewpoint rewritten according to the instruction. Rather than generating in pixel space like a world model, it first reconstructs the video into an explicit, read-write scene graph, and then lets every agent (reasoning-capable, driven by an LLM/VLM) and tool (no reasoning, atomic operations only) cooperate around that graph.

The scene graph writes the scene at frame \(t\) as one static background plus \(K\) dynamic objects:

\[SG(t) = \{\, N_{\mathrm{bg}},\ \{N^{i}_{\mathrm{asset}}(t)\}_{i=1}^{K} \,\}\]

\(N_{\mathrm{bg}}\) is the static background Gaussian shared across the whole video, and \(N^{i}_{\mathrm{asset}}(t)\) is the time-dependent Gaussian of the \(i\)-th dynamic object node (vehicle, pedestrian) after applying its own pose. Each object node consists of a canonical Gaussian plus a motion trajectory, and rendering places the canonical node at the corresponding pose along that trajectory. This representational choice decides two things outright: editing an object means editing a node, so the background cannot move; editing behavior means editing a trajectory, so appearance is left untouched. The scene graph is built once by the scene reconstruction tool following OmniRe's 3DGS decomposition (static background Gaussians plus canonical object nodes with trajectory-based transformations), and is subsequently shared by all modules — which is precisely what makes the pipeline interpretable and able to host verification checkpoints.

On top of it sits an orchestrator agent: it uses an off-the-shelf LLM (GPT-4) configured by in-context learning as the brain, implemented in a very direct way — the capabilities of each module are wrapped as modular functions, and the LLM emits executable Python scripts that call them. The script first decomposes the user instruction into sub-instructions per target object, then walks each object through the flow: object grounding and localization → modify the node according to the edit type (removal/insertion/replacement) and update the scene graph → if the instruction involves trajectories, invoke behavior editing and behavior review → once all objects are processed, coarse render → video diffusion harmonization → video review loop. There are two feedback points on this chain, acting at the trajectory level and the pixel level respectively, and that is the essential difference from feed-forward agentic pipelines.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Driving video + instruction + map"] --> B["Scene graph construction<br/>static background + dynamic object nodes"]
    B --> C["Multimodal Object Grounding & Node Editing<br/>appearance / behavior / position evidence"]
    C --> D["Counterfactual Behavior Combination & Multi-object Trajectory Simulation<br/>combination as LangTraj text condition"]
    D --> E["Behavior Reviewer Loop<br/>tune CFG and avoidance guidance"]
    E -->|misaligned / collision / off-road| D
    E -->|trajectories pass| F["Coarse rendering<br/>3DGS rasterization + mesh depth compositing"]
    F --> G["Video Reviewer Loop<br/>harmonize, then retune denoising strength and L2 guidance"]
    G -->|still unsatisfactory, re-run diffusion| G
    G -->|pass| H["Edited video"]

Key Designs¶

1. Multimodal Object Grounding & Node Editing: landing open-vocabulary descriptions on the right scene-graph node

Open-vocabulary detection/tracking algorithms (Grounded-DINO, Grounded-SAM and relatives) and 3DGS language fields (LangSplat, 4D LangSplat) are already adequate at category-level recognition, but what actually discriminates in a user instruction is "that white car on the left" or "the black SUV in front of ego" — color, vehicle type, spatial relation, even motion state. Category-level methods routinely pick the wrong object under such attribute-level distinctions, and once grounding is wrong every downstream edit acts on the wrong target, an error the renderer then amplifies into a video that looks "correctly edited".

The key here is that the grounding agent is not handed a single image but three complementary sources of evidence: for appearance, each node is projected back into pixel space and segmented with SAM to extract its visual appearance; for behavior, heuristic rules turn the trajectory into motion descriptions such as speed, heading and lane changes; for position, trajectory coordinates plus lane information establish spatial relations. Localization then proceeds in two stages: the query is first decomposed into a triplet (e.g. "the black SUV on the left of ego" → ), the reference node is located by matching appearance and behavior, candidates are filtered by the spatial relation, and the target node is finally picked from the candidates through the same matching process. The paper reports in the supplementary a grounding comparison against 4D LangSplat and Grounded-SAM (that supplementary material is not covered by this note; refer to the original paper for the numbers).

Once localized, rewriting the node is operationally light in all three cases: the removal tool simply deletes all Gaussians belonging to the node; replacement chains removal and insertion and lets the new node inherit the original object's trajectory, so appearance changes while behavior does not; insertion is the involved one, requiring a text-to-3D tool (Hunyuan3D) to generate a mesh, whereupon the agent computes the mesh's bounding box to rescale it to the scene's real size and resolves orientation alignment by rendering the mesh from a fixed axis, reading off which way it faces in that rendering and deriving the local axes, before attaching it to the scene graph as a new node. This is where language actually becomes a 3D asset in the system, and it is also why video diffusion is later needed to rescue image quality: a composited mesh is inherently unnatural in lighting and material.

2. Counterfactual Behavior Combination & Multi-object Trajectory Simulation: changing only the behavior the instruction asks for

When a user says "speed up", they do not mean to cancel the existing "change from the middle lane to the left lane and turn left". If a language-conditioned generator is simply asked to regenerate the whole trajectory from the new instruction, it readily rewrites behaviors that were never meant to change, and can even produce actions that do not exist on the map (turning left where there is no intersection). This is the most fragile joint in the language-to-trajectory interface.

The behavior editing agent starts by translating the original trajectory into a string of behavior descriptions, which is essentially one behavior combination (e.g. "slow down, change from the middle lane to the left lane, turn left"). It then performs counterfactual generation: each behavior in the combination is subjected to replace / remove / keep / add operations, enumerating a candidate list of behavior combinations; the map filters out unreasonable combinations (no left turn without an intersection), and mutually contradictory ones are removed as well (a combination containing both "going straight" and "static"). Finally the agent selects the single best-matching combination rather than a single behavior: if the instruction says "speed up" and the original behavior is "slow down, change lane, turn left", the selected combination is "speed up, change lane, turn left". This buys two things directly — unreasonable behaviors are blocked before generation, and the object's original behaviors are preserved as far as possible. The selected combination is passed as the text condition to LangTraj, a diffusion-based, language-conditioned multi-object trajectory simulator, which generates the trajectories of several objects at once, so inter-object interaction (who yields to whom) is reasoned about jointly during generation rather than stitched together afterwards.

3. Behavior Reviewer Loop: pushing failed trajectories back to controllable guidance parameters

LangTraj's output can still misalign with the instruction, or go off-road, or collide, and a feed-forward pipeline carries that error straight into the video — ChatSim's inserted vehicle colliding with traffic is the canonical failure. The reviewer's approach is not to change the simulator itself but to adjust a few controllable knobs around it at inference time.

The reviewer agent evaluates instruction alignment and traffic-rule compliance with a set of trajectory validation functions, and splits a multi-object scene into objects that are "already right" and "still wrong". Objects that already satisfy all requirements have their trajectories stored and reused as guidance for subsequent rounds — which helps twice over: an already-correct object is no longer disturbed by resampling, and other objects can genuinely interact with it. For objects that fail, the agent adjusts LangTraj's guidance configuration according to the failure type: if behavior misaligns with the instruction it increases the classifier-free guidance (CFG) weight; for off-road or collision violations it adds the corresponding off-road or collision-avoidance guidance and tunes its weight. The loop runs for at most 5 iterations. The value of this design is that "failure mode → which parameter to turn" becomes an explicit mapping, and successful sub-results are reused, making the loop both cheap and stable.

4. Video Reviewer Loop: trading off denoising strength against L2 guidance from feedback

The coarse rendering tool renders the edited scene graph — 3DGS background by rasterization, inserted meshes through PyVista, composited by depth — and the result is visibly insufficient: newly inserted objects tend to look pasted on, and when the new viewpoint differs substantially from the original (for instance after editing the ego trajectory) 3DGS rendering quality degrades quickly. A video diffusion tool is therefore needed for harmonization, but it introduces a new opposition: a higher denoising strength yields more realism yet discards more information from the conditioning input, so the inserted vehicle loses its roof light or changes type or color. Photorealism and appearance preservation pull against each other at this step.

The video diffusion tool uses CogVideoX as its backbone, initialized from TrajectoryCrafter's pretrained weights, and takes the coarse video as a condition to generate the enhanced output. To learn realistic vehicle appearance and effective denoising it relies on two fine-tuning strategies: during training the Gaussian primitives in the 3DGS representation are replaced with object meshes so the model learns photorealistic vehicle appearance, and noisy Gaussian rendering pairs are curated via a cycle reconstruction strategy so the model learns to denoise. At inference time, the trade-off between the two objectives is handled by the video reviewer agent, a VLM-powered agent that first applies diffusion with a relatively high denoising strength (empirically the most photorealistic), then reviews the output: if appearance is compromised (missing key parts, changed shape or color) it increases the weight of the L2 guidance loss; if the inserted vehicle looks unrealistic (e.g. lighting mismatched with the environment) it increases the denoising strength. The L2 guidance loss is computed at each denoising step and measures, in latent space, the L2 distance between the inserted-vehicle regions of the predicted and the condition video — a rope tied to the diffusion process saying "do not change this region too much":

\[\mathcal{L}_{\mathrm{L2}} = \bigl\| (z^{\mathrm{pred}}_{t} - z^{\mathrm{cond}}_{t}) \odot M_{\mathrm{ins}} \bigr\|_2^2\]

⚠️ The original paper describes this loss in words only; the expression above is reconstructed from that description, so the exact form and notation refer to the original paper. The loop continues until both photorealism and appearance preservation are satisfied, or the maximum iteration count (5) is reached.

A Worked Example¶

Take the instruction from Figure 3 of the paper: "Insert a yellow taxi on the right of the ego vehicle, 6 meters ahead, and make it go straight." The orchestrator first decomposes it into object-level sub-tasks — the target is a taxi to be inserted, the positional constraint is 6 meters ahead on the right of ego, and the behavioral constraint is going straight. The insertion agent calls Hunyuan3D to generate a yellow taxi mesh, rescales it to real size, aligns its orientation and attaches it as a new node 6 meters ahead on the right. Because a behavior is specified, the behavior editing agent turns it into the behavior combination "keep going straight" and hands it to LangTraj; the behavior reviewer confirms straight motion and no collision via its validation functions and lets it through. Coarse rendering yields the first video: the position is right, but the taxi looks pasted on.

The video reviewer loop then alternates in exactly the way Figure 3 shows. Iteration 1 runs diffusion at a comparatively high denoising strength: the taxi looks realistic but its roof light has disappeared — photorealism passes, appearance preservation fails, so the reviewer increases the L2 guidance weight. Iteration 2 restores the appearance, but the whole vehicle is over-bright and the lighting does not match the environment — appearance preservation passes, photorealism fails, so the reviewer increases the denoising strength. Iteration 3 satisfies both and the loop stops. The example shows why a single "more realistic" or "more faithful" knob is not enough: the two failure modes point at parameters that move in opposite directions.

For a case that genuinely needs the loop, take the multi-object example of Figure 6: "Make the ego vehicle change to the middle lane and make car 2 change to the right lane." In iteration 1 neither trajectory changes lanes, so the reviewer increases the CFG weight for both vehicles. In iteration 2 car 2 changes lanes correctly while ego still fails, so the reviewer stores car 2's successful trajectory as guidance for later rounds and keeps increasing only ego's CFG. In iteration 3 both trajectories are correct and the loop terminates. Here "successful trajectories as guidance" is not a nicety: it keeps already-correct objects from being resampled and perturbed, so the remaining budget is spent entirely on the object that is still wrong.

Loss & Training¶

There is no training in the inference stage, only in-context calls to an LLM (GPT-4) and a VLM (GPT-4o), with the maximum iteration count of both reviewer loops set to 5. What is trained is the video diffusion tool: with CogVideoX as backbone and initialization from TrajectoryCrafter's pretrained weights, it is fine-tuned in two stages — 40k iterations on 33-frame short videos, then 20k iterations on 81-frame long videos, both with batch size 4 and each taking 48 hours on a 4×H100 workstation. The supervision is not conventional paired data but the two construction strategies above: replacing Gaussians in the 3DGS representation with object meshes yields photorealistic vehicle-appearance samples, while cycle reconstruction builds pairs of noisy Gaussian renderings and clean targets so the model learns to wash the coarse rendering clean. Inference runs on a single A6000, and end-to-end editing of one 8-second, 10 fps scene takes about 17.1 minutes.

Key Experimental Results¶

Main Results¶

Experiments use a self-built evaluation set on the Waymo Open Dataset: 30 scenes covering different times of day, road types and weather, deliberately chosen with few pedestrians since the paper focuses on vehicle editing. Each scene yields four instruction types generated by GPT-4 (removal / replacement / behavior editing / insertion, 2-3 instructions each) followed by human filtering. The baselines are ChatSim (LLM-agent line) and Cosmos (world-model line), both state-of-the-art open-source methods in their respective categories, evaluated strictly under their original settings; the authors additionally build a naive baseline by chaining the image editing model ChronoEdit with the image-to-video model Wan 2.2 (its numbers are not listed in the main table; details are in the supplementary). Evaluation covers five aspects: photorealism (FID / FVD), instruction alignment (appearance alignment has GPT-4o compare the original and edited videos to decide whether the target object was correctly removed, inserted or replaced; behavior alignment first tracks the edited object with Grounded-SAM-2, back-projects its trajectory to world coordinates and checks it against the instruction given the map), structure preservation (difference of DINO self-similarity matrices, lower is better), traffic realism (collision rate and off-road rate, detected by GPT-4o on sampled frames), and a user study (26 participants choosing the best of three methods or "none" per aspect).

Method	FID ↓	FVD ↓	Photo. User (%) ↑	App. Align. (%) ↑	Beh. Align. (%) ↑	Align. User (%) ↑	Struct. Dist. ↓	Struct. User (%) ↑	Collision (%) ↓	Off-road (%) ↓	Traffic User (%) ↑	Time (min) ↓
Cosmos	33.42	797.51	34.60	46.25	32.86	10.50	74.07	19.10	3.16	3.95	35.5	16.9
ChatSim	47.70	605.69	4.80	42.33	26.64	3.90	46.52	7.40	27.62	24.71	5.60	19.3
Ours	32.85	467.20	54.20	82.19	71.67	60.40	34.62	65.00	0.58	1.73	48.30	17.1

Editing time is measured on a single A6000 for 8-second, 10 fps videos. Note that both ChatSim and Ours require a one-time ~2-hour 3D reconstruction preprocessing step per scene, which is not counted in the editing time above.

Ablation Study¶

The behavior reviewer loop is ablated on the same 30 scenes but with 70 newly generated, harder instructions (specifying target behaviors for several objects simultaneously), and the generated trajectories are evaluated directly rather than the edited videos:

Behavior Review	Behavior Align. (%) ↑	Collision (%) ↓	Off-road (%) ↓	Overall Success (%) ↑
✗	54.29	35.71	30.00	34.29
✓	70.00	22.86	14.29	51.43

The video reviewer loop is ablated using only insertion and replacement instructions, so that the diffusion model's tendency to alter inserted vehicles is exposed:

Video Diffusion Tool	Video Reviewer	FID ↓	FVD ↓	Appearance Align. (%) ↑
✗	✗	43.54	613.72	87.69
✓	✗	36.83	501.84	65.47
✓	✓	36.78	493.25	85.28

The authors also analyze hallucinations of the diffusion tool: after harmonization, 3D Consistency (WorldScore) drops slightly from 74.07 to 72.64 and the lane NTL-IoU (DriveDreamer4D) from 52.11 to 50.96, yet both remain above ChatSim (71.32 / 50.13) and Cosmos (69.85 / 48.76).

Key Findings¶

Instruction alignment is where the gap opens: appearance alignment 82.19% vs Cosmos 46.25% (about 1.8×) and ChatSim 42.33%; behavior alignment 71.67% vs 32.86% (about 2.2×) and 26.64%. The "nearly 2×" claim in the abstract comes mainly from behavior alignment.
Photorealism has to be read in two parts: the FIDs are actually close (32.85 / 33.42 / 47.70), but the FVDs are far apart (467.20 vs 797.51 / 605.69). Cosmos has good single-frame quality and loses on temporal consistency, while ChatSim is poor at both; the advantage here is mainly in temporal consistency.
The structure-preservation ordering is telling: Ours has the best structure distance at 34.62, ChatSim 46.52, and Cosmos the worst at 74.07. Editing in pixel space makes it easy to alter the background as a side effect of making frames realistic, and the user votes (65.00 / 7.40 / 19.10) agree with that ordering.
Traffic realism shows the largest gap of all: collision rate 0.58% vs 27.62% (ChatSim) and 3.16% (Cosmos); off-road rate 1.73% vs 24.71% and 3.95%. The ablation attributes this almost entirely to the behavior reviewer loop — with it, behavior alignment rises from 54.29% to 70.00%, off-road from 30.00% down to 14.29%, and overall success from 34.29% to 51.43% (about +50% relative).
Video diffusion is the textbook "trade appearance for realism" step: one diffusion pass drops FID from 43.54 to 36.83 and FVD from 613.72 to 501.84, but appearance alignment collapses from 87.69% to 65.47%. With the video reviewer attached, FID/FVD stay essentially unchanged (36.78 / 493.25) while appearance alignment returns to 85.28% — the reviewer loop buys back appearance fidelity at almost zero cost in image quality, which is the best value-per-design in the paper.
Timing: 17.1 minutes per scene end to end, on par with Cosmos (16.9) and ChatSim (19.3). Per module, object node editing takes 6.1 minutes (dominated by the text-to-3D tool) and video iterative refinement 7.6 minutes, while object query takes 1.4 minutes, coarse rendering 0.5 minutes and behavior editing (feedback loop included) 1.5 minutes. The feedback loops themselves are therefore cheap; what costs time is 3D asset generation and diffusion refinement.
The paper acknowledges that diffusion harmonization introduces mild hallucinations (both 3D consistency and lane IoU decrease slightly), but the effect is limited and still better than both baselines.

Highlights & Insights¶

Anchoring editing on the scene graph compresses the role of language into "pick a node + change a trajectory", which is the root reason structure preservation reaches 34.62. A pixel-space world model lacks that hard constraint no matter how realistic a single frame looks.
The two reviewer agents are one idea reused twice: leave the generator untouched, tune only controllable inference-time knobs (CFG weight, avoidance guidance, denoising strength, L2 weight), and freeze already-successful sub-results as the next round's condition. That pattern transfers to any pipeline with "a generator that has several knobs but no feedback", for example motion-magnitude control in video generation or geometry-material consistency adjustment in 3D asset generation.
Mapping failure cause to knob is done cleanly: misaligned behavior → CFG, off-road/collision → avoidance guidance, lost appearance → L2 weight, unrealistic lighting → denoising strength. Four failure modes map to four parameters in an explicit, non-interfering way, which is why the loop converges steadily instead of oscillating.
Counterfactual behavior combination is a very cheap but effective language-to-trajectory interface: enumerate replace/remove/keep/add, filter by the map and by contradiction, then match whole combinations instead of single behaviors. It blocks the common failure of "changing one behavior destroys the others" before generation rather than repairing it afterwards.
The scene graph plus multi-agent interface makes the pipeline inherently interpretable: every intermediate artifact (nodes, trajectories, coarse video) can be inspected and replaced on its own, which is exactly what allows verification checkpoints to be inserted at two levels.

Limitations & Future Work¶

The method assumes the original object trajectories and the scene map are known; without them, neither scene decomposition nor the reviewer's validation functions can be established.
Hallucinations from video diffusion cannot be fully removed; both 3D consistency and lane NTL-IoU decrease slightly after harmonization, and the authors can only keep the impact bounded.
The evaluation is small: 30 scenes and 26 participants. The authors explicitly chose scenes with few pedestrians and focus on vehicle editing, so pedestrians and more complex interactions are not covered.
The user study is a four-way preference vote including "none", yielding a relative ranking rather than an absolute quality score, so cross-method comparisons should be read with care.
Cost and dependencies: a one-time ~2-hour 3D reconstruction plus 17 minutes of editing per scene, with the pipeline heavily reliant on GPT-4/GPT-4o API calls, limiting both reproducibility and cost.
Concrete directions: replace the hand-written "failure mode → knob" mapping with a learned controller; turn 3D reconstruction from offline preprocessing into on-demand incremental reconstruction; extend evaluation to pedestrians, non-motorized traffic and denser multi-object interaction; and report the missing ChronoEdit + Wan 2.2 baseline numbers in the main table so that the gains of the "explicit 3D + closed loop" route over purely generative routes become more directly comparable.

vs ChatSim: Also LLM-agent-based scene editing, but ChatSim reasons only in the text modality, composites naively, and verifies nothing in between. The differences here are precisely those three: grounding with appearance/behavior/position evidence, rendering delegated to a fine-tuned video diffusion model, and a feedback loop each for behavior and video. The resulting gap — behavior alignment 71.67% vs 26.64%, collision rate 0.58% vs 27.62% — comes mostly from the latter two.
vs Cosmos: A pixel-space world model with single-frame realism comparable to this work (FID 33.42 vs 32.85) and no need for 3D reconstruction, but weak controllability: it inserts unrequested pedestrians, alters the background (worst structure distance at 74.07 of the three), and exposes no object-level editing interface. This paper trades a one-time 3D reconstruction preprocessing step for controllability and structure preservation via the explicit scene graph plus two closed loops.
vs LangTraj: LangTraj is a language-conditioned multi-object trajectory simulator from the same group, and this paper uses it as the behavior generation tool while wrapping counterfactual behavior combination and a behavior reviewer around it. The division of labor is: LangTraj solves how to generate several interacting trajectories from language at once, while this paper solves how to guarantee that the generated trajectories really match this particular instruction and violate no traffic rule.
vs the ChronoEdit + Wan 2.2 combination: The authors chain image editing with image-to-video as an extra baseline. Such split pipelines impose no constraint on cross-frame consistency or 3D structure and represent the "replace explicit 3D with general generative models" route; the paper mentions it in the experimental setup but reports no numbers for it in the main table.

Rating¶

Novelty: ⭐⭐⭐⭐ The 3DGS scene graph, language-conditioned trajectory simulation and video diffusion harmonization all have predecessors; the novelty lies in the combination of three-evidence grounding with two reviewer loops, and in mapping failure modes explicitly to inference-time knobs.
Experimental Thoroughness: ⭐⭐⭐⭐ Four edit types, five metric families, a user study, ablations of both loops and a hallucination analysis give full coverage, but only 30 scenes are used and the ablations of grounding and counterfactual behavior generation live in the supplementary material rather than the main table.
Writing Quality: ⭐⭐⭐⭐ The pipeline is explained clearly and Figures 2/3/6 map neatly onto the overall framework, the video loop and the behavior loop respectively; the cost is that several details (validation functions, combination filtering rules) are pushed into the supplementary.
Value: ⭐⭐⭐⭐⭐ An interpretable multi-agent framework that resolves the long-standing controllability-versus-realism tension while matching baselines in end-to-end latency, with directly usable value for synthetic data generation.