Boba: Batched Simulation for Physics-Based Gaussian Digital Twins

Conference: ECCV2026
Paper: Official page · PDF
Project: Boba
Area: 3D Vision / Physics Simulation / Robotics
Keywords: physical digital twins, spring-mass models, Gaussian skinning, batched simulation, model predictive control

TL;DR

Boba jointly optimizes the physical model, skinning, and GPU execution of PhysTwin-based deformable Gaussian digital twins, reducing single-instance Orin latency from 330.9 ms to 32.5 ms and reaching 3309.9 aggregate FPS on RTX 4090 while retaining reconstruction and prediction quality close to the baseline.

Background & Motivation

A digital twin must do more than support novel-view rendering: it should deform plausibly when pulled, brought into contact, or manipulated by a robot. PhysTwin fits spring-mass dynamics from real human-object interaction videos and transfers physical-node motion to a Gaussian visual representation through skinning. This connects appearance to interaction, but every step requires physical integration, local rotation estimation, Gaussian deformation, and rendering. A single instance is already expensive, making the evaluation of many candidate actions inside a controller even harder.

Launching more processes does not resolve the underlying bottlenecks. Each process duplicates the same object's topology, rest geometry, and Gaussian assets, consuming GPU memory before useful parallel work scales. Meanwhile, many springs add forces to shared endpoints, creating atomic contention. Edge XR devices primarily need low frame latency and low device-side power, whereas robot planning needs many independent rollouts per unit time. A single FPS number cannot adequately describe both objectives.

Boba therefore lowers the compute and memory cost of one instance before sharing static assets across multiple interactions with the same object. Core idea: separate the reusable digital-twin template from each instance's changing state, and co-design physics, skinning, and execution so that real-time edge interaction and batched server-side planning reuse the same efficient foundation.

Method

Overall Architecture

Boba takes an existing PhysTwin digital twin and per-instance interaction actions as input, rather than starting from unprocessed video. It retains the Gaussian appearance representation while using a more compact physical system to drive deformation. Each step advances physical nodes, updates Gaussian poses, and then renders or visualizes the result according to the deployment configuration. Batched instances represent different interactions with the same reconstructed object, each maintaining its own positions, velocities, and rotation cache.

The four design groups address shared data, physical computation, skinning, and execution scheduling. Boba-Local runs the optimized pipeline locally; Boba-Distributed performs simulation and skinning on a server and sends dynamic poses to the edge for rendering; Boba-Batched uses single-process batched arrays and specialized kernels for high throughput. These are alternative deployment configurations, not three sequential stages traversed by every sample.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Existing twin and actions"] --> Template["Template-State Separation"]
    Template --> Physics["Compact Physical Model"]
    Physics --> Skinning["Efficient Gaussian Skinning"]
    Skinning --> Execution["Execution Scheduling<br/>and Delivery"]
    Execution --> Output["Local interaction / Distributed XR<br/>Batched robot planning"]
    Execution -.->|Batched atomic-free force assembly| Physics

Key Designs

1. Template-State Separation: avoid copying an entire asset for every trajectory

The shared template contains spring topology, rest geometry, physical attributes, skinning metadata, Gaussian parameters, and rest-derived precomputations. Instance-specific data include node positions, velocities, and cached rotations. These dynamic quantities are packed into contiguous batched arrays, and kernels locate an instance's state through offsets while reading the same static template. Sharing immutable assets does not force instances into identical states: every instance can still receive a different action.

This boundary explains why shrinking the dynamic state and eliminating asset replication are complementary. The former reduces each instance's changing workload; the latter prevents duplicated templates from dominating batch capacity. Rest-state collision masks, spring adjacency, and node ordering can all be prepared once for the template. The intended setting is repeated trials of the same twin, so the results should not be extrapolated directly to hundreds of arbitrary heterogeneous objects.

2. Compact Physical Model: reduce nodes and substeps, then refit the resulting motion

A dense spring-mass graph does not necessarily yield proportionally better visible motion. Nearby nodes often move together, and Gaussian-level deformation may be insensitive to modest changes in physical resolution. Boba reduces the node count and increases the integration timestep, then refits the compact model's physical parameters and skinning relations so that its rendered rollouts remain close to the original model. This is not node deletion followed by reuse of unchanged parameters: both spatial and temporal discretization have changed, requiring the resulting dynamics to be recalibrated. The Gaussian appearance asset is retained; coarsening primarily affects the physical representation driving it.

Self-collision introduces many additional node-pair tests. Boba precomputes a symmetric mask from rest-space distances, skipping pairs closer than \(\alpha d_{\mathrm{col}}\), where \(d_{\mathrm{col}}\) is the collision distance and \(\alpha>1\) is a margin. Such pairs often represent structural neighbors and contribute little meaningful collision impulse. Training and runtime use the same pruning rule to avoid fitting one collision model and deploying another. This is a structural prior, not a rule that removes all collisions based on current distance, nor a proof of error-free behavior under arbitrary large deformation.

The remaining spring computation can still be memory-bound. Boba orders nodes by Morton keys derived from rest coordinates, remaps spring endpoint indices, and organizes springs into spatial blocks. Nodes frequently accessed together become closer in memory. This does not reduce the number of springs or change the complexity of the physical equations; it improves coalesced memory access and cache locality. Because connectivity indices remain fixed as the object deforms, the ordering need not be recomputed every frame.

3. Efficient Gaussian Skinning: reserve expensive precision for sensitive deformation operations

Node translations follow directly from simulation, but rotations must be extracted from a local deformation matrix \(F\). General-purpose SVD implementations create working copies and temporary buffers, making data movement expensive across many nodes and instances despite the small matrices. Boba instead uses the polar-decomposition relation in a fused rotation-only kernel:

\[ C=F^{\top}F,\qquad R=F C^{-1/2}. \]

The inverse square root is obtained through symmetric eigendecomposition with eigenvalue clamping. The kernel reuses neighborhood data already loaded into memory and writes quaternions directly. Each node also caches its previous deformation matrix and rotation; rotation extraction is repeated only when \(\lVert F-F_{\mathrm{cache}}\rVert_F\) exceeds a threshold. Substeps with small changes reuse the cached rotation, while larger changes trigger an update. The main text does not specify the numerical threshold.

For each Gaussian, Boba selects only the \(K_{\mathrm{lbs}}\) nearest rest-space nodes and normalizes skinning weights over that set, avoiding low-value distant influences in smooth deformation. Mixed precision does not mean converting the entire pipeline to FP16. Cached rotations and rest-pose skinning quantities use FP16, while deformation-matrix construction, the rotation-reuse test, eigendecomposition, and translation blending remain in FP32. Final Gaussian poses are cast back to FP32 for the renderer. Sparse influence sets reduce arithmetic, and selective precision reduces bandwidth without indiscriminately weakening sensitive decisions and accumulations.

4. Execution Scheduling and Delivery: eliminate physics write conflicts and reduce display-side transfers

Each spring contributes forces with opposite signs to its two endpoints. If spring threads write directly into node forces, springs incident to the same node contend on atomic additions. Boba separates this into two passes: a spring-parallel kernel computes each spring force once and writes a temporary buffer; a node-parallel kernel then gathers incident forces using precomputed signed adjacency indices. Each node's thread performs its own accumulation, with the signs specifying whether to add or subtract each contribution, removing scatter-write contention.

This does not eliminate spring-force evaluation or make total system work depend only on the degree of one node. It changes ownership of accumulation from many springs competing to update endpoints to each node gathering its own contributions. Extra indices and temporary data consume memory, explaining why the full configuration slightly reduces average batch capacity while improving throughput. Template sharing and atomic-free assembly therefore need to be evaluated separately.

For rendering and display, Boba keeps compositing and display transfers on the GPU where possible, reducing CPU-GPU synchronization and copies. The distributed configuration caches static templates on both server and edge and transmits only compact dynamic Gaussian pose updates; the edge applies those updates and renders. The batched configuration also uses a specialized batched rendering/visualization path, so its 16.2-fold gain is system-level, not attributable entirely to physics kernels or atomic removal. Communication encoding and batched-display implementation details are deferred to the supplement and cannot be reconstructed from the current cache.

A Worked Example

Consider the paper's rope MPC application. The controller must select a candidate action sequence that moves the rope toward a target. Multiple instances share the same rope-twin template, but each receives its own candidate actions and begins from the corresponding state. The main text does not provide the number of candidates or the planning horizon.

Within a rollout, the compact spring network advances positions and velocities. Batched atomic-free assembly avoids repeating the same endpoint-contention bottleneck across candidates. Efficient skinning converts physical motion into Gaussian poses, and the batched path produces a frame for each instance. MPC evaluates the candidates, executes the action that best approaches the target, and replans. The reported rope-planning speedup is 26-fold, but the main text does not report a success-rate improvement. The supported conclusion is faster candidate evaluation, not automatically more accurate control.

Loss & Training

Boba is not a neural dynamics predictor trained from scratch. It starts from a PhysTwin twin, compresses its physical system, and refits that system. The main text states that the compact model is optimized directly on rendered rollouts to reproduce the original model's visual behavior. This differs from simply matching node positions before and after coarsening, because the underlying node sets have changed.

The compact model uses \(M\ll N\) nodes and an integration timestep \(\Delta t'>\Delta t\), with refitted physical parameters \(\theta'\) and skinning relations \(w'\). However, the main text does not fully specify the refitting loss weights, optimizer, training iterations, numerical node counts, timesteps, or \(K_{\mathrm{lbs}}\). Those values are not invented here, and the generic inverse-simulation objective in the preliminaries should not be mistaken for Boba's complete training loss.

Key Experimental Results

Main Results

Experiments use PhysTwin objects and its single-instance action sequences. CPU/GPU resources on Orin are restricted to emulate an XR compute budget; the desktop system uses an Intel i9 and RTX 4090. In the distributed configuration, Orin connects over Wi-Fi and the server uses Ethernet. The following latency and power values come from Section 5.2 and Figure 5. "Not listed" means that the quantity is not reported in this comparison.

Configuration	Execution device	End-to-end latency / ms	Orin incremental power / mW
PhysTwin	Orin	330.9	Not listed
Boba-Local	Orin	32.5	6112
PhysTwin	RTX 4090	59.5	Not applicable
Boba-Local	RTX 4090	4.9	Not applicable
Boba-Distributed	RTX 4090 server + Orin edge	25.1	4758

Local latency includes physics, skinning, and rendering/visualization; distributed latency also includes transmission and edge-side update/display. The ratio 330.9/32.5 gives approximately 10.2-fold latency acceleration, whereas Figure 3 reports 10.3-fold from rounded FPS values. These are different rounding conventions. The 22.2% power reduction compares workload-induced Orin power above its idle baseline and excludes server power. It is not a 22.2% reduction in total edge-server power.

The following reorganizes Table 1. CD is single-direction Chamfer Distance, and Track measures tracking error against PhysTwin's manually annotated ground-truth points; lower is better for both. The cached table does not identify their physical units. IoU is shown on the original percentage scale, and PSNR is in dB. Image metrics are evaluated at the center viewpoint and averaged across frames and cases.

Task	Method	CD ↓	Track ↓	IoU ↑	PSNR ↑	SSIM ↑	LPIPS ↓
Reconstruction & Resimulation	PhysTwin	0.005	0.009	87.4	28.7	0.966	0.025
Reconstruction & Resimulation	Boba-Local / Boba-Batched	0.006	0.009	89.6	28.4	0.963	0.031
Reconstruction & Resimulation	Boba-Distributed	0.006	0.009	90.2	28.6	0.964	0.030
Future Prediction	PhysTwin	0.012	0.022	71.4	23.3	0.950	0.049
Future Prediction	Boba-Local / Boba-Batched	0.014	0.022	75.5	23.6	0.953	0.050
Future Prediction	Boba-Distributed	0.014	0.022	75.6	23.7	0.953	0.050

The results support comparable quality, not losslessness on every metric. Local reconstruction PSNR decreases from 28.7 to 28.4, LPIPS rises from 0.025 to 0.031, and future-prediction CD rises from 0.012 to 0.014. At the same time, tracking error is unchanged and silhouette IoU improves.

Ablation Study

Table 2 compares parallelization strategies on RTX 4090. Each batch element has independent actions and state, and one batch step includes physical advancement, skinning, and one rendered/visualized frame per instance. FPS is aggregated across all instances. The authors sweep batch sizes separately for each case, select the highest aggregate throughput for each case, and then average these maxima, rather than averaging performance at a single fixed batch size.

Parallel configuration	Average batch capacity	Minimum batch capacity	Maximum batch capacity	Aggregate throughput / FPS
PhysTwin-MultiProc	11.0	4	27	44.7
Boba-Batched, template sharing only	369.2	213	688	2392.8
Boba-Batched, plus atomic-free scheduling	368.3	210	688	3309.9

"Template sharing only" ablates Boba's batching-specific extensions while retaining the optimized single-instance stack. Its 53.5-fold throughput advantage over PhysTwin-MultiProc cannot therefore be attributed entirely to template sharing in isolation. The cleaner local comparison is between the last two rows: atomic-free scheduling adds 38.3% throughput, with average capacity decreasing slightly from 369.2 to 368.3 and maximum capacity remaining at 688.

Key Findings

• Edge optimizations progressively remove different bottlenecks. In Figure 3, Orin rises from 3.0 FPS to 8.1 after physics optimization, then to 21.5 after skinning optimization, and finally to 30.8 after rendering/visualization optimization. These are cumulative additions, not independent leave-one-component-out effects.
• Batched throughput of 3309.9 FPS is a 16.2-fold system-level gain over Boba-Local's 204.5 FPS on the same GPU. It does not mean one instance runs at 3309.9 FPS, and average capacity cannot be used to infer a universal batch-step latency.
• Rope planning has a measured 26-fold speedup. For cloth with matched self-collision settings, the PhysTwin baseline takes over 24 hours and the speedup above 2000-fold is estimated from the first 100 samples. Without self-collision, the speedup still exceeds 16-fold. The extrapolated cloth result is not evidence of the same strength as the measured rope result.

Highlights & Insights

• Physical resolution can be redesigned around final visible motion instead of preserving the original node density by default. Refitting rendered rollouts connects computational compression to output fidelity, although visually similar results can still conceal dynamical differences.
• Replacing spring scatter with node gather is a useful parallelization pattern for repeated topology. Adjacency is precomputed once and reused across instances, exchanging modest indexing overhead for contention-free force accumulation.
• Precision selection and temporal reuse are tied to specific operations rather than lowering precision everywhere. FP16 caches paired with FP32 threshold decisions and rotation extraction allocate bandwidth according to where numerical errors are most consequential.

Limitations & Future Work

• Evidence and reproducibility boundary: the main text has no dedicated limitations section, and communication, batched rendering, networking, and some hyperparameter details are deferred to supplementary material absent from the current cache. This limits this note's evidence; it does not establish that the complete paper omits those details.
• Fidelity is not mechanical equivalence: the authors report quality close to PhysTwin, but several geometric and perceptual metrics degrade slightly. Long rollouts, strong impacts, extreme folding, and out-of-distribution manipulation need more direct dynamical and task-level evaluation. These are reviewer-style validation needs identified in this note.
• Structural priors have conditions: rest-space neighbors are pruned from collision evaluation, and small deformation changes reuse cached rotations. Sensitivity to collision thresholds, rotation thresholds, and skinning-neighbor counts should characterize where these approximations fail; average quality alone does not establish universal safety.
• Scope and system accounting are limited: capacity gains primarily concern instances sharing one template, power is measured only on Orin, and throughput uses each case's best batch size. Heterogeneous objects, network jitter, tail latency, and total edge-server energy remain important engineering evaluations.
• Planning claims mix measurement and estimation: the cloth speedup above 2000-fold is not a completed end-to-end planning measurement, and the main text does not provide an associated control-success table. Future evaluation should report full-run time and task success under matched budgets and collision settings.

Related Work & Insights

• Compared with PhysTwin: PhysTwin reconstructs simulation-ready twins from interaction video, while Boba builds on those twins to reduce execution and scaling costs. Their relationship is asset construction followed by efficient execution, not two independent reconstruction methods.
• Compared with PhysGaussian and VR-GS: these systems couple physical motion with Gaussian appearance, whereas Boba additionally emphasizes low-power deployment and parallel interactions with a shared asset. Its advantage comes from system co-design, not a new general-purpose Gaussian rendering formula.
• Compared with Brax, Isaac Gym, MJX, and Madrona: these provide foundational ideas in batched simulation and data-oriented execution, which Boba extends to high-dimensional deformable Gaussian twins. The transferable combination is static-topology precomputation, contiguous dynamic arrays, and atomic-free gathering, rather than workload-agnostic comparisons of FPS.

Rating

These are the note author's subjective ratings out of 5, not conference review scores.

• Novelty: 4/5. Individual optimizations build on established systems ideas, but their integration into a batched deformable Gaussian-twin pipeline is a clear contribution.
• Experimental Thoroughness: 4/5. The paper covers latency, power, quality, capacity, and planning with informative batch ablations; extreme-deformation behavior and completed cloth runs need more evidence.
• Writing Quality: 4/5. Deployment configurations and metric definitions are clear, but important implementation details rely on the supplement and coarsening-refit details are incomplete in the main text.
• Value: 4/5. The work offers a practical acceleration direction for expensive deformable-twin rollouts, particularly robot planning with one asset and many candidate actions.