HoloTetSphere: Unified TetSphere Mesh Reconstruction for Physical Simulations

Conference: ECCV 2026
Paper: ECCV Official
Area: 3D Vision
Keywords: Tetrahedral Mesh Reconstruction, Physical Simulation, Topology Optimization, Gaussian Splatting, HC-Laplacian Smoothing

TL;DR

Addressing the error-prone nature of decoupled two-stage surface extraction and tetrahedralization pipelines, as well as the topological inflexibility and disjoint fragmentation caused by homeomorphic constraints in existing Lagrangian volumetric primitives, HoloTetSphere introduces a tightly coupled tetrahedral-Gaussian representation that achieves end-to-end differentiable element pruning via a vertex-shared continuous opacity field and alternates geometric refinement with two-stage HC-Laplacian smoothing to reconstruct high-fidelity, single-connected tetrahedral meshes ready for physical simulations.

Background & Motivation

Accurate volumetric 3D shape modeling serves as a critical foundation for physical simulation, robotic manipulation, and virtual reality. However, standard pipelines for generating physics-ready meshes predominantly rely on a decoupled two-stage paradigm: first extracting a surface mesh from multi-view images or neural implicit representations (e.g., NeRF/NeuS), and subsequently tetrahedralizing the bounded interior using discrete meshing tools such as TetGen. This pipeline is notoriously brittle; minor geometric imperfections on the reconstructed surface—such as non-manifold edges, micro-holes, or self-intersections—frequently cause tetrahedralization algorithms to crash or produce severely inverted and fragmented elements.

To bypass the fragility of discrete meshing, explicit Lagrangian neural representations such as 3D Gaussian Splatting and TetSphere Splatting have emerged. TetSphere Splatting incorporates tetrahedral elements as deformable volumetric primitives, uniting rendering efficiency with geometric deformation. Nonetheless, existing Lagrangian formulations rely on homeomorphic mappings over initial primitives, imposing a rigid topology-preservation constraint: optimization cannot create cavities, holes, or adapt to topology variations. Consequently, TetSphere is forced to initialize with a complex collection of pre-clustered, disjoint tetrahedral spheres. The final reconstruction inevitably remains an aggregate of dozens of disconnected clusters, completely forfeiting the volumetric structural coherence required for finite element method (FEM) physical simulations and shattering under dynamic stress.

The core tension lies in the conflict between explicit volumetric connectivity and dynamic topological adaptation in Lagrangian representations. Core idea: by coupling Gaussian spheres to tetrahedral mesh vertices and formulating a vertex-shared continuous scalar field for differentiable element pruning, combined with alternating geometry refinement under two-stage normal-decomposed HC-Laplacian smoothing, HoloTetSphere directly reconstructs a watertight, single-connected, physics-ready tetrahedral mesh from a single convex hull.

Method

Overall Architecture

HoloTetSphere unifies multi-view volumetric reconstruction and downstream simulation-ready mesh generation into an end-to-end framework. The pipeline begins by extracting a coarse point cloud from multi-view inputs via 2D Gaussian Splatting (2DGS), computing its 3D convex hull, and performing Delaunay tetrahedralization to produce a watertight, single-connected initial mesh \(\mathcal{M}_0\) enclosing the target object.

Throughout the optimization, Gaussian spheres are tightly coupled to individual tetrahedral elements through vertex barycentric interpolation, synchronizing Gaussian movements with tetrahedral deformation. The system alternates between two synergistic optimization phases: in the topology optimization stage, a continuous vertex implicit field predicts a spatially coherent opacity field, leveraging edge-based smoothness and Eikonal regularizers for differentiable pruning of redundant exterior elements; in the coupled geometry optimization stage, addressing surface roughness and volume shrinkage caused by internal elements becoming newly exposed boundary faces, the system employs Gaussian-guided normal supervision alongside a two-stage HC-Laplacian smoothing strategy and weighted bi-harmonic energy to achieve fine detail recovery while strictly preserving global volume.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multi-view Image Inputs"] --> B["Convex Hull Initialization<br/>2DGS point cloud → convex hull → Delaunay mesh"]
    B --> C["Continuous Opacity Field Pruning<br/>Vertex-shared scalar field + Eikonal/smoothness regularizers"]
    C --> D["Alternating Gaussian-Tetrahedral Geometry Refinement<br/>Barycentric interpolation + normal consistency supervision"]
    D --> E["Two-stage HC-Laplacian Smoothing<br/>Stage 1 concavity pushing + Stage 2 global smoothing"]
    E --> F["Unified Physical Tetrahedral Mesh<br/>FEM simulation-ready and high-fidelity novel view rendering"]

Key Designs

1. Continuous Opacity Field Pruning: Escaping Homeomorphic Constraints without Topological Fragmentation

Traditional Lagrangian volumetric primitives optimize independent opacity parameters or binary masks per cell, which under noisy multi-view photometric gradients leads to scattered deletions, floating artifacts, and chaotic topological fragmentation. To enable topology adaptation on a unified mesh without sacrificing connectivity, HoloTetSphere adapts principles from continuous topology optimization by grounding element opacities upon a shared vertex implicit field.

Specifically, each vertex in the tetrahedral mesh is assigned a learnable scalar field value \(\phi\). For any tetrahedron \(m_i=(v_{i,1}, v_{i,2}, v_{i,3}, v_{i,4})\), the opacity \(\alpha_i\) of its coupled Gaussian sphere is derived from the barycentric mean of its four vertices via a scaled sigmoid function:

\[\bar{\phi}_i = \frac{1}{4} \sum_{j=1}^4 \phi_{i,j}, \quad \alpha_i = \sigma(-s \cdot \bar{\phi}_i \cdot \kappa)\]

where \(\sigma(\cdot)\) denotes the sigmoid function, \(s\) is a fixed scale factor, and \(\kappa\) is a learnable sharpness parameter governing transition steepness. Because adjacent tetrahedra inherently share vertices, updates to any vertex smoothly propagate to its neighbor cells. To prevent field explosion or high-frequency oscillations across the mesh, the topology optimization objective augments the multi-view rendering loss \(L_{\text{render}}\) with discrete Eikonal and smoothness regularizers defined over mesh edges \(E\):

\[L_{\text{topo}} = L_{\text{render}} + \lambda_{\text{eik}} L_{\text{eik}} + \lambda_{\text{smooth}} L_{\text{smooth}}\]

\[L_{\text{eik}} = \frac{1}{|E|} \sum_{(u,v)\in E} \left( \frac{|\phi_u - \phi_v|}{\|x_u - x_v\|_2} - 1 \right)^2, \quad L_{\text{smooth}} = \frac{1}{|E|} \sum_{(u,v)\in E} (\phi_u - \phi_v)^2\]

\(L_{\text{eik}}\) enforces a unit gradient magnitude to ensure signed-distance-like smooth spatial variation, while \(L_{\text{smooth}}\) penalizes local discontinuities. Together, they guarantee that threshold-based element deletion yields smooth, connected boundary surfaces rather than isolated clusters.

2. Alternating Gaussian-Tetrahedral Geometry Refinement: Micro-Surface Normal Consistency Guidance

During geometric refinement, Gaussian spheres and tetrahedral elements provide complementary advantages: Gaussians excel at capturing high-frequency radiance and micro-geometry, while the tetrahedral mesh enforces structural continuity and volume coherence. The framework establishes a bidirectional alternating optimization loop.

In the forward pass, the position of each coupled Gaussian sphere \(g_k\) is strictly determined by barycentric interpolation of its host tetrahedron's vertices: \(g_k = \sum_{i=1}^4 \beta_{ik} v_i\). The photometric loss rapidly optimizes Gaussian appearances, yielding high-quality rendered surface normals \(n_{\text{GS}}\). In the backward feedback pass, to guide coarse tetrahedral boundary facets toward fine geometric details, the rasterized tetrahedral surface normals \(n_{\text{Tet}}\) are directly aligned with the Gaussian normals using a masked cosine distance loss across visible pixels \(M_{\text{mask}}\):

\[L_{\text{norm}} = \sum_p M_{\text{mask}}(p) \left( 1 - \langle n_{\text{GS}}(p), n_{\text{Tet}}(p) \rangle \right)\]

This normal transfer mechanism enables tetrahedral boundary vertices to adaptively deform to intricate surface details without getting trapped in the overly smooth local minima typical of pure mesh rasterization.

3. Two-Stage HC-Laplacian Smoothing: Mitigating Volume Shrinkage and Decoupling Surface Deformation

When exterior tetrahedra are pruned, unrefined internal tetrahedra abruptly form new boundary surfaces. Directly applying uniform bi-harmonic energy or standard Laplacian smoothing causes severe volume shrinkage and erases high-frequency features. To resolve this, the authors introduce a two-stage normal-decomposed HC-Laplacian energy coupled with weighted bi-harmonic regularization.

The HC-Laplacian formulation decomposes displacements on boundary vertices \(S = \{v \in F_\partial \mid \text{count}(f)=1\}\) into tangential components \(d_i^t\) and normal components \(d_i^n\), further separating normal movement into outward \((d_i^n)^+\) and inward \((d_i^n)^-\) directions:

\[L_{\text{HC}} = \sum_{i \in S} \left( \|d_i^t\|_2^2 + \lambda_{\text{cond}}(t) \left( (d_i^n)_-^2 + (d_i^n)_+^2 \right) \right)\]

In Stage 1 (\(\lambda_{\text{cond}}=0\)), normal displacement damping is disabled, allowing tangential distortion relaxation while pushing concave surfaces outward to restore original object volume. In Stage 2 (\(\lambda_{\text{cond}}=1\)), symmetric normal regularization is activated to achieve comprehensive surface fairness. Furthermore, deformation flexibility is balanced across the volume using a weighted bi-harmonic energy \(L_w = w(x) \|L F_x\|_2^2\), assigning reduced stiffness \(w(x_i)=\gamma < 1\) to surface vertices \(S\) for geometric fidelity while preserving strong interior stiffness to prevent element inversion and preserve element regularity.

Loss & Training

The framework is optimized on a single NVIDIA RTX 4090 GPU over 30k iterations. Initial convex hull and feature alignment run for the first 15k steps, after which topology and geometry alternate in 5k-iteration blocks.

• Topology Pruning Objective: \(L_{\text{topo}} = L_{\text{render}} + \lambda_{\text{eik}} L_{\text{eik}} + \lambda_{\text{smooth}} L_{\text{smooth}}\), optimizing the vertex scalar field \(\phi\).
• Coupled Geometry Objective: \(L_{\text{geo}} = \lambda_m L_{\text{mask}} + \lambda_n L_{\text{norm}} + \lambda_{\text{HC}} L_{\text{HC}} + \lambda_w L_w\), where \(L_{\text{mask}} = \|\hat{O} - M_{\text{gt}}\|_1\) penalizes opacity silhouette discrepancies. In rare cases where localized topological changes induce inverted tetrahedra, an automatic post-processing step fixes 98.3% of them through simple vertex index reordering (as they exhibit neither penetration nor degenerate volume), while only 0.14% truly degenerate surface slivers are culled via a lightweight volume threshold filter, ensuring a 100% inversion-free mesh for stable FEM simulations.

Key Experimental Results

Main Results

Quantitative evaluations are conducted across a combined 3D benchmark encompassing both closed- and open-surface objects (Thingi10k, DeepFashion3D, Objaverse, Google Scanned Objects). HoloTetSphere is benchmarked against Eulerian (NeuS2) and Lagrangian (2DGS, DMesh, TetSphere Splatting) representations.

Surface mesh reconstruction accuracy is presented in the table below:

Method	Geo. Rep.	Rendering	Chamfer ↓	Chamfer L2 ↓	Rec. ↑	Haus. ↓	Vol. IoU ↑
NeuS2	Eulerian	✓	0.0118	0.0209	0.7754	0.2419	0.7275
2DGS	Lagrangian	✓	0.0130	0.0190	0.6678	0.1337	0.6557
DMesh	Lagrangian	✗	0.0152	0.0213	0.6720	0.1173	0.6572
TetSphere	Lagrangian	✓	0.0173	0.0291	0.6644	0.1784	0.5712
Ours	Lagrangian	✓	0.0102	0.0163	0.7442	0.1118	0.7268

Tetrahedral mesh quality and physical simulation suitability metrics are reported below (top: cell geometric regularity; bottom: global connectivity; Vol. denotes direct volumetric reconstruction, "-" indicates no external TetGen step required):

Method	Min Dihedral (°) ↑	Aspect Ratio ↓	Inverted Ratio ↓	MR (Manifoldness) ↑
TetSphere	32.000	2.587	0.057	100.0%
Ours	39.503	2.554	0.017	96.7%

Method	Vol.	Single-Comp. ↑	Multi-Comp. ↓	Avg. Comp. ↓	Failed Ratio ↓
NeuS2 + TetGen	✗	60.0%	30.0%	1.48	10.0%
2DGS + TetGen	✗	40.0%	26.7%	2.10	33.3%
DMesh + TetGen	✗	53.3%	46.7%	7.60	0.0%
TetSphere	✓	0.0%	100.0%	61.21	-
Ours	✓	96.7%	3.3%	1.03	-

In novel view synthesis, HoloTetSphere attains 40.68 dB PSNR, 0.987 SSIM, and 0.024 LPIPS, outperforming 2DGS (40.25 dB / 0.985 / 0.035) and TetSphere (34.75 dB / 0.977 / 0.045) while rendering at 50 FPS. In gravity drop and collision tests evaluated against Isaac Sim FEM ground truth, surface Chamfer distance remains bounded below 0.02 throughout the entire trajectory (\(t=0.0\): 0.0096, \(t=0.25\): 0.0119, \(t=0.5\): 0.0136, \(t=0.75\): 0.0171, \(t=1.0\): 0.0184).

Ablation Study

The ablation experiments isolate the contributions of the two-stage HC-Laplacian, weighted bi-harmonic energy, and the continuous opacity field:

Config	Normal Consistency ↑	F1 score ↑	Edge F1 score ↑	Edge precision ↑	IOU ↑	Max Connection Rate ↑	Num. components ↓
Full model (Ours)	0.819	0.428	0.285	0.379	0.620	1.0	1
w/o two-stage HC	0.784	0.317	0.132	0.227	0.558	1.0	1
w/o W-Biharm.	0.684	0.444	0.274	0.178	0.645	1.0	1
w/o Con-opacity	0.597	0.405	0.202	0.119	0.585	0.98	133

Key Findings

• Cause of Topological Shattering: Disabling the continuous opacity field (w/o Con-opacity) prompts independent cell deletions, fragmenting the mesh into 133 disjoint components and collapsing edge precision from 0.379 to 0.119, proving that vertex-shared field interpolation is essential for topological coherence.
• Surface Fairness vs. Contraction: Removing the weighted bi-harmonic formulation (w/o W-Biharm.) causes severe loss of fine boundary fidelity: although global IoU appears marginally higher due to excessive uniform contraction (0.645 vs 0.620), normal consistency drops dramatically to 0.684 and edge precision falls by over half to 0.178.
• Volume Preservation via Two-Stage HC: Replacing the two-stage HC-Laplacian with standard smoothing leads to severe boundary shrinkage and feature erasure, with F1 score dropping to 0.317. The concavity pushing stage is indispensable for maintaining correct volumetric profiles.

Highlights & Insights

• Bridging Continuous Topology Optimization and Explicit Primitives: Mapping continuous density optimization techniques to vertex-shared implicit fields over a deformable tetrahedral mesh elegantly resolves the long-standing challenge of non-differentiable element deletion in explicit Lagrangian frameworks.
• Cross-Representation Normal Supervision Loop: Leveraging the photometric fidelity of Gaussian splatting to supervise rasterized tetrahedral surface normals creates a mutually reinforcing loop, bypassing local geometric minima while retaining strict volumetric topology.
• Direct Physics-Ready Asset Generation: Eliminating the brittle post-reconstruction tetrahedralization step marks a major engineering leap, yielding single-connected volumetric meshes that directly support FEM simulation out of the box.

Limitations & Future Work

• Subtractive-Only Topological Modification: The current pruning strategy exclusively removes material from an initial convex hull, lacking the ability to dynamically spawn new geometry or subdivide elements during later training stages.
• Degradation on Ultra-Thin Plates: For structures with a relative thickness below 0.34, tetrahedral elements are prone to geometric slivering and inversion, necessitating adaptive hierarchical remeshing.
• Homogeneous Material Assumption: Physical simulations currently treat the entire reconstructed volume as a uniform elastic medium; integrating multi-modal physical priors to estimate spatially varying Young's modulus and mass density represents an exciting future avenue.

Related Work & Insights

• vs TetSphere Splatting: TetSphere relies on homeomorphic deformation over pre-clustered sphere primitives, resulting in an average of 61 disconnected fragments incapable of continuous force transmission; HoloTetSphere achieves single-connected, unified meshes via continuous field pruning.
• vs NeuS2 / 2DGS + TetGen: Standard surface-first approaches extract intermediate triangular meshes before invoking TetGen, frequently failing due to non-manifold self-intersections (e.g., 2DGS exhibits a 33.3% tetrahedralization failure rate); HoloTetSphere optimizes tetrahedral elements directly to ensure watertight simulation compatibility.

Rating

• Novelty: ⭐⭐⭐⭐⭐ Pioneering framework combining vertex-shared continuous implicit field pruning with explicit volumetric Gaussian splatting for topology-adaptive reconstruction.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive validation covering geometric accuracy, tetrahedral mesh quality, dynamic FEM collision simulation, rendering fidelity, and extensive topological ablations.
• Writing Quality: ⭐⭐⭐⭐⭐ Rigorous mathematical formulations, clear problem motivation, and coherent progression from architectural intuition to empirical evidence.
• Value: ⭐⭐⭐⭐⭐ Offers an invaluable paradigm shift toward directly generating interactive, simulation-ready 3D digital assets from multi-view images for robotics and embodied AI.

Related Papers

• [CVPR 2026] F²HDR: Two-Stage HDR Video Reconstruction via Flow Adapter and Physical Motion Modeling
• [ECCV 2026] CloSeR: Unified Relational Distillation from Closed-Set Teachers for Category Discovery
• [ECCV 2026] MambaRaw: Selective State Space Modeling for Efficient 4K Raw Image Reconstruction
• [ACL 2026] IMPACT: Importance-Aware Activation Space Reconstruction
• [ECCV 2026] Learning to Tessellate: Point Cloud Generation via Recursive Spectral Partitioning