NeuralGarSim: Geometry-agnostic Garment Simulation with Neural Fields

Conference: ECCV 2026
Paper: ECCV Official
Code: Project Page
Area: 3D Vision
Keywords: neural physics simulation, neural fields, tangential differential calculus, thin-shell mechanics, geometry-agnostic representation

TL;DR

Addressing the mesh-resolution sensitivity of conventional solvers and the strict 2D parameterisation barrier in existing neural simulators, NeuralGarSim reformulates nonlinear Kirchhoff–Love thin-shell mechanics directly in 3D Euclidean space via Tangential Differential Calculus (TDC), achieving continuous, parameterisation-free quasistatic garment simulation seamlessly across multi-panel topologies and diverse geometric representations including meshes, UDFs, point clouds, and 3D Gaussians.

Background & Motivation

Physics-based cloth simulation has long relied on discrete triangle meshes and finite element methods (FEM) to resolve internal elastic potential and dynamic equilibrium. However, explicit discretisation imposes fundamental limitations: predicted fold trajectories and wrinkle patterns remain intrinsically sensitive to mesh resolution and initial triangulation topology. Achieving fine wrinkle fidelity routinely demands adaptive remeshing or running on prohibitive high-resolution meshes, which drastically escalates computational and memory overhead while breaking the differentiability required for downstream inverse physics and optimization tasks.

To eliminate resolution dependency, pioneer works such as NeuralClothSim introduced continuous coordinate-based multilayer perceptrons (MLPs) to represent non-rigid garment deformations, minimizing potential energy functionals via analytic derivatives. Nevertheless, this line of neural simulators remains fundamentally constrained by classical parametric surface differential geometry, which strictly necessitates an underlying globally bijective and differentiable 2D curvilinear parametric domain \(\tau \subset \mathbb{R}^2\). Real-world garments consist of multiple stitched panels, darts, seams, and geometric cutouts such as collars and armholes. Constructing a continuous 2D parametric chart that retains analytic differentiability across disparate boundaries is notoriously intractable in neural fields, effectively confining prior neural approaches to isolated single cloth panels. Furthermore, this 2D dependency locks simulators to clean parameterised meshes, rendering them incompatible with unsigned distance fields (UDFs), raw point clouds, or 3D Gaussian Splats produced directly by off-the-shelf 3D reconstruction pipelines.

The angle of attack in this work is to bypass 2D parametric mappings altogether and formulate thin-shell deformation directly on the 3D embedded surface. Core idea: by casting the nonlinear Kirchhoff–Love thin-shell model into Tangential Differential Calculus (TDC), membrane and bending strains are computed entirely in 3D Euclidean space using only surface normals and Weingarten curvature tensors, enabling unified, parameterisation-free neural cloth simulation across arbitrary multi-panel garment topologies and multimodal geometry representations.

Method

Overall Architecture

NeuralGarSim takes an undeformed reference garment geometry \(\mathcal{M}_\Gamma\) (provided as a neural UDF, triangle mesh, point cloud, or 3D Gaussian field) and optimizes a continuous 3D neural deformation field (NDF) \(\mathbf{u}(\mathbf{x}_\Gamma; \Theta_\mathbf{u}) : \Gamma \to \mathbb{R}^3\) without constructing any intermediate 2D flattening. The workflow comprises four synchronized phases: geometric prior extraction, neural deformation field prediction with hard boundary modulation, TDC-based thin-shell strain evaluation directly in \(\mathbb{R}^3\), and total potential energy minimization via Monte Carlo sampling.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multimodal Undeformed Input<br/>UDF / Mesh / Point Cloud / 3D Gaussians"] --> B["Geometric Prior Extraction<br/>Surface normals & Weingarten map"]
    B --> C["3D Neural Deformation Field<br/>SIREN mapping with boundary modulation"]
    C --> D["TDC Thin-Shell Strain Evaluation<br/>Nonlinear membrane & bending projection"]
    D --> E["Quasistatic Potential Energy Minimization<br/>Elastic internal energy vs external work"]
    E --> F["Simulated Deformed Surface<br/>Arbitrary topology & continuous resolution"]

Key Designs

1. Geometric Prior Extraction: Unified Differential Operators Across Multimodal Inputs

Previous simulation pipelines could not ingest non-mesh geometries because classical finite element models rely on discrete face elements and local shape functions, while parametric neural fields demand regular 2D UV charts. NeuralGarSim recognizes that a thin-shell formulation based on Tangential Differential Calculus requires only two local geometric operators at each surface evaluation point: the unit surface normal \(\mathbf{n}_\Gamma\) and the Weingarten curvature map \(\mathbf{H} = \nabla_\Gamma^{\text{dir}} \mathbf{n}_\Gamma\). For differentiable unsigned distance fields (DUDF), an MLP \(\phi(\mathbf{x}; \Theta_\phi)\) fits the implicit surface whose zero level-set denotes the cloth mid-surface, providing analytic normals and second derivatives via standard automatic differentiation. For triangle meshes, \(\mathbf{n}_\Gamma\) is computed via area-weighted vertex normals and \(\mathbf{H}\) via cotangent-weighted Laplacians. For point clouds, local principal component analysis (PCA) on nearest neighborhoods provides tangent plane normal estimates. For 3D Gaussians, the unit normal is assigned to the axis of minimal variance derived from each Gaussian covariance matrix. The orthogonal projection operator onto the tangent space \(T_{\mathbf{x}_\Gamma}\) is then formulated uniformly as:

\[\mathbf{P} = \mathbf{I} - \mathbf{n}_\Gamma \otimes \mathbf{n}_\Gamma\]

Any 3D vector \(\mathbf{v}\) projected via \(\mathbf{P} \cdot \mathbf{v}\) yields an exact tangential vector, decoupling intrinsic geometry from any ambient parametric coordinates.

2. 3D Neural Deformation Field: Parameterisation-Free Mapping with 3D Hard Boundary Modulation

To overcome chart boundary singularities and tearing across stitched multi-panel garments, NeuralGarSim defines the neural deformation field directly over 3D coordinates: \(\mathbf{x}_\Gamma \in \mathbb{R}^3 \mapsto \mathbf{u} \in \mathbb{R}^3\). The backbone is implemented as a 5-layer coordinate-based MLP with 256 hidden units and periodic SIREN activations, whose \(C^\infty\) smoothness enables reliable second-order analytic differentiation required by thin-shell bending energies. To enforce Dirichlet boundary conditions (e.g., pinning shoulders or waists), parametric models require cumbersome 2D UV masks; here, boundary constraints are imposed directly in 3D Euclidean space via a continuous distance scalar field \(\mathcal{B}(\mathbf{x}_\Gamma)\):

\[\mathbf{u}(\mathbf{x}_\Gamma; \Theta_\mathbf{u}) = \mathcal{F}_{\Theta_\mathbf{u}}(\mathbf{x}_\Gamma) \cdot \mathcal{B}(\mathbf{x}_\Gamma)\]

The modulation function evaluates strictly to zero on the pinned boundary subset \(\partial\Gamma\) and transitions smoothly to one elsewhere. This design guarantees that multi-panel garments with complex cuts, seams, and necklines can be trained end-to-end without seam stitching heuristics or artificial periodic boundary conditions.

3. TDC Thin-Shell Strain Evaluation: Fully Nonlinear Mechanics in 3D Space

To capture finite rotations and large stretching without relying on classical parametric fundamental forms, the framework formulates Green–Lagrange strain tensors entirely within 3D Euclidean space under the Kirchhoff–Love hypothesis. In the absence of transverse shear, the mid-surface director difference vector \(\mathbf{w}\) (capturing normal rotation) is derived as:

\[\mathbf{w} = \mathbf{H} \cdot \mathbf{u} - \nabla_\Gamma(\mathbf{u} \cdot \mathbf{n}_\Gamma)\]

Using directional surface gradients of displacement \(\mathbf{F}_u = \nabla_\Gamma^{\text{dir}} \mathbf{u}\) and director difference \(\mathbf{F}_w = \nabla_\Gamma^{\text{dir}} \mathbf{w}\), the directional membrane strain \(\boldsymbol{\epsilon}_M^{\text{dir}}\) and directional bending strain \(\boldsymbol{\epsilon}_B^{\text{dir}}\) are established as:

\[\boldsymbol{\epsilon}_M^{\text{dir}} = \frac{1}{2} \left( \mathbf{F}_u + \mathbf{F}_u^\mathsf{T} + \mathbf{F}_u^\mathsf{T} \mathbf{F}_u \right)\]

\[\boldsymbol{\epsilon}_B^{\text{dir}} = \frac{1}{2} \left( \mathbf{F}_w + \mathbf{F}_w^\mathsf{T} + \mathbf{F}_u^\mathsf{T} \mathbf{F}_w + \mathbf{F}_w^\mathsf{T} \mathbf{F}_u \right)\]

Because these tensors reside in ambient space \(\mathbb{R}^{3 \times 3}\), applying the tangential projector \(\mathbf{P}\) yields the true intrinsic in-plane strains: \(\boldsymbol{\epsilon}_M = \mathbf{P} \cdot \boldsymbol{\epsilon}_M^{\text{dir}} \cdot \mathbf{P}\) and \(\boldsymbol{\epsilon}_B = \mathbf{P} \cdot \boldsymbol{\epsilon}_B^{\text{dir}} \cdot \mathbf{P}\). This formulation retains full geometric nonlinearity while completely bypassing Christoffel symbols, streamlining backpropagation into standard tensor contractions.

4. Quasistatic Potential Energy Minimization: Dual Constitutive Laws and Monte Carlo Integration

The equilibrium configuration of the deformed garment is discovered by minimizing the total potential energy \(\Pi = \Pi_{\text{int}} + \Pi_{\text{ext}}\). The external potential accounts for gravity \(\Pi_{\text{ext}} = - \int_\Gamma \rho \mathbf{g} \cdot \mathbf{u} \, d\mathbf{x}_\Gamma\). For internal strain energy \(\Pi_{\text{int}}\), the architecture supports two distinct material models: - Isotropic Linear Elasticity: Designed for benchmark mechanical validation, where stress resultants across thickness \(h\) are pre-integrated analytically, yielding mid-surface double-contraction energy between membrane/bending resultants and strain tensors. - Orthotropic St. Venant–Kirchhoff (StVK) Model: Employed for realistic textiles with distinct warp and weft fiber stiffnesses, governed by parameters \(\boldsymbol{\Phi} = \{a_{11}, a_{22}, a_{12}, G_{12}\}\). Because StVK energy density is nonlinear across the shell thickness, numerical integration through \([-h/2, h/2]\) is computed using composite Simpson quadrature.

During training, the continuous surface integral is evaluated using Monte Carlo quadrature over 1,000 points uniformly sampled on \(\mathcal{M}_\Gamma\) per iteration. The network converges to physical equilibrium within 2,000–5,000 Adam optimization steps.

Key Experimental Results

Main Results

Physical accuracy was validated on standard structural mechanics benchmarks (Belytschko shell tests), covering membrane-dominated deformation (Square Plate), mixed membrane-bending deformation (Scordelis-Lo Roof), and bending-dominated deformation (Pinched Cylinder), comparing target point maximum displacements against analytical solutions.

Benchmark Case	Analytical [5, 81]	Guo et al.	Bastek et al.	NeuralClothSim	NeuralGarSim (Ours)	Relative Error
Square Plate	0.4870	2.566*	N/A	0.4870	0.4870	< 0.01%
Scordelis-Lo Roof	0.3024	N/A	0.2970	0.2965	0.2987	1.22%
Pinched Cylinder	1.825e-5	N/A	N/A	1.825e-5	1.820e-5	0.27%

Note: Guo et al. adopted alternative material constants matched to their analytical benchmark.

In terms of computational efficiency, computing differential quantities directly on 3D embedded samples eliminates the costly differentiation through Christoffel symbols on 2D parametric domains, delivering a consistent 2–4× training speedup over NeuralClothSim under identical optimization schedules.

Benchmark / Scenario	NeuralClothSim Time	NeuralGarSim Time (Ours)	Speedup
Square Plate	13m 01s	6m 26s	2.02×
Scordelis-Lo Roof	4h 49m	2h 21m	2.05×
Pinched Cylinder	56m 04s	27m 17s	2.05×
Napkin Moving Handles	6m 09s	1m 35s	3.88×
Skirt Twist	21m 45s	7m 14s	3.01×
Sleeve Buckle	11m 03s	5m 55s	1.87×
Average Training Time	1h 06m 10s	31m 35s	2.10×

Ablation Study

1. Discretisation Independence Across Mesh Resolutions

Evaluating a T-shirt simulated across three distinct isotropic remeshings (\(M_1, M_2, M_3\)) with identical pinned boundaries. The bidirectional wrinkle Chamfer distance \(d_W\) (extracted from vertices exceeding the 75th percentile mean curvature) and full-surface Chamfer distance \(d_C\) confirm discretisation independence.

Method	\(d_W(M_1, M_2)\)	\(d_W(M_1, M_3)\)	\(d_C(M_1, M_2)\)	\(d_C(M_1, M_3)\)	Characteristic
DiffCloth (Fixed-Mesh FEM)	0.035	0.034	0.020	0.021	Wrinkle patterns shift with element connectivity
NeuralGarSim (Continuous NDF)	0.011	0.013	0.009	0.009	Highly consistent wrinkles across resolutions

2. Cross-Modal Geometric Representation Consistency

Taking the simulation on a fine triangle mesh as reference \(M_{\text{ref}}\), NeuralGarSim was executed independently on UDF, point cloud, and 3D Gaussian representations of the identical reference asset.

Input Modality	Wrinkle Distance \(d_W(M_{\text{ref}}, M_i)\)	Full Chamfer Distance \(d_C(M_{\text{ref}}, M_i)\)	Modality Handling
Triangle Mesh (Reference)	ref.	ref.	Vertex normals and cotangent Laplacian curvature
Differentiable UDF	0.017	0.013	Analytic gradients and Weingarten map from MLP
Point Cloud	0.018	0.015	Local PCA tangent estimation, zero-curvature assumption
3D Gaussians	0.017	0.014	Smallest variance axis of covariance matrix

Key Findings

• TDC Eliminates the Failure Modes of Parametric Stitching: An ablation extending NeuralClothSim to multi-panel garments via per-panel UV charts (NeuralClothSim+) proved that non-conformal parameterisations distort per-face Jacobians, transmitting inconsistent parametric gradients that cause training divergence. In contrast, NeuralGarSim operates directly in \(\mathbb{R}^3\) and converges robustly across complex geometries.
• Continuous Fields Decouple Wrinkles from Meshing: While adaptive remeshing solvers like DiffARCSim exhibit wrinkle drift due to dynamic edge flipping, NeuralGarSim retains spatial wrinkle correspondence because folds are governed by the global continuous coordinate MLP rather than local mesh edges.
• Cross-Modal Consistency Within 2%: Simulation outputs across UDFs, meshes, point clouds, and 3D Gaussians show cross-modal Chamfer distances below 0.018, proving that high-order physical simulation can be performed directly on unmeshed reconstruction outputs.

Highlights & Insights

• First Fusion of Tangential Differential Calculus and Neural Physics: While TDC has been utilized in linear FEM structural mechanics, this paper is the first to marry TDC with continuous neural fields for fully nonlinear thin-shell simulation, avoiding metric tensors and Christoffel symbols.
• Reconstruction-Ready Physics Pipeline: By bypassing surface meshing, manifold hole-filling, and UV unwrapping, NeuralGarSim can directly drape raw 3D Gaussian Splats or laser-scanned point clouds produced by multi-view reconstruction methods.
• Simultaneous Speedup and Generalization: Formulating derivatives in 3D Euclidean space significantly trims the automatic differentiation computational graph, speeding up neural network convergence by 2–4× over parametric counterparts.

Limitations & Future Work

• Quasistatic Restriction Without Dynamics: The framework solves static equilibrium states under gravity and boundary displacements; it lacks time-stepping, inertia (\(\mathbf{M}\ddot{\mathbf{u}}\)), and velocity-dependent damping required for dynamic garment fluttering.
• Absence of Explicit Collision and Self-Contact Handling: Without an active penalty barrier or Incremental Potential Contact (IPC) formulation, severe folding or multi-layer draping can result in unphysical garment interpenetration.
• Per-Garment Optimization Latency: Training the deformation field takes several minutes per garment configuration; extending this toward generalized feed-forward networks or grid-accelerated architectures (\(\partial^\infty\)-Grid) remains a key future direction.

Related Work & Insights

• vs NeuralClothSim: Both employ neural fields to minimize thin-shell potential energies, but NeuralClothSim is bounded by 2D parametric domains and single cloth panels; NeuralGarSim leverages TDC to handle multi-panel garments with arbitrary seams and cutouts while training 2–4× faster.
• vs DiffARCSim / DiffCloth: Classical differentiable FEM simulators depend heavily on mesh connectivity, causing wrinkle patterns to fluctuate across resolutions and consuming substantial memory on fine meshes; NeuralGarSim offers continuous, resolution-agnostic drape.
• vs GaussianGarments: GaussianGarments reconstructs dynamic clothing from multi-view video but relies on empirical motion tracking; NeuralGarSim provides rigorous thin-shell physics directly over registered 3D Gaussians.

Rating

• Novelty: ⭐⭐⭐⭐⭐ [Pioneers the use of Tangential Differential Calculus in neural physics simulation, eliminating the fundamental 2D parameterisation barrier]
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive validation across analytical Belytschko benchmarks, multi-resolution remeshing, and four distinct geometric modalities]
• Writing Quality: ⭐⭐⭐⭐⭐ [Rigorous mathematical formulation with clear, intuitive exposition connecting differential geometry to deep neural fields]
• Value: ⭐⭐⭐⭐⭐ [Bridges the gap between modern 3D neural surface reconstruction and continuous physics-based simulation]

Related Papers

• [ECCV 2026] PhysConvex: Physics-Informed Dynamic Convex Fields for Reconstruction and Simulation
• [CVPR 2026] ReWeaver: Towards Simulation-Ready and Topology-Accurate Garment Reconstruction
• [ECCV 2026] MLP Splatting: Object-Centric Neural Fields
• [ECCV 2026] GrowFields: Compositional 4D Neural Fields for Topology-Changing Plant Growth
• [ECCV 2026] MotionAnymesh: Physics-Grounded Articulation for Simulation-Ready Digital Twins