SENC: Handling Self-collision in Neural Cloth Simulation

Conference: ECCV2024
arXiv: 2407.12479
Code: zycliao/senc
Area: Graph Learning
Keywords: cloth simulation, self-collision, graph neural network, self-supervised learning, Global Intersection Analysis
Institution: The University of Hong Kong

TL;DR

This paper proposes SENC, which effectively addresses the cloth self-collision problem in self-supervised neural cloth simulation for the first time, using a self-collision loss based on Global Intersection Analysis (GIA) and a self-collision-aware graph neural network.

Background & Motivation

Self-supervised neural cloth simulation is attractive as it does not require expensive ground-truth data. However, existing methods generally face a critical issue: cloth self-collision. Self-collision frequently occurs in various scenarios, such as self-penetration of clothing in the underarm area when arms are close to the body, and the overlap of the front and back of a skirt.

Existing collision detection and response techniques cannot be directly applied to self-supervised neural simulators, primarily due to the following reasons:

1. Incremental Potential Contact (IPC): Adopts a barrier method to prevent collisions, assuming simulation starts from a collision-free state. The collision energy tends to infinity as collision pairs approach, leading to gradient explosion, which is unsuitable for neural simulators where clothing states are randomly instantiated during training.
2. Penalty-based methods based on edge-edge collisions: Similarly require starting from a collision-free state.
3. Signed Distance Function (SDF) based methods: SDF gradients only locally push particles towards the nearest surface, making them prone to local optima—when one part deeply penetrates another, the intermediate region may get trapped inside the mesh.

Core Problem

How can cloth self-collision be effectively handled in a self-supervised neural cloth simulation framework, without requiring collision-free initialization, while correctly guiding vertices in the penetrated regions to resolve collisions?

Method

1. Self-collision-aware GNN

The architecture is extended based on MeshGraphNets and HOOD. Previous methods build graphs solely based on mesh topological connections, where messages propagate along the topology, failing to handle self-collisions that are far topologically but close spatially.

Key Improvement: Construct additional self-collision edges based on the spatial distance of vertices. For each cloth vertex \(\mathbf{v}_i\), other vertices \(\mathbf{v}_j\) within a spatial distance of \(\|\mathbf{v}_i - \mathbf{v}_j\| < r\) (\(r = 2\) cm) are searched, and original mesh edges (as adjacent vertices do not self-collide) are excluded to construct the self-collision edges \(\mathbf{e}^{\text{self-col}}_{ij}\).

The node feature update formula is extended to:

\[\mathbf{x}'_i \leftarrow f_{\text{node}}\left(\mathbf{x}, \sum_j \mathbf{e}'^{\text{ body}}_{ij}, \sum_j \mathbf{e}'^{\text{ cloth}}_{ij}, \sum_j \mathbf{e}'^{\text{ self-col}}_{ij}\right)\]

In addition, variable external forces are modeled by appending them to the input node features. During training, a force of random direction and magnitude is generated at each iteration, which is superimposed onto gravity.

2. Self-collision Loss Based on GIA

The self-collision loss is based on the volume formed by cloth self-penetration, calculated through the following four steps:

Step 1: Garment Closure. Fill the openings of the garment (such as cuffs) to form a closed mesh for the penetrated regions. This is achieved by calculating the average position of each open boundary and adding fan triangles to connect them.

Step 2: Remeshing. Remesh configurations with self-intersections so that all intersection points lie exactly on the remeshed edges. The intersection points are represented by barycentric coordinates to support backpropagation.

Step 3: Global Intersection Analysis. Find all intersection paths and distinguish between two types of self-collisions: - A single intersection path generated by the folding of a single region (e.g., a severely bent elbow) - Two intersection paths generated by the intersection of two distinct regions (e.g., torus self-intersection)

These two cases are distinguished via loop vertices (vertices shared by two intersecting triangles). A parallel flood-fill algorithm is used to extract the set of penetrated faces: traversing from both sides of the intersection path simultaneously, the side that completes traversal first is considered the penetrated region.

Step 4: Penetration Volume Computation. For the penetrated faces extracted by GIA, the penetration volume is computed by summing the signed volume of the tetrahedrons formed by each penetrated face and the origin. Since the new vertices in the remeshing step are represented by the barycentric coordinates of the original vertices, the volume loss can be backpropagated through the neural network.

3. Overall Energy Model

The total loss function consists of seven terms:

\[\mathcal{L} = \mathcal{L}_{\text{self-col}} + \mathcal{L}_{\text{col}} + \mathcal{L}_{\text{stretching}} + \mathcal{L}_{\text{bending}} + \mathcal{L}_{\text{ext-force}} + \mathcal{L}_{\text{inertia}} + \mathcal{L}_{\text{friction}}\]

where \(\mathcal{L}_{\text{self-col}}\) penalizes the self-penetration volume, \(\mathcal{L}_{\text{col}}\) prevents garment-body collisions based on the human body SDF, and \(\mathcal{L}_{\text{stretching}}\) and \(\mathcal{L}_{\text{bending}}\) are based on the St.Venant-Kirchhoff material model and bending energy, respectively.

4. Training Strategy

• Trained using 52 motion capture sequences from the AMASS dataset.
• Two-stage training: pre-training for 120K steps without self-collision loss, followed by training for 70K steps with all losses included (due to noisy network output in early stages, GIA computation can be time-consuming and prone to numerical errors).
• Total training takes approximately 48 hours on a single RTX 4090.

Key Experimental Results

Evaluated on 4 test sequences (2175 frames) from the AMASS dataset, using the mean self-collision loss and the ratio of frames exceeding a threshold as metrics:

Method	T-shirt \(\mathcal{L}_{\text{self-col}}\) (×10⁻³)	T-shirt %(0.01)	Skirt \(\mathcal{L}_{\text{self-col}}\) (×10⁻³)	Skirt %(0.01)
SNUG	52.00	38.76	N/A	N/A
NCS	75.05	70.58	476.03	100
HOOD	26.10	22.62	7.33	9.84
SENC	1.80	3.72	1.59	2.71

SENC significantly outperforms existing methods across all metrics, reducing the self-collision loss by approximately 14 times on T-shirts (vs HOOD) and about 4.6 times on skirts.

Ablation Study

Variant	T-shirt \(\mathcal{L}_{\text{self-col}}\)	Skirt \(\mathcal{L}_{\text{self-col}}\)
Area loss instead of volume loss	28.81	8.80
W/o self-collision edges	1.14	7.92
Keep mesh edges	2.37	2.02
SENC (Full)	1.80	1.59

• Area loss is almost ineffective: gradients for reducing penetration area cannot effectively guide vertices to resolve collisions.
• Removing self-collision edges is effective on T-shirts but fails on skirts: collision regions in skirts have larger topological distances.
• Keeping mesh edges performs slightly worse than the excluded version and increases computational overhead.

Highlights & Insights

1. First to address self-collision in neural cloth simulation: By extracting the penetration volume via GIA to construct the loss, the gradients naturally act as collision-resolving forces.
2. Self-collision-aware GNN: Constructs additional edges based on spatial distance, covering the blind spots of topological graphs regarding long-range collisions.
3. Differentiable design: All new vertices are represented by barycentric coordinates, allowing the penetration volume loss to be backpropagated end-to-end.
4. External force modeling: Supports user-applied external forces such as wind, enhancing simulation dynamics.
5. Generality: Can be seamlessly integrated into existing neural cloth simulation frameworks.

Limitations & Future Work

1. Computational efficiency: The remeshing and GIA steps in penetration volume calculation are computationally expensive.
2. Garment closure limitations: Can only handle meshes that are easy to close; closing garments with complex topology (such as the bottom opening of a long skirt) may suppress plausible deformations or miss collisions.
3. Dilemma of closure heuristics: Taking a long skirt as an example—closing the bottom prevents the hem from moving upward correctly, whereas leaving it open makes it impossible to detect collisions between the front and back of the hem.
4. Not extended to multi-layer cloth and 3D deformable characters.

Related Work & Insights

Method	Self-supervised	Handles Self-collision	Dynamic Simulation	Arbitrary Topology
SNUG	✓	✗	Limited	✗
NCS	✓	✗	✓	✗
HOOD	✓	✗	✓	✓
ClothCombo	✓	Simple repulsion	Quasi-static	✓
ContourCraft	✓	Contour length (potentially reversed)	✓	✓
SENC	✓	✓ (GIA Volume)	✓	✓

Unlike the simple repulsive loss of ClothCombo, SENC's gradient based on penetration volume correctly guides the collision resolution direction. Compared with ContourCraft's loss based on self-intersection contour length, SENC does not exacerbate collisions when the true resolution direction opposes the direction that reduces contour length.

Insights & Connections

• The idea of extracting penetration volume via GIA can be generalized to other tasks requiring mesh self-intersection handling, such as mesh quality optimization in 3D generation.
• The strategy of constructing additional graph edges based on spatial distance is valuable for any GNN task that requires long-range interactions across topologies.
• Future work could explore winding numbers or electronic flux to compute approximate penetration volumes without closed boundaries.

Rating

• Novelty: ⭐⭐⭐⭐ — First to introduce GIA into neural cloth simulation to handle self-collision; the self-collision-aware GNN is well-designed.
• Experimental Thoroughness: ⭐⭐⭐⭐ — Comprehensive quantitative and qualitative comparisons across multiple garment types, with thorough ablation studies.
• Writing Quality: ⭐⭐⭐⭐ — Clear motivation, detailed method descriptions, and intuitive illustrations.
• Value: ⭐⭐⭐⭐ — Resolves a long-standing key issue in neural cloth simulation, showing strong practicality.

Related Papers

• [NeurIPS 2025] Self-Supervised Discovery of Neural Circuits in Spatially Patterned Neural Responses with Graph Neural Networks
• [ECCV 2024] Confidence Self-Calibration for Multi-Label Class-Incremental Learning
• [ICML 2025] HGOT: Self-supervised Heterogeneous Graph Neural Network with Optimal Transport
• [ACL 2026] Graph-Based Alternatives to LLMs for Human Simulation
• [ICLR 2026] Self-Consistency Improves the Trustworthiness of Self-Interpretable GNNs