ETCH-X: Robustify Expressive Body Fitting to Clothed Humans with Composable Synthetic Data¶

Conference: ECCV 2026
Paper: ECCV Official
Code: https://xiaobenli00.github.io/ETCH-X
Area: 3D Vision
Keywords: Clothed Human Body Fitting, SMPL-X, Implicit Dense Correspondences, Tightness Disentanglement, Hand Pose Refinement

TL;DR¶

ETCH-X establishes an "undress first, then dense fit" modular paradigm that unifies local SE(3)-equivariant tightness vector prediction with implicit dense neural fields, enabling scalable training on composable synthetic datasets (CLOTH3D garments, AMASS poses, and InterHand2.6M gestures) for expressive and robust SMPL-X body fitting under diverse clothing, complex motions, and partial point cloud scans.

Background & Motivation¶

Fitting parametric human body models (such as SMPL or SMPL-X) to raw 3D point clouds of clothed humans is a cornerstone for computer vision and graphics downstream tasks, including character animation, garment retargeting, and human-scene interaction. However, raw point clouds captured by 3D scanners and consumer depth sensors are intrinsically noisy, sparse, unorganized, and plagued by severe occlusions and partial viewpoints. Furthermore, the large variability in clothing geometries and dynamic wrinkles creates severe discrepancies between outer garment surfaces and underlying anatomical human bodies. Collecting dense 3D clothed scans paired with perfectly aligned, expressive ground-truth body meshes remains prohibitively costly and labor-intensive.

Existing body fitting methodologies broadly fall into two categories, both suffering from structural bottlenecks. Dense correspondence methods such as Neural ICP (NICP) directly align parametric body templates to clothed surfaces via neural field mappings; because loose or hanging garments deviate arbitrarily from internal body geometry, dense matching becomes mathematically ill-posed, yielding unnatural, overweight, or biomechanically implausible fitted bodies. Conversely, sparse correspondence approaches (e.g., the original ETCH) leverage tightness vectors to strip away clothing displacements but rely heavily on explicit sparse surface markers and skeleton keypoints. Consequently, sparse approaches lack expressiveness for articulate extremities like hands and faces, and critically, suffer catastrophic failure on partial single-view point clouds where key markers fall entirely within unobserved regions.

The strengths and weaknesses of these two paradigms are complementary: dense correspondences capture expressive fine details across all topological regions and gracefully handle arbitrary spatial coordinate queries, yet require an undressed inner surface to avoid clothing ambiguity; tightness vectors reliably decouple garments from the body, yet downstream fitting requires continuous dense representations. Core idea: completely disentangle clothed body fitting into two standalone stages—masked undressing followed by implicit dense fitting—leveraging composable synthetic datasets (simulated clothing, full-body mocap, and fine-grained hand gestures) to train each module independently and fit expressive SMPL-X bodies via local neural fields and re-sampled hand refinement.

Method¶

Overall Architecture¶

ETCH-X adheres to the modular "undress first, then dense fit" paradigm, decomposing the end-to-end fitting task into two decoupled sequential stages. In the first stage, Masked Undress, the network processes the input clothed point cloud to predict pointwise SE(3)-equivariant tightness vectors along with skin-aware classification masks, projecting the outer garment surface points inward to recover a bare-body point cloud. In the second stage, Dense Fit, an implicit feature volume is constructed from the recovered inner points, where a localized neural field (LoVD) predicts ordered vertex offset vectors to the SMPL-X template under test-time iterative fine-tuning. Finally, a dedicated hand re-sampling and classification module refines the hand gestures using MANO before re-integrating them into the global SMPL-X mesh.

The complete information flow is illustrated in the vertical flowchart below:

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Clothed Human Scan X"] --> B["Equivariant & Invariant Encoding<br/>EPN feature extraction + Point Transformer"]
    B --> C["Masked Undress Prediction<br/>Direction Head + Magnitude Head + Tightness Mask"]
    C --> D["Recovered Inner Body Points ŷ"]
    D --> E["Implicit Dense Fitting & Test-Time Tuning<br/>IF-Nets Feature Volume + LoVD Neural Field"]
    E --> F["Global SMPL-X Parameter Optimization"]
    F --> G["Hand Refinement by Re-sampling<br/>Hand Classifier + Hand LVD + MANO Fitting"]
    G --> H["Output: Expressive Posed SMPL-X Mesh"]

Key Designs¶

1. Masked Undress: Local SE(3)-Equivariant Tightness Vectors with Skin-Aware Masking Directly establishing correspondences on loose garments introduces substantial geometrical ambiguity. ETCH-X projects each outer point $x_i$ to an underlying inner body point via a tightness vector $y_i = x_i + v_i$. The vector $v_i = b_i d_i$ is decomposed into a directional unit vector $d_i$ and a scalar displacement magnitude $b_i$. The direction correlates intrinsically with local articulated body poses and is learned via an Equivariant Point Network (EPN) followed by self-attention across rotation group dimensions, guaranteeing that predicted vectors consistently point toward the anatomical skeleton regardless of global orientation. The magnitude reflects clothing drape and looseness and is regressed using a Point Transformer operating on rotation-invariant features. Crucially, ETCH-X introduces tightness masking: regions such as exposed skin, the face, and bare hands exhibit zero physical tightness. Forcing a regression network to estimate zero displacements often produces boundary leakage and unnatural shrinkage; therefore, a lightweight classifier predicts a binary mask $l_i \in \{0, 1\}$. The inner point is recovered during inference via $\hat{y}_i = x_i + \hat{l}_i \hat{b}_i \hat{d}_i$, preserving exposed skin geometry intact.

2. Implicit Dense Correspondence Field: Local Neural Field with Test-Time Fine-Tuning To overcome the vulnerability of sparse markers on incomplete scans, ETCH-X adopts a continuous implicit formulation. The undressed inner point cloud $\hat{Y}$ is voxelized and encoded by IF-Nets into a multi-scale 3D feature volume $Z = \mathcal{IF}(\hat{Y})$. Rather than using a single global MLP, ETCH-X employs Localized Vertex Descent (LoVD), partitioning the SMPL-X template into 16 anatomical clusters and assigning specialized MLP heads. Given any arbitrary 3D continuous query coordinate $q$, the field predicts ordered coordinate offsets $o \in \mathbb{R}^N$ toward canonical SMPL-X vertices. At inference time, ETCH-X executes test-time fine-tuning: it iteratively draws query points from $\hat{Y}$, determines corresponding template vertex indices, and updates the neural field weights $\theta$ via gradient descent to minimize surface fitting residuals. After convergence, 1,051 dense correspondence anchors are queried to drive SMPL-X parameter optimization. Because the continuous implicit field spans the entire coordinate volume, it reliably hallucinates corresponding anchors even for unobserved anatomical parts under single-view partial scans.

3. Hand Refinement by Re-sampling: Self-Contact Filtering and Decoupled MANO Fitting Because hands occupy a minute fraction of total body surface area, uniform point cloud sampling leaves hand regions under-sampled, resulting in imprecise finger articulations and severe pose jitter. Furthermore, hands frequently make direct self-contact with the torso, thighs, or arms, contaminating local geometric neighborhoods. ETCH-X introduces a two-step refinement strategy: following initial full-body fitting, the 3D bounding sphere of each wrist is computed, and the raw input point cloud is densely re-sampled in this local coordinate frame. To eliminate points belonging to adjacent body regions during self-contact, a hand binary classifier (trained on the MTP dataset) prunes non-hand points. The purified hand points are processed by a dedicated Hand LVD network to generate detailed hand markers, which are fitted to the articulated MANO model. Crucially, the hand LVD model is trained with visibility data augmentation derived from the empirical occlusion distribution of MTP. The optimized MANO parameters are subsequently transformed back and seamlessly merged into the SMPL-X kinematic hierarchy.

4. Composable Scalable Training: Decoupled Multi-Source Synthetic Data Ingestion Real-world 3D scans with ground-truth SMPL-X registrations are rare and expensive to capture. ETCH-X leverages its modular architecture to scale training using diverse, readily available synthetic datasets: - The Masked Undress stage requires paired cloth-to-body correspondences across clothing styles; it is trained on ~150k frames from CLOTH3D, capturing diverse dynamic draping, fabrics, and topologies. - The Dense Fit stage requires rich anatomical pose variations; it is trained on ~120k SMPL-X meshes from AMASS, spanning over 11,000 optical motion capture sequences without requiring garments. - The Hand Refinement module is independently trained on InterHand2.6M and MTP to specialize in complex dexterous manipulations and self-contact occlusions. By avoiding end-to-end monolithic training, each module scales independently with domain-specific assets.

A Worked Example¶

Consider a single-view LiDAR scan of a subject wearing an oversized trench coat with their left hand resting against their hip: 1. Input Sampling: A raw, single-view point cloud $X$ is sampled from the untextured scan. 2. Masked Undressing: EPN extracts rotation-equivariant features. The mask classifier flags the head and exposed right hand as $l_i = 0$, while the trench coat points receive $l_i = 1$. The trench coat points are displaced inward along predicted equivariant vectors $\hat{v}_i$, producing clean inner body points $\hat{Y}$. 3. Implicit Fitting: IF-Nets encodes $\hat{Y}$ into a feature volume. LoVD undergoes test-time optimization on observed front points, while its learned continuous spatial prior accurately infers correspondence anchors on the completely unobserved back. Global SMPL-X optimization aligns body pose and shape. 4. Local Hand Refinement: The wrist location of the resting left hand is extracted, and points within its bounding sphere are densely re-sampled. The hand classifier strips away points belonging to the hip and coat hem. The Hand LVD module predicts dense finger landmarks under simulated self-occlusion priors, guiding precise MANO fitting. 5. Assembly: The refined MANO hand rotations overwrite the crude hand joints in SMPL-X, yielding an expressive, physically plausible mesh in under 10 seconds.

Loss & Training¶

The Masked Undress stage is trained end-to-end with a composite multi-task objective: $$\mathcal{L} = w_d \mathcal{L}_d + w_b \mathcal{L}_b + w_l \mathcal{L}_l$$ where the loss components are defined as: $$\mathcal{L}_d = -\frac{1}{N} \sum_{i=1}^N l_i \frac{\hat{\mathbf{d}}_i \cdot \mathbf{d}_i}{\|\hat{\mathbf{d}}_i\| \|\mathbf{d}_i\|}, \quad \mathcal{L}_b = \frac{1}{N} \sum_{i=1}^N l_i (\hat{b}_i - b_i)^2, \quad \mathcal{L}_l = -\frac{1}{N} \sum_{i=1}^N \log(\mathcal{P}(\mathbf{x}_i, l_i))$$ Here, directional loss $\mathcal{L}_d$ maximizes cosine similarity exclusively on clothed points ($l_i = 1$), $\mathcal{L}_b$ minimizes MSE on displacement lengths, and $\mathcal{L}_l$ optimizes cross-entropy on skin classification. The Dense Fit stage trains LoVD by minimizing Euclidean distance between queried neural offsets and SMPL-X template vertices, supplemented by test-time optimization during inference.

Key Experimental Results¶

Main Results¶

Quantitative evaluations are conducted on standard in-distribution benchmarks: CAPE (form-fitting clothing, 26,004 train / 1,021 val frames) and 4D-Dress (loose dynamic clothing, 59,395 train / 1,943 val frames). All baselines are adapted to fit SMPL-X for equitable evaluation. Metrics include Chamfer Distance (CD, cm), Vertex-to-Vertex error (V2V, cm), and Mean Per Joint Position Error (MPJPE, cm).

Dataset	Method	Group	CD ↓	V2V-All ↓	V2V-Hands ↓	V2V-Head ↓	MPJPE-All ↓	MPJPE-Hands ↓
CAPE	NICP	Tightness-agnostic (A)	-	1.736	2.741	1.184	2.074	2.565
CAPE	ArtEq	Tightness-agnostic (A)	-	2.202	3.417	2.011	2.405	3.055
CAPE	IPNet	Tightness-aware (B)	1.077	5.529	7.454	5.485	5.611	6.600
CAPE	PTF	Tightness-aware (B)	1.194	2.341	3.880	2.038	2.641	3.377
CAPE	ETCH	Tightness-aware (B)	1.040	1.567	3.449	1.236	2.002	2.833
CAPE	ETCH-X (Ours)	Tightness-aware (B)	1.015	1.484	2.215	1.120	1.764	2.148
4D-Dress	NICP	Tightness-agnostic (A)	-	4.085	6.224	3.323	4.862	6.142
4D-Dress	ArtEq	Tightness-agnostic (A)	-	3.072	4.537	3.145	3.378	4.170
4D-Dress	IPNet	Tightness-aware (B)	1.187	7.495	8.881	7.378	7.380	8.606
4D-Dress	PTF	Tightness-aware (B)	1.207	3.297	4.938	3.338	3.567	4.607
4D-Dress	ETCH	Tightness-aware (B)	1.134	2.408	5.108	1.997	3.459	4.695
4D-Dress	ETCH-X (Ours)	Tightness-aware (B)	1.060	1.897	3.101	1.836	2.317	3.065

To rigorously evaluate out-of-distribution (OOD) generalization, models trained on 4D-Dress are tested on BEDLAM2.0 (1,000 paired synthetic frames across 20 subjects with varied BMI and clothing):

Test Set (OOD)	Method	CD ↓ (cm)	V2V-All ↓ (cm)	MPJPE-All ↓ (cm)
BEDLAM2.0	NICP	-	5.178	6.238
BEDLAM2.0	ArtEq	-	4.136	4.447
BEDLAM2.0	IPNet	1.369	8.641	9.471
BEDLAM2.0	PTF	1.288	3.974	4.668
BEDLAM2.0	ETCH	1.454	12.209	15.031
BEDLAM2.0	ETCH-X (Ours)	1.265	3.429	4.033

Ablation Study¶

Ablation on single-view partial input simulation and partial data augmentation (50% random replacement during training):

Input Type	Augmentation	CAPE CD ↓	CAPE V2V ↓	CAPE MPJPE ↓	4D-Dress CD ↓	4D-Dress V2V ↓	4D-Dress MPJPE ↓
Full Scan	w/o Aug	0.894	1.484	1.764	0.951	1.897	2.317
Full Scan	w/ Aug	0.918	1.644	2.027	0.917	2.135	2.677
Degradation ($\Delta$)	-	+2.7%	+10.8%	+14.9%	-3.6%	+12.5%	+15.5%
Partial Scan	w/o Aug	1.149	10.056	10.403	2.261	13.861	16.662
Partial Scan	w/ Aug	0.951	2.898	3.516	0.978	3.808	5.273
Error Reduction ($\Delta$)	-	17.2% ↓	71.2% ↓	66.2% ↓	56.7% ↓	72.5% ↓	68.4% ↓

Ablation on the hand refinement components (Hand LVD, hand classifier, and occlusion data augmentation):

Setting	Hand LVD	Hand Classifier	Hand Data Aug	CAPE V2V-Hands ↓	CAPE MPJPE-Hands ↓	4D-Dress V2V-Hands ↓	4D-Dress MPJPE-Hands ↓
A	✗	✗	✗	2.607	2.467	3.417	3.367
B	✓	✗	✗	2.417	2.277	3.321	3.293
C	✓	✓	✗	2.278	2.203	3.167	3.125
ETCH-X	✓	✓	✓	2.215	2.148	3.101	3.065

Ablation on tightness masking trained on CLOTH3D + AMASS: On 4D-Dress, introducing the tightness mask drops CD from 1.515 cm to 1.493 cm, V2V from 4.256 cm to 4.169 cm, and MPJPE from 4.200 cm to 4.083 cm. On CAPE, V2V drops from 1.975 cm to 1.894 cm.

Key Findings¶

Partial Augmentation Unlocks Extreme Robustness: On partial inputs from 4D-Dress, partial data augmentation slashes V2V error from 13.861 cm down to 3.808 cm (a 72.5% reduction) with a negligible trade-off on full scans (12.5% increase), demonstrating the effectiveness of implicit neural fields under single-view occlusion.
Hand Refinement Pipeline is Mutually Reinforcing: Transitioning from global fitting (Setting A) to dedicated Hand LVD drops hand V2V on 4D-Dress from 3.417 cm to 3.321 cm; filtering self-contact artifacts further reduces it to 3.167 cm, and occlusion-aware augmentation yields the best result of 3.101 cm.
Overcoming Sparse Marker Fragility in OOD Scenarios: On the unseen BEDLAM2.0 dataset, baseline ETCH suffers catastrophic misclassification of sparse markers, exploding to 12.209 cm V2V and 15.031 cm MPJPE. ETCH-X achieves 3.429 cm V2V and 4.033 cm MPJPE—an error reduction exceeding 70-80%.
Scaling Dynamics Differ Between Garments and Poses: Increasing CLOTH3D garment simulation data quickly saturates on loose clothing benchmarks (4D-Dress), highlighting limited topological diversity in synthetic clothing. In contrast, scaling AMASS body poses yields monotonic, steady improvements in fitting accuracy across all benchmarks.

Highlights & Insights¶

Decoupled Architecture Unlocks Composable Synthetic Data Scaling: Rather than hunting for non-existent full-body scans with ground-truth SMPL-X registrations, ETCH-X separates undressing from fitting. This enables training on independent, highly scalable synthetic modalities: 150k simulated garments for clothing physics and 120k mocap meshes for anatomical variations.
Implicit Neural Fields Replace Fragile Sparse Markers: Moving from explicit marker voting to continuous implicit neural fields (LoVD with test-time fine-tuning) ensures that partial or occluded scans can query global topological correspondences without marker dropouts.
Self-Contact Aware Hand Parsing: Identifying and filtering out spurious torso/thigh vertices via a trained classifier prevents hand fitting from collapsing into adjacent body surfaces during touching gestures.

Limitations & Future Work¶

Inference Latency Bottleneck: Iterative test-time optimization of the local neural field and secondary hand re-sampling require approximately 10 seconds per scan, which precludes real-time interactive avatar tracking.
Synthetic Garment Diversity Bounds: Current cloth simulators are biased toward standard shirts and pants. Extreme, non-manifold attire like long dresses, capes, or open winter jackets still cause saturation and undressing artifacts.
Single-Person Focus: The current formulation operates on cropped, isolated individuals, leaving multi-person close interactions and complex scenes with cluttered indoor background LiDAR for future work.

vs NICP (Marin et al., ECCV 2024): NICP pioneered test-time implicit neural ICP for naked or skin-tight scans, but directly fitting loose clothing leads to bloated, overweight body shapes. ETCH-X retains NICP's neural field fitting power while preceding it with a masked undressing stage to remove clothing bias.
vs ETCH (Li et al., ICCV 2025): ETCH introduced SE(3)-equivariant tightness vectors, but coupled them with sparse surface markers and SMPL. It fails catastrophically when single-view scans hide sparse markers, and cannot express fine hand/facial nuances. ETCH-X upgrades to SMPL-X, substitutes sparse markers with continuous implicit fields, and adds skin masking and hand refinement.
vs IPNet / PTF (Bhatnagar et al., ECCV 2020; Wang et al., CVPR 2021): These double-layer occupancy methods attempt joint outer/inner surface prediction. Their tightly entangled architectures struggle with generalization; ETCH-X's modular pipeline achieves far superior cross-dataset transfer on BEDLAM2.0.

Rating¶

Novelty: ⭐⭐⭐⭐ [Decoupled masked undress and dense fit pipeline with skin masking and re-sampling hand refinement]
Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Exhaustive benchmarks across CAPE, 4D-Dress, and OOD BEDLAM2.0, with dedicated partial scan and scaling studies]
Writing Quality: ⭐⭐⭐⭐⭐ [Clear structural narrative, comprehensive figures, and well-motivated ablation analyses]
Value: ⭐⭐⭐⭐⭐ [Provides a practical, robust blueprint for high-fidelity 3D digital human capture from imperfect real-world scans]