DiGS-Avatar: Single-Image Animatable 3D Human Reconstruction via UV-Space Diffusion

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/KLMAV-CUC/DiGS-Avatar
Area: 3D Vision
Keywords: 3D Human Reconstruction, Animatable Avatars, 3D Gaussian Splatting, Latent Diffusion Models, UV-Space Completion

TL;DR

DiGS-Avatar reformulates single-image animatable 3D human reconstruction as an efficient 2D UV-latent completion task, supervising a single-view diffusion student via a multi-view teacher and injecting rich appearance cues through cross-attention to reconstruct a photorealistic, rig-ready 3D Gaussian avatar in only 0.71 seconds.

Background & Motivation

Reconstructing high-fidelity, animatable 3D human avatars from a single unconstrained image is a core problem for virtual reality, gaming, visual effects, and embodied AI. However, due to the severe ill-posedness of monocular input, unobserved back-side occlusions, intricate garment deformations, and varied articulations, inferring complete 3D geometry and realistic appearance remains an arduous challenge. Pure feed-forward deterministic regressors such as IDOL and LHM achieve rapid inference, yet they intrinsically lack strong generative hallucination priors, which frequently results in over-smoothed geometry and washed-out textures in unseen or occluded regions. Conversely, multi-view diffusion methods attempt to synthesize multi-angle images prior to 3D fitting, which introduces heavy multi-stage pipelines and subtle geometric or photometric inconsistencies across viewpoints.

To guarantee rigorous 3D consistency, recent native 3D generative architectures such as SyncHuman and TRELLIS operate directly on 3D volumetric structures or triplane feature fields. However, these methods depend heavily on dense 3D supervisory ground truth and computationally prohibitive volumetric processing, leading to cumbersome training requirements and sluggish inference times reaching tens of seconds per character. This reveals a fundamental trilemma in current literature: existing solutions must choose between the over-smoothing of deterministic regression, the multi-view inconsistencies of 2D image diffusion, or the computational bottlenecks of native 3D volumetric generative modeling.

DiGS-Avatar circumvents this dilemma by observing that the SMPL-X parametric human mesh defines a continuous, unique 2D surface parameterization where every UV coordinate corresponds to a physical body surface point. Core idea: reformulate single-image 3D human reconstruction as an efficient 2D UV-latent diffusion completion task within a canonical surface-aligned domain, supervising a single-view diffusion student with geometrically aligned pseudo-ground-truth latents from a multi-view teacher, and injecting semantic details via a Geometry-Aligned Semantic Aggregation (GASA) module into 3D Gaussian primitives.

Method

Overall Architecture

DiGS-Avatar constructs a canonical surface-aligned pipeline where single-view 3D reconstruction is formulated as pure 2D UV-space inpainting. During inference, given a single reference image \(I_\text{ref}\) along with estimated SMPL-X parameters, the model extracts compact structural VAE latents and high-level semantic tokens from a frozen DINOv3 appearance encoder. The VAE features are projected into canonical UV coordinates via SMPL-X inverse texture mapping, yielding a partial UV latent covering only visible regions. A conditional 2D latent diffusion model then infers the missing geometry and texture to produce a complete canonical UV latent. Subsequently, the Geometry-Aligned Semantic Aggregation (GASA) module injects multi-level appearance cues into this structural skeleton through progressive cross-attention. Finally, a UV-to-3DGS decoder predicts position, scale, rotation, and color offsets relative to the SMPL-X canonical vertices, producing a fully rigged 3D Gaussian avatar driven via standard Linear Blend Skinning (LBS).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Single Input Image + SMPL-X Parameters"] --> B["Multi-View Teacher Distillation & Single-View Completion<br/>Inverse texture mapping & conditional 2D diffusion"]
    B --> C["Geometry-Aligned Semantic Aggregation (GASA)<br/>Hierarchical cross-attention for deep/mid/shallow cues"]
    C --> D["Canonical UV-to-3DGS Decoding & Animation<br/>Relative Gaussian attribute regression & LBS driving"]
    D --> E["Photorealistic Animatable 3D Avatar"]

Key Designs

1. Multi-View Teacher Distillation & Single-View Completion: Built-in 3D Consistency via Pure 2D Latents

Directly training a single-view diffusion model on partial UV latents is severely under-constrained because the complete, view-consistent UV ground truth for an arbitrary single image cannot be observed directly. DiGS-Avatar overcomes this by introducing a multi-view teacher network during training. Given \(N\) calibrated multi-view images of a subject, the teacher fuses their VAE representations using a UV-Alignment Transformer conditioned on Plücker ray embeddings, explicitly resolving view ambiguities to output a view-consistent pseudo-ground-truth UV latent \(Z_\text{VAE} \in \mathbb{R}^{32 \times 32 \times 32}\). The single-view student then learns to complete the partially projected UV latent by predicting the denoising target under teacher supervision: $$ \mathcal{L}\text{diff} = \mathbb{E} \left[ |\epsilon - \epsilon_\theta(x_t, t, \xi)|^2_2 \right] $$ where the conditioning vector \(\xi = F_\text{ref}^\text{VAE} \oplus F_\text{ref}^\text{app} \oplus N_\text{ref}^\text{SMPL-X}\) incorporates structural VAE features, DINOv3 semantic tokens, and the rendered SMPL-X surface normal map. This pure 2D formulation reduces computationally heavy 3D volumetric generation to an efficient 2D inpainting task that is 3D-consistent by design.

2. Geometry-Aligned Semantic Aggregation (GASA): Overcoming VAE Bottlenecks with Coarse-to-Fine Cross-Attention

Although the compact \(32 \times 32\) VAE latent provides a stable topological skeleton for diffusion modeling, its strong spatial downsampling inevitably strips away high-frequency appearance details such as fine fabric wrinkles, buttons, and detailed facial features. To restore rich textures without disrupting spatial consistency, the GASA module splits frozen DINOv3 features into deep (\(F_\text{deep}^\text{app}\)), mid (\(F_\text{mid}^\text{app}\)), and shallow (\(F_\text{shallow}^\text{app}\)) subsets, and progressively injects them into the UV latent across three spatial stages: $$ Z^{(1)} = \text{CrossAttn}{32}(Z\text{VAE}, F_\text{deep}^\text{app}) $$ $$ Z^{(2)} = \text{CrossAttn}{64}(\text{Upsample}(Z^{(1)}), F\text{mid}^\text{app}) $$ $$ F_\text{UV} = \text{CrossAttn}{128}(\text{Upsample}(Z^{(2)}), F\text{shallow}^\text{app}) $$ Using the UV structural latent as queries and multi-scale appearance features as keys and values, GASA enriches the representation up to a \(128 \times 128\) resolution while strictly anchoring appearance cues to the underlying 3D geometry.

3. Canonical UV-to-3DGS Decoding & Animation: Topology-Anchored Relative Offset Parameterization

To ensure instantaneous animation without relying on complex, test-time deformation fields or neural blend weights, the decoder converts the high-resolution feature map \(F_\text{UV}\) into 3D Gaussian attributes anchored directly to SMPL-X surface points. Each Gaussian primitive's center position \(\mu_k\), scale \(s_k\), and rotation \(r_k\) are regressed as explicit offsets relative to its corresponding canonical SMPL-X vertex \((\hat{\mu}_k, \hat{s}_k, \hat{r}_k)\): $$ \mu_k = \hat{\mu}_k + \delta \mu_k, \quad s_k = \hat{s}_k \odot \delta s_k, \quad r_k = \hat{r}_k \cdot \delta r_k $$ Because all Gaussian primitives inherit the underlying body mesh topology, arbitrary novel poses can be animated immediately using standard Linear Blend Skinning (LBS), eliminating common artifacts such as texture sliding or floating geometric tear-offs under extreme skeletal articulations.

Loss & Training

The framework is trained in a staged progression. First, the multi-view teacher is trained alongside the GASA module and 3DGS decoder on HuGe100K and THuman 2.1 using a photometric objective combined with Gaussian offset regularization: \(\mathcal{L}_\text{teacher} = \mathcal{L}_\text{photo} + \lambda_\text{off} \|\delta \mu_k\|_2^2\), where \(\mathcal{L}_\text{photo} = \mathcal{L}_\text{MSE} + \lambda_\text{VGG} \mathcal{L}_\text{VGG}\). Second, the student's diffusion backbone is trained to predict the clean pseudo-ground-truth latent \(Z_\text{VAE}\) under the joint objective: $$ \mathcal{L}\text{student} = \mathcal{L}\text{diff} + \lambda_\text{photo} \mathcal{L}_\text{photo} $$ Inference uses a Soft Blending Euler sampler with 20 sampling steps and CFG=0.1, requiring merely 0.71 seconds on a single NVIDIA A100 GPU.

Key Experimental Results

Main Results

The authors benchmark DiGS-Avatar against leading baselines across three standard datasets: HuGe100K (synthetic), THuman 2.1 (high-quality scans), and 2K2K (high-resolution captures), evaluating novel view synthesis quality using PSNR, SSIM, and LPIPS metrics.

Table 1: Quantitative comparison on in-distribution datasets (from original paper Table 1, * denotes animatable representations)

Method	HuGe100K (PSNR ↑ / SSIM ↑ / LPIPS ↓)	THuman 2.1 (PSNR ↑ / SSIM ↑ / LPIPS ↓)	2K2K (PSNR ↑ / SSIM ↑ / LPIPS ↓)
SIFU	18.44 / 0.921 / 0.095	16.82 / 0.926 / 0.095	18.54 / 0.931 / 0.091
Human3Diffusion	18.09 / 0.893 / 0.101	15.67 / 0.902 / 0.105	19.29 / 0.908 / 0.079
IDOL*	22.75 / 0.926 / 0.084	18.82 / 0.937 / 0.077	18.17 / 0.916 / 0.092
LHM*	21.76 / 0.919 / 0.065	19.62 / 0.931 / 0.072	20.67 / 0.918 / 0.073
TRELLIS	18.24 / 0.895 / 0.093	16.52 / 0.910 / 0.094	18.72 / 0.907 / 0.084
SyncHuman	17.52 / 0.893 / 0.092	17.04 / 0.911 / 0.089	20.27 / 0.917 / 0.069
DiGS-Avatar (Ours)*	25.32 / 0.940 / 0.039	21.45 / 0.938 / 0.060	22.22 / 0.919 / 0.064

Table 2: Zero-shot generalization on SIZER and computational efficiency (from original paper Table 2)

Method	SIZER PSNR ↑	SIZER SSIM ↑	SIZER LPIPS ↓	Train Cost (GPU hours)	Test Time (s) ↓
SIFU	19.30	0.905	0.101	-	30.0
Human3Diffusion	19.14	0.904	0.088	~960	28.0
IDOL*	21.90	0.922	0.075	~768	2.25
LHM*	23.45	0.929	0.064	~2,496	5.13
TRELLIS	19.94	0.916	0.078	~10,000	7.43
SyncHuman	19.56	0.916	0.074	-	51.0
DiGS-Avatar (Ours)*	23.54	0.932	0.065	~60	0.71

Ablation Study

The authors systematically evaluate the core design components on the HuGe100K benchmark to demonstrate their individual contributions.

Table 3: Quantitative ablation of key components on HuGe100K (from original paper Table 3)

Variant Config	PSNR ↑	SSIM ↑	LPIPS ↓	Description
Teacher Upper Bound	26.18	0.942	0.037	Multi-view observation upper bound
w/o Teacher Supervision	19.03	0.804	0.168	Removing multi-view pseudo-ground-truth targets causes geometric collapse
w/o GASA Module	22.52	0.915	0.084	Relying solely on compressed VAE latents causes blurry textures
w/ Standard LDM-UNet	17.39	0.790	0.204	Standard UNet struggles with highly compressed UV representations
w/o Conditioning Branch	23.26	0.936	0.061	Removing normal and appearance conditioning impairs alignment
w/o Denoising Loss \(\mathcal{L}_\text{diff}\)	15.98	0.785	0.226	Removing diffusion objective prevents valid latent completion
Ours (Full Model)*	25.32	0.940	0.039	Single-view student closely approaches the teacher upper bound

Key Findings

1. Teacher distillation provides irreplaceable 3D structural guidance: Removing the teacher's multi-view supervision causes PSNR to drop dramatically from 25.32 to 19.03 dB and LPIPS to deteriorate by more than \(4\times\) (0.039 to 0.168), confirming that single-view self-supervision cannot resolve UV-space geometric ambiguity alone.
2. GASA resolves the compression dilemma: Ablating the GASA module leads to a 2.80 dB drop in PSNR and increases perceptual error to 0.084, validating that progressive cross-attention is crucial for injecting high-frequency DINOv3 details without distorting geometric stability.
3. Exceptional computational speedup: Requiring only 0.71 seconds per image and ~60 GPU hours of total training, DiGS-Avatar runs over \(70\times\) faster than native 3D generative baselines like SyncHuman (51s) and \(3\times\) faster than deterministic feed-forward models like IDOL (2.25s).

Highlights & Insights

• Formulating 3D human reconstruction as 2D UV inpainting: Exploiting the topological unrolling of SMPL-X eliminates the need for expensive 3D volumetric diffusion or inconsistent multi-view stitching, ensuring native 3D consistency by design.
• Decoupling geometric skeleton and semantic appearance: Separating the pipeline into a low-dimensional structural VAE latent and rich DINOv3 semantic tokens preserves spatial structural integrity while recovering high-frequency textural realism.
• Topology-anchored relative offsets for instant animation: Predicting Gaussian attributes as relative offsets from canonical SMPL-X mesh vertices enables direct, post-processing-free skeletal animation via standard LBS.

Limitations & Future Work

• Constrained by SMPL-X topological limits: Because Gaussians are anchored to SMPL-X body vertices via local offsets, the model struggles with extremely loose or floating garments (e.g., voluminous dresses, long coats) and poses that drastically violate human kinematics.
• Sensitivity to monocular pose estimation: Accurate inverse texture mapping and normal conditioning depend on initial SMPL-X fitting (via Multi-HMR); inaccurate pose predictions can cause cascading geometric misalignment.
• Potential extensions: Future efforts could integrate higher-resolution tiled UV latents or relaxed, unconstrained Gaussian layers around the body surface to better handle extreme garments and accessory objects.

Related Work & Insights

• vs IDOL / LHM (Deterministic Regressors): IDOL and LHM directly regress Gaussian attributes, leading to over-smoothed occluded regions due to the lack of generative capacity. DiGS-Avatar introduces generative diffusion completion, improving PSNR by 2.57 dB and 3.56 dB on HuGe100K.
• vs Human3Diffusion / SIFU (Multi-view / Pixel-domain Diffusion): These methods synthesize multi-view 2D images or refine pixel textures, often causing visual seam artifacts and cross-view disparity mismatches. DiGS-Avatar operates on a single canonical UV unfold where each pixel uniquely maps to a 3D surface point.
• vs SyncHuman / TRELLIS (Native 3D Generative Models): Native 3D methods operate on volumetric grids or 3D latents, taking 7 to 51 seconds per avatar. DiGS-Avatar completes the generation in 0.71 seconds, providing an ideal balance between fidelity and real-time efficiency.

Rating

• Novelty: ⭐⭐⭐⭐ [Clever reformulation of 3D human reconstruction into UV-latent inpainting with a teacher-student geometric distillation framework]
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive comparisons across three benchmarks, out-of-distribution SIZER evaluation, in-the-wild testing, thorough ablations, and efficiency analysis]
• Writing Quality: ⭐⭐⭐⭐⭐ [Clear motivation, well-organized technical sections, transparent implementation details, and self-contained figures]
• Value: ⭐⭐⭐⭐⭐ [High practical impact by delivering sub-second animatable 3D avatar creation on a single commercial GPU]

Related Papers

• [ECCV 2026] Monocular Avatar Reconstruction via Cascaded Diffusion Priors and UV-Space Differentiable Shading
• [CVPR 2026] Zero-Shot Reconstruction of Animatable 3D Avatars with Cloth Dynamics from a Single Image
• [ECCV 2026] Identity-Preserving Human Reconstruction from a Single Image via 3D Token Inference
• [ECCV 2026] PixGS: Pixel-Space Diffusion for Direct 3D Gaussian Splat Generation
• [ICCV 2025] AdaHuman: Animatable Detailed 3D Human Generation with Compositional Multiview Diffusion