HairOrbit: Multi-view Aware 3D Hair Modeling from Single Portraits

Conference: ECCV 2026
Paper: ECCV Official
Project Page: Pinscreen / MBZUAI
Code: Coming soon
Area: 3D Vision / Video Generation
Keywords: 3D hair modeling, single-view reconstruction, video diffusion prior, neural orientation extractor, hybrid implicit field

TL;DR

HairOrbit reformulates strand-level 3D hair reconstruction from a single portrait into a calibrated multi-view reconstruction problem using cross-view video diffusion priors, enabling complete, high-fidelity, and view-consistent 3D hair strand synthesis even in previously invisible side and back regions.

Background & Motivation

Reconstructing strand-level 3D hair geometry from a single portrait is one of the most intractable challenges in digital human creation and avatar modeling. Human hair possesses extraordinarily intricate 3D topological structures characterized by severe self-occlusions, complex tangling, and subtle geometric attributes such as curliness, partings, and layered lengths. Because single-view reconstruction is an inherently ill-posed inverse problem, the visible front view offers only partial geometric cues, leaving the occluded side and back regions to be hallucinated entirely from learned priors. Unfortunately, existing synthetic 3D hair datasets (such as USC-HairSalon and Difflocks) are severely bottlenecked in sample size, physical realism, and hairstyle diversity. As a result, current single-view methods either produce over-smoothed, synthetic-looking strands or reconstruct hair that looks plausible only from the frontal viewpoint while suffering from catastrophic geometric breakdown, length mismatch, or curl distortion when viewed from side and rear angles.

In contrast, recent multi-view and monocular video-based hair capture methods achieve remarkable fidelity and geometric consistency by aggregating dense observations across orbiting camera viewpoints. However, these multi-view setups require cumbersome multi-camera arrays or user-captured 360-degree videos, requiring hours or days of per-scene optimization that cannot serve casual single-portrait scenarios. Hence, the core tension lies in the severe gap between the demand for consistent multi-view geometric cues and the scarcity of input observations in single-portrait applications.

Inspired by recent advances in video diffusion transformers (such as WAN) that naturally encode rich 3D spatiotemporal and multi-view priors through temporal consistency, this paper fundamentally rethinks the single-view paradigm. Core idea: reformulate single-view 3D hair reconstruction into a calibrated multi-view problem by employing a video diffusion model as a generative bridge from a single portrait to a 360-degree orbital sequence, combined with a full-view neural orientation extractor and an efficient hybrid implicit field for parallel strand synthesis.

Method

Overall Architecture

The HairOrbit pipeline comprises two primary stages: calibrated multi-view synthesis with geometric proxy extraction, followed by implicit field prediction and hybrid strand generation. Given an unconstrained frontal portrait \(I_{in}\), the face is first aligned with a standard template mesh via landmark fitting to yield \(I_{aligned}\). Next, a video diffusion transformer fine-tuned with lightweight LoRA generates a 360-degree orbital rotation video \(\{I_i^m\}\), which is sharpened using a high-resolution diffusion upscaler to restore fine hair strand textures. From these synthetic views, 2D Gaussian Splatting and Marching Cubes extract a base surface mesh \(M\) to render multi-view depth maps \(\{D_i\}\), while a dedicated neural orientation extractor predicts clean, pixel-aligned 2D orientation maps \(\{O_i\}\). Finally, multi-view feature maps are fused to query a unified hybrid implicit field \(\mathcal{F}\), from which full 3D hair strands are grown via a dual-stage parallel algorithm.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Single Frontal Portrait Input<br/>Landmark Alignment to Template"] --> B["Orbital Multi-view Video Generation<br/>WAN-DiT LoRA + Super-Resolution"]
    B --> C1["Geometry & Depth Reconstruction<br/>2DGS Surface Extraction & Depth Rendering"]
    B --> C2["Full-view Neural Orientation Extraction<br/>Undirectional Pixel-aligned U-Net"]
    C1 --> D["Hybrid Implicit Field Construction<br/>Joint Occupancy & Orientation Vector Field"]
    C2 --> D
    D --> E["Dual-stage Strand Growing Algorithm<br/>Scalp-rooted Strands + Gap Filling & Buzz-cut Recovery"]
    E --> F["High-fidelity 3D Hair Strands Model"]

Key Designs

1. Orbital video diffusion multi-view generation: bridging single-view and multi-view via generative spatiotemporal priors

Directly predicting 3D hair from a single image easily leads to flat, over-smoothed geometry on unseen sides due to the limited scale of 3D synthetic datasets. To convert the problem into calibrated multi-view reconstruction, HairOrbit adapts WAN, an open-source video diffusion transformer. The authors render synthetic 360-degree orbital rotation videos (97 views) from curated 3D hair models spanning diverse lengths, partitions, and curl patterns. Lightweight LoRA adapters are injected strictly into the attention projection weights (Q, K, V, O) and the feed-forward network boundary layers (FFN.0, FFN.2) of each DiT block. This locks the generative trajectory to a circular 360-degree orbit while preserving the foundational model's real-world prior regarding hair texture, specular highlights, and natural clump distribution. To satisfy consumer GPU memory limits during sampling, lower-resolution frames are first synthesized and then refined through the Flux.1-dev Upscaler, restoring crisp strand boundaries critical for subsequent orientation extraction.

2. Full-view neural orientation extractor: eliminating viewpoint domain shift via local pixel-aligned targets

Accurate 2D orientation maps \(\{O_i\}\) are essential for guiding strand reconstruction, yet classical Gabor filters suffer from severe high-frequency noise and shadow sensitivity, requiring costly multi-view optimization. Conversely, existing neural estimators like HairStep learn "directional strand maps" that encode global root-to-tip flow. Because global semantics depend heavily on frontal facial anchors and are annotated only on front-facing portraits, HairStep fails catastrophically on profile and rear views. HairOrbit observes that 2D strand orientation is fundamentally a local, pixel-aligned perceptual task that does not require global flow awareness. By reformulating the learning target from a directional vector to an undirectional 2D orientation map within the hair mask, a U-Net trained exclusively on augmented frontal annotations generalizes seamlessly to full 360-degree views, bypassing Gabor noise without any slow post-processing.

3. Hybrid implicit field: unifying occupancy and direction for fully parallelized strand tracing

Conventional implicit hair modeling frameworks maintain two detached fields: an occupancy field defining exterior boundaries and an orientation field predicting tangent vectors inside. This separation incurs heavy computational overhead from repeated ray-casting and per-point occupancy queries; more critically, because strands exit the volume at irregular step counts, GPU thread execution diverges. HairOrbit unifies both representations into a single hybrid implicit field \(\mathcal{F}\): inside the hair volume, the field predicts unit-length 3D tangent vectors \((d_x, d_y, d_z)\); outside the volume, it predicts exact zero vectors \((0, 0, 0)\). Deep features extracted by a Stacked Hourglass network from concatenated depth and orientation maps are retrieved via perspective projection and fed alongside spatial positional encodings into a shallow MLP decoder. Because growth vectors outside the hair volume vanish, any strand that reaches the boundary naturally stops advancing, and further iteration steps converge to the exact same boundary coordinates. This allows hair growth to be executed as a fixed-iteration, fully parallel tensor operation, accelerating strand synthesis by orders of magnitude.

4. Dual-stage hybrid strand growth and buzz-cut recovery: balancing completeness and thin-layer geometry

Growing strands solely from scalp roots is computationally fast but vulnerable to small orientation deviations near boundaries, which can cause premature termination and leave visible hollows. Conversely, growing thousands of unconstrained short segments and stitching them geometrically is computationally prohibitive. HairOrbit adopts an efficient hybrid strategy: main guide strands are first grown in parallel from scalp roots, successfully populating over 90% of the visible volume. Next, lightweight supplementary segments are grown exclusively in detected unfilled boundary regions and merged smoothly into the nearest primary scalp strand. Furthermore, for extremely thin geometry such as buzz-cuts where implicit vector fields tend to collapse, candidate roots are validated through multi-view mask reprojection, and collapsed roots are recovered by estimating the local mesh surface tangent vector across visible views before regrowing, preserving clean and crisp short-hair contours.

Loss & Training

1. Video Diffusion LoRA Training: With pretrained WAN parameters frozen, the LoRA weights are optimized via standard noise prediction loss: $\(\mathcal{L}_{\text{denoise}} = \mathbb{E}_{\mathbf{x}_0, \boldsymbol{\epsilon}, t} \left[ \left\| \boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t, \mathbf{c}) - \boldsymbol{\epsilon} \right\|_2^2 \right]\)$ where \(\mathbf{c}\) represents the VAE latent code of the aligned portrait \(I_{aligned}\).
2. Hybrid Implicit Field Training: Supervised with multi-axis mean \(L_1\) loss against ground-truth vectors: $\(\mathcal{L}_{\text{ori}} = \frac{1}{N} \sum_{i=1}^N \sum_{k \in \{x,y,z\}} \left| d_{i,k}^* - d_{i,k} \right|\)$ Exterior query points are explicitly supervised with target vectors \((0,0,0)\). At inference time, a simple vector magnitude threshold of \(0.1\) separates hair interior from empty space.

Key Experimental Results

Main Results

The authors evaluated orientation estimation accuracy on their newly annotated full-view hair dataset comprising 395 images from 7 human subjects across multi-view video captures, reporting Mean Angular Error (MAE in degrees, lower is better):

Method	nastya (62 v)	ksyusha (66 v)	jenya (70 v)	shortCurly (58 v)	whiteCurly (42 v)	midWavy (33 v)	midCurly (64 v)	Overall (395 v) ↓
Gabor Filter [44]	11.88	16.56	18.88	18.87	24.56	11.31	10.79	16.05
HairStep [42]	6.28	10.98	13.16	13.68	21.57	6.32	5.54	10.88
HairOrbit (Ours)	1.79	4.97	6.09	6.72	13.50	2.15	2.12	5.13

On the single-view 3D hair strand reconstruction benchmark (196 annotated diverse hairstyles), performance is evaluated via 2D projected orientation error HairSale (degrees, lower is better), relative depth accuracy HairRida (%, higher is better), and silhouette IoU (higher is better):

Method	HairSale (°) ↓	HairRida (%) ↑	IoU ↑
DiffLocks [25]	26.50	69.67	0.593
HairStep [42]	17.38	77.01	0.639
Im2Haircut [30]	16.24	79.38	0.757
HairOrbit (Ours)	12.83	80.52	0.847

Ablation Study

The paper systematically benchmarks four configurations (Table 3): - C0: Replacing the neural orientation extractor with traditional Gabor filters. - C1: Growing hair strands purely from scalp roots without supplementary segment stitching. - C2: Disabling the multi-view bridging strategy and reconstructing the 3D field from the single input view only. - Full: The complete HairOrbit pipeline.

Configuration	Input View HairSale (°) ↓	Input View IoU ↑	8 Synthesized Views HairSale (°) ↓	8 Synthesized Views IoU ↑	Note
Full (Ours)	12.83	0.847	9.97	0.900	Complete HairOrbit pipeline
w/o Neural Extractor (C0)	16.42 (-27.98%)	0.794	13.45 (-34.90%)	0.878	Gabor noise degrades fine parting and flow
w/o Segment Growing (C1)	13.70 (-6.78%)	0.828	10.37 (-4.01%)	0.887	Mild strand gaps and incomplete tips
w/o Multi-view Bridging (C2)	19.10 (-48.87%)	0.680	24.14 (-142.1%)	0.658	Severe collapse on unobserved side/rear views

Key Findings

• Multi-view video bridging is decisive for 360-degree fidelity: Eliminating the video-generated multi-view bridge (C2) causes angular error across synthesized novel views to spike by 142.1% (from 9.97° to 24.14°) while novel-view IoU drops from 0.900 to 0.658. This proves that temporal video generation models successfully impart true 3D spatial coherence to single-view reconstruction.
• Local undirectional formulation solves orientation generalization: Across 395 real-world multi-view frames, the proposed neural extractor slashes mean angular error from Gabor's 16.05° and HairStep's 10.88° down to 5.13° (over 68% improvement), establishing an exceptionally clean intermediate representation for field prediction.
• Hybrid field unlocks massive parallelism without boundary leakage: Encoding occupancy as zero-magnitude velocity vectors eliminates branching divergence and expensive cross-field boundary queries, allowing tens of thousands of hair curves to converge in parallel within fixed step counts.

Highlights & Insights

• Repurposing video generation as a calibrated geometric bridge: Instead of attempting to learn ill-posed 3D hair priors directly from scarce 3D assets, HairOrbit recognizes that large-scale video diffusion models inherently capture temporal-spatial consistency, transforming an ill-posed single-view problem into a robust multi-view reconstruction problem via lightweight LoRA tuning.
• Simplifying representation targets to unlock zero-shot generalization: Prior art assumed hair orientation required global, directional root-to-tip trajectories, which inadvertently coupled the network to frontal facial semantics. By reducing the objective to local, undirectional tangent lines, the network trained only on frontal crops effortlessly generalizes to 360-degree orientations.
• Zero-vector boundary design for unified implicit tracing: Multiplying occupancy into the orientation vector field turns spatial boundaries into an implicit velocity damper. Strands naturally stop upon boundary exit, establishing mathematical convergence that enables fully vectorized, branch-free tensor operations during hair growth.

Limitations & Future Work

• Severe occlusions and complex head accessories: When subjects wear hats, turbans, or large hairpins, the video diffusion model can occasionally synthesize inconsistent textures or hallucinate geometry across rotated views, polluting the implicit field.
• Highly irregular afro-textured and frizzy hair: Extremely voluminous curls or random flyaway strands can be smoothed out during 2D Gaussian Splatting surface extraction, leading to slight volume compression.
• Multi-step decoupled inference latency: The overall pipeline involves sequential steps: video generation, super-resolution upscaling, 2DGS depth extraction, orientation estimation, and implicit field querying. A unified, end-to-end conditional diffusion-to-strand framework remains an exciting future direction.

Related Work & Insights

• vs HairStep (CVPR 2023): HairStep pioneered implicit hair reconstruction guided by predicted strand maps from a single image, but relied on directional maps that fail outside frontal poses and provided no rear-view cues. HairOrbit solves rear-view hallucination via video diffusion and achieves superior front-view accuracy (HairSale 12.83° vs 17.38°).
• vs Difflocks (CVPR 2025): Difflocks synthesizes scalp texture latent maps directly from DINOv2 features, lacking dense spatial pixel alignment with the source portrait. HairOrbit anchors the reconstruction to pixel-aligned multi-view deep features and explicit depth, yielding vastly superior silhouette overlap (IoU 0.847 vs 0.593).
• vs Im2Haircut (ICCV 2025): Im2Haircut relies on slow differentiable rendering optimization and monocular depth priors (DepthPro) that lack multi-view consistency. HairOrbit reconstructs consistent 3D geometry via orbital synthesis and 2DGS, ensuring seamless frontal-to-side strand flow.

Rating

• Novelty: ⭐⭐⭐⭐⭐ [Pioneering integration of video diffusion spatiotemporal priors into single-view 3D hair modeling]
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive quantitative evaluation on 395 newly annotated multi-view frames and 196 test hairstyles]
• Writing Quality: ⭐⭐⭐⭐⭐ [Crystal-clear motivation, coherent prose, and well-structured technical formulations]
• Value: ⭐⭐⭐⭐⭐ [Offers a robust, highly practical paradigm for realistic 3D avatar and digital human hair synthesis]

Related Papers

• [ECCV 2026] GeoNVS: Geometry Grounded Video Diffusion for Novel View Synthesis
• [ECCV 2026] Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Joints
• [ECCV 2026] Learning Transferable Dynamics Priors from Action to World Modeling
• [ECCV 2026] MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation
• [ECCV 2024] SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion