Skip to content

Discovering Geometric Biases in 3D Face Reconstruction: A Curvature-Aware Spectral Framework for Fairness Evaluation

Conference: ECCV 2026
Paper: ECCV 2026 Poster
Code: https://github.com/artefactory/3dface-fairness
Area: 3D Vision
Keywords: 3D Face Reconstruction, Fairness Evaluation, Curvature Error, Laplace-Beltrami Operator, Spectral Clustering

TL;DR

Addressing the insensitivity of traditional Euclidean metrics to local high-frequency morphological discrepancies that conceal algorithmic bias, this paper introduces a Curvature Reconstruction Error (CRE) based on locally fitted Laplace-Beltrami operators alongside a spectral error clustering framework, uncovering and quantifying systematic age and ethnicity geometric biases in 3D Morphable Models and downstream reconstruction methods.

Background & Motivation

Monocular 3D face reconstruction plays an indispensable role in safety-critical domains such as virtual reality, biometric security, cinematic rendering, and maxillofacial surgery. Current state-of-the-art pipelines ubiquitously rely on 3D Morphable Models (3DMMs), such as the Basel Face Model (BFM) and FLAME, as parametric shape priors. However, these foundational statistical bases were constructed from limited pools of registered 3D face scans, naturally inheriting the morphological and demographic skews of their underlying training data. With modern regulatory frameworks like the European Union AI Act establishing strict mandates for accuracy auditing and bias transparency in high-risk AI deployments, discovering and quantifying demographic disparities across age, gender, and ethnicity in 3D face reconstruction has emerged as an urgent imperative.

Conventional evaluation protocols in 3D face reconstruction predominantly measure global surface alignment via point-to-point or point-to-surface Euclidean metrics (such as ICP-based Root Mean Square Error or Normalized Mean Square Error). These first-order distance metrics focus exclusively on the macro-level "outer shell" of the face and remain largely blind to second-order differential surface features, including subtle folds, nasolabial creases, skin undulations, and tissue sagging. Consequently, Euclidean evaluations produce misleadingly tiny performance gaps across demographics, masking the reality that linear 3DMM shape spaces drastically smooth out distinctive anatomical traits of elderly subjects and minority ethnicities.

To overcome the diagnostic blindness of Euclidean distance metrics to local anatomical distortions, this paper reframes reconstruction auditing through differential geometry and manifold spectral analysis. Core idea: estimate an asymmetric Laplace-Beltrami operator via local quadratic regressions to formulate a perceptually aligned Curvature Reconstruction Error (CRE), coupled with spectral projection onto 3DMM manifold harmonic bases for unsupervised clustering, delivering an end-to-end framework to discover, quantify, and explain systematic demographic biases.

Method

Overall Architecture

The proposed framework is structured into two sequential stages: localized geometric curvature error measurement and spectral-space failure mode discovery. First, reconstructed face meshes are rigidly aligned to ground truth scans using ICP, and an anatomical voting procedure establishes standardized morphological subregions across disparate topologies. Second, the Laplace-Beltrami operator (LBO) is locally approximated via quadratic polynomial regressions on tangent planes to compute vertex-wise mean curvature; point-to-surface mapping then produces a dense curvature error map, which is integrated over dual barycentric cell areas into a scalar CRE metric. Finally, by solving the generalized Helmholtz eigenproblem on the canonical 3DMM template, high-dimensional error maps are projected onto orthonormal manifold harmonics, enabling robust K-means clustering in a compact spectral domain to isolate recurring geometric failure patterns and associate them with sensitive demographic factors.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Ground Truth Scan M and Reconstruction M'"] --> B["Topological Voting Partition: Robust Anatomical Segmentation"]
    B --> C["Local Quadratic Regression LBO: Vertex Mean Curvature Estimation"]
    C --> D["Curvature Reconstruction Error CRE: Local and Area-Weighted Metric"]
    D --> E["3DMM Template Spectral Projection: Orthonormal Manifold Basis Reduction"]
    E --> F["Unknown Geometric Bias Spectral Clustering: Failure Mode Discovery & Attribution"]

Key Designs

1. Topological Voting Partition: Robust Anatomical Segmentation Evaluating fidelity across specific facial subregions (nose, mouth, cheek, forehead) requires establishing an anatomically coherent correspondence between the 3DMM parameter space and unconstrained 3D ground truth scans. Because direct point-to-point mapping on noisy, individual meshes causes boundary artifacts and topological misalignment, the authors formulate a consensus voting protocol across the entire benchmark. For each ground truth scan \(M\) in REALY, the template mesh \(\mu\) is aligned via ICP, and each vertex \(v_i \in \mu\) retrieves the nearest-neighbor anatomical label from \(M\). By accumulating votes across all \(N\) subjects in the dataset, each template vertex is permanently assigned its modal label: $\(l(\mathbf{v}_i, \mathcal{M}) = \text{Label}\left(\arg\min_{\mathbf{v} \in \mathcal{M}} \|\mathbf{v}_i - \mathbf{v}\|\right)\)$ This consensus partition yields an invariant, robust regional mask over the 3DMM topology, preventing semantic boundary drift during subsequent localized error calculations.

2. Local Quadratic Regression LBO: Vertex Mean Curvature Estimation Triangular meshes represent piecewise linear 2-manifolds lacking continuous analytical derivatives. Fitting local bivariate surfaces directly is notoriously susceptible to surface normal inaccuracies and high-frequency discretization noise. To capture underlying intrinsic differential properties, the authors estimate the continuous Laplace-Beltrami operator \(\Delta\) over discrete neighborhoods. For each vertex \(v_i\), an \(R\)-ring neighborhood \(\mathcal{N}_R(v_i)\) is projected onto its tangent plane \((\xi, \eta)\), where a second-order polynomial \(\hat{f}_{\mathbf{a}}(\xi, \eta) = a_1 \xi^2 + a_2 \xi \eta + a_3 \eta^2 + a_4 \xi + a_5 \eta + a_6\) is fitted via least squares. The closed-form pseudoinverse \(C(v_i) = (X^\top X)^{-1} X^\top\) evaluates the Laplacian at the origin \((0, 0)\) as \(2a_1 + 2a_3 = \sum_{v_j \in \mathcal{N}_R(v_i)} 2(C_{1j} + C_{3j}) f_{v_j}\). Assembling these entries yields a sparse operator \(\tilde{\Delta} \in \mathbb{R}^{|V| \times |V|}\), computing the mean curvature \(H\) via the fundamental differential geometry identity: $\(H^M(v_i) = -\frac{1}{2} \langle \mathbf{n}_i, (\tilde{\Delta} \mathbf{v})_i \rangle\)$ This formulation decouples curvature estimation from tangent plane displacement distances, with the user-defined radius \(R\) tuning the optimal balance between high-frequency detail retention and numerical noise suppression.

3. Curvature Reconstruction Error CRE: Local and Area-Weighted Metric Standard Euclidean distance fails to penalize smoothed-out wrinkles and artificial surface creasing when global vertex positions remain approximately centered. The framework resolves this by mapping the ground truth mean curvature \(H^M\) onto reconstructed vertices via point-to-surface projection \(T_{M' \to M}^{\text{pts}}\) with barycentric interpolation, defining the localized error map \(\epsilon_H(v_i) = |H^{M'}(v_i) - H^M(T_{M' \to M}^{\text{pts}}(v_i))|\). To guarantee invariance to irregular mesh tessellation and non-uniform vertex density, a diagonal mass matrix \(\mathbf{M}\) storing barycentric dual cell areas is applied, formalizing the scalar Curvature Reconstruction Error (CRE): $\(\mathcal{E}_{\text{CRE}}(\mathcal{M}, \mathcal{M}') = \frac{1}{\text{Tr}(\mathbf{M})} \|\mathbf{M} \epsilon_H\|_{L_1}\)$ This metric acts as an area-normalized, perceptually grounded objective that isolates high-frequency structural deviations overlooked by Euclidean norms.

4. 3DMM Template Spectral Projection: Orthonormal Manifold Basis Reduction Direct unsupervised clustering on raw error maps \(\epsilon_H \in \mathbb{R}^{|V'|}\) is fundamentally ill-conditioned because typical high-fidelity validation sets contain few samples (\(N \sim 100\)) relative to high mesh vertex counts (\(|V'| \sim 10^4\)). The authors leverage manifold harmonics by solving the generalized eigenvalue problem on the canonical 3DMM template \(\mu\): $\(\mathbf{L} \boldsymbol{\phi}_k = \lambda_k \mathbf{M} \boldsymbol{\phi}_k\)$ where \(\mathbf{L}\) is the cotangent stiffness matrix representing the weak Laplacian and \(\mathbf{M}\) is the lumped mass matrix. Truncating to the lowest \(K\) non-trivial eigenfunctions yields an orthonormal basis \(\boldsymbol{\Phi} = [\boldsymbol{\phi}_1, \dots, \boldsymbol{\phi}_K] \in \mathbb{R}^{|V'| \times K}\) satisfying \(\boldsymbol{\Phi}^\top \mathbf{M} \boldsymbol{\Phi} = \mathbf{I}\). Projecting any error map \(\epsilon_H^{(n)}\) into this spectral domain: $\(\mathbf{c}_{\epsilon_H}^{(n)} = \boldsymbol{\Phi}^\top \mathbf{M} \epsilon_H^{(n)} \in \mathbb{R}^K\)$ effectively filters high-frequency sampling jitter and compresses structural discrepancies into a compact, geometrically interpretable spectral descriptor.

5. Unknown Geometric Bias Spectral Clustering: Failure Mode Discovery & Attribution Rather than restricting bias detection to pre-annotated classification bins, the framework performs K-means clustering directly on the low-dimensional spectral representations \(\{\mathbf{c}_{\epsilon_H}^{(n)}\}_{n=1}^N\) to discover latent, unlabeled geometric failure modes. Each cluster centroid \(\bar{\mathbf{c}}_{\epsilon_H}\) is projected back to the spatial mesh domain via the adjoint synthesis transform \(\bar{\epsilon}_H = \boldsymbol{\Phi} \bar{\mathbf{c}}_{\epsilon_H}\) for intuitive 3D heat-map visualization. Correlating these clusters with demographic attributes (age, sex, ethnicity) and projecting ground-truth scans onto regional morphological PCA bases exposes exactly which phenotypic dimensions (such as nasal bridge length or nostril curvature) are systematically compromised by the 3DMM basis.

Key Experimental Results

Main Results

To demonstrate that Curvature Reconstruction Error (CRE) mirrors human qualitative assessment of 3D facial shape fidelity far better than traditional Euclidean metrics, a rigorous user study was conducted on the REALY benchmark. Nineteen annotators evaluated 1,111 high-consensus pairs (\(>75\%\) inter-annotator agreement) in a two-alternative forced-choice (2AFC) fidelity preference test:

Metric Mapping Protocol Hyperparameters \((R, K, \text{Norm})\) Pairwise-Accuracy (%) Pairwise-ROC-AUC (%) Note
\(E_{\text{REALY}}^{\text{NMSE}}\) Bidirectional Non-Rigid ICP – 53.92 56.53 Barely exceeds random chance (50%)
\(E_{\text{pts}}^{\text{NMSE}}\) Point-to-surface Euclidean – 49.32 55.02 Insensitive to local geometric quality
\(E_{\text{vtx}}^{\text{NMSE}}\) Vertex-to-vertex Euclidean – 42.03 42.47 Degraded by vertex resampling density
\(E_{\text{CRE}}\) (Ours) Point-to-surface Differential \(R=3, K=128, L_1\) 73.63 82.66 Substantially outperforms Euclidean metrics (+19.71% Acc)

Ablation Study

The lower section of Table 1 from the paper details the ablation across neighborhood rings \(R\), spectral cutoff dimensions \(K\), and error norm formulations:

Config / Norm Ring \(R\) Spectral Dim \(K\) Pairwise-Accuracy (%) Pairwise-ROC-AUC (%) Note
\(L_1\) Norm 3 128 73.63 82.66 Optimal operational parameter set
\(L_1\) Norm 1 128 71.38 80.63 1-ring is more vulnerable to fine mesh noise
\(L_1\) Norm 1 1024 66.52 74.35 Excessive spectral cutoff retains high-frequency artifacts
\(L_1\) Norm 2 128 72.46 81.86 Stable across intermediate ring radii
\(L_1\) Norm 4 128 73.00 81.85 Large rings slightly oversmooth sharp ridges
\(L_2\) Norm 3 128 72.28 81.17 \(L_2\) over-penalizes isolated curvature spikes
Huber Loss 3 128 67.60 75.82 Quadratic smoothing blunts fine structural errors

Furthermore, the authors computed Pearson correlation coefficients \(r\) between subject age and regional CRE across both foundational 3DMM bases and state-of-the-art monocular 2D-to-3D reconstruction networks:

Model / Method Category Nose @nose (All/F/M) Mouth @mouth (All/F/M) Cheek @cheek (All/F/M) Forehead @forehead (All/F/M)
BFM 3DMM Basis Prior 0.39 / 0.32 / 0.59 0.33 / 0.21 / 0.67 0.77 / 0.72 / 0.88 0.52 / 0.49 / 0.59
FLAME 3DMM Basis Prior 0.78 / 0.79 / 0.78 -0.03 / -0.16 / 0.18 0.82 / 0.84 / 0.82 0.41 / 0.48 / 0.25
Deep3D Monocular 2D Model 0.24 / 0.18 / 0.41 -0.23 / -0.19 / -0.38 0.69 / 0.81 / 0.62 0.58 / 0.54 / 0.66
3DDFA-v3 Monocular 2D Model 0.67 / 0.65 / 0.73 0.31 / 0.30 / 0.37 0.79 / 0.78 / 0.88 0.53 / 0.55 / 0.48
MGCNet Monocular 2D Model 0.54 / 0.50 / 0.67 0.30 / 0.30 / 0.32 0.80 / 0.79 / 0.89 0.51 / 0.54 / 0.47
DECA Monocular 2D Model 0.73 / 0.75 / 0.70 -0.32 / -0.30 / -0.35 0.76 / 0.80 / 0.85 0.40 / 0.46 / 0.28
MICA Monocular 2D Model 0.69 / 0.72 / 0.65 -0.31 / -0.31 / -0.30 0.79 / 0.76 / 0.88 0.43 / 0.48 / 0.31

Key Findings

  • Severe Age Bias Rooted in Subspace Under-Representation: In the cheek region (@cheek), reconstruction error exhibits a strong positive correlation with subject age across all tested models (\(r = 0.77\) for BFM, \(r = 0.82\) for FLAME, reaching \(r = 0.88\) for males in BFM). Linear PCA bases lack the capacity to express mature morphological structures (folds, jowls, creases), coercively smoothing older subjects toward a youthful mean. Downstream 2D networks inherit this bias directly from the 3DMM prior without exacerbating it.
  • Ethnic Disparities in Geometric Failure Modes: Caucasian error profiles disperse evenly across different spectral clusters, whereas African and Asian subjects cluster heavily in specific failure modes (Cluster 1 for BFM; Clusters 1 and 2 for FLAME). Phenotypic PCA reveals that Cluster 1 corresponds to shorter nasal bridges and wider alar bases, which 3DMMs distort into pinched, upward-drawn nasal tips with unnatural lateral creases.
  • Euclidean Metrics Obscure Algorithmic Inequity: When standard Euclidean NMSE is used instead of CRE, the correlation between reconstruction error and age drops to near zero (\(r < 0.1\)), falsely suggesting that current 3DMMs are demographically fair.

Highlights & Insights

  • Curvature as a Diagnostic Lens for 3D Fairness: Demonstrates for the first time that algorithmic bias in 3D vision manifests as second-order differential surface loss rather than macro-alignment displacement, establishing differential geometry as a core tool for geometric fairness auditing.
  • Manifold Spectral Decomposition for Low-Sample Clustering: Solving the generalized LBO eigenproblem on the template enables meaningful pattern discovery in severe small-sample regimes (\(N \approx 100\) vs. \(|V| \sim 10^4\)), avoiding overfitting to vertex-level registration noise.
  • Decoupling Prior Limitations from Inference Networks: By optimizing 3DMM coefficients directly against 3D ground truth scans (eliminating 2D image ambiguity), the study proves conclusively that geometric bias originates from the statistical expressive bottlenecks of the PCA basis itself.

Limitations & Future Work

  • Small Demographic Sample Sizes: High-resolution 3D facial scans with comprehensive demographic annotations remain scarce; REALY contains only 100 scans, meaning ethnic subgroup conclusions remain exploratory and warrant validation on larger cohorts.
  • Linearity of Shape Models: Current parametric models rely on linear PCA subspaces, fundamentally constraining their ability to capture non-linear morphological aging patterns.
  • Audit Tool without Closed-Loop Mitigation: CRE currently functions as an ex-post evaluation and discovery metric; formulating a differentiable Laplace-Beltrami curvature loss for end-to-end neural network fine-tuning remains an open direction for future work.
  • vs. REALY Benchmark (Chai et al., ECCV 2022): REALY established regional masks and bidirectional alignment but relied on Euclidean NMSE. This work demonstrates that NMSE exhibits near-random correlation with human visual perception (53.92% vs. 73.63% for CRE), providing the missing curvature-aware evaluation dimension.
  • vs. 3D Face Fairness Auditing (Cui & Yu, IJCNN 2024): Previous attempts to analyze 3D reconstruction bias used Euclidean metrics and concluded that demographic disparities were marginal. By introducing differential curvature analysis, this paper reveals acute, statistically significant age (\(r > 0.8\)) and ethnic failure modes.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Pioneering introduction of differential geometry and manifold spectral analysis into 3D facial reconstruction fairness evaluation.
  • Experimental Thoroughness: ⭐⭐⭐⭐☆ Extensive user study, comprehensive parameter ablations, and cross-model regional correlation analyses; slightly bounded by 3D benchmark scan counts.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical derivations, well-motivated geometric formulations, and articulate exposition of failure mechanisms.
  • Value: ⭐⭐⭐⭐⭐ Highly timely for regulatory compliance (EU AI Act) and foundation model development in 3D computer vision and biometric avatars.