Skip to content

Learning Manifolds in High-D Point Embedding for Anisotropic Surface Approximation from Unstructured Point Clouds

Conference: ECCV 2026
Paper: ECCV 2026
Area: 3D Vision
Keywords: High-D Euclidean embedding, anisotropic surface mesh, unstructured point clouds, Riemannian metric, surface reconstruction

TL;DR

To tackle the challenge of estimating anisotropic metric fields on unstructured point clouds without connectivity, HD-PEA introduces a barycentric-invariant isometric transfer scheme to train Point Transformer V3 for 8D manifold embedding, followed by tangent subspace particle relaxation and high-dimensional restricted Voronoi cell (RVC) clipping, directly reconstructing compact, curvature-aligned anisotropic meshes from raw point clouds.

Background & Motivation

In robotics, autonomous driving, and 3D vision, dense acquisition sensors produce massive 3D point clouds characterized by substantial geometric redundancy. For downstream numerical simulation and geometric processing, anisotropic surface meshes are highly advantageous: their triangular elements elongate along principal curvature directions—stretching across flat planar regions and densely refining near sharp features—thus yielding superior surface approximation accuracy with significantly fewer vertices and faces compared to uniform isotropic counterparts. However, traditional anisotropic meshing and remeshing algorithms rely heavily on pre-existing clean triangle meshes to compute explicit face normals, edge vectors, and local parameterizations, rendering them incapable of operating directly on unstructured, discrete point sets.

Recent advances in neural implicit surface reconstruction (such as SDF-based marching cubes variants NMC and NDC) and adaptive meshing methods (such as LMR) generate resolution-adaptive meshes, but they fundamentally remain isotropic or locally adaptive without strict control over curvature-aligned directional stretching ratios. Multi-stage pipelines—which first extract an intermediate watertight mesh from point clouds and then apply mesh-based anisotropic remeshing (e.g., NDC followed by NASM)—suffer from severe cascading artifacts: discretization errors and aliasing in the intermediate mesh are amplified during remeshing, leading to degraded surface fidelity. Although the foundational Nash Embedding Theorem proves that any smooth Riemannian manifold can be isometrically embedded into a higher-dimensional Euclidean space, classical formulations assume continuous manifolds and cannot handle unorganized, discrete point sets lacking tangent vectors.

The core idea is to transfer precomputed high-dimensional embeddings from mesh faces to dense surface points via barycentric coordinate invariance to create ground-truth supervision, train a network with a pairwise neighborhood distance loss to predict an 8D manifold embedding, and optimize isotropic particle systems under local tangent subspace projections to extract restricted Voronoi cells that naturally project back into 3D as curvature-aligned anisotropic meshes.

Method

Overall Architecture

The HD-PEA framework consists of three primary stages: 1. Neural High-D Euclidean Point Embedding: Barycentric-invariant weight transfer generates ground-truth point-wise mappings from 3D points to 8D embeddings. A Point Transformer V3 backbone takes point coordinates and normal vectors \([P, n] \in \mathbb{R}^6\) to predict the \((d-3)\)-dimensional coordinate extension. For arbitrary large-scale scenes, Patch-based Meta-Embedding Inference (PMEI) aligns local patch embeddings via a minimum spanning tree. 2. High-D Embedding Manifold Approximation via Tangent Subspace: Because a 2D surface embedded in 8D ambient space lacks a unique normal vector, local Principal Component Analysis (PCA) on high-dimensional neighbor sets estimates an orthonormal basis for the 2D tangent subspace. Repulsive particle dynamics are constrained to this subspace to reach isotropic equilibrium. 3. Anisotropic Manifold Reconstruction via High-D RVC: At each optimized sampling seed, a local 2D intrinsic disk is mapped into 8D ambient space and iteratively clipped by bisecting hyperplanes of neighboring seeds using re-entrant polygon clipping. Truncating the resulting high-dimensional mesh back to 3D automatically produces an optimal anisotropic surface triangulation.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Unstructured Point Cloud<br/>[P, n] ∈ R^(N×6)"] --> B["Neural High-D Point Embedding<br/>Point Transformer V3 predicts 8D embedding"]
    B --> C["Patch-Based Meta-Embedding (PMEI)<br/>MST affine alignment across overlapping patches"]
    C --> D["Tangent Subspace Approximation & Particle Relaxation<br/>PCA 2D tangent basis constraints for uniform seeds"]
    D --> E["High-D Restricted Voronoi Cell Clipping<br/>Tangential complex triangulation and 3D projection"]
    E --> F["Compact Anisotropic Surface Mesh"]

Key Designs

1. Barycentric-invariant isometric data generation: bridging unorganized points to high-d Nash embeddings
Prior mesh-based high-dimensional embeddings rely on explicit triangle face connectivity, preventing direct application to unstructured point clouds. To resolve this, the authors exploit the geometric property that the mapping from an intrinsic 2D Riemannian surface to the target 8D Euclidean space is locally affine, and barycentric coordinates remain invariant under affine transformations. Points \(p\) sampled inside an original triangle \(f = \{v_1, v_2, v_3\}\) are expressed as \(p = w_1 v_1 + w_2 v_2 + w_3 v_3\) with \(w_1+w_2+w_3=1, w_i \ge 0\). The ground-truth 8D embedding \(\bar{p}\) is directly synthesized using the identical weights over the embedded mesh vertices: \(\bar{p} = w_1 \bar{v}_1 + w_2 \bar{v}_2 + w_3 \bar{v}_3\). This bypasses differential operator estimation on point clouds, establishing clean ground-truth pairs \((p, \bar{p})\) encoding local curvature and metric anisotropy.

2. Neighborhood distance embedding loss and Patch-based Meta-Embedding Inference (PMEI): scalable robust representation
Unlike mesh formulations that optimize dot products of face edge vectors, point clouds lack defined edge vectors. HD-PEA uses a neighborhood distance-based Mean Absolute Error (MAE) loss over local point pairs, which implicitly forces predicted Euclidean distances in 8D to match intrinsic Riemannian geodesic metrics while offering robustness against non-uniform point distributions: $\(\mathcal{L}_{\text{NDHDE}} = \frac{1}{N} \sum_{i=1}^N \sum_{j \in \mathcal{N}(i)} \left| \|f(p_i) - f(p_j)\| - \|\bar{p}_i - \bar{p}_j\| \right|\)$ To process large-scale shapes and scenes beyond fixed network input sizes, PMEI decomposes point clouds into overlapping patches via farthest point sampling (FPS). It formulates patch alignment as a minimum spanning tree (MST) problem initialized from an arbitrary root and solved with Prim's algorithm, sequentially optimizing affine transformations \(T_{ij}^* = \arg\min_{T_{ij}} \sum_k \|p_i^k - T_{ij}(p_j^k)\|^2\) across shared regions to eliminate non-equivariant attention drift.

3. Tangent subspace estimation and particle equilibrium: handling high codimension constraints
When a 2D manifold is embedded into \(d=8\) dimensional Euclidean space, its orthogonal complement has dimension 6, meaning no unique surface normal exists. To constrain particle optimization strictly to the underlying manifold, the algorithm constructs the local covariance matrix \(C_p = \frac{1}{k} \sum (p_i - \hat{p})(p_i - \hat{p})^T\) over \(k\) nearest neighbors in 8D, deriving an orthonormal basis \(V \in \mathbb{R}^{8 \times 2}\) corresponding to the two largest eigenvalues. During distance-based particle repulsion, particle coordinate updates are projected onto this local tangent space: \(\tilde{x} = x_p + V V^T (x - x_p)\). At physical equilibrium, this yields an intrinsically uniform, isotropic sampling across the high-dimensional manifold.

4. High-dimensional tangential Delaunay clipping and truncation projection: generating anisotropic elements
Using the equilibrated seeds, a 2D intrinsic disk \(D^2\) of radius \(r = \alpha \cdot l_{\text{diag}}\) is defined and mapped to 8D ambient space as \(D^8(x) = V D^2 + x\). Sutherland-Hodgman re-entrant polygon clipping iteratively clips this disk using the bisector hyperplanes between each seed and its high-dimensional neighbors to construct Restricted Voronoi Cells (RVCs). Because all clipping operations are confined within the 2D tangent plane, computational complexity scales with intrinsic dimension \(m=2\) rather than ambient dimension \(d=8\). Truncating the resulting high-dimensional mesh vertices to their first 3 coordinates maps high-dimensional isotropic disks into elongated Riemannian metric ellipses, producing a clean, geometry-aligned anisotropic mesh.

Loss & Training

The backbone is Point Transformer V3 taking point positions and normals \([P, n] \in \mathbb{R}^{N \times 6}\) computed via Winding Number Normal Consistency (WNNC). A single linear head outputs the 5 additional coordinate dimensions. The network is trained on 2,400 augmented models from Thingi10K with \(N=40\text{K}\) points per model and neighborhood size \(|\mathcal{N}(i)| = 40\) in 8D space (\(d=8\)). Optimization uses Adam with the \(\mathcal{L}_{\text{NDHDE}}\) loss without auxiliary direct coordinate loss.

Key Experimental Results

Main Results

Quantitative evaluations are conducted across the Thingi10K benchmark (80 testing shapes) and synthetic scanning point clouds (75 models from Myles et al.'s dataset simulated with Blensor scanner noise). Metrics include Chamfer distance (CD, \(\times 10^{-5}\)), F1-Score (F1), Normal Consistency (NC), Hausdorff distance (HD, \(\times 10^{-2}\)), anisotropic mesh triangle quality \(G\), and total runtime in seconds.

Table 1: Quantitative comparison on Thingi10K testing set (80 models) | Method | #Vout | #fout | CD \(\downarrow\) | F1 \(\uparrow\) | NC \(\uparrow\) | HD \(\downarrow\) | Time (s) \(\downarrow\) | |---|---|---|---|---|---|---|---| | Poisson | 38,346 | 76,684 | 39.128 | 0.912 | 0.970 | 1.821 | 19.27 | | NDC (\(64^3\)) | 4,134 | 8,327 | 0.798 | 0.986 | 0.978 | 0.921 | 4.97 | | NDC (\(128^3\)) | 16,638 | 33,470 | 0.676 | 0.991 | 0.987 | 0.716 | 5.53 | | POCO (\(64^3\)) | 8,237 | 16,497 | 29.817 | 0.853 | 0.951 | 6.521 | 4.37 | | POCO (\(128^3\)) | 33,590 | 67,196 | 33.107 | 0.888 | 0.960 | 6.862 | 15.59 | | PoNQ | 15,760 | 31,531 | 11.729 | 0.967 | 0.982 | 1.813 | 31.05 | | LMR | 7,848 | 15,721 | 2.936 | 0.978 | 0.978 | 0.778 | 2,710.60 | | HD-PEA (Ours) | 5,180 | 10,358 | 0.496 | 0.996 | 0.987 | 0.678 | 14.28 |

Table 2: Quantitative comparison on synthetic scanned shapes from Myles et al.'s dataset (75 models) | Method | #Vout | #fout | CD \(\downarrow\) | F1 \(\uparrow\) | NC \(\uparrow\) | HD \(\downarrow\) | Quality \(G \uparrow\) | Time (s) \(\downarrow\) | |---|---|---|---|---|---|---|---|---| | Poisson | 77,296 | 154,594 | 26.346 | 0.922 | 0.966 | 2.147 | 0.528 | 39.30 | | NDC (\(64^3\)) | 5,182 | 10,575 | 1.308 | 0.939 | 0.956 | 1.230 | 0.589 | 5.28 | | NDC (\(128^3\)) | 21,011 | 42,721 | 0.749 | 0.982 | 0.971 | 0.928 | 0.588 | 5.93 | | NDC + NASM | 6,964 | 14,210 | 0.792 | 0.980 | 0.975 | 1.265 | 0.576 | 24.70 | | POCO (\(128^3\)) | 58,871 | 117,837 | 90.462 | 0.626 | 0.875 | 11.601 | 0.530 | 22.87 | | PoNQ | 24,685 | 49,368 | 7.610 | 0.963 | 0.973 | 1.511 | 0.596 | 35.83 | | NKSR | 12,023 | 24,040 | 1.461 | 0.954 | 0.966 | 1.745 | 0.525 | 0.71 | | SAP | 31,692 | 63,320 | 2.081 | 0.976 | 0.972 | 1.447 | 0.526 | 6.58 | | SIREN | 696,845 | 1,393,631 | 58.960 | 0.843 | 0.953 | 3.593 | 0.522 | 4,658.97 | | LMR | 7,590 | 15,124 | 10.048 | 0.964 | 0.972 | 2.471 | 0.629 | 3,153.80 | | HD-PEA (Ours) | 7,028 | 13,989 | 0.649 | 0.989 | 0.979 | 0.798 | 0.702 | 14.59 |

Ablation Study

The ablation investigates the high-dimensional embedding loss function on 80 Thingi10K models, contrasting point-wise coordinate MSE/MAE against pairwise neighborhood distance formulations (NDHDE).

Table 3: Ablation on loss functions for high-d embedding (Thingi10K 80 models) | Config / Loss | #Vout | #fout | CD \(\downarrow\) | F1 \(\uparrow\) | NC \(\uparrow\) | HD \(\downarrow\) | |---|---|---|---|---|---|---| | Point-wise MSE | 12,632 | 48,368 | 30.864 | 0.953 | 0.949 | 2.148 | | Point-wise MAE | 12,116 | 48,421 | 29.737 | 0.958 | 0.952 | 2.044 | | NDHDE w/ MSE | 8,688 | 48,991 | 30.072 | 0.984 | 0.979 | 1.065 | | NDHDE w/ MAE (Full Model) | 5,180 | 10,358 | 0.496 | 0.996 | 0.987 | 0.678 |

Table 4: Robustness evaluation under noise and non-uniform sampling (Myles et al. 75 models) | Perturbation / Sampling | #Vout | #fout | CD \(\downarrow\) | F1 \(\uparrow\) | NC \(\uparrow\) | HD \(\downarrow\) | Time (s) | |---|---|---|---|---|---|---|---| | Real scan + 0.1% Gaussian noise | 8,684 | 17,255 | 0.822 | 0.978 | 0.974 | 0.893 | 14.81 | | Real scan + 0.3% Gaussian noise | 9,341 | 18,521 | 1.000 | 0.966 | 0.959 | 1.125 | 14.97 | | Real scan + 0.5% Gaussian noise | 9,601 | 19,043 | 1.417 | 0.933 | 0.947 | 1.358 | 15.00 | | Gradient density sampling | 7,526 | 14,907 | 0.730 | 0.982 | 0.976 | 0.970 | 14.63 | | Striped density sampling | 7,186 | 14,296 | 0.688 | 0.985 | 0.978 | 0.894 | 14.60 |

Key Findings

  1. End-to-end anisotropic meshing outclasses two-stage pipelines: HD-PEA achieves an anisotropic triangle quality \(G\) of 0.702 directly from raw point clouds, compared to 0.576 for NDC+NASM. Grid aliasing in intermediate meshes misguides downstream edge optimizers, whereas direct high-dimensional metric learning avoids cascading errors.
  2. Pairwise neighborhood distance is critical: Direct coordinate regression yields large approximation errors (CD \(\sim 30\)), while NDHDE w/ MAE constrains local metric tensors effectively, lowering CD to 0.496 and improving F1 to 0.996.
  3. Efficiency and scalability on large scenes: On ScanNet scenes, HD-PEA achieves a lower CD than NDC (0.786 vs. 0.814) with less than one-third of the mesh face count, completing inference in seconds rather than the hours required by per-shape implicit optimization baselines like LMR.

Highlights & Insights

  • Barycentric transfer unlocks Nash embedding for point sets: Using affine-invariant barycentric interpolation over pre-embedded mesh triangles provides ground-truth 8D embeddings without requiring discrete differential operators on raw point clouds.
  • Dimension reduction naturally induces anisotropy: Instead of solving ill-posed second fundamental forms and principal curvature directions on noisy 3D points, the method solves isotropic particle equilibrium and Voronoi clipping in 8D, letting truncation back to 3D naturally stretch elements along principal directions.
  • Curvature tensor estimation as a free downstream byproduct: By solving a least-squares system \(Ax = B\) relating 3D neighborhood vectors to predicted 8D displacement vectors and performing SVD, principal curvature directions and stretching ratios are directly extracted for point clouds.

Limitations & Future Work

  • Vulnerability to severe regional incompleteness: In thin cylindrical structures with missing interior scanning points, high-dimensional tangent estimation suffers from directional distortion, leading to missing elements or holes.
  • Lack of explicit global intersection checks: Tangential Delaunay complexes operate locally; when thin opposing sheets are in extreme proximity, topological self-intersections are not explicitly detected.
  • Future directions: Integrating neural implicit priors into the tangent subspace projection to bridge large holes and exploring self-supervised metric learning on raw point clouds without pre-meshed training data.
  • vs NASM (SIGGRAPH Asia 2024): NASM requires an initial clean triangle mesh and optimizes edge dot products; HD-PEA operates directly on unstructured point clouds using barycentric supervision and neighborhood distance loss.
  • vs LMR (CVPR 2025): LMR relies on per-shape optimization of implicit fields taking over 45 minutes per model without strict stretching control; HD-PEA is a feedforward neural framework completing in 14 seconds with high-quality anisotropic stretching.
  • vs Voxel Meshing (NMC / NDC / PoNQ): These methods are bounded by voxel grid resolution and produce zigzag aliasing along smooth surfaces; HD-PEA works off-grid via particle relaxation, achieving higher geometric fidelity with minimal face counts.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Elegant adaptation of the Nash Embedding Theorem to point clouds via barycentric coordinate invariance.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Extensive evaluations on Thingi10K, Myles et al., ScanNet, synthetic scans, and noise perturbations, alongside comprehensive ablations.
  • Writing Quality: ⭐⭐⭐⭐⭐ Rigorous mathematical formulation of high-codimension tangent spaces and clean geometric exposition.
  • Value: ⭐⭐⭐⭐⭐ Bridges the gap between raw point cloud acquisition and compact, simulation-ready anisotropic surface meshes.