Learning Geometry-Aware Embedding Fields for Intrinsic Riemannian Mappings¶
Conference: ECCV 2026
Paper: ECCV Paper
Code: https://circle-group.github.io/research/LearningRiemannianGeometry
Area: 3D Vision
Keywords: Riemannian Geometry, Exponential and Logarithmic Maps, Neural Fields, Triplane Hyper-Modulation, Geometric Deep Learning
TL;DR¶
To tackle the challenge of computing intrinsic Riemannian mappings on non-manifold and noisy point sets, this paper presents a data-driven framework that learns a continuous geometric embedding field via sparse octree convolutions and decodes exponential and logarithmic maps in a single forward pass using triplane hyper-modulation.
Background & Motivation¶
Differential and Riemannian geometry provide the rigorous computational foundation for reasoning about curved manifolds, formalizing geodesic distances, tangent spaces, and intrinsic topological structures. On idealized, smooth continuous surfaces, the exponential map (which propels tangent vectors along geodesic paths onto the manifold) and the logarithmic map (its inverse, which projects surface points back to their initial tangent vectors) admit closed-form expressions or can be accurately solved using discrete operators on clean meshes. However, raw 3D data in modern vision and graphics pipelines—such as unordered point clouds, triangle soups with self-intersections, or implicit surface representations—rarely provide clean 2-manifold connectivity and are frequently corrupted by sensor noise and missing regions. Consequently, classical differential solvers (like the Vector Heat Method or global Eikonal solvers) that rely on high-quality mesh topology fail catastrophically.
Existing methods for implicit surface navigation typically resort to iterative projection schemes or numerical geodesic tracing. Such heuristics scale poorly with resolution, suffer from severe numerical instability around sharp features or high-curvature regions, and lack awareness of global shape semantics. As a result, pure geometric heuristics frequently suffer from boundary leakage—incorrectly short-circuiting across the opposing sheets of thin geometric plates—or fail across sparsely sampled gaps. Furthermore, contemporary geometric deep learning frameworks demand intrinsic operators that are fully differentiable and capable of batch-parallel evaluation on GPUs, which traditional whole-shape global differential solvers cannot deliver.
To bridge this fundamental gap, this paper introduces a data-driven formulation that treats Riemannian mappings as continuous neural fields over raw geometric observations. The authors capitalize on two key intrinsic properties: locality and spatial coherence, recognizing that local metric tensors and geodesic propagation rules are largely determined by local geometry while varying smoothly over the surface. Core idea: encode unstructured raw point sets into a continuous geometric embedding field using a sparse octree convolutional network, retrieve local latent embeddings to dynamically modulate lightweight triplane neural fields, and directly predict bidirectional, isometric, and conformal Riemannian mappings in ambient 3D coordinates in a single forward pass.
Method¶
Overall Architecture¶
The framework processes an unstructured point set \(\mathcal{P} \subset \mathbb{R}^3\) without mesh connectivity and enables differentiable evaluation of the exponential map \(\text{Exp}_p: T_p\mathcal{M} \to \mathcal{M}\) and logarithmic map \(\text{Log}_p: \mathcal{M} \to T_p\mathcal{M}\) at any source point \(p\). The overall pipeline consists of two primary stages: (1) learning a continuous Geometric Embedding Field (GEF) over the shape using a 3D sparse octree CNN, and (2) performing single-pass inference via dynamically modulated lightweight decoders (ExpNet and LogNet) conditioned on local geometric embeddings.
The overall workflow is illustrated below:
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Raw Point Cloud Input<br/>(Unstructured/Noisy Geometry)"] --> B["Sparse Octree Geometric Encoder<br/>(3D OCNN U-Net Continuous Field)"]
B --> C["Source Point Trilinear Interpolation<br/>(Extract Local Latent zp)"]
C --> D["Triplane Hyper-Modulation<br/>(Generate Orthogonal Feature Planes)"]
D --> E["Ambient 3D Relative Coordinates<br/>(Bypass Local 2D Basis Ambiguity)"]
E --> F["Lightweight MLP Decoding<br/>(Instant ExpMap and LogMap Output)"]Key Designs¶
1. Sparse Octree Geometric Embedding Field: Mesh-Agnostic Continuous Encoding
Traditional differential geometry algorithms require watertight 2-manifold triangulations, failing when applied to point clouds or non-manifold triangle soups. This method employs an octree-based convolutional neural network (OCNN) with a 3D U-Net backbone. By hierarchically aggregating local geometric patterns and global contextual shape semantics across adaptive octree cells, the network outputs a continuous volumetric feature field. For any arbitrary source point \(p \in \mathcal{M}\), the local geometric latent descriptor \(z_p\) is retrieved via standard trilinear interpolation. Because the OCNN operates on relative hierarchical spatial grids, the learned continuous field is fully translation-invariant and requires only a single forward pass to encode the entire shape.
2. Triplane Hyper-Modulation: Geometry-Adaptive Efficient Neural Field Synthesis
While the global shape of \(\mathcal{M}\) varies considerably, the behavior of geodesic mappings is governed primarily by local curvature and metric structure. Inspired by FiLM and HyperNetworks, the retrieved source embedding \(z_p\) is linearly projected to generate the parameter volume for three orthogonal feature planes (\(XY, XZ, YZ\)). For any query coordinate, feature representations are sampled via bilinear interpolation across the three planes and concatenated. The resulting compact vector is fed into an ultra-lightweight MLP decoder (ExpNet or LogNet), requiring only approximately 5 matrix multiplications and 3 linear interpolations per query. This hyper-modulation decouples local geometric adaptation from the global backbone, enabling massive GPU-parallel evaluation.
3. Ambient 3D Relative Coordinate Formulation: Eliminating Gauge Ambiguity
Differential geometry on surfaces traditionally defines local 2D tangent charts. However, defining a canonical 2D orthonormal basis on a tangent plane is inherently ambiguous up to an arbitrary \(SO(2)\) rotation, which introduces gauge inconsistency and hinders neural network convergence. To bypass this fundamental issue, ExpNet and LogNet operate directly within ambient 3D Euclidean space using relative displacement coordinates (\(v \in \mathbb{R}^3\) and \(q - p\)). This design eliminates rotational chart ambiguity, ensures global coordinate coherence across heterogeneous surfaces, and retains translation invariance.
4. Cut Locus Sample Filtering: Discarding Non-Bijective Topological Ambiguities
On general Riemannian manifolds, the exponential map is locally surjective but not globally injective due to intersecting geodesics at the cut locus, rendering the logarithmic map ill-defined beyond the injectivity radius (as shown in Figure 3 on the Stanford Bunny). Directly supervising on uncurated global geodesic paths leads to severe multi-modal oscillation and trivial training collapse. To address this, the pipeline identifies and removes target points that correspond to multiple distinct tangent vectors (due to geodesics wrapping around the shape). This filtering eliminates approximately 29.6% ambiguous and redundant training samples, allowing the network to focus strictly on well-defined, bijective diffeomorphic patches.
Loss & Training¶
The overall network is trained using a multi-objective loss function combining empirical reconstruction, differential geometric regularization, and cycle consistency: $\(\mathcal{L}_{\mathrm{total}} = \lambda_{\mathrm{rec}}\mathcal{L}_{\mathrm{rec}} + \lambda_{\mathrm{geo}}\mathcal{L}_{\mathrm{geo}} + \lambda_{\mathrm{cyc}}\mathcal{L}_{\mathrm{cycle}}\)$ with weights \(\lambda_{\mathrm{rec}} = 1.0\), \(\lambda_{\mathrm{geo}} = 0.01\), and \(\lambda_{\mathrm{cyc}} = 0.01\).
- Reconstruction Loss \(\mathcal{L}_{\mathrm{rec}}\): Supervised by ground-truth geodesic data computed on clean meshes using potpourri3D via robust \(L_1\) norms: $\(\mathcal{L}_{\mathrm{Exp}} = \|\hat{q} - q_{\mathrm{gt}}\|_1, \quad \mathcal{L}_{\mathrm{Log}} = \|\hat{v} - v_{\mathrm{gt}}\|_1, \quad \mathcal{L}_{\mathrm{rec}} = \mathcal{L}_{\mathrm{Exp}} + \mathcal{L}_{\mathrm{Log}}\)$
- Intrinsic Geometric Loss \(\mathcal{L}_{\mathrm{geo}}\): Enforces local isometry and conformality on the learned mapping fields. Using PyTorch auto-differentiation, the \(3 \times 3\) spatial Jacobian \(J\) is computed, yielding the metric tensor \(M = J^T J\). The isometric loss \(\mathcal{L}_{\mathrm{iso}}\) constrains diagonal elements toward 1 (preserving tangent vector norms), while the conformal loss \(\mathcal{L}_{\mathrm{conf}}\) penalizes off-diagonal elements (preserving orthogonality): $\(\mathcal{L}_{\mathrm{iso}} = \sum_i |M_{ii} - 1|, \quad \mathcal{L}_{\mathrm{conf}} = \sum_{i \neq j} |M_{ij}|, \quad \mathcal{L}_{\mathrm{geo}} = \mathcal{L}_{\mathrm{iso}} + \mathcal{L}_{\mathrm{conf}}\)$
- Cycle Consistency Loss \(\mathcal{L}_{\mathrm{cycle}}\): Enforces the inverse duality between ExpNet \(g(\cdot; z_p)\) and LogNet \(h(\cdot; z_p)\): $\(\mathcal{L}_{\mathrm{cycle}} = \|h(g(v; z_p); z_p) - v\|_1\)$
Key Experimental Results¶
Main Results¶
The framework is evaluated on clean ShapeNet models, low-quality raw meshes (RawShapeNet with non-manifold edges and self-intersections), non-uniform point distributions, and noisy point clouds (uniform jitter in \([0, 0.015]\)). Metrics include Mean Distance Distortion (\(E_{\mathrm{MDD}} \downarrow\)) and Mean Angular Error (\(E_\theta \downarrow\), degrees) for the Logarithmic map, Average Euclidean Error (\(E_{\mathrm{Exp}} \downarrow\)) for the Exponential map, and single-query runtime (Time \(\downarrow\), seconds) evaluated on an Apple M4 Max CPU.
| Category / Benchmark | Samples / #Verts | Log Map: Method | \(E_{\mathrm{MDD}} \downarrow\) | \(E_\theta (^\circ) \downarrow\) | Time(s) \(\downarrow\) | Exp Map: Method | \(E_{\mathrm{Exp}} \downarrow\) | Time(s) \(\downarrow\) |
|---|---|---|---|---|---|---|---|---|
| ShapeNet | 1000 / 16.6k | Heat Method Projection Ours |
0.250 0.115 0.110 |
16.7 14.6 11.1 |
0.178 0.001 0.012 |
Tracing Proj. Ours |
0.036 0.049 0.026 |
0.396 0.003 0.019 |
| RawShapeNet | 200 / 10.0k | Heat Method Projection Ours |
0.314 0.166 0.169 |
21.3 17.1 15.4 |
0.101 0.001 0.013 |
Tracing Proj. Ours |
0.050 0.048 0.036 |
0.374 0.003 0.020 |
| Non-uniform | 200 / 15.0k | Heat Method Projection Ours |
0.339 0.155 0.152 |
22.4 16.3 14.0 |
0.051 0.0005 0.011 |
Tracing Proj. Ours |
0.043 0.047 0.030 |
0.415 0.003 0.019 |
| Noisy | 200 / 16.6k | Heat Method Projection Ours |
0.326 0.203 0.202 |
21.1 19.4 17.8 |
0.101 0.001 0.013 |
Tracing Proj. Ours |
0.060 0.055 0.038 |
0.374 0.003 0.020 |
Zero-shot transfer evaluations on non-rigid human motions (DFAUST) and real-world sensor scans (Kinect-v2) confirm strong out-of-distribution generalization without fine-tuning: on DFAUST, Log angular error drops from \(25.47^\circ\) (Heat) to \(16.51^\circ\) (Ours), and Exp error reaches \(0.043\); on Kinect-v2 scans, Log angular error reaches \(8.37^\circ\) while Exp error drops to \(0.060\) (outperforming Tracing at \(0.063\) and Proj. at \(0.064\)).
Ablation Study¶
Ablation experiments conducted across 100 ShapeNet shapes quantify the individual contributions of the filtering strategy and geometric loss terms:
| Variant Config | \(E_{\mathrm{MDD}}\) (Log) \(\downarrow\) | \(E_\theta (^\circ)\) (Log) \(\downarrow\) | \(E_{\mathrm{Exp}}\) (Exp) \(\downarrow\) | Rel. Cost \(\downarrow\) | Key Observation |
|---|---|---|---|---|---|
| No Filtering | 0.112 | 11.3 | 0.027 | 1.42× | Achieves comparable precision but incurs 42% higher computational overhead due to cut locus ambiguities. |
| Simple Loss | 0.162 | 11.5 | 0.025 | 1.00× | Omitting Jacobian isometry and conformality constraints drastically degrades distance preservation (\(0.112 \to 0.162\)). |
| Full Model | 0.112 | 11.2 | 0.028 | 1.00× | Delivers optimal balance between metric accuracy, conformality, and training efficiency. |
Key Findings¶
- Robustness Against Boundary Leakage on Thin Structures: Classical numerical methods that discretize the Laplace-Beltrami operator suffer from metric bleed across thin geometric sheets (e.g., vessel walls or container rims), improperly short-circuiting between opposing sides. In contrast, the learned embedding field correctly enforces geodesic continuity around thin plate boundaries.
- Handling Extreme Normal Variations: While local Euclidean projection baselines degrade severely whenever surface normals flip rapidly, the triplane neural representation retains intrinsic curved coordinate fidelity, yielding artifact-free UV parameterizations on noisy Kinect-v2 point scans.
- Substantial Runtime Advantage: Compared to iterative geodesic tracing (\(0.37 \sim 0.41\) s per query), this model resolves queries in approximately \(0.019\) s on CPU, achieving an approximate \(20\times\) speedup while supporting instant GPU batch parallelization.
Highlights & Insights¶
- Operator Learning for Manifold Mappings: Extends geometric deep learning beyond static classification or reconstruction by learning continuous, vector-valued inverse Riemannian mapping operators directly from point sets.
- Ambient 3D Coordinate Parameterization: Resolves the classic \(SO(2)\) tangent chart gauge ambiguity by performing all queries directly in extrinsic 3D coordinates, stabilizing neural optimization across arbitrary complex topologies.
- Jacobian-Based Metric Regularization: Elegantly translates classical differential geometric properties (isometry and conformality) into auto-differentiable Jacobian penalties, enforcing physically sound geodesic properties in unconstrained neural fields.
Limitations & Future Work¶
- Supervision Bound by Numerical Solvers: The model relies on ground-truth training paths generated by classical tools (e.g., potpourri3D) on watertight meshes, making its peak performance inherently bounded by traditional discretization artifacts.
- Resolution Smoothing in Fine Details: Octree voxelization and triplane grids can smooth out high-frequency micro-scale geometric wrinkles and sharp creases.
- Non-End-to-End Global Parameterization: Due to injectivity radius limitations, large-scale surface parameterization currently relies on an external greedy weighted rigid alignment stitching algorithm rather than an end-to-end global bijective formulation.
Related Work & Insights¶
- vs Vector Heat Method (Sharp et al., 2019): The Vector Heat Method requires clean, manifold triangular meshes to solve vector heat diffusion and Poisson equations; this approach bypasses all mesh connectivity requirements and processes raw, noisy point clouds directly.
- vs NeuroGF (Zhang et al., 2023) & GeodesicEmbedding (Xia et al., 2021): Prior works primarily regress scalar geodesic distances; this framework directly learns the vector-valued bidirectional mapping between tangent spaces and the ambient surface.
- vs NRDF (He et al., 2024): NRDF focuses on articulated human pose priors in configuration manifolds; this work provides general-purpose intrinsic differential operators for arbitrary non-rigid 3D shapes.
Rating¶
- Novelty: ⭐⭐⭐⭐☆ Pioneering neural operator formulation for bidirectional Riemannian mappings using ambient coordinates and triplane hyper-modulation.
- Experimental Thoroughness: ⭐⭐⭐⭐☆ Evaluated across clean ShapeNet, raw non-manifold models, non-uniform/noisy point clouds, DFAUST, and real Kinect-v2 scans.
- Writing Quality: ⭐⭐⭐⭐⭐ Rigorous geometric mathematical formulation, clear conceptual motivation, and well-designed experimental validation.
- Value: ⭐⭐⭐⭐☆ Highly impactful for discrete geometry processing, differentiable surface parameterization, and deep learning on unstructured 3D shapes.