VNC: A Scale-Space Foundation for Learnable 3D Surface Evolution

Conference: ECCV 2026
Paper: ECCV Official
Code: https://media.eventhosts.cc/Conferences/ECCV2026/pdfs/10686.pdf
Area: Interpretability
Keywords: 3D Surface Evolution, Scale-Space Theory, Variational Neighborhood Curvature, Structured Latent Space, Neural Surface Flow

TL;DR

Addressing the numerical irregularity and lack of geometric interpretability in conventional 3D surface evolution, this paper introduces a dimensionless, differentiable Variational Neighborhood Curvature (VNC) operator alongside a spacetime-balanced VNC-Flow algorithm, training a time-conditioned Controllable VAE that spontaneously organizes a physically interpretable radial latent structure.

Background & Motivation

Modern 3D generative modeling and neural representations have demonstrated remarkable capabilities in synthesizing highly detailed static meshes from scratch. However, controlling the continuous, geometrically valid evolution of these surfaces—such as progressively smoothing fine geometric wrinkles under physical laws while preserving macroscopic topology—remains a fundamental obstacle. In classical computer graphics, this challenge was elegantly addressed via scale-space theory, where physical geometric flows such as mean curvature flow (MCF) act like a "smart iron" to systematically iron out high-frequency details across scales. Nevertheless, classical differential geometric computations rely on expensive spatial queries, exhibit severe numerical irregularity, and have long remained isolated from modern deep generative architectures.

Current deep-learning-driven paradigms for 3D deformation fall into two main categories, both carrying fundamental deficiencies. Iterative methods driven by Neural ODEs deform explicit meshes or implicit level-sets directly, but they struggle with unbalanced spatial deformation rates, suffer from topological singularities such as pinching or self-intersections, and fail to respect frequency-ordered physical decay. Conversely, latent interpolation methods operate within unstructured latent spaces; linear operations on abstract vectors lack local geometric interpretability, typically yielding non-geometric artifacts or discrete hallucinated transitions. The root tension stems from the mismatch between irregular, unstable discrete physical curvature flows and deep neural networks that require predictable, identically distributed, and singularity-free intermediate representations.

The paper resolves this tension by replacing unstable discrete physical simulations with a continuous neural surrogate trained directly on physics-grounded trajectories. Core idea: by establishing a fully tensorized, scale-stable Variational Neighborhood Curvature (VNC) operator, the method drives a spacetime-balanced flow (VNC-Flow) to sample singularity-free deformation trajectories, enabling a time-conditioned Controllable VAE to self-organize a physically interpretable radial latent structure where base shapes anchor at the center and high-frequency details radiate toward the periphery.

Method

Overall Architecture

The framework bridges classical physical surface smoothing and deep generative modeling through three synergistic components. First, the Variational Neighborhood Curvature (VNC) operator leverages integral invariants and Gaussian soft-boundary relaxations to deliver a differentiable, scale-stable metric of geometric complexity. Second, the spacetime-balanced VNC-Flow algorithm enforces spatial information equilibrium via adaptive scales and applies an equidistant geometric information decay budget, converting unpredictable physical flows into singularity-free trajectories indexed by normalized time \(t \in [0, 1]\). Finally, a Controllable VAE (C-VAE) conditioned on evolution time acts as a continuous neural surrogate, inducing a geometrically structured radial latent organization.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input 3D Mesh M(0)<br/>High-frequency wrinkles and details"] --> B["Variational Neighborhood Curvature<br/>Tensorized matrix ops & scale-stable shape index"]
    B --> C["Spacetime-Balanced Curvature Flow<br/>Adaptive spatial scale & information decay budget"]
    C --> D["Controllable Variational Autoencoder<br/>Deep cross-attention & radial latent structure"]
    D --> E["Continuous Surface Evolution & Abstraction<br/>Bidirectional mapping between base shapes and details"]

Key Designs

1. Variational Neighborhood Curvature Operator: Reformulating Discrete Spatial Intersections into Dense Matrix Multiplications

Classical normal cycle theory and discrete curvature estimators require constructing explicit ball neighborhoods and querying intersected line segments using spatial data structures like R-Trees, which completely breaks GPU parallelization on high-resolution meshes. The authors propose VNC-soft by relaxing the sharp spherical boundary into a continuous Gaussian spatial weighting kernel over the entire edge set \(E\): \(w_{kj}(\sigma) = \exp(-d_j(\mathbf{v}_k)^2 / (2\sigma^2))\), utilizing full topological edge lengths \(L_j\) and dihedral angles \(\alpha_j\):

\[\hat{H}_{\text{soft}}(\mathbf{v}_k, \sigma) = \frac{\sum_{j \in E} \delta_j \alpha_j w_{kj}(\sigma) L_j}{2 \sum_{j \in E} w_{kj}(\sigma) L_j}\]

This mathematical relaxation transforms non-differentiable geometric intersections into global, dense tensor multiplications. Crucially, normalizing the curvature integration by the weighted length sum yields a dimensionless shape index \(\hat{H}\), ensuring that the global mean response \(\bar{\hat{H}}\) remains exceptionally stable across varying observation scales. Based on this, the VNC-AUC metric accumulates multi-scale complexity: \(s(\mathbf{v}_k, \tau) = \int_{\tau_{\min}}^\tau |\hat{H}(\mathbf{v}_k, x) - \bar{\hat{H}}(x)| dx\), serving as an information indicator for flow balancing.

2. Spacetime-Balanced Curvature Flow Sampling: Eliminating Spatial Imbalance and Topological Singularities via Information Budgets

Standard mean curvature flow accelerates rapidly in flat areas while lagging in wrinkled regions, causing premature collapse and singularities. VNC-Flow formulates the vertex update as \(\mathbf{v}_k \leftarrow \mathbf{v}_k - \lambda (\hat{H}_{\text{soft}}(\mathbf{v}_k, \tau_k) - \bar{\hat{H}}) \hat{\mathbf{n}}_{\text{soft}}(\mathbf{v}_k, \tau_k)\) and achieves complete spacetime balance through three key mechanisms: - Spatial Information Balance: The VNC-AUC metric dynamically determines an adaptive scale \(\tau_{\text{adpt}}(\mathbf{v}_k)\) for each vertex. Geometrically complex regions are assigned smaller scales to temporarily preserve fine details, whereas smooth regions receive larger scales to accelerate global smoothing, enforcing uniform information decay. - Temporal Rhythm Normalization: The total geometric complexity decay is partitioned into \(N_{\text{stages}}\) equidistant stages with a strict information decay budget \(\phi_{\text{stage}} = S_{\text{init}} / N_{\text{stages}}\). This remaps the flow into a normalized evolution time \(t^{(i)} = \bar{S}_{\text{cur}}^{(i)} / \bar{S}_{\text{init}} \in [0, 1]\), where \(t = 1\) denotes the initial detailed mesh and \(t \to 0\) corresponds to the simplified spherical base shape. - Implicit Topology Regularization: At each stage transition, periodic SDF-based remeshing smoothly eliminates topological pinching and self-intersections while preserving face counts and surface areas, maintaining the manifold conditions required for VNC scale stability.

3. Controllable VAE for Structured Latent Space: Spontaneous Emergence of Radial Latent Organization

To bypass expensive online physical simulations during inference, the framework introduces a Controllable VAE based on the VecSet architecture, conditioned on normalized time \(t\). Two conditioning pathways are developed: C-VAE-I injects temporal embeddings into intermediate encoder feature layers via cross-attention for dataset-level generalization; C-VAE-II freezes the pretrained geometric encoder and deeply modulates the extracted latent codes \(\mathbf{z}_{\text{geo}}\) for high-fidelity, single-instance trajectory distillation.

Supervised by the VNC-Flow sequences, this temporal modulation forces the latent space to spontaneously organize into an interpretable radial latent structure. Minimum-information base shapes (\(t \to 0\)) densely cluster at the latent origin, while complex high-frequency features (\(t \to 1\)) radiate outward along linear rays. Consequently, a linear traversal in latent space or time conditioning corresponds directly to a physically grounded, frequency-ordered scale-space smoothing in 3D Euclidean space.

Loss & Training

The C-VAE is trained using a composite objective: \(\mathcal{L} = \mathcal{L}_{\text{tsdf}} + \beta_{\text{kl}} \mathcal{L}_{\text{kl}} + \lambda_{\text{eik}} \mathcal{L}_{\text{eik}}\), where \(\mathcal{L}_{\text{tsdf}}\) is the truncated signed distance field reconstruction loss, \(\mathcal{L}_{\text{kl}}\) regularizes the latent distribution, and \(\mathcal{L}_{\text{eik}}\) enforces the Eikonal constraint on distance gradients. For C-VAE-I, a curriculum learning schedule dynamically adjusts \(\beta_{\text{kl}}\) to promote radial latent structuring; for C-VAE-II, the encoder weights remain frozen while optimizing solely the attention layers and decoder for precise trajectory distillation.

Key Experimental Results

Main Results

The experiments were conducted on THuman2.0 and 2K2K human scan datasets using an RTX 3090 GPU and an Intel Xeon Gold CPU. The computational efficiency of VNC was evaluated across different mesh resolutions against the CPU-based integral curvature baseline and DiffCurvature:

Operator	Multi-scale	Parallel	Differentiable	V=10k Time	V=25k Time	V=50k Time
Baseline [2]	✓	×	×	25 s	37 s	256 s
DiffCurvature [18]	×	✓	✓	3 ms	4 ms	7 ms
VNC-hard (Ours)	✓	✓	✓	149 ms	761 ms	3.0 s
VNC-soft (Ours)	✓	✓	✓	57 ms	358 ms	1.4 s

For trajectory reconstruction, C-VAE-II was quantitatively benchmarked against Neural Mesh Flow (NMF), Neural Implicit Evolution (NIE), and MeshSDF at the target shape (\(t = 1.0\)) using Chamfer Distance (CD), Earth Mover's Distance (EMD), and Normal Consistency Error (NCE):

Method	Evolution Paradigm	State Definition	CD \(\times 10^{-4}\) (↓)	EMD \(\times 10^{-2}\) (↓)	NCE (rad) (↓)
NMF [8]	Explicit mesh deformation	iter. 15k	1.92	5.60	0.518
NIE [19]	Implicit level-set iteration	iter. 1k	4.77	3.17	0.405
MeshSDF [27]	Latent code interpolation	\(\alpha = 1.0\)	1.61	2.55	0.341
C-VAE-II (Ours)	Temporal neural surrogate	\(t = 1.0\)	1.05	1.94	0.257

Ablation Study

The evolution behavior of VNC-Flow under different scale selection strategies was ablated against classical mean curvature flow baselines:

Config	Scale Strategy	Convergence Behavior & Phenomenon	Intermediate Manifold Validity
Fixed small scale	\(r = 0.01\)	Uneven local degeneration in flat areas; fails to reach spherical target	Poor (localized collapse)
Fixed large scale	\(r = 0.32\)	Incomplete geometric abstraction; distorted residual geometry on limbs	Moderate (incomplete smoothing)
Classical MCF Baseline A	\(\to H_+ = 0\)	Lacks scale-stable reference; surface rapidly contracts into singularity	Extremely Poor (complete collapse)
Classical MCF Baseline B	\(\to H_{r \to 0} = 0\)	Cannot preserve shell topology; contracts into mean curvature skeleton	Poor (skeletal contraction)
VNC-Flow (Full model)	Adaptive scale \(\tau_{\text{adpt}}\)	Smoothly reaches spherical target under frequency-ordered decay	Superior (consistently valid)

Key Findings

• Computational Parallelism: VNC-soft achieves a 50–180× speedup over the CPU baseline and runs 2–3× faster than VNC-hard while maintaining identical multi-scale perception at \(r = 6\sigma\).
• Latent Space Organization: UMAP and PCA visualizations confirm that latent codes organize radially based on physical complexity; base shapes cluster at the center while high-frequency details expand toward the outer shell along a shared temporal trajectory.
• Out-of-Domain Generalization: Trained exclusively on THuman2.0, C-VAE-I generalizes zero-shot to unseen scans from 2K2K, accurately predicting continuous abstraction sequences driven purely by the scalar condition \(t\).

Highlights & Insights

• Modernizing Differential Geometry for Neural Pipelines: Formulating curvature estimation as Gaussian-weighted dense matrix multiplication bridges classical scale-space differential geometry with GPU tensor computation.
• Information-Preserving Spacetime Balancing: Transforming an irregular, step-size-sensitive geometric partial differential equation into an equidistant sequence over \(t \in [0, 1]\) provides structured supervision for deep generative models.
• Interpretable Radial Latent Space: Replacing entangled latent distributions with an explicit radial organization offers a physics-grounded inductive bias where radial distance represents physical geometric entropy.

Limitations & Future Work

• Watertight Mesh Assumption: The reliance on SDF-based remeshing requires closed surfaces, necessitating preprocessing for raw, non-watertight point clouds or open meshes.
• Category Generalization Scope: Current evaluations focus primarily on digital human meshes; cross-category transfer across diverse rigid objects and complex indoor scenes remains to be verified.
• Direct Gradient Optimization: While VNC-Flow currently operates as an offline trajectory sampler, the differentiable VNC-soft operator holds promise as an online geometric loss within Flow Matching generative frameworks.

Related Work & Insights

• vs NMF [8]: NMF is constrained by a fixed mesh topology and suffers from face self-intersections during large deformations; VNC-Flow utilizes adaptive scales and SDF remeshing to achieve singularity-free topological simplification.
• vs NIE [19]: NIE tracks implicit level sets via a hybrid Eulerian-Lagrangian formulation with heavy computational overhead; the proposed C-VAE acts as a continuous neural surrogate that predicts intermediate geometry in a single forward pass.
• vs MeshSDF [27]: MeshSDF interpolates across unstructured latent codes, often causing non-geometric artifacts and abrupt jumps; C-VAE enforces physical temporal conditioning that guarantees smooth, frequency-ordered transitions.

Rating

• Novelty: ⭐⭐⭐⭐⭐ Seamlessly unifies classical scale-space curvature flow with deep generative modeling via a tensorized, scale-stable operator.
• Experimental Thoroughness: ⭐⭐⭐⭐☆ Comprehensive operator benchmarks, flow ablations, reconstruction metrics, and latent dimensionality reductions, with potential room for cross-category expansion.
• Writing Quality: ⭐⭐⭐⭐⭐ Rigorous mathematical formulation, clear conceptual motivation, and well-structured empirical validation.
• Value: ⭐⭐⭐⭐⭐ Establishes a foundational paradigm for physics-guided 3D shape abstraction, level-of-detail editing, and interpretable generative modeling.

Related Papers

• [ICLR 2026] Latent Planning Emerges with Scale
• [ECCV 2026] Weight-Space Mixture-of-Experts for Implicit Neural Representation Classification
• [ECCV 2026] CFM: Language-aligned Concept Foundation Model for Vision
• [ICLR 2026] Evolution of Concepts in Language Model Pre-Training
• [ICLR 2026] Paradigm Shift of GNN Explainer from Label Space to Prototypical Representation Space