Sparse-View Surface Reconstruction using Gaussian Splatting through High-Confidence Depth Propagation with Normal Priors

Conference: ECCV 2026
Paper: ECCV Official
Code: https://hanl2010.github.io/DP-GS
Area: 3D Vision
Keywords: 3D Gaussian Splatting, Sparse-View Surface Reconstruction, Normal Prior, Depth Propagation, Edge-Aware Smoothness

TL;DR

Addressing ambiguous geometry and surface tearing caused by Gaussian discreteness under sparse views, this paper identifies high-confidence depth anchors via multi-view geometric and photometric consistency, iteratively propagates depths outward guided by normal priors, and introduces an abnormal depth edge-aware smoothing regularizer to reconstruct continuous, high-fidelity surfaces.

Background & Motivation

Reconstructing high-fidelity 3D geometric surfaces from sparse views (such as only three images with minimal overlap) remains a fundamental yet difficult problem in computer vision. While 3D Gaussian Splatting (3DGS) has achieved groundbreaking efficiency and quality in dense novel view synthesis, its surface reconstruction capability deteriorates rapidly when the input viewpoints are extremely sparse. Due to the scarcity of cross-view geometric cues and the unorganized, discrete nature of 3D Gaussians, relying solely on sparse photometric rendering losses creates an under-constrained optimization landscape. This leaves Gaussian primitives misplaced in free space, yielding corrupted rendered depths and fractured meshes.

Existing literature largely branches into generalizable models and scene-specific optimization. Generalizable frameworks demand expensive multi-view pretraining and suffer severe degradation whenever camera trajectories or scene distributions deviate from their training setups. Conversely, scene-specific optimization schemes relying on neural implicit representations (such as NeRF- or SDF-based architectures) suffer from sluggish convergence. Although recent 3DGS optimization attempts have sought guidance from monocular depth and normal cues, direct monocular depth estimation suffers from scale ambiguity and perspective distortion, failing to strictly align with the true 3D surface. Furthermore, monocular normals are conventionally restricted to direct normal regularization without unlocking their potential for spatial depth extrapolation. Compounding this challenge, in low-texture and occluded regions where multi-view consistency checks fail, Gaussians optimized with normal regularization alone tend to rotate toward the target normal but split along depth directions, creating abnormal depth steps and broken surfaces.

This paper tackles this issue with a fresh perspective: rather than blindly imposing monocular depth across the entire image, one can verify reliable depth anchors using multi-view geometric reprojection and photometric patch consistency, and then leverage monocular surface normals as local tangent planes to propagate reliable depth outward into uncertain, textureless regions. Core idea: identify high-confidence depth anchors via dual geometric-photometric consistency, iteratively propagate depths outward across local tangent planes guided by monocular normal priors, and enforce abnormal depth edge-aware smoothing using inverted normal edge masks to suppress discrete Gaussian tearing.

Method

Overall Architecture

The proposed framework reconstructs continuous 3D surfaces from sparse images initialized with sparse COLMAP point clouds. 3D Gaussians render color, depth, and surface normal maps at each viewpoint, alongside monocular normal priors inferred from a pretrained estimator (Metric3Dv2). The system first computes a depth confidence map by evaluating round-trip reprojection errors and patch-level normalized cross-correlation (NCC) between reference and source views, isolating high-confidence anchors via thresholding. Next, an iterative normal-guided depth propagation algorithm extends accurate depth values from confident anchors to surrounding low-confidence regions under local planarity constraints. Finally, an abnormal depth edge detector isolates non-physical depth discontinuities by filtering depth edges against inverted normal edges, applying an orthogonal tangential smoothing loss in 3D back-projected space.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Sparse Views Input<br/>COLMAP Point Cloud Initialization"] --> B["3DGS Differentiable Rendering<br/>RGB / Depth / Normal Maps"]
    B --> C["High-Confidence Depth Anchor Selection<br/>Geometric Reprojection + NCC Photometric Map"]
    C --> D["Normal-Guided Depth Propagation<br/>Iterative Local Tangent Plane Outward Extension"]
    B --> E["Abnormal Depth Edge Detection & Smoothing<br/>Depth Edges × Inverted Normal Edge Mask"]
    D --> F["Joint Optimization<br/>RGB Loss + Propagated Depth + Edge Smoothness + Normal Prior"]
    E --> F
    F --> G["Continuous High-Fidelity 3D Mesh Output"]

Key Designs

1. High-Confidence Depth Anchor Selection via Dual Consistency: Rejecting Unchecked Monocular Priors
Directly optimizing monocular depth priors introduces scale mismatch and geometric drift, while optimizing multi-view consistency losses directly under sparse views injects heavy gradient noise. This method instead leverages multi-view consistency purely as a confidence filter. For a pixel \(p_r(u_r, v_r)\) with rendered depth \(d_r\) in the reference view, depth-aware homography maps it to \(p_s\) in the source view, and back-projects it to \(p'_r\) via interpolated depth \(d_s\). The geometric consistency confidence is formulated as:

\[C_{geo} = \begin{cases} \frac{1}{e^{\|p_r - p'_r\|}}, & \text{if } \|p_r - p'_r\| \le 1 \\ 0, & \text{if } \|p_r - p'_r\| > 1 \end{cases}\]

Concurrently, a patch \(P_r\) centered at \(p_r\) is warped onto the source view as \(P_s\), and photometric consistency is quantified via Normalized Cross-Correlation: \(C_{pho} = \text{NCC}(P_r, P_s)\). The overall depth confidence is defined as the product \(C = C_{geo} \cdot C_{pho}\). Applying a threshold \(\tau\) produces a binary mask \(M_C = \mathbb{I}(C > \tau)\) that strictly preserves verified, high-precision surface anchors.

2. Normal-Guided Iterative Depth Propagation: Extrapolating Depth into Texture-Less Regions
Low-texture surfaces and non-overlapping regions cannot yield reliable multi-view consistency scores, leaving them vulnerable to depth drift. Guided by monocular normal priors \(\boldsymbol{n}\), this paper enforces local planarity to transfer depth analytically. Given a target pixel \(p_i\) and a neighboring pixel \(p_j\) with depth \(d_j\), assuming both lie on the same tangent plane with normal \(\boldsymbol{n}_i\), and given ray unit vectors \(\boldsymbol{r}_i\) and \(\boldsymbol{r}_j\) from the camera optical center, the geometric depth relationship is derived as:

\[d_i = \frac{\boldsymbol{n}_i^\top \boldsymbol{r}_j}{\boldsymbol{n}_i^\top \boldsymbol{r}_i} d_j\]

Filtered by the confidence mask \(M_C\), valid neighboring depth predictions are averaged at pixel \(p_i\). This propagation is executed iteratively for \(t \in [1, 10]\) steps, progressively expanding both the propagated depth \(d^{t+1}\) and the confidence mask \(M_C^{t+1}\). The newly confirmed regions supervise the 3DGS rendered depth via \(\mathcal{L}_{dp} = M_C^{t+1} |D - d^{t+1}|\), anchoring regions that previously lacked multi-view constraints.

3. Abnormal Depth Edge-Aware Smoothing Regularization: Eliminating Discrete Gaussian Tearing
Due to curvature variations and propagation range limits, local planar propagation cannot extend indefinitely across distant or occluded zones. In such under-constrained regions, regularizing surface normals alone encourages Gaussians to orient properly while shearing along depth rays, producing step-like tears where depth jumps abruptly despite continuous normals. To suppress these artifacts, the paper extracts edges from rendered depth and normal prior maps, multiplying the depth edge mask by the inverted normal edge mask. Because genuine physical silhouettes exhibit simultaneous depth and normal discontinuities, this product precisely isolates non-physical depth cracks. For sampled edge points \(i\) and neighbor \(j\), their back-projected 3D spatial vector \(\boldsymbol{v}_{ij} = \boldsymbol{a}_j - \boldsymbol{a}_i\) is penalized against normal \(\boldsymbol{n}_i\):

\[\mathcal{L}_{ds} = \frac{1}{N} \sum_{i,j \in \mathcal{N}} |\boldsymbol{n}_i^\top \boldsymbol{v}_{ij}|\]

This drives adjacent Gaussians to align seamlessly along the tangent plane, eliminating mesh fragmentation.

Loss & Training

The overall training loss balances rendering fidelity and geometric constraints:

\[\mathcal{L} = \mathcal{L}_{rgb} + \lambda_1 \mathcal{L}_{dp} + \lambda_2 \mathcal{L}_{ds} + \lambda_3 \mathcal{L}_{normal}\]

where \(\mathcal{L}_{rgb}\) combines photometric \(\ell_1\) and D-SSIM losses, and \(\mathcal{L}_{normal}\) is the \(\ell_1\) distance between rendered normals and Metric3Dv2 predictions. The implementation builds upon the differentiable Gaussian rasterizer of PGSR, incorporating Gaussian opacity decay to prune floating artifacts near surfaces. Optimization runs for 10K iterations on a single RTX 3090 GPU in approximately 10 minutes. Hyperparameters are set to \(\tau = 0.6\), \(t = 10\), \(\lambda_1 = 0.5\), \(\lambda_2 = 0.03\), and \(\lambda_3 = 0.1\).

Key Experimental Results

Main Results

Evaluation is conducted on the DTU dataset under the challenging 3 small-overlapping views setting across 15 scans (reporting Chamfer Distance, CD ↓ in mm) and on the Tanks and Temples (TNT) dataset across 360-degree scenes (reporting F1 score ↑).

Table 1: Quantitative Chamfer Distance (CD ↓, mm) comparison on DTU with 3 small-overlapping views (excerpt from original Table 1; # denotes initialization using dense point clouds from MASt3R):

Category	Method	Mean CD ↓	Scan 24	Scan 65	Scan 110
Dense 3DGS Baselines	2DGS (SIGGRAPH 2024)	2.45	3.54	2.35	2.97
Dense 3DGS Baselines	PGSR (CVPR 2024)	2.38	4.01	2.84	1.91
Generalizable Sparse	SparseNeuS-ft (ECCV 2022)	3.34	4.81	3.88	3.12
Generalizable Sparse	VolRecon (CVPR 2023)	3.02	3.05	3.68	2.77
Generalizable Sparse	UFORecon (CVPR 2024)	1.40	1.51	1.81	0.93
Scene-Specific Implicit	NeuSurf (3DV 2024)	1.35	1.35	2.35	1.21
Scene-Specific Implicit	SparseRecon (ECCV 2024)	1.06	1.26	2.38	0.77
Scene-Specific 3DGS	FatesGS (ECCV 2024)	1.37	1.32	2.08	0.85
Scene-Specific 3DGS	MAtCha (NeurIPS 2024)	1.48	1.38	1.58	1.24
Scene-Specific 3DGS	MAtCha# (MASt3R init)	1.16	0.88	1.35	1.02
Ours	Ours (COLMAP init)	1.02	1.45	1.09	0.61
Ours	Ours# (MASt3R init)	0.85	0.57	1.11	0.65

Table 2: Quantitative F1 score comparison on Tanks and Temples (TNT) with sparse viewpoints (excerpt from original Table 2; MASt3R initialization):

Method	5 Views	10 Views	20 Views
FatesGS	0.014	0.025	0.033
MAtCha	0.072	0.156	0.218
Ours	0.081	0.173	0.293

Ablation Study

Ablation on DTU with 3 small-overlapping views examining normal prior loss \(\mathcal{L}_{normal}\), opacity decay (O.D.), depth propagation \(\mathcal{L}_{dp}\), and abnormal depth edge smoothing \(\mathcal{L}_{ds}\) (excerpt from original Table 4):

Config Index	\(\mathcal{L}_{normal}\)	Opacity Decay (O.D.)	Depth Propagation \(\mathcal{L}_{dp}\)	Edge Smoothing \(\mathcal{L}_{ds}\)	Mean CD ↓ (mm)	Note
(1) Baseline	-	-	-	-	2.79	Color loss only; collapsed geometry
(2)	✓	-	-	-	2.28	Normal smoothing with heavy distortion
(3)	✓	✓	-	-	1.55	Reduces floaters; coarse surfaces remain
(4)	✓	✓	✓	-	1.13	Significant gain in low-texture regions
(5)	✓	✓	-	✓	1.38	Reduces tearing, lacks accurate depth anchors
(6) Full Model	✓	✓	✓	✓	1.02	Optimal synergy across all modules

Key Findings

• Depth propagation provides the primary performance boost: comparing Config (3) and (4), adding normal-guided depth propagation reduces mean CD from 1.55 mm to 1.13 mm (a 27.1% error reduction), showing that extending anchor depths fundamentally rectifies under-constrained zones.
• Edge-aware smoothing specifically removes mesh tearing: while \(\mathcal{L}_{ds}\) refines global CD from 1.13 mm to 1.02 mm, qualitative inspection (original Fig. 9 & 10) confirms that without \(\mathcal{L}_{ds}\), meshes exhibit sharp cracks and fragmentations where local planar assumptions reach their spatial limits.
• Iteration efficiency and prior robustness: CD improves steadily as propagation steps \(t\) rise from 4 to 10 (1.126 mm to 1.021 mm) and plateaus beyond 10 steps (20 steps consume a negligible 0.008 s). Furthermore, evaluating against different normal estimators demonstrates consistent performance: even with older Omnidata normals, the method attains a 1.14 mm CD, highlighting strong algorithmic robustness.

Highlights & Insights

• Repurposing multi-view consistency from optimization loss to confidence gating: in dense reconstruction, photometric and reprojection errors serve as direct losses; under sparse views, however, direct optimization injects noisy gradients. Demoting consistency to an anchor filter prevents bad gradients while providing unshakeable depth seeds.
• Isolating non-physical artifacts via inverted mask multiplication: multiplying depth edges by inverted normal edges offers an elegant, closed-form geometric trick to isolate unnatural step discontinuities from true object silhouettes without neural segmentation heuristics.
• Transferable normal-guided depth propagation: surface normals are scale-invariant and less prone to monocular distortion than monocular depth maps. Leveraging normals as local geometric bridges to extrapolate sparse verified depths is highly transferable to sparse LiDAR-camera fusion and indoor RGB-D completion.

Limitations & Future Work

• Breakdown of initial confidence in untextured regions: confidence scores rely on photometric NCC and geometric reprojection. On expansive, untextured surfaces (e.g. solid white walls), NCC scores can become unreliable or produce false-positive matches, propagating corrupt depth seeds into adjacent regions.
• Local planarity assumption on sharp geometric features: analytical propagation assumes local co-planarity. On highly intricate, thin structures or complex foliage, propagation can over-smooth sharp geometric features.
• Future directions: developing curvature-aware non-local propagation weights or integrating monocular curvature priors could help dynamically attenuate propagation around fine boundaries.

Related Work & Insights

• vs PGSR / 2DGS: dense 3DGS methods struggle under sparse small-overlapping viewpoints, producing disconnected fragments and high error (DTU CD of 2.38 mm and 2.45 mm); the proposed propagation and edge smoothing suppress these artifacts, lowering CD to 1.02 mm.
• vs SparseRecon / NeuSurf: implicit neural surface approaches deliver accurate geometry but require several hours of optimization (3.5 hours for SparseRecon); this method requires only ~10 minutes and 3 GB memory, combining 3DGS speed with state-of-the-art geometric fidelity.
• vs MAtCha / FatesGS: existing sparse 3DGS works suffer from incomplete meshes and severe tearing on 360-degree scenes (TNT 5-view F1 of 0.014 for FatesGS and 0.072 for MAtCha); our approach achieves 0.081 (5 views) and 0.293 (20 views), yielding smooth and complete reconstructions.

Rating

• Novelty: ⭐⭐⭐⭐☆ (Ingenious integration of normal-guided depth propagation and inverted edge mask filtering)
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Comprehensive evaluations across DTU 15 scenes, TNT 360-degree scenes, real self-captured scans, and detailed component ablations)
• Writing Quality: ⭐⭐⭐⭐⭐ (Clean narrative arc, rigorous mathematical formulas, and intuitive pipeline design)
• Value: ⭐⭐⭐⭐☆ (Highly practical blueprint for stabilizing 3DGS geometry under extreme viewpoint scarcity)

Related Papers

• [ECCV 2026] Confidence-Based Mesh Extraction from 3D Gaussians
• [CVPR 2026] Confidence-Guided Multi-Scale Aggregation for Sparse-View High-Resolution 3D Gaussian Splatting
• [AAAI 2026] MeshSplat: Generalizable Sparse-View Surface Reconstruction via Gaussian Splatting
• [CVPR 2026] 3D Gaussian Splatting with Self-Constrained Priors for High Fidelity Surface Reconstruction
• [AAAI 2026] SparseSurf: Sparse-View 3D Gaussian Splatting for Surface Reconstruction