Denoising-GS: Gaussian Splatting with Spatial-aware Denoising

Conference: ECCV 2026
Paper: ECCV Official Link
Code: https://github.com/
Area: 3D Vision
Keywords: 3D Gaussian Splatting, Spatial-aware Denoising, Momentum-biased Stochastic Exploration, Fisher Information Uncertainty, Novel View Synthesis

TL;DR

Denoising-GS reformulates 3D Gaussian Splatting optimization as a spatial-aware primitive denoising process, simultaneously refining mean positions and covariance structures while leveraging Fisher information uncertainty pruning to achieve SOTA novel view synthesis with 10%+ fewer primitives.

Background & Motivation

3D Gaussian Splatting (3DGS) has revolutionized novel view synthesis (NVS) by enabling real-time, photorealistic radiance field rendering through explicit anisotropic Gaussian primitives. Because of its compelling balance between rendering fidelity and efficiency, 3DGS has been rapidly extended to surface reconstruction, SLAM, 3D generative modeling, and human body capture. However, 3DGS relies inherently on sparse and incomplete point clouds from Structure-from-Motion (SfM) for initialization. Standard SGD-based optimization easily falls into suboptimal local minima, generating abundant noisy Gaussian primitives and floating artifacts that severely impair structural fidelity.

Recent methods like 3DGS-MCMC attempt to escape local minima by introducing opacity-guided relocation alongside Stochastic Gradient Langevin Dynamics (SGLD) perturbations. Nevertheless, these stochastic updates introduce two critical shortcomings: first, vanilla SGLD relies solely on instantaneous gradients and white noise perturbations, exhibiting severe oscillations and sluggish convergence around high-curvature regions and saddle points; second, SGLD perturbs only the mean positions while leaving the covariance (spatial structure) untouched. As a consequence, positional translations along an axis are frequently counteracted by concurrent scaling expansions along the same axis, enlarging noisy regions rather than denoising them.

This paper bridges 3DGS optimization with 3D point cloud denoising, observing that noisy Gaussian primitives scattered around true surfaces require joint refinement of both their centroids and their spatial extents. Core idea: reformulate 3DGS optimization as a spatial-aware Gaussian primitive denoising process, introducing momentum-biased stochastic exploration to maintain optimization flow, local coordinate gradient denoising to harmonize mean and scaling updates, and Fisher-information uncertainty pruning paired with spatial coherence refinement to ensure a compact, high-fidelity scene representation.

Method

Overall Architecture

Starting from noisy Gaussian primitives initialized by SfM, Denoising-GS executes a progressive multi-stage denoising pipeline. First, momentum-biased stochastic exploration injects exponentially decaying historical gradients into the Langevin dynamics, enabling directed, coherent exploration across saddle points. Next, the spatial gradient-based denoising module transforms mean gradients into each Gaussian's local coordinate system to detect alignment between mean movement and principal scaling expansion, applying an adaptive denoising displacement. Finally, an uncertainty-based denoising stage approximates the Fisher Information Matrix to prune low-confidence redundant primitives, followed by a spatial coherence refinement strategy that selectively splits Gaussians in under-represented sparse regions via \(k\)-nearest-neighbor metrics.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Noisy Initial Primitives<br/>Sparse SfM Point Clouds"] --> B["Momentum-biased Stochastic Exploration<br/>Directed exploration with gradient EMA"]
    B --> C["Spatial Gradient-based Denoising<br/>Local frame alignment & displacement offset"]
    C --> D["Uncertainty-based Denoising<br/>Singular-value pruning via Fisher Information"]
    D --> E["Spatial Coherence Refinement<br/>Selective densification via k-NN distance"]
    E --> F["Compact High-Fidelity 3DGS<br/>Noise-free novel view synthesis"]

Key Designs

1. Momentum-biased Stochastic Exploration: Stabilizing Exploration with Historical Trajectory Flow Vanilla SGLD updates in 3DGS-MCMC rely exclusively on instantaneous gradients and random noise, often yielding noisy, erratic trajectories that slow down convergence near complex geometry. To stabilize stochastic exploration while retaining the asymptotic correctness of the sampling process, this module incorporates an exponential moving average (EMA) momentum vector \(\mathbf{m}_t\): $\(\mathbf{m}_t = \beta_1 \mathbf{m}_{t-1} + (1 - \beta_1) \nabla_{\mathcal{M}_{t-1}}\)$ The Gaussian centroid \(\mathcal{M}_t\) is then updated with both Langevin stochastic diffusion and a momentum drift offset scaled by bias factor \(\alpha\): $\(\mathcal{M}_{t+1} = \mathcal{M}_t - \lambda_t \nabla_{\mathcal{M}_t} \mathcal{L}_{\mathrm{total}} + \sqrt{2 \lambda_t \tau} \boldsymbol{\xi}_t - \alpha \lambda_t \mathbf{m}_t\)$ where \(\boldsymbol{\xi}_t \sim \mathcal{N}(0, \mathbf{I})\) denotes injected Gaussian noise and \(\tau\) is the temperature parameter. By endowing the exploration with directional inertia, the primitives transition smoothly across energy barriers rather than suffering from random Brownian jitter.

2. Spatial Gradient-based Denoising: Coordinated Updates in the Local Gaussian Frame In standard 3DGS, the gradients of the centroid \(\mathcal{M}\) and scaling factors \(\mathbf{S} = \mathrm{diag}(s_x, s_y, s_z)\) are evaluated independently. When a primitive shifts along an axis while simultaneously expanding along that same axis, the outer boundary expands into empty space, amplifying noisy regions. To prevent this destructive competition, this design first rotates the global mean gradient into the local coordinate system of the Gaussian via rotation matrix \(\mathbf{R}\): $\(\nabla \mathcal{M}^G = \mathbf{R}^\top \nabla \mathcal{M}\)$ It then identifies the dominant principal axis \(j \in \{x, y, z\}\) for both the local mean gradient \(\nabla \mathcal{M}^G\) and the scaling gradient \(\nabla \mathbf{S}\). When these dominant axes coincide, a directional denoising term \(\Delta \mu\) is synthesized along that axis: $\(\Delta \mu = |\nabla \mathbf{S}_j| \cdot \mathrm{sign}(\nabla \mathcal{M}_j^G)\)$ The updated mean position incorporates this correction rotated back to the world frame: $\(\mathcal{M}_{t+1} = \mathcal{M}_t - \lambda_t \nabla_{\mathcal{M}_t} \mathcal{L}_{\mathrm{total}} + \Delta D + \beta_2 \mathbf{R} \Delta \mu\)$ where \(\Delta D = \sqrt{2 \lambda_t \tau} \boldsymbol{\xi}_t - \alpha \lambda_t \mathbf{m}_t\) encapsulates the exploration term, and \(\beta_2\) controls denoising strength. This formulation actively counteracts superfluous dilation, pulling primitives toward the true underlying surface.

3. Uncertainty-based Denoising and Spatial Coherence Refinement: Pruning Redundancy and Densifying Sparse Cavities Even with coordinated gradient descent, many primitives contribute negligibly to view synthesis or represent low-confidence noise. Simply pruning by opacity thresholds risks carving holes in low-texture regions. To overcome this, the framework estimates primitive-wise uncertainty using Fisher Information. For primitive parameters \(\theta_i = \{\mathcal{M}_i, \mathbf{S}_i\}\), the Fisher Information Matrix is approximated via the outer product of multi-view \(L_2\) rendering gradients: $\(\mathbf{F}_i = \sum_{v \in \mathcal{V}} \nabla_{\theta_i} \mathcal{L}_v (\nabla_{\theta_i} \mathcal{L}_v)^\top\)$ The total information magnitude \(u_i\) is computed by summing the six singular values obtained from singular value decomposition (SVD) of \(\mathbf{F}_i\): $\(u_i = \sum_{j=1}^{6} \sigma_j(\mathbf{F}_i)\)$ Primitives with minimal \(u_i\) carry low information sensitivity and are pruned as uncertain noise. Subsequently, the spatial coherence refinement step computes the mean Euclidean distance \(d_i\) from each primitive centroid \(\mathbf{x}_i\) to its \(k\) nearest neighbors: $\(d_i = \frac{1}{k} \sum_{j=1}^{k} \| \mathbf{x}_i - \mathbf{x}_{i, j} \|_2\)$ Primitives exhibiting high \(d_i\) reside in under-represented, sparse regions and are selectively split and densified, ensuring geometric completeness without bloating the overall primitive count.

Loss & Training

The framework optimizes the standard composite photometric loss combining \(L_1\) color error and D-SSIM: \(\mathcal{L}_{\mathrm{total}} = (1 - \lambda_{\mathrm{sim}}) \mathcal{L}_1 + \lambda_{\mathrm{sim}} \mathcal{L}_{\mathrm{D-SSIM}}\). Key hyperparameters are set to \(\alpha = 0.05\), \(\beta_1 = 0.9\), \(\beta_2 = 0.5\), and \(k = 3\). Uncertainty-based denoising prunes the top 10% most uncertain primitives, while spatial coherence refinement is executed 5 times throughout training, each time splitting the top 0.05% most sparse primitives. Optimization runs for 35K iterations on Mip-NeRF 360 and 30K iterations on Tanks and Temples and Deep Blending on an NVIDIA H100 GPU.

Key Experimental Results

Main Results

Quantitative evaluations across three standard benchmarks (Mip-NeRF 360, Tanks and Temples, and Deep Blending) demonstrate that Denoising-GS achieves state-of-the-art performance against 10 modern 3DGS baselines (Table 1 in original paper):

Dataset	Metric	Denoising-GS (Ours)	3DGS-MCMC (Prev. SOTA)	Vanilla 3DGS	Gain / Margin
Mip-NeRF 360	PSNR ↑ / SSIM ↑ / LPIPS ↓	29.925 / 0.897 / 0.142	29.893 / 0.896 / 0.186	29.258 / 0.880 / 0.168	+0.032 dB / LPIPS improved by 0.044
Tanks and Temples	PSNR ↑ / SSIM ↑ / LPIPS ↓	24.561 / 0.868 / 0.149	24.290 / 0.860 / 0.190	23.649 / 0.845 / 0.178	+0.271 dB / SSIM +0.008 / LPIPS improved by 0.041
Deep Blending	PSNR ↑ / SSIM ↑ / LPIPS ↓	29.872 / 0.906 / 0.239	29.665 / 0.895 / 0.320	29.475 / 0.900 / 0.247	+0.207 dB / SSIM +0.011 / LPIPS improved by 0.081

Regarding representation compactness on the Mip-NeRF 360 benchmark (Table 2 in original paper): - Vanilla 3DGS uses an average of 3.004M primitives with 29.258 dB PSNR; - 3DGS-MCMC requires 3.093M primitives with 29.893 dB PSNR; - Mip-Splatting and Pixel-GS expand to 3.783M and 4.684M primitives respectively; - Denoising-GS requires only 2.789M primitives on average—saving more than 10% in model size compared to 3DGS-MCMC while reaching a higher PSNR of 29.925 dB (e.g., Bicycle drops from 5.90M to 5.32M, and Counter drops from 1.20M to 1.08M).

Ablation Study

The paper conducts extensive ablation studies across coefficients and pruning/densification settings across all three benchmarks (Table 3 in original paper):

Configuration & Hyperparameters	Prune %	Densify %	Mip-NeRF 360 (PSNR/SSIM/LPIPS)	Tanks & Temples (PSNR)	Note
\(\alpha=0.05, \beta_1=0.9, \beta_2=0.5\) (Default)	10%	0.05%	29.925 / 0.897 / 0.142	24.561	Best balanced full framework
\(\alpha=0.05, \beta_1=0.9, \beta_2=0.1\)	10%	0.05%	29.930 / 0.897 / 0.142	24.606	Weaker gradient denoising term
\(\alpha=0.05, \beta_1=0.9, \beta_2=1.0\)	10%	0.05%	29.924 / 0.897 / 0.142	24.499	Aggressive denoising leads to over-shrinking
\(\alpha=0.1, \beta_1=0.9, \beta_2=0.5\)	10%	0.05%	29.901 / 0.897 / 0.143	24.458	Excessive momentum slows down discovery
Default hyperparameters	20%	0.05%	29.864 / 0.896 / 0.144	24.496	Over-pruning damages subtle thin structures
Default hyperparameters	10%	0.1%	29.900 / 0.897 / 0.143	24.521	Over-densification introduces redundancy

Key Findings

• Substantial Perceptual Quality Gain: Across all benchmarks, LPIPS exhibits substantial drops (e.g., from 0.186 to 0.142 on Mip-NeRF 360, and from 0.320 to 0.239 on Deep Blending), confirming that spatial-aware denoising effectively eliminates floating artifacts and blur.
• Harmonious Cooperation of Momentum and Gradient Denoising: Adding momentum prevents wandering near saddle points, while local principal axis alignment eliminates the conflicting dynamics between positional shift and scaling spread.
• Balanced 10% Pruning and 0.05% Refinement: Pruning approximately 310K uncertain primitives and compensating with 5.6K well-targeted points per scene produces the optimal tradeoff between memory footprint and textureless surface fidelity.

Highlights & Insights

• Local Frame Gradient Alignment: Rotating global mean gradients into the Gaussian ellipsoid's principal coordinate system directly tackles the longstanding blind spot where translation along an axis is counteracted by scaling expansion along the same axis.
• Fisher Information via SVD Singular Values: Approximating the intractable Fisher Information expectation via rendering loss gradient outer products and summing the six singular values provides a principled, physical uncertainty metric for 3DGS.
• Pruning-Guided Compactness: Demonstrates that intelligent spatial denoising and sparse replenishment produce superior visual quality with 10%+ fewer Gaussian primitives, avoiding brute-force primitive inflation.

Limitations & Future Work

• Periodic Computational Overhead: Evaluating parameter gradients and computing SVD for the Fisher Information matrix across millions of primitives incurs temporary computation and GPU memory spikes during pruning iterations.
• Static Neighborhood Heuristic: The spatial coherence refinement relies on a fixed \(k=3\) nearest-neighbor distance, which does not adapt to non-uniform camera trajectory densities across foreground and distant background.
• Future Directions: Exploring lightweight online Fisher approximations and adapting spatial-aware denoising to dynamic, deformable 3D Gaussian Splatting scenarios.

Related Work & Insights

• vs 3DGS-MCMC: While 3DGS-MCMC pioneered SGLD stochastic noise and opacity-based relocation to escape local minima, its perturbations are purely random and strictly positional. Denoising-GS adds momentum memory and jointly denoises positions and scaling extents in the local reference frame.
• vs Pixel-GS / AbsGS: While Pixel-GS and AbsGS analyze gradient magnitudes to guide densification and split oversized primitives, they do not coordinate the geometric interactions between centroid translation and scaling expansion along principal axes.
• vs PUP 3D-GS / FisherRF: PUP 3D-GS uses Fisher Information only for post-training compression, and FisherRF uses it for active next-best-view selection. Denoising-GS integrates Fisher uncertainty directly into the in-training primitive denoising loop.

Rating

• Novelty: ⭐⭐⭐⭐☆ (Insightful reformulation of 3DGS optimization as joint spatial denoising with elegant local frame gradient alignment)
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Comprehensive validation across three standard benchmarks with detailed ablation of hyperparameters and primitive counts)
• Writing Quality: ⭐⭐⭐⭐⭐ (Clear problem definition, clean mathematical formulation, and well-designed explanatory diagrams)
• Value: ⭐⭐⭐⭐☆ (Offers an effective, lightweight optimization paradigm for generating compact, artifact-free 3D Gaussian representations)

Related Papers

• [ECCV 2026] JanusMesh: Fast and Zero-Shot 3D Visual Illusion Generation via Cross-Space Denoising
• [ECCV 2026] StructSplat: Generalizable 3D Gaussian Splatting from Uncalibrated Sparse Views
• [ECCV 2026] AirSplat: Alignment and Rating for Robust Feed-Forward 3D Gaussian Splatting
• [ECCV 2026] MVGS: Multi-view Regulated Gaussian Splatting for Novel View Synthesis
• [ECCV 2026] Geometry-Aware Style Transfer in 3D Gaussian Splatting