title: >- [Paper Note] GlassGS: Geometry and Concept-Aware 3D Gaussian Splatting for Reflective Enclosures description: >- [ECCV 2026][3D Vision][3D Gaussian Splatting] Proposes GlassGS, an iterative two-stage framework combining geometry-consistency perception and reflection-free concept learning to achieve high-fidelity 3DGS reconstruction of glass-enclosed objects. tags: - ECCV 2026 - 3D Vision - 3D Gaussian Splatting - Specular Reflection Removal - Concept Learning date: 2026-09-19 content_hash: 192d660545be9e18

GlassGS: Geometry and Concept-Aware 3D Gaussian Splatting for Reflective Enclosures¶

Conference: ECCV 2026
Paper: ECCV Official Link
Area: 3D Vision
Keywords: 3D Gaussian Splatting, High Specular Reflections, Geometry Consistency, Concept Learning, Reflective Enclosures

TL;DR¶

To tackle severe multi-view inconsistencies caused by high specular reflections on glass display cases, GlassGS presents an iterative two-stage framework coupling geometry-consistency perception with reflection-free concept learning, achieving artifact-free, high-fidelity 3D reconstruction on a dedicated benchmark dataset.

Background & Motivation¶

In cultural heritage preservation, virtual museum tours, augmented reality (AR), and robotic perception, precious artifacts and exhibits are predominantly housed behind transparent protective glass enclosures. While Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have demonstrated exceptional capabilities in standard novel-view synthesis, reconstructing objects enclosed in glass cases presents fundamental bottlenecks. High specular reflections (HSR) from smooth glass surfaces exhibit dramatic appearance shifts across viewing perspectives: a specific region can appear overwhelmingly bright due to directional flashes in one viewpoint, dim in another, or even superimpose virtual reflections of surrounding visitors and exhibits. This severe view-dependency inherently shatters the core multi-view appearance consistency assumption of 3DGS, forcing optimization routines to mistakenly interpret transient reflections as actual physical structures, thereby generating severe floaters, fractured surfaces, and geometric distortions.

Existing reflection-handling pipelines typically suffer from distinct shortcomings. Physically-based ray tracing or planar reflection priors fail when encountering non-planar enclosures, irregular geometry, or complex ambient illumination. Appearance-decomposition 3DGS variants struggle with severe residual artifacts due to insufficient explicit geometric supervision. Meanwhile, 2D reflection suppression and dynamic distractor removal approaches (such as single-image reflection removal or SpotLessSplats) merely treat specular flares as transient occluders; lacking multi-view 3D geometric cross-validation, they fail to disentangle semi-transparent reflections from underlying object surfaces, inevitably introducing blurred or compromised geometries into 3D space. Crucially, systematic investigation unveils two key properties of HSR: first, the "geometry-inconsistency property", where reflective regions exhibit extreme cross-view color variance; second, the "concept-preservation property", where despite severe local surface degradation, high-level semantic representations (such as global topology, category identity, and material concepts) captured by pretrained vision models remain remarkably robust.

Bridging these complementary physical and semantic insights, this paper departs from purely geometric or isolated 2D suppression heuristics to establish a closed-loop coupling of cross-view geometric inconsistency analysis and robust object concept priors. Core idea: by constructing pixel-wise geometry-consistency maps to explicitly identify and downweight specular reflections, and driving reflection-free textual concept inversion alongside geometry-aware ControlNet diffusion to restore authentic textures, an iterative self-cycling framework progressively suppresses specular flare artifacts while faithfully recovering the true 3D geometry and appearance of glass-enclosed objects.

Method¶

Overall Architecture¶

The GlassGS framework comprises two mutually reinforcing stages unified in an iterative feedback mechanism. Stage 1 is "Geometry-Consistency Perception (GCP)", which unprojects pixel observations into 3D world space, tracks cross-view color variance across adjacent camera poses to build pixel-wise geometry-consistency maps, and downweights reflection-corrupted regions in the 3DGS loss. Stage 2 is "Reflection-Free Concept Learning (RFCL)", which leverages Stable Diffusion textual inversion to extract an uncorrupted scene concept token and employs a 4-channel ControlNet (conditioned on rendered images concatenated with the consistency map) to synthesize clean, de-reflected views. Finally, a "Self-Cycling Refinement" process alternates 3DGS optimization with de-reflected synthesis until multi-view consistency converges.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multi-view Input Images + Camera Poses"] --> B["Geometry-Consistency Perception (GCP)<br/>3D projection & cross-view variance tracking"]
    B --> C["Reflection-Aware 3DGS Optimization<br/>Consistency-weighted loss suppressing artifacts"]
    C --> D["Render Current Viewpoint Images"]
    D --> E["Reflection-Free Concept Learning (RFCL)<br/>Textual inversion + 4-channel ControlNet de-reflection"]
    E --> F["Self-Cycling Refinement<br/>Joint optimization with de-reflected views & updated weights"]
    F -->|Unconverged / Consistency < Threshold| B
    F -->|Mean Consistency > Threshold| G["High-Fidelity Reflection-Free 3D Gaussians"]

Key Designs¶

1. Geometry-Consistency Perception: Pinpointing High Specular Reflections via Cross-View Projection Variance

Standard 3DGS assumes stable Lambertian appearance across views; when confronted with glass specular flares, the optimizer aggressively extrudes spurious Gaussians into free space. GCP addresses this failure mode by explicitly quantifying appearance variation across viewpoints. For pixel $x$ in image $I_i$ under camera pose $p_i$, it is back-projected to world coordinates $w_x$ via rendered depth $D_i$. The point $w_x$ is then tracked across adjacent viewpoints $P' \subset P$, evaluating the projected color $c(w_x, p_j)$. The geometric consistency score $\text{cons}_x$ is formulated as the mean similarity between observed colors across views: $$\text{cons}_x = \frac{1}{|P'|} \sum_{p_j \in P'} \text{Similarity}\left( c(w_x, p_j), \frac{1}{|P'|} \sum_{p_k \in P'} c(w_x, p_k) \right)$$ Pixel regions dominated by specular reflections experience drastic color shifts under minimal viewpoint rotation, yielding low $\text{cons}_x$ values, whereas genuine object surfaces sustain high consistency. Aggregating these values forms the full-resolution map $\text{Cons}_i$. During 3DGS parameter optimization, the consistency map weights the photometric and structural reconstruction losses: $$\mathcal{L}_{geo} = \lambda_1 \left\| \text{Cons}_i \odot (I_i - \hat{I}_i) \right\|_1 + \lambda_2 \text{D-SSIM}\left( \text{Cons}_i \odot I_i, \text{Cons}_i \odot \hat{I}_i \right)$$ where $\odot$ denotes element-wise multiplication. Downweighting inconsistent regions prevents spurious geometry expansion, ensuring that Gaussians are fitted solely to physically reliable observations.

2. Reflection-Free Concept Learning: Textual Inversion Disentanglement and Geometry-Aware ControlNet

Simply downweighting reflective zones leaves severely contaminated regions under-constrained, causing surface holes and missing textures. Grounded in the insight that specular noise does not corrupt global semantic concepts, RFCL first optimizes a reflection-free scene concept embedding $s_{\text{scene}}$ via textual inversion over corrupted patches. Governed by a frozen Stable Diffusion denoiser, this process captures underlying object topology, material identity, and global shape while filtering high-frequency reflection corruptions.

To inject these high-level semantics into local texture restoration without disturbing authentic geometry, a 4-channel geometry-aware ControlNet is introduced. The conditioning input $C$ concatenates the 3-channel reflection-contaminated rendered image $\hat{I}_{p_i}$ with the 1-channel geometry-consistency map $\text{Cons}_i$: $C = \text{Concat}(\hat{I}_{p_i}, \text{Cons}_i)$. To overcome the scarcity of paired 3D real-world data, the authors design a 2D Gaussian blur simulation on single images to mathematically approximate 3D spatial neighborhood color sampling, pretraining the ControlNet on 100,000 synthetic pairs to selectively reconstruct corrupted low-consistency zones while strictly preserving intact high-consistency textures.

3. Self-Cycling Refinement: Progressive Closed-Loop Consistency Convergence

Single-pass 2D reflection removal frequently introduces subtle geometric drift across perspectives, causing multi-view blurring. GlassGS resolves this with a four-step Self-Cycling Refinement loop: starting with initial contaminated images $I_0$ and initial consistency maps $C_0$, an initial field $G_0$ is optimized to render initial views $\hat{I}_0$. At iteration $k$, rendered views $\hat{I}_{k-1}$ pass through RFCL to produce de-reflected views $\tilde{I}_k$. Assigning newly de-reflected regions with complementary reliability weights $\tilde{C}_k = 1 - \text{Cons}_{k-1}$, the 3DGS scene is re-optimized by minimizing the joint reflection-aware loss: $$\mathcal{L}_{\text{refine}} = \mathcal{L}_{geo}(I_0, C_0) + \mathcal{L}_{geo}(\tilde{I}_k, \tilde{C}_k)$$ Finally, multi-view images $\hat{I}_k$ are re-rendered to update consistency maps $C_k$. As 3D Gaussians become cleaner, consistency detection becomes sharper; in turn, refined consistency maps yield superior de-reflected views. The loop terminates when the mean consistency satisfies $\text{Mean}(C_k) > \tau$ (stabilizing around 0.92 in practice) or reaches the iteration limit, yielding an artifact-free 3D representation.

Loss & Training¶

During ControlNet pretraining, latent variables $z_0 = \phi(I)$ are perturbed across diffusion timesteps $t$ to form $z_t$. Conditioning on text guidance $h_{\text{text}}$ and 4-channel control map $C$, the network minimizes noise prediction and latent alignment objectives: $$\mathcal{L}_{\text{control}} = \mathbb{E}_{z_0, t, C, \epsilon} \left[ \|\epsilon - \hat{\epsilon}\|_2^2 + \lambda_{x_0} \|z_0 - \hat{x}_0\|_2^2 \right]$$ During 3DGS optimization, standard learning rate schedules, densification, and opacity resetting are maintained. Across iterations, metrics show dramatic improvement across the first 3 cycles before plateauing, confirming rapid convergence.

Key Experimental Results¶

Main Results¶

Evaluation is conducted on the real-world GlassFlash3D benchmark, which comprises 23 sets of objects displayed behind glass cases (each featuring 80 glass-obstructed training images and 20 clean, glass-free ground-truth test images with target masks). Qualitative verification is also performed on the NeuS-HSR real-world dataset. Quantitative comparisons are summarized below:

Method	PSNR ↑	SSIM ↑	MAE ↓	Mask PSNR ↑	Mask SSIM ↑	Mask MAE ↓
Vanilla 3DGS	15.38	0.62	36.05	17.17	0.65	32.59
DSRNet + 3DGS	15.90	0.65	33.30	16.91	0.68	32.67
YTMT + 3DGS	15.89	0.64	33.32	17.10	0.66	33.21
SpotLessSplats	15.90	0.66	33.92	17.75	0.69	30.82
Ours (GlassGS)	15.90	0.68	31.99	18.94	0.71	24.68

Ablation Study¶

Systematic ablations on GlassFlash3D evaluate the contribution of individual framework components:

Config	PSNR ↑	SSIM ↑	MAE ↓	Mask PSNR ↑	Mask SSIM ↑	Mask MAE ↓	Note
Vanilla 3DGS	15.38	0.62	36.05	17.17	0.65	32.59	Baseline without reflection processing
Ours w/o GCP	15.90	0.62	33.44	17.86	0.65	28.22	Degrades to 2D reflection suppression without cross-view checks
Ours w/o RFCL	15.89	0.64	33.88	17.62	0.68	30.31	Lacks high-level semantic concept guidance
Ours w/o $\text{Cons}$ in Eq.(2)	15.29	0.62	36.73	17.44	0.65	30.15	Unweighted 3DGS loss allows reflections to corrupt geometry
Ours w/o $\text{Cons}$ in Eq.(3)	15.90	0.67	32.03	18.78	0.70	24.95	Concept inversion without consistency masking
Ours w/o $\mathcal{L}_{x_0}$ in Eq.(9)	15.75	0.62	34.33	17.70	0.65	28.81	Removing latent alignment drops generation fidelity
Ours (Full Model)	15.90	0.68	31.99	18.94	0.71	24.68	All components combined achieve superior object-centric fidelity

Key Findings¶

Substantial object-centric fidelity gains: Mask MAE drops markedly from 32.59 (Vanilla 3DGS) to 24.68 (a 24.3% reduction), and Mask PSNR increases by 1.77 dB, indicating that GlassGS successfully restores genuine object surface details while eliminating specular contaminations.
Critical role of GCP in geometric stability: Removing GCP causes Mask MAE to degrade to 28.22 and Mask SSIM to drop to 0.65. Completely omitting the consistency map from 3DGS loss (Eq. 2) worsens global MAE to 36.73 (worse than Vanilla 3DGS), underscoring that explicit geometric downweighting is vital to prevent geometry collapse.
Fast and reliable convergence: Iteration curves demonstrate rapid metric improvements across the first 3 iterations; as multi-view consistency stabilizes, the mean consistency metric converges to 0.92, preventing over-editing of pristine regions.

Highlights & Insights¶

Dual intrinsic properties of HSR: Formalizes the dichotomy between geometric inconsistency (cross-view color variance) and concept preservation (semantic invariance under heavy specular corruption), offering a principled foundation for reflection-robust neural rendering.
Equivalence of 2D Gaussian blur and 3D neighborhood projection: Ingeniously bridges 2D and 3D training data scarcity by establishing an analogy between 2D Gaussian smoothing and 3D neighbor sampling, enabling effective ControlNet pretraining on 100K synthetic pairs.
Closed-loop self-cycling refinement: Establishes a symbiotic flywheel between geometric perception and generative concept inpainting, where improved geometry purifies 2D synthesis, and refined 2D images reinforce 3D reconstruction.

Limitations & Future Work¶

Domain gap in complex real-world specular flares: While GlassFlash2D covers 100K synthetic composite pairs, real-world multi-source reflections and non-uniform glass curvatures can still lead to partial restoration imperfections.
Potential false-positive inconsistency detections: Anisotropic, view-dependent material highlights on genuine metallic or polished object surfaces may be misidentified as glass reflections, prompting unintended diffusion inpainting.
Future directions: Integrating polarization imaging cues or foundation-model segmentation masks into consistency map estimation holds strong promise for resolving ambiguous material highlights.

vs Vanilla 3DGS & Standard Radiance Fields: Vanilla 3DGS relies on spherical harmonics (SH) to capture view dependency, which fails under sharp specular reflections and produces fragmented floaters. GlassGS suppresses specular noise explicitly via geometry-consistency filtering.
vs SpotLessSplats: SpotLessSplats treats reflections as opaque dynamic distractors, whereas specular glass reflections are semi-transparent and alter ambient object lighting. GlassGS utilizes generative concept learning to recover obscured surface textures.
vs NeRFReN & NeuS-HSR: NeuS-HSR enforces strict planar reflection assumptions, failing on curved glass cases or ambient multi-source reflections. GlassGS relies purely on cross-view variance tracking, remaining robust across arbitrary enclosure geometries.

Rating¶

Novelty: ⭐⭐⭐⭐☆ Grounding reflection removal in the concept-preservation property and pairing it with geometric consistency perception is highly original.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Establishes the GlassFlash3D benchmark with real-world ground truth, accompanied by extensive baselines and ablation studies.
Writing Quality: ⭐⭐⭐⭐⭐ Rigorously organized with clear mathematical formulations and intuitive visual motivation.
Value: ⭐⭐⭐⭐☆ Highly valuable for museum digitalization, cultural relic archiving, and robot vision in glass-enclosed environments.