Tackling Misattribution in 3D Intrinsic Decomposition via Proximity Attention Point Rendering¶

Conference: ECCV 2026
Paper: ECCV Original
Area: 3D Vision
Keywords: 3D Intrinsic Decomposition / Neural Rendering / Proximity Attention Point Rendering / Inverse Rendering / 3D Appearance Editing

TL;DR¶

Reveals the root cause of the misattribution issue in point-based intrinsic decomposition due to volume rendering aggregation over translucent primitives, and proposes Intrinsic PAPR, which leverages Proximity Attention Point Rendering for direct per-point supervision along with cIMLE ambiguity-aware priors and space carving loss to achieve multi-view consistent, high-fidelity, and editable 3D representations.

Background & Motivation¶

Recent breakthroughs in point-based neural scene representations, notably 3D Gaussian Splatting (3DGS), have dramatically improved the quality and speed of novel view synthesis. To support downstream applications that require fine-grained control over material properties and illumination, subsequent research has explored 3D intrinsic decomposition and inverse rendering, aiming to disentangle scenes into explicit albedo and shading components. However, when users perform localized point-level edits—such as copying learned albedo features from a source region to a target surface—the resulting rendered target appearance often deviates drastically from the source, exhibiting severe color drift, blotchy artifacts, and view-dependent inconsistencies.

This unexpected failure exposes a critical flaw: individual scene primitives learn incorrect, degenerate appearance attributes despite producing plausible aggregated renderings, a phenomenon the authors term the "misattribution issue." The root cause lies squarely in the volume rendering aggregation formulation ubiquitous in splatting and neural radiance field methods. Because splatting primitives are constrained to ellipsoidal footprints, they must remain semi-transparent and overlap along camera rays to model opaque, non-ellipsoidal surfaces. Volume rendering then composites transmittance-weighted features across all primitives along each ray, supervising only the final aggregated pixel color. Consequently, individual primitives are free to adopt erroneous, mutually compensating values so long as their weighted sum matches the ground truth. Crucially, capturing more training viewpoints cannot resolve this ambiguity, as spatially co-located primitives maintain near-identical relative depths and transmission weights from all camera angles.

To eliminate this ill-posed compensatory behavior, the authors recognize that neural intrinsic decomposition requires a fundamental paradigm shift away from volumetric accumulation toward surface-aligned, single-point dominance. Core idea: repurpose Proximity Attention Point Rendering (PAPR) to replace semi-transparent volume rendering with opaque surface-ray intersection predictions, enabling direct per-point photometric supervision that eliminates misattribution, combined with a cIMLE-adapted multi-hypothesis 2D albedo prior and a mode-selecting space carving loss to resolve single-view ambiguities into a unified 3D representation.

Method¶

Overall Architecture¶

Intrinsic PAPR represents a 3D scene using a parsimonious set of surface points $\mathcal{P} = \{(p_k, a_k, h_k, \tau_k)\}_{k=1}^N$, where each point stores its 3D coordinates $p_k$, an albedo feature vector $a_k$, a shading feature vector $h_k$, and a scalar influence score $\tau_k$. For a camera ray $r_{ij}$, the system retrieves a compact set of $K$ nearest points $\mathcal{N}(r_{ij})$ based on perpendicular ray distance. Proximity-based attention scores are computed and softmax-normalized to weight the point features into separate albedo and shading feature maps, which are then passed through lightweight convolutional rendering heads to synthesize the decoupled albedo image and the composite color image.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multi-view RGB images and ray rij"] --> B["Proximity Attention Point Selection<br/>Retrieve Top-K points & compute softmax weights"]
    B --> C["Orthogonal Feature Aggregation & Decoding<br/>Render distinct albedo & shading feature maps"]
    C --> D["cIMLE Ambiguity-Aware Prior Modeling<br/>Sample M plausible pseudo-ground-truth albedos"]
    D --> E["Space Carving Loss & Scale Alignment<br/>Mode-selecting distillation with per-view scalar"]
    E --> F["Consistent Decoupled 3D Scene<br/>Supports robust albedo/shading editing & NVS"]

Key Designs¶

1. Proximity Attention Point Rendering: restoring per-primitive identifiability Unlike volume rendering that blurs supervision across multiple depth samples, PAPR computes ray-surface intersection features using learned proximity attention over spatial points without explicit volume thickness. For ray $r_{ij}$ and candidate point $p_k \in \mathcal{N}(r_{ij})$, the normalized weight is calculated as: $$s_{ijk} = f_\theta(p_k, r_{ij}, \tau_k), \quad w_{ijk} = \frac{\exp(s_{ijk})}{\sum_{k' \in \mathcal{N}(r_{ij})} \exp(s_{ijk'})}$$ where $f_\theta$ is a learned scoring function conditioned on positional encodings and point influence $\tau_k$. Albedo and shading feature maps are aggregated via $A_{ij}^C = \sum_{k} w_{ijk} a_k$ and $S_{ij}^C = \sum_{k} w_{ijk} h_k$. Because the proximity attention weight decays rapidly with perpendicular distance, each surface point possesses a non-trivial viewing cone of rays where its normalized weight approaches unity ($w_{ijk} \approx 1$). Photometric reconstruction residuals backpropagate almost exclusively into that single dominant point, providing direct per-primitive supervision and mechanically precluding the compensatory feature errors that plague volume rendering.

2. Minimalist Intrinsic Factorization: implicit shading as a geometric regularizer While many state-of-the-art inverse rendering techniques optimize complex SVBRDF parameters, specular lobes, and incident lighting, these formulations are severely under-constrained in casual multi-view captures and frequently bake residual shadows into the albedo. The authors intentionally adopt the simpler intrinsic formulation $I \approx A \odot S$. In this framework, supervising the reconstructed color $I$ and the decoupled albedo $A$ algebraically determines the shading $S$ without requiring a dedicated shading objective. This design prevents numerical instabilities in near-black regions ($A \approx 0$) where direct shading supervision is ill-posed. In turn, multi-view consistency on $I$ and $A$ forces the implicit shading to capture true illumination and acts as an effective geometric regularizer, yielding cleaner surface reconstructions.

3. Ambiguity-Aware Prior via cIMLE: capturing multi-hypothesis albedos Single-view 2D intrinsic decomposition models inherently suffer from monocular ambiguities—for instance, dark spots on a surface could represent either geometric cast shadows or painted texture patterns. Deterministic 2D prior models produce conflicting single predictions across different views that break 3D multi-view fusion. To resolve this, the authors adapt a pre-trained 2D diffuse intrinsic network into an ambiguity-aware probabilistic generator using conditional Implicit Maximum Likelihood Estimation (cIMLE). By conditioning on sampled latent vectors $z \sim \mathcal{N}(0, I)$ via AdaIN layers, the prior yields $M$ distinct, plausible pseudo-ground-truth albedo hypotheses $\{p_{ijk}^{*v}\}_{k=1}^M$ for each training view $v$. Furthermore, to handle the multiplicative scale ambiguity intrinsic to reflectance estimation ($A \odot S \equiv (\gamma A) \odot (S/\gamma)$), a learnable per-view scalar $\lambda_v$ is jointly optimized to align the global scale before loss evaluation.

4. Space Carving Loss: probabilistic multi-view mode selection Rather than averaging the multiple 2D albedo hypotheses (which would blur sharp reflectance boundaries and flatten textures), the authors formulate a mode-selection objective inspired by space carving for ambiguous depth: $$\mathcal{L}_{\text{SC}} = \sum_{(i,j) \in \Omega} \min_{k \in \{1,\dots,M\}} \left\| \hat{p}_{ij} - p_{ijk}^* \right\|_2^2$$ where $\hat{p}_{ij}$ denotes the predicted pixel albedo and $p_{ijk}^*$ is the $k$-th scaled hypothesis under $\lambda_v$. This loss penalizes deviations against the closest valid mode per pixel, effectively allowing multi-view geometric consistency to "carve away" inconsistent 2D hypotheses across intersecting rays. The spatial continuity of the lightweight albedo convolutional decoder ensures local smoothness across selected modes without requiring ad-hoc smoothness regularizers.

Loss & Training¶

The framework is trained end-to-end using the combined objective: $$\mathcal{L} = \mathcal{L}_{\text{recon}} + w_{\text{SC}} \mathcal{L}_{\text{SC}}$$ where the reconstruction loss $\mathcal{L}_{\text{recon}}$ optimizes synthesized RGB colors against ground truth using a weighted combination of Mean Squared Error and LPIPS: $$\mathcal{L}_{\text{recon}} = \alpha \cdot \text{MSE}(\hat{I}, I_{\text{gt}}) + \beta \cdot \text{LPIPS}(\hat{I}, I_{\text{gt}})$$ Supervising both final rendering and mode-selected albedos ensures stable gradient flow, allowing the scene point cloud, proximity weights, and intrinsic features to converge simultaneously.

Key Experimental Results¶

Main Results¶

Intrinsic PAPR is benchmarked against leading point-based inverse rendering baselines (DPIR, GS-IR, IRGS, DiscretizedSDF), NeRF-based methods (Intrinsic-NeRF), diffusion-guided approaches (MaterialFusion), and vanilla PAPR across both synthetic benchmarks (NeRF Synthetic, TensoIR) and challenging real-world captures (Tanks & Temples, Mip-NeRF 360).

Task / Benchmark Dataset	Evaluation Metric	Intrinsic PAPR (Ours)	Best Baseline (DiscretizedSDF / IRGS)	Improvement
Novel View Synthesis (Synthetic)	PSNR ↑ SSIM ↑ LPIPS ↓	35.00 0.978 0.021	31.72 (DiscretizedSDF) 0.959 0.038	+3.28 dB (+10.3%) +0.019 -0.017
Novel View Synthesis (Real-world)	PSNR ↑ SSIM ↑ LPIPS ↓	29.99 0.912 0.094	27.86 (DiscretizedSDF) 0.876 0.147	+2.13 dB (+7.6%) +0.036 -0.053
Albedo Reconstruction (Synthetic GT)	PSNR ↑ SSIM ↑ LPIPS ↓	31.21 0.949 0.056	28.89 (DiscretizedSDF) 0.921 0.095	+2.32 dB (+8.0%) +0.028 -0.039

To rigorously evaluate whether primitives learn genuine physical attributes rather than compensatory artifacts, the authors design a per-point intrinsic transfer benchmark across $N=100$ random trials, reporting the mean and standard deviation of Mean Squared Error (MSE):

Scene	Evaluated Task	Intrinsic PAPR (Ours)	GS-IR (Leading 3DGS Baseline)	DPIR (Point IR Baseline)
Lego	Albedo Transfer (MSE: Avg / STD ↓) Shading Transfer (MSE: Avg / STD ↓)	0.053 / 0.016 0.074 / 0.029	0.381 / 0.219 0.414 / 0.157	0.236 / 0.179 0.328 / 0.223
Materials / Chair	Albedo Transfer (Materials, MSE ↓) Shading Transfer (Chair, MSE ↓)	0.067 / 0.026 0.059 / 0.022	0.318 / 0.175 0.319 / 0.101	0.221 / 0.106 0.425 / 0.135
Hotdog	Albedo Transfer (MSE: Avg / STD ↓) Shading Transfer (MSE: Avg / STD ↓)	0.071 / 0.033 0.041 / 0.033	0.280 / 0.141 0.540 / 0.187	0.291 / 0.135 0.482 / 0.119

Ablation Study¶

The ablation study validates both the progressive component additions and the impact of physical decomposition complexity. Cross-view albedo stability is tracked using the Mean Albedo Consistency Error (MACE) across consecutive trajectory frames:

Variant / Configuration	NVS PSNR (dB) ↑	Albedo PSNR (dB) ↑	MACE (Drift Error) ↓	Configuration Analysis & Insight
Vanilla PAPR	32.84	—	—	Baseline point renderer without intrinsic factor split
+ albedo/shading split + 2D prior	33.71	29.11	0.019	Standard deterministic prior provides baseline decomposition
+ cIMLE-adapted prior ($M=10$)	34.36	29.74	0.018	Multi-hypothesis generation mitigates single-view bias
+ Space Carving Loss (SCL)	34.83	30.58	0.016	Delivers largest single albedo jump (+0.84 dB) and consistency gain
+ Per-view learnable scalar (Full)	35.00	31.21	0.015	Absorbs exposure/scale discrepancies across views
Alternative 1: + specular branch	33.20	30.95	0.017	Extra degree of freedom degrades ill-posed inverse optimization
Alternative 2: Cook-Torrance + SH lighting	33.40	29.40	0.021	Full microfacet BRDF suffers from severe under-constraint

Key Findings¶

Misattribution is an inherent defect of volume rendering: Volumetric 3DGS baselines incur transfer MSE errors of 0.28–0.54 with large variances, demonstrating that their primitive features are unphysical compensations. Intrinsic PAPR reduces transfer error by more than $4\times$ (down to 0.04–0.07), verifying true per-point attribution.
Increasing training views cannot cure volumetric misattribution: Sweeping training views from 25 to 200 maintains a 5.04–5.23$\times$ error gap between GS-IR and Intrinsic PAPR; even at 25 views, Intrinsic PAPR achieves a lower transfer error (0.078) than GS-IR trained with 200 views (0.314).
Parsimonious decomposition outperforms over-parameterized PBR: Adding microfacet BRDF and spherical harmonics degrades albedo PSNR from 31.21 dB to 29.40 dB. By respecting the bias-variance tradeoff in ill-posed inverse problems, recovering a minimal set of parameters reliably proves vastly superior for scene editability.

Highlights & Insights¶

Formal diagnosis of volumetric misattribution: Clearly pinpoints why volume rendering's transmittance-weighted summation fundamentally prevents unique parameter identification at the primitive level, establishing an important theoretical caveat for 3DGS editing workflows.
Direct supervision via proximity attention: Elegantly demonstrates that changing the underlying rendering mechanism to opaque, surface-dominant attention solves the inverse problem at the source, allowing standard gradients to act directly on individual 3D points.
Probabilistic distillation of 2D priors into 3D geometry: Combines cIMLE hypothesis generation with a mode-selecting space carving loss, offering an effective general blueprint for lifting multi-modal, ambiguous monocular 2D priors into geometrically consistent 3D representations.

Limitations & Future Work¶

Opaque surface assumption: The framework assumes non-translucent geometry; volumetric, semi-transparent phenomena such as thin smoke, glassware, and complex participating media cannot be rendered accurately.
Diffuse Lambertian approximation: To prevent optimization instability, the formulation omits high-frequency specular highlights, rough mirror reflections, and subsurface scattering, limiting its capability for full environmental relighting under arbitrary HDR maps.
Extensions to hybrid reflectance models: Future research may investigate combining PAPR's per-point identifiability with constrained specular roughness models or diffusion-based generative illumination priors to expand physical fidelity without sacrificing editability.

vs GS-IR / IRGS / DiscretizedSDF: These 3DGS-based inverse rendering methods composite semi-transparent ellipsoids via volume rendering and fit complex BRDFs. They suffer from severe compensatory misattribution in casual captures, causing editing failures; Intrinsic PAPR resolves this through surface point attention, reducing feature transfer error by over $4\times$.
vs IntrinsicNeRF: IntrinsicNeRF optimizes continuous coordinate MLPs with iterative reflectance clustering, which is computationally heavy and lacks explicit, discrete point primitives for interactive localized editing; Intrinsic PAPR enables fast, precise point-level material manipulation.
vs 2D Intrinsic Decomposition (Careaga & Aksoy / IDT): Pure 2D methods lack explicit 3D structure and suffer from multi-view flicker and scale drift. Intrinsic PAPR distills their monocular generative priors into a persistent 3D point cloud via cIMLE and space carving, achieving robust multi-view consistency.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ [Identifies the core misattribution bottleneck in volumetric neural editing and introduces surface point attention to resolve it]
Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive synthetic and real-world benchmarks, rigorous transfer error metric, and complete component ablations]
Writing Quality: ⭐⭐⭐⭐⭐ [Crystal-clear problem formulation, well-structured mathematical arguments, and highly intuitive visualizations]
Value: ⭐⭐⭐⭐⭐ [Provides a crucial paradigm shift for editable 3D representations and robust inverse rendering in casual captures]