GAINS: Gaussian-based Inverse Rendering from Sparse Multi-View Captures¶

Conference: ECCV2026
Paper: ECCV 2026
Area: 3D Vision
Keywords: inverse rendering, sparse views, Gaussian Splatting, material-light disentanglement, relighting

TL;DR¶

GAINS augments a two-stage 2D Gaussian Splatting inverse-rendering pipeline with five systematically integrated foundation-model priors (monocular depth/normal, geometry-stage SDS, SAM subpart segmentation, monocular intrinsic image decomposition, and multi-illuminated SDS), disentangling geometry, materials and environment lighting from only 4-32 input views and reaching relighting PSNR of 28.16 on Synthetic4Relight (Ref-Gaussian: 24.29) and 22.29 on Shiny Blender (Ref-Gaussian: 17.26).

Background & Motivation¶

Inverse rendering recovers the intrinsic properties of a scene — geometry, materials (BRDF), and lighting — from multi-view images, and it is the prerequisite for relightable rendering, material and shape editing, and related downstream applications. The field has moved quickly over the past decade: from early methods restricted to surface normals, albedo and spherical-harmonic lighting, to today's 3D Gaussian primitives, physically based BRDFs and realistic illumination. But that progress has always been tied to an implicit assumption — dense multi-view capture. Only dense observations provide constraints strong enough to separate the shape, reflectance and lighting that are entangled in the image formation model; as soon as the views become sparse those constraints collapse, and the optimizer starts "painting lighting into albedo and baking reflections into diffuse" in order to fit the handful of training images, producing materials that fall apart under any new illumination.

This paper attacks exactly that ill-posed sparse-view regime. The authors observe that recent sparse-view reconstruction (in both the NeRF and the 3DGS line) improved dramatically by combining learning-based priors — monocular depth, normals, diffusion models — with explicit geometric representations; but transplanting the recipe into inverse rendering is not trivial, because the priors themselves are noisy, hallucination-prone and cross-view inconsistent, and a misused prior contaminates material and lighting estimation. Rather than adding a single prior, the paper selects complementary priors for each stage and makes them coexist inside one optimization: the geometry stage gets pixel-aligned depth/normal constraints plus a diffusion regularizer that only acts on high frequencies; the material stage then analyzes the gains and failure modes of three priors individually (segmentation improves cross-view consistency of specular parameters but hurts albedo; IID yields high-quality albedo but is view-inconsistent; diffusion generalizes well but is materially inconsistent) and uses all three together.

The core idea is to make "which prior to trust where" an explicit division of labour inside a two-stage optimization: on a 2DGS representation, first pin the geometry down with monocular depth, normals and a delayed diffusion SDS term, then, while jointly optimizing materials and lighting, constrain specular, diffuse and generalization simultaneously with three complementary regularizers — segmentation-driven intra-class consistency, a monocular IID albedo prior, and multi-illuminated SDS. The payoff is a large margin over existing Gaussian inverse-rendering methods in the high-ambiguity 4-16 view regime.

Method¶

Overall Architecture¶

The input is N sparse RGB images (as few as 4 in the experiments, with 8 used for the main evaluation) plus known intrinsics and extrinsics; the output is a relightable 2DGS scene in which every Gaussian carries geometry (position, scale, rotation, opacity), BRDF parameters (albedo, roughness, metallicity), and a 128×128 environment cubemap. For the representation the authors deliberately choose 2D Gaussian Splatting [10] over 3DGS [16] — in inverse rendering geometry error propagates directly into material estimation, and the surfel parameterization of 2DGS together with a depth-distortion loss yields more accurate normals and surfaces.

The pipeline follows the field's standard "geometry first, materials and lighting second" scheme, and the contribution lies entirely in how foundation-model priors are injected into the two stages. Stage I optimizes geometry for 16 000 iterations: on top of the original 2DGS colour, depth-distortion and normal-consistency losses it adds monocular depth and normal guidance, plus a diffusion SDS regularizer that only becomes active at iteration 10 000; if the dataset provides ground-truth alpha masks an outlier-removal term is added as well. Stage II freezes the geometry and jointly optimizes materials and environment lighting for 7 000 iterations, augmenting the re-rendering loss with three complementary priors: segmentation-driven intra-class consistency, a monocular intrinsic image decomposition (IID) albedo prior, and multi-illuminated SDS (MI-SDS, active from step 3 000) that renders under random environments before distillation. All hyper-parameters are fixed; no per-scene tuning and no fine-tuning of any prior network.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Sparse input views (4-32)"] --> B["Stage I: geometry estimation<br/>2DGS + differentiable rasterization"]
    B --> C["Monocular depth and normal guidance<br/>constrains unseen-view geometry"]
    B --> D["Delayed diffusion guidance in geometry<br/>only removes high-frequency artifacts"]
    C --> E["Converged geometry and normals"]
    D --> E
    E --> F["Stage II: joint material and lighting optimization"]
    F --> G["Segmentation-driven intra-class consistency<br/>constrains specular parameters"]
    F --> H["Intrinsic decomposition prior<br/>reliable starting point for albedo"]
    F --> I["Multi-illuminated score distillation<br/>disentangles material from environment"]
    G --> J["Relighting / novel view synthesis"]
    H --> J
    I --> J

Key Designs¶

1. Monocular depth and normal guidance: making geometry serve more than the input views

With sparse input, 2DGS rapidly overfits the training viewpoints, producing geometric collapse and floaters in unseen views — and since inaccurate geometry dooms the subsequent BRDF estimation, this has to be fixed first. The crux is using the priors without letting them backfire: monocular depth networks (the paper uses the diffusion-based depth estimator of [15]) carry systematic bias and even hallucinations, so a hard L1/L2 constraint would drag the whole geometry toward the prior's bias and contaminate materials and lighting. Depth is therefore split into two complementary losses — a scale-invariant correlation loss \(L_{DC}=1-\text{PCC}(D_i,\hat D_i)\) that only asks the rendered depth to agree with the monocular depth in relative structure without locking absolute scale, and a local depth ranking loss \(L_{DR}\) (following FSGS/FateGS-style practice) whose soft local ordering constraints prevent the geometric collapse and long-range ambiguity that hard constraints induce. Normals are handled more firmly, since they directly determine BRDF shading: an L1 term between rendered and geometric normals, an L1 term between rendered and monocular normals (weight 0.1), and a total-variation term to suppress noise:

\[L_N = L_1(\tilde N_i, \hat N_i) + 0.1\cdot L_1(\tilde N_i, N_i) + TV(\tilde N_i, \hat N_i)\]

An outlier-removal (OR) strategy accompanies this (only when ground-truth alpha masks exist): a BCE loss on the reconstructed alpha mask plus KNN-based floater removal every 1 000 iterations. The ablation localizes the gain of this design to high-frequency geometric fidelity: removing depth and normal guidance blows normal MAE up from 11.64 to 32.23 and drops NVS PSNR from 23.67 to 16.73, the largest degradation in Stage I.

2. Delayed diffusion guidance in the geometry stage: only removing high-frequency artifacts after convergence

Depth and normals provide pixel-aligned constraints, so they cannot say anything about regions no input view ever covers; those need a view-agnostic "does this look like a real object" prior, which is where Score Distillation Sampling (SDS) with a diffusion model comes in. The implementation samples 100 novel viewpoints orbiting the scene, renders them, adds noise at a random timestep \(t\sim U(0.02,0.98)\) to form latents, and back-propagates the diffusion model's noise-prediction error. What keeps it from derailing is delayed activation: early on the geometry is still far from the target shape and SDS would treat the diffusion model's imagination as supervision, directly provoking hallucination; so SDS only starts after iteration 10 000 of 16 000, by which point the geometry has essentially converged and the diffusion prior at guidance scale 100 mainly suppresses high-frequency artifacts, adding only limited hallucinated detail. The ablation shows removing SDS alone costs just 0.1 PSNR (23.67 → 23.57), whereas removing SDS together with outlier removal drops to 21.75 — i.e. it polishes an already-standing geometric skeleton rather than providing a safety net.

3. Segmentation-driven intra-class consistency: constraining specular parameters by semantic subparts

Diffuse albedo is supposed to vary sharply across space because of texture, but specular material parameters (roughness, metallicity) are typically uniform within semantically similar regions — a prior that is especially valuable under sparse views. Each Gaussian is extended with a one-hot class vector \(E_j\in\{0,1\}^K\), and 2D segmentation masks are lifted to 3D via the training-free mask-lifting idea: images with spherical-harmonic colours are rendered from 100 sampled novel views orbiting the scene, SAM produces a mask per image, CLIP and DINOv2 features are extracted for each mask region, each mask is assigned to the Gaussian with the largest alpha-blending contribution, and newly lifted objects are merged with previously accumulated ones by geometric and feature similarity. Because mask boundaries are imprecise and a single semantic object may contain several materials, the constraint is not "parameters must be equal within a subpart" but variance minimization inside each mask, with a weight scaled by mask size to damp the noisy statistics of small masks.

The regularizer sums three variance terms: the first reduces the variance of roughness and metallicity inside a mask, directly suppressing multi-view noise in specular parameters; the second targets the ambiguity in which mirror-like materials reflect environment texture into the diffuse channel — in low-roughness, high-metallicity regions the optimization is biased to attribute texture detail to specular reflection rather than albedo; the third gives high-specularity regions a small extra weight, since those regions tend to carry larger geometry errors and hence larger material errors. One telling observation in the ablation: removing segmentation guidance leaves albedo PSNR marginally higher (21.73 → 21.92) but lowers both NVS (23.90 → 23.75) and relighting (22.29 → 22.23), exactly matching the authors' claim that this prior governs specular consistency, not albedo.

4. Intrinsic decomposition prior: giving diffuse albedo a reliable starting point

Segmentation guidance is powerless on high-frequency textured diffuse albedo, and albedo is precisely what determines relighting quality: once lighting (especially shadows and specular reflections) is baked into albedo, it reappears verbatim under new illumination. The authors therefore use a monocular intrinsic image decomposition network as a teacher for albedo — Teamwork [26] for indoor data and RGB2X [43] for outdoor data (the latter empirically more reliable outdoors) — adding an L2 loss between rendered and IID-predicted albedo. Because monocular IID is itself view-inconsistent, the loss is weighted by a linearly decaying factor \(\beta(\tau)\):

\[L_{IID} = \beta(\tau)\cdot L_2(A_i, \hat A_i)\]

That is, the optimization borrows the IID decomposition heavily early on to pull albedo into a plausible range, then lets go as the model converges and the multi-view re-rendering loss takes over. Notably, the authors deliberately do not apply IID supervision to roughness and metallicity — those channels proved less robust and more cross-view inconsistent. In the ablation this prior is the single largest contributor to albedo: removing it drops albedo PSNR from 21.73 to 21.38 and relighting from 22.29 to 22.07.

5. Multi-illuminated score distillation: freeing materials from the learned environment map

Segmentation and IID improve the accuracy of specular and diffuse parameters respectively, but neither guarantees that they hold at unseen viewpoints and under unseen lighting. The authors reuse the SDS idea from Stage I but change what is scored: instead of rendering a novel view under the current lighting, they render a relit image \(R_{mat}(C_j, G, M, E_l)\) from a novel view \(C_j\) under an environment map drawn at random from a predetermined set \(\{E_l\}\), and apply SDS to that relit image:

\[L_{MI-SDS} = \mathbb{E}_{t,\epsilon}\big[w(t)\,\|\epsilon_\phi(Z_j^l; t)-\epsilon\|_2^2\big],\quad Z_j^l = \alpha_t R_{mat}(C_j, G, M, E_l) + \sigma_t\epsilon\]

The point is to break the collusion between materials and the learned environment map: if supervision only ever uses the learned lighting \(L\), materials can offload error onto lighting and lighting can compensate back, so together they fit the training views while generalizing to neither; rendering under a random third-party illumination instead means any lighting baked into the diffuse channel exposes itself immediately. As in Stage I, MI-SDS is delayed until step 3 000 of 7 000 with the same guidance scale of 100, mainly suppressing high-frequency artifacts. The ablation shows it has the smallest metric effect (relighting 22.29 → 22.26), but the authors stress that its benefit is stability under new views and lighting — visible qualitatively rather than in the numbers.

Loss & Training¶

The full Stage I (geometry) loss is a weighted sum of the original 2DGS losses and the added priors:

\[L_{stage1} = L_{col} + \lambda_{DC}L_{DC} + \lambda_{DR}L_{DR} + \lambda_{NC}L_{NC} + \lambda_{N}L_{N} + \lambda_{SDS}L_{SDS} + \lambda_{BCE}L_{BCE}\]

where \(L_{col}\) is the re-rendering loss mixing L1 and D-SSIM, and \(L_{NC}\) is the normal-consistency loss between rendered normals and depth-derived normals. The weights are \(\lambda_{DC}=0.005,\ \lambda_{DR}=10,\ \lambda_{NC}=1,\ \lambda_{N}=0.25,\ \lambda_{SDS}=0.0001,\ \lambda_{BCE}=0.75\). Stage II (materials and lighting) jointly optimizes for 7 000 iterations with geometry frozen:

\[L_{mat} = L_{color} + \lambda_{ICC}L_{ICC} + \lambda_{IID}L_{IID} + \lambda_{MI\text{-}SDS}L_{MI\text{-}SDS} + \lambda_{TV}L_{TV}\]

where \(L_{TV}\) is a total-variation smoothing term on the reconstructed lighting, and \(\lambda_{ICC}=0.1,\ \lambda_{IID}=2,\ \lambda_{MI\text{-}SDS}=0.0001,\ \lambda_{TV}=0.1\). All hyper-parameters are fixed across every scene with no per-scene search and no fine-tuning of any prior network; the SDS guidance scale is 100 throughout. The tiny magnitudes of \(\lambda_{SDS}\) and \(\lambda_{MI\text{-}SDS}\) (1e-4) reflect the large scale of SDS gradients — what actually matters is the delayed-activation schedule.

A Worked Example¶

Take the sedan scene from Ref-Real: the input is 8 sparse images captured around the car (plus ground-truth alpha masks). Stage I first builds an initial 2DGS geometry while every image is passed through the monocular depth and normal networks. Before iteration 10 000 the optimization is driven entirely by the colour loss, the depth correlation/ranking losses, the normal L1+TV terms and outlier removal, and the geometry converges onto the car body. After iteration 10 000 SDS enters at weight 1e-4 and cleans high-frequency floaters around the roof and wheel arches over 100 orbiting novel views, and by iteration 16 000 the geometry is fixed. Stage II then starts joint material and 128×128 environment-lighting optimization: for the first 3 000 steps segmentation-driven intra-class consistency flattens the roughness/metallicity of the body and the glass separately, while the IID prior (the indoor Teamwork variant here) pulls the diffuse albedo inside the doors into a sensible range; after step 3 000 MI-SDS begins scoring relit renders under random environment maps, peeling the sky reflection that had been baked into albedo back out. The end product is the material set shown in Fig. 3, relightable under three novel environment maps — compared with GI-GS's noisy normals and Ref-Gaussian's floaters on the car, the albedo, roughness and metallicity are markedly cleaner.

Key Experimental Results¶

Main Results¶

The baselines cover six Gaussian inverse-rendering methods (Ref-Gaussian [39], GI-GS [3], GaussianShader [12], R3DG [6], MaterialRefGS [46], SVG-IR [32]), evaluated on Synthetic4Relight [48], Shiny Blender [33] (extended with ground-truth albedo and relighting, with the ball object removed) and the real-world Ref-Real [33]. The main tables train on 8 sparse views uniformly sampled from the dense captures; albedo is evaluated with a scale-invariant normalisation to sidestep the inherent albedo-lighting ambiguity. GIR [30] failed to reconstruct meaningful geometry and reflectance on real scenes while taking over 7 hours per scene, so it appears only in the qualitative comparison.

Dataset	Metric	Ours	Prev. SOTA	Gain
Synthetic4Relight (relighting)	PSNR ↑ / SSIM ↑ / LPIPS ↓	28.16 / 0.93 / 0.09	27.42 / 0.90 / 0.10 (R3DG)	+0.74 PSNR
Synthetic4Relight (NVS)	PSNR ↑	29.32	27.04 (GI-GS)	+2.28
Synthetic4Relight (albedo)	PSNR ↑	22.34	22.15 (R3DG)	+0.19
Synthetic4Relight (roughness)	MSE ↓	0.04	0.03 (SVG-IR)	slightly behind SVG-IR
Shiny Blender (relighting)	PSNR ↑ / SSIM ↑ / LPIPS ↓	22.29 / 0.89 / 0.12	17.26 / 0.66 / 0.20 (Ref-Gaussian)	+5.03 PSNR
Shiny Blender (albedo)	PSNR ↑	21.73	21.37 (GI-GS)	+0.36
Shiny Blender (normal)	MAE ↓	11.64	24.43 (R3DG)	error cut by more than half
Ref-Real (NVS, real data)	PSNR ↑ / SSIM ↑	20.28 / 0.56	19.64 / 0.44 (GI-GS)	+0.64 PSNR / +0.12 SSIM
Runtime	per scene	≈1h20m	≈1h30m (R3DG / GI-GS)	same order as the fastest methods

For the two relighting columns specifically: on Synthetic4Relight the competitors score 23.92 (GI-GS) and 24.29 (Ref-Gaussian) against 28.16 for GAINS; on Shiny Blender they score 15.44 (GI-GS) and 17.26 (Ref-Gaussian) against 22.29.

Ablation Study¶

Stage I ablation on Shiny Blender with 8 views (geometry stage only, no Stage II components):

Config	NVS PSNR ↑	Albedo PSNR ↑	Relight PSNR ↑	Normal MAE ↓	Note
Full (depth, normal, SDS, OR)	23.67	21.18	21.96	11.64	complete Stage I
w/o SDS	23.57	21.09	21.91	11.70	smallest effect; SDS refines rather than rescues
w/o SDS, OR	21.75	19.98	21.26	13.79	clear drop once outlier removal is gone
w/o Normal, SDS, OR	19.42	18.79	19.41	19.38	normal prior contributes substantially
w/o Depth, Normal, SDS, OR	16.73	16.41	16.71	32.23	depth prior is the foundation of geometry

Stage II ablation, also on Shiny Blender with 8 views (full Stage I geometry, only the material-stage prior combination changes):

Config	NVS PSNR ↑	Albedo PSNR ↑	Relight PSNR ↑	Note
Full	23.90	21.73	22.29	all three priors on
w/o MI-SDS	23.88	21.66	22.26	smallest metric effect; benefit is cross-view stability
w/o Seg	23.75	21.92	22.23	albedo rises slightly, but NVS/relighting fall
w/o IID	23.84	21.38	22.07	largest albedo drop (−0.35)
w/o Seg, IID, MI-SDS	23.67	21.18	21.96	degrades to the geometry-only level

Key Findings¶

Depth is the foundation of geometry; diffusion is the polish. In Stage I, removing depth/normal guidance worsens normal MAE from 11.64 to 32.23 and costs 6.9 PSNR of NVS, whereas removing SDS alone costs only 0.1. This matches SDS being delayed to iteration 10 000 of 16 000 — it performs post-convergence high-frequency cleanup, not damage control.
The three material priors have non-overlapping benefits. IID is the largest albedo contributor (removing it costs 0.35 albedo PSNR and 0.22 relighting PSNR), segmentation governs specular consistency (removing it actually raises albedo PSNR by 0.19 while lowering NVS by 0.15 and relighting by 0.06), and MI-SDS barely moves any metric yet is responsible for stability under unseen views and lighting. The authors use this to argue that only the combination covers parameter accuracy, multi-view consistency and generalization at once.
The advantage narrows with more views, but starts from a higher baseline. In the extremely sparse 4-8 view regime GAINS leads by a wide margin on NVS, albedo and relighting; at 16-32 views GI-GS catches up or overtakes on NVS while GAINS keeps the relighting lead; at 64-100 views Ref-Gaussian surpasses GAINS on NVS and matches it on relighting — which the authors attribute to Ref-Gaussian's looser objective (it does not explicitly optimize albedo, relying instead on the Stage I spherical-harmonic representation for appearance fitting, and keeps densifying during Stage II). In other words, the value of this work is concentrated in the sparse regime.
Differences beyond the metrics. GI-GS is second-best on NVS but its normals are visibly noisier (the car body in the sedan scene), leading the authors to conclude its NVS score comes from overfitting rather than genuinely recovered shape; in the qualitative comparison Ref-Gaussian shows floaters, GI-GS often recovers reasonable geometry but unstable albedo and roughness, and the main visual gain of GAINS is that reflections are no longer baked into the materials — ground and sky reflections change correctly with illumination during relighting.
An honest caveat: on roughness MSE, SVG-IR (0.03) beats this paper (0.04), so the method is not ahead on every metric.

Highlights & Insights¶

The "complementary priors" judgement is turned into verifiable experimental claims. The paper does not vaguely assert that it uses foundation-model priors; it states one by one that segmentation improves specular consistency but hurts albedo, that IID improves albedo but is cross-view inconsistent, and that diffusion improves generalization but is materially inconsistent — and every ablation number lines up with that claim (the counter-intuitive albedo rise when segmentation is removed is the best evidence). This style of prior-benefit analysis is worth transplanting into any multi-prior fusion work.
Delayed activation of the diffusion prior is a cheap and effective trick. SDS readily hallucinates in sparse-view geometry tasks, and the fix here is extremely simple — switch it on only after the geometry has essentially converged (10 000/16 000 and 3 000/7 000) and set the guidance scale to 100 so it mostly smooths high frequencies. The schedule costs nothing yet converts SDS from "can derail" into "stable refinement".
MI-SDS cutting the material-lighting collusion is the most elegant step in the paper. Under the self-learned environment map, materials and lighting can cover for each other; rendering under a random third-party environment and scoring that instead exposes any reflection baked into the materials. The idea transfers directly to any inverse problem with two mutually compensating variables — intrinsic decomposition, shadow removal, white balancing.
A soft prior schedule with iteration-decaying weight (the linear \(\beta(\tau)\) on IID) is a general recipe for exploiting a strong but cross-view-inconsistent prior: use it as initialisation early, hand over to multi-view data later, far more robust than a hard constraint throughout.
Presence or absence of alpha masks is handled explicitly: with masks, BCE plus KNN floater removal; without masks, the scene is reconstructed together with the background — avoiding a mechanism designed for synthetic data that would silently fail on real captures.

Limitations & Future Work¶

The first limitation the authors admit is that they ignore global illumination: only direct light is modelled, objects receive directional incident light from the environment map, and there are no secondary bounces or inter-reflections. Acceptable for isolated objects, clearly wrong for full indoor scenes (which is exactly what the baseline GI-GS targets).
The second is slightly wavy geometric normals: despite smoothness constraints the geometry normals still undulate, noticeably on highly specular objects, and wavy normals cause part of the specular highlight to be baked into diffuse albedo. The authors suggest separating geometric from shading normals to alleviate this, but do not implement it.
A limitation I notice myself: every hyper-parameter, prior network and iteration budget is fixed, which is both the "no per-scene tuning" selling point and a lack of adaptability to radically different scenes (large translucent areas, strong anisotropy). In addition, all three Stage II priors depend on external large models (SAM, CLIP, DINOv2, Teamwork/RGB2X, diffusion), yet the paper never reports the resulting inference overhead or practical deployment cost.
Improvement directions: extend direct lighting toward an approximate global illumination with ambient occlusion or inter-reflection; introduce an explicit separation of geometric and shading normals; make the IID and segmentation weights adapt to geometric confidence (segmentation weight currently scales only with mask size, ignoring reconstruction quality in that region).

vs Ref-Gaussian [39] / GI-GS [3]: the two principal baselines, both sharing the recipe of Gaussian primitives plus physically based BRDF and deferred shading, and both assuming dense views. GI-GS additionally models global illumination and is strong on NVS (able to beat this paper with dense views), but the experiments here show it has noisy normals and unstable albedo/roughness under sparse views, trailing badly on relighting; Ref-Gaussian overtakes GAINS on NVS at 64-100 views, but because it fits spherical-harmonic appearance without explicitly optimizing albedo it shows floaters and reflection baking in the sparse regime. The fundamental difference is that this paper changes no part of the rendering pipeline and instead augments the optimization objective with learning-based priors.
vs sparse-view geometry reconstruction (FSGS, SparseGS, GaussianObject, FateGS, etc. [11,37,38,49]): that line already demonstrated that monocular depth/normal and diffusion priors dramatically improve sparse-view geometry, but it optimizes only appearance and shape without recovering materials or lighting. This paper carries the same priors into inverse rendering and adds an observation of its own: the bias of a depth prior is tolerable in a geometry task, but in inverse rendering it propagates through geometric error into the materials and lighting, which is why depth must be softened into correlation and ranking losses that lock structure without locking scale.
vs early single-view / sparse-view inverse rendering [17,28,29] and RelitLRM [45]: the former learn priors from synthetic data without explicit geometry or light-transport modelling and generalize poorly to complex real scenes; the latter uses large reconstruction models for object relighting but does not estimate intrinsic components (material parameters) and does not generalize to complex scenes. This paper sits between them: it keeps physical light transport and an explicit BRDF, and uses foundation models to supply the constraints that sparse observations lack.
vs neural inverse rendering (NeRFactor [47], TensoIR [13], GaNI [35], etc.): the NeRF line uses implicit representations with simplified BRDF/lighting models for high quality at high training cost; this paper stays with a Gaussian representation for efficiency (about 1h20m per scene), at the price of not being able to incorporate near-field lighting and neural radiance caching as naturally as GaNI does.

Rating¶

Novelty: ⭐⭐⭐ [the framework is an engineered combination of 2DGS with foundation-model priors and each individual component (depth/normal/SDS/IID/segmentation) has been used before; the novelty lies mainly in the deliberate division of labour among complementary priors and in the MI-SDS reformulation]
Experimental Thoroughness: ⭐⭐⭐⭐ [two synthetic datasets plus one real dataset, six baselines, a view-count sweep from 4 to 100, and separate ablations for both stages; but the real dataset is evaluated on NVS only, with no quantitative material or relighting metrics]
Writing Quality: ⭐⭐⭐⭐ [the prior-benefit analysis is clear and the ablations support the claims, with formulas and loss weights fully reported; weaknesses are some typographically broken equations and the fact that Stage II priors only get single-variable ablations without interaction effects]
Value: ⭐⭐⭐⭐ [provides a reproducible recipe for sparse-view inverse rendering (prior selection + delayed scheduling + soft weight decay) with direct practical value for 3D reconstruction and relighting engineering]