[ECCV 2026][3D Vision][In-the-Wild Novel View Synthesis] WildSplat decouples appearance-invariant geometry from reference-conditioned appearance with a dual-branch architecture: a VGGT-style geometry branch jointly predicts poses and 3D Gaussian attributes, while an appearance branch injects reference-image appearance into content features via [CLS] global pre-modulation plus cross-attention. Joint multi-reference supervision and geometry-guided view sampling stabilize training; in a single forward pass it surpasses AnySplat, WorldMirror, and optimization-based wild methods on MegaScenes and Phototourism, with Phototourism PSNR 19.57/21.15/21.47 at 4/8/12 views.

Tags: ECCV 2026 · 3D Vision · Novel View Synthesis · 3D Gaussian Splatting · Feedforward Reconstruction · Appearance Decoupling · Unposed Input

WildSplat: Feedforward Gaussian Splatting from Unposed In-the-Wild Images¶

Conference: ECCV 2026
Paper: ECCV paper page
Area: 3D Vision
Keywords: novel view synthesis, 3D Gaussian Splatting, feedforward reconstruction, in-the-wild images, appearance editing

TL;DR¶

WildSplat is the first feedforward 3DGS framework for unposed in-the-wild images: a geometry branch (DINOv2 encoder + 24 alternating-attention layers) extracts appearance-invariant content features and jointly predicts camera poses, while an appearance branch uses [CLS] pre-modulation plus cross-attention to inject reference-image appearance before predicting conditioned spherical-harmonics colors. Joint multi-reference supervision prevents geometry-appearance entanglement; on MegaScenes/Phototourism at 4/8/12 views it comprehensively beats AnySplat, WorldMirror, and optimization-based methods (Phototourism 12-view PSNR 21.47), and supports single-pass appearance editing and interpolation.

Background & Motivation¶

Internet photo collections capture the same scene under different lighting, weather, and exposure, clearly violating the static-illumination assumption. NeRF-W and subsequent appearance-aware 3DGS methods (GS-W, WildGaussians, Splatfacto-W, Wild-GS) decouple shared structure from photometric variation using per-image appearance embeddings; interpolation works well, but these methods strictly require precomputed poses, dense collections, and hours of per-scene optimization, and per-image embeddings become severely under-constrained under sparse views. The other line, feedforward reconstruction (pixelSplat, MVSplat, NoPoSplat, AnySplat), learns generalizable priors from large multi-view data and outputs a 3D representation in one forward pass, but it assumes photometric consistency; when input appearance diverges, geometry cannot be separated from illumination changes, and reconstructions show obvious appearance mixing.

The core tension is that decoupling capability and sparse-unposed efficiency have so far been held by two separate method families — optimization-based methods decouple without efficiency or generalization, feedforward methods are efficient and general without any decoupling mechanism. The authors propose designating one reference image as the appearance condition: geometry is learned once, appearance is injected per reference. Core idea: split "appearance-invariant geometry + joint pose estimation" from "reference-conditioned appearance" in a dual-branch design, where the [CLS] global descriptor first pre-modulates content features into the target appearance domain before cross-attention gathers local appearance details, and joint multi-reference supervision stabilizes training and blocks appearance leakage into geometry.

Method¶

Overall Architecture¶

The inputs are N context images with varying appearances plus one designated reference image \(I_{\text{ref}}\); the outputs jointly include camera poses \(\{p_i\}\), appearance-invariant Gaussian geometry \(G=\{\mu, s, q, \alpha\}\) (position/rotation/scale/opacity), and conditioned color attributes \(C_{\text{ref}}\in\mathbb{R}^{G\times 3\times(k+1)^2}\) (degree-k spherical harmonics conditioned on the reference). Inference is a single forward pass: the geometry branch encodes context images, refines them through L alternating-attention layers, then DPT heads output depth and Gaussian attributes, a camera head outputs poses, and back-projection places Gaussian centers in global coordinates; the appearance branch encodes the reference image and injects global and local appearance tokens into content features to predict colors; conditioned rasterization finally renders target views.

Training has two stages: first a 30k-iteration warm-up on photometrically consistent DL3DV with the appearance injector disabled (stabilizing the Gaussian attribute heads), then 60k end-to-end iterations on DL3DV + MegaScenes + MegaDepth. MegaDepth/MegaScenes batches are assembled with geometry-guided view sampling, DL3DV with interval sampling; color jittering on inputs and random cropping of references prevent appearance leakage during training.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["In-the-wild context images<br/>+ reference image Iref"] --> B["Dual-branch encoding<br/>DINOv2 geometry/appearance encoders"]
    B --> C["Appearance-invariant geometry<br/>depth + Gaussian attributes + poses"]
    B --> D["Global pre-modulation<br/>[CLS] shifts into target appearance domain"]
    D --> E["Appearance injector<br/>cross-attention + self-attention"]
    C --> F["Decoupled 3D representation<br/>G and Cref kept separate"]
    E --> F
    F --> G["Conditioned rasterization<br/>renders at target poses"]

Key Designs¶

1. Dual-branch decoupling: appearance-invariant geometry and conditioned appearance predicted separately

Prior feedforward methods (AnySplat, WorldMirror) regress Gaussian geometry and color jointly from one entangled representation; once the output must match a reference appearance, the conditioning signal can perturb geometry in return. WildSplat's geometry branch is built on a VGGT-style architecture: N context images are first encoded independently by a DINOv2-large geometry encoder into dense patch tokens, five learnable camera tokens are concatenated, and the combined tokens are refined through L=24 alternating-attention layers (intra-frame self-attention plus cross-frame global attention). Content features \(F_{\text{content}}\) are aggregated from selected multi-layer representations \(\mathcal{K}\) — defined as the geometric scaffold before any appearance conditioning. DPT heads regress per-pixel depth and Gaussian attributes \((s, q, \alpha)\), the camera head predicts poses from refined camera tokens, Gaussian centers \(\mu\) are back-projected from depth and poses, and the predicted poses transform Gaussians into a global, view-independent frame. This branch's output stays consistent across arbitrary reference images — same context images, different reference, unchanged geometry.

2. Global pre-modulation + appearance injector: shift into the target domain first, then gather local details

The reference image passes through a separate DINOv2-base appearance encoder producing a global [CLS] token and local dense tokens. If cross-attention ran directly — content features as query, reference tokens as key/value — the content features would still carry the context views' own photometric properties, so modulation would apply local corrections over the wrong appearance base. The authors therefore pre-modulate globally: the [CLS] token is decoded by an AdaLN-Zero layer into a scale γ, a shift β, and an adaptive gate η, modulating each layer's content features as

\[\tilde{\mathbf{T}}_i^{(k)} = \gamma \odot \text{LayerNorm}(\mathbf{T}_i^{(k)}) + \text{MLP}_{\text{cond}}(\mathbf{T}_{\texttt{[CLS]}}),\]

moving content features wholesale into the target appearance domain; the modulated features then serve as queries aggregating condition-relevant details from reference tokens, followed by self-attention for spatial regularization:

\[\bar{\mathbf{T}}_i^{(k)} = \mathbf{T}_i^{(k)} + \eta \odot \text{CrossAttn}(\tilde{\mathbf{T}}_i^{(k)}, \mathbf{T}_{\text{ref}}),\quad \hat{\mathbf{T}}_i^{(k)} = \bar{\mathbf{T}}_i^{(k)} + \text{SelfAttn}(\bar{\mathbf{T}}_i^{(k)}),\]

\[\mathbf{T}_{\text{app},i}^{(k)} = \hat{\mathbf{T}}_i^{(k)} + \text{MLP}(\hat{\mathbf{T}}_i^{(k)}).\]

The updated appearance features are routed to a color head predicting conditioned spherical-harmonics coefficients \(C_{\text{ref}}\). Zero-initialized AdaLN-Zero keeps injection near-identity at the start of training, protecting the pretrained geometry backbone.

3. Joint multi-reference supervision: one geometry, multiple appearances rendered simultaneously

With only one reference per training iteration, the model entangles 3D geometry with that image's illumination — appearance conditioning degenerates into memorizing a specific lighting. WildSplat fixes the appearance-invariant geometry and content features within an iteration, then randomly samples M images from the batch to serve simultaneously as appearance references and ground truth. The appearance branch produces M sets of conditioned colors, rendering M images from each reference's own viewpoint for unified supervision. The same geometry is forced to accommodate multiple appearance conditions, so entanglement is structurally discouraged. Input color jittering plus random reference cropping further severs the shortcut between "reference identity" and "input appearance."

4. Geometry-guided view sampling: in-the-wild batches cannot be assembled by pose order alone

Feedforward training requires sufficient overlap among input views for reliable correspondences; in-the-wild focal-length variation means even small pose shifts can drastically change image content, so sampling by extrinsics or sequence adjacency assembles invalid contexts. The authors use SfM points to build an overlap matrix (shared 3D-point ratio between view pairs) and a scale matrix (minimum depth ratio between views), filter pairs failing overlap or scale-consistency thresholds, then iteratively aggregate valid neighbors breadth-first from a strongly connected candidate view (the paper's Algorithm 1), ensuring every training sample's multi-view context is genuinely registrable.

Loss & Training¶

The total loss sums reconstruction losses over the M rendered images: \(\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{rgb}} + \lambda_1\mathcal{L}_{\text{pose}}\), where the photometric loss \(\mathcal{L}_{\text{rgb}}\) is a weighted combination of MSE, SSIM, and LPIPS, and the pose loss reuses AnySplat's distillation loss to regularize camera estimation. Balancing weights are \(\lambda_i=\{10, 1.0, 0.05, 0.05\}\) (corresponding to pose/MSE/SSIM/LPIPS respectively; ⚠️ the original equation is corrupted in text extraction — defer to the paper for this ordering). Training uses Adam on 4 NVIDIA RTX H20 GPUs; the geometry backbone loads pretrained VGGT weights, AdaLN-Zero layers are zero-initialized, and everything else is randomly initialized.

Key Experimental Results¶

Main Results¶

Evaluation covers two in-the-wild datasets: four Phototourism scenes (following the NeRF-W protocol — the left half of each ground-truth image serves as the reference input while metrics are computed only on the right half, preventing photometric leakage) and ten scenes sampled from the MegaScenes test set, all evaluated at 4/8/12-view sparse configurations on a fixed test set. Baselines come in two families: feedforward AnySplat and WorldMirror (evaluation-time pose alignment applied uniformly — Gaussian parameters fixed while target camera poses are optimized photometrically, identically for baselines and this method), and optimization-based FSGS, GS-W, and WildGaussians (which require accurate poses and dense point clouds; the authors estimate their geometric priors with VGGT).

The table below excerpts the original Table 2, Phototourism novel view synthesis (PSNR↑/SSIM↑/LPIPS↓):

Method	4-view PSNR	8-view PSNR	12-view PSNR	12-view SSIM	12-view LPIPS
FSGS	13.63	14.85	15.39	0.573	0.346
GS-W	9.40	10.19	12.63	0.399	0.437
WildGaussians	14.95	17.02	17.65	0.616	0.353
AnySplat	15.61	17.35	18.00	0.700	0.228
WorldMirror	15.38	16.71	17.24	0.724	0.234
WildSplat	19.57	21.15	21.47	0.788	0.137

The table below excerpts the original Table 1, MegaScenes novel view synthesis (same metric system):

Method	4-view PSNR	8-view PSNR	12-view PSNR	12-view SSIM	12-view LPIPS
FSGS	12.88	13.28	13.43	0.376	0.453
GS-W	9.54	10.76	9.81	0.311	0.526
WildGaussians	14.38	15.32	15.68	0.448	0.452
AnySplat	14.58	16.16	16.31	0.601	0.273
WorldMirror	15.33	16.08	16.05	0.629	0.287
WildSplat	17.91	18.65	19.20	0.683	0.210

WildSplat leads on all metrics across both datasets, and gains grow steadily with input view count, whereas WorldMirror drops from 8 to 12 views on MegaScenes (16.08→16.05) with visible fluctuation. Optimization-based methods fare worst under sparse inputs. Pose evaluation (original Table 4, 20 MegaScenes scenes × 24 views): WildSplat reaches AUC@5/10/30 = 0.310/0.443/0.624 with ATE 1.559, beating AnySplat (0.249/0.392/0.596) on all three AUC metrics while AnySplat keeps a slight ATE edge (1.525); overall close to the dedicated pose prior VGGT (0.314/0.451/0.632, ATE 1.525). Note the table shows complete numbers only for AnySplat and Ours; the VGGT row's digits are fused with AnySplat's row in text extraction, ⚠️ defer to the original PDF.

Ablation Study¶

The table below excerpts the original Table 3, component ablation on Phototourism (PSNR/SSIM/LPIPS at 4 views; the full model corresponds to 18.52/0.705/0.212, 19.90/0.748/0.180, and 20.15/0.766/0.169 across the three view counts):

Variant	4-view PSNR	8-view PSNR	12-view PSNR
w/o Multi-Ref Supervision	16.88	18.77	19.35
w/o Geo-guided Sampling	17.99	19.24	19.58
w/o Pre-Modulation	18.25	19.70	20.11
Full	18.52	19.90	20.15

Note the ablation's Full row (18.52/19.90/20.15) differs from the main-table WildSplat row (19.57/21.15/21.47); the paper does not explain this gap (possibly a subset or protocol difference), ⚠️ defer to the original, and this note does not reconcile the numbers on its own.

Key Findings¶

Joint multi-reference supervision is the single largest contributor. Removing it drops 4-view PSNR from 18.52 to 16.88 (−1.64), the largest ablation drop — single-reference iteration indeed lets geometry memorize a specific lighting.
Global pre-modulation matters more for appearance consistency than for coarse geometry. At 12 views, removing pre-modulation costs only 0.04 PSNR (20.15→20.11), but LPIPS rises from 0.169 to 0.175 — its effect concentrates on appearance consistency.
Geometry-guided sampling mainly rescues the sparse end. Removing it costs 0.53 PSNR at 4 views (17.99 vs 18.52) and 0.66/0.57 at 8/12 views; in-the-wild focal-length variance makes pose-ordered sampling unreliable at any view count.
Optimization-based methods collapse under sparse inputs. FSGS/GS-W/WildGaussians depend on dense collections, reaching at best 17.65 PSNR at 12 views, clearly behind the feedforward family.
Appearance editing is a free byproduct of decoupling. Swapping the reference transfers color tone (even across building types), and feature interpolation yields continuous appearance transitions — all without re-optimization.

Highlights & Insights¶

Two-level appearance injection: pre-modulate, then cross-attend. Shifting features wholesale into the target domain with [CLS] before gathering local details matches the "global tone + local materials" structure of appearance better than single-level injection; the pattern transfers to any architecture where conditioning features would contaminate a backbone representation.
Solve entanglement with a training strategy, not just architecture. Multi-reference supervision forces one geometry to fit multiple appearances within one iteration, breaking entanglement at the data-organization level — cheaper than stacking more modules.
The evaluation protocol takes sparse-unposed fairness seriously. VGGT supplies geometric priors for optimization-based baselines, and evaluation-time pose alignment is applied uniformly to feedforward baselines, avoiding wins attributable to crippled baselines.
Use SfM points as a sampling prior, not a reconstruction input. Only overlap/scale statistics are consumed for batching; full per-scene SfM output is not needed at training time.

Limitations & Future Work¶

Depends on a reference image to define appearance. Without a reference, no "target appearance" exists; for implausible references (a completely different building type), tone transfers but semantic plausibility is not guaranteed.
Ablation and main-table numbers disagree. The Table 3 Full row is below Table 2; the text offers no explanation, so reproduction comparisons need care.
Pose advantage has boundaries. All three AUC metrics beat AnySplat but ATE is slightly worse (1.559 vs 1.525), and neither reaches the dedicated pose prior VGGT — the geometry branch's poses are merely "good enough."
Training needs synthesized relighting data. Appearance decoupling relies on TC-Light-generated multi-illumination paired data, capping the diversity of real illumination variation at the relighting model's fidelity.
Poses are not fully optimization-free. Evaluation uses evaluation-time pose alignment (fixed Gaussians, photometric optimization of target poses); "unposed" refers to training and the forward pass, while inference retains this light alignment step.
Dynamic objects are not handled. Transient occluders in wild photos (pedestrians, vehicles) are not explicitly modeled and are not shown in qualitative figures.

vs NeRF-W / GS-W / WildGaussians / Splatfacto-W / Wild-GS: they decouple appearance too, but all rely on per-scene optimization with precomputed poses; WildSplat moves decoupling into a feedforward network completed in one pass, at the cost of specifying appearance explicitly via a reference image instead of implicit per-image embeddings.
vs AnySplat / WorldMirror: also feedforward 3DGS/reconstruction, but appearance is entangled with geometry, so wild inputs produce mixed appearance; WildSplat's dual branch plus pre-modulation directly targets this failure mode.
vs NoPoSplat: also pose-free feedforward, but assumes photometric consistency; WildSplat inherits its "canonical frame + pose head" idea and adds appearance conditioning.
vs SWAG: also appearance-conditioned Gaussians, but optimization-based; the contrast highlights how fragile optimization-based conditioning is under sparse inputs.
vs TC-Light: a training-data augmentation tool synthesizing paired multi-appearance data; this hints that "generative models manufacturing supervision" may be a general recipe for wild 3D disentanglement.

Rating¶

Novelty: 4/5. First feedforward in-the-wild 3DGS with explicit appearance conditioning; the dual-branch plus multi-reference supervision combination is well targeted, though individual components (VGGT backbone, AdaLN, cross-attention) are not new.
Experimental Thoroughness: 4/5. Two in-the-wild benchmarks, 4/8/12-view settings, pose evaluation, and three ablations give broad coverage; the unexplained ablation/main-table gap and missing runtime comparison are gaps.
Writing Quality: 4/5. The method narrative and formulas are clear; some prose statements (AUC, ablation gap) do not fully sync with the tables.
Value: 4/5. A practical combination for wild sparse reconstruction plus appearance editing; single-forward deployment with light pose alignment is realistic, though no code link accompanies the main text (arXiv/code to be confirmed).