AVSplat:Dense-View Feed-Forward 3D Gaussian Splatting with Assist-View Preconditioning

Conference: ECCV 2026
Paper: ECCV Official
Area: 3D Vision
Keywords: 3D Gaussian Splatting, Feed-Forward Reconstruction, Pose-Free Reconstruction, Dense-View Scaling, Voxel Fusion

TL;DR

Addressing dense-view reconstruction degradation in pose-free feed-forward 3D Gaussian Splatting caused by attention dispersion and naive voxel averaging, AVSplat introduces Assist-View Preconditioning and Occupancy-guided Voxel Fusion with adaptive temperature, restoring positive view scaling as input views increase.

Background & Motivation

Pose-free feed-forward 3D Gaussian Splatting (3DGS) has emerged as a promising paradigm for novel view synthesis from uncalibrated multi-view collections. Building on geometry foundation models such as DUSt3R, MASt3R, and VGGT, feed-forward models predict camera intrinsics, extrinsics, and scene Gaussians in a single neural network pass. Unlike classical 3DGS, which requires per-scene optimization taking tens of minutes or hours, feed-forward frameworks achieve instant 3D reconstruction and real-time rendering. Intuitively, supplying more input views to such a system should provide tighter geometric constraints, resolve visual occlusions, and monotonically enhance novel view synthesis fidelity.

However, existing feed-forward architectures—such as AnySplat—frequently exhibit a counterintuitive performance plateau or degradation when scaling from sparse inputs (3 to 16 views) to dense inputs (32 to 72 views). This breakdown stems from two structural bottlenecks. First, in the Transformer-based multi-view global aggregation stage without prior camera poses, the network relies on global Softmax attention across all visual tokens to discover 3D correspondences. As the input sequence length grows, the attention distribution diffuses across an inflating pool of unrelated candidate tokens, diluting the attention mass on true geometrically matching patches and undermining multi-view consistency. Second, in the voxel fusion stage used to control Gaussian redundancy, all Gaussians mapped into the same spatial voxel are averaged uniformly. Under dense inputs, voxel occupancy escalates dramatically; uniform averaging acts as an aggressive low-pass filter, where a few highly accurate, confident Gaussians are washed out by numerous weak contributors, sacrificing high-frequency geometric and textural details.

To resolve these bottlenecks, the goal of this work is to convert additional dense input views into supportive geometric signals rather than destructive noise. The core idea is to introduce a lightweight Assist-View Preconditioning step before global aggregation to inject compact, complementary scene context into each view, paired with an Occupancy-guided Voxel Fusion mechanism that dynamically sharpens fusion temperatures in crowded voxels, completely preventing dense-view performance degradation.

Method

Overall Architecture

AVSplat takes an uncalibrated multi-view image sequence as input. A geometry-aware encoder first extracts local patch tokens and computes a global descriptor for each view. Before entering expensive multi-view global aggregation, the framework applies Assist-View Preconditioning (AVP): using the view descriptors and a token coverage objective, it greedily selects a compact set of complementary, highly relevant assist views and performs a single lightweight cross-view interaction to pre-condition each view with focused scene context. Next, the preconditioned tokens are processed by global geometric attention layers and decoding heads to predict camera poses, depth maps, and a dense set of 3D Gaussian primitives. Finally, the predicted Gaussians are passed to Occupancy-guided Voxel Fusion (OVF), where fusion temperature is adaptively modulated based on spatial dispersion and voxel occupancy before differentiable Gaussian rasterization renders the target views.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}%%
flowchart TD
    A["Uncalibrated Multi-View Input Images"] --> B["Geometry-Aware Feature Extraction<br/>Patch Tokens & View Descriptors"]
    B --> C["Coverage-Driven Assist View Selection<br/>Descriptor Filtering + Submodular Greedy Search"]
    C --> D["Lightweight Assist-View Preconditioning<br/>Cross-Attention Injection & Gated Residuals"]
    D --> E["Global Multi-View Geometry Aggregation & Decoding<br/>Camera Poses, Depth Maps & Gaussian Primitives"]
    E --> F["Adaptive Temperature Voxel Fusion<br/>Occupancy Modulation & Geometric Variance Truncation"]
    F --> G["Fused 3D Gaussian Primitives & Differentiable Rasterization"]

Key Designs

1. Assist-View Preconditioning: Structuring local complementary context to focus global correspondence

Global attention tends to disperse over long sequences, leading to ambiguous correspondence matching. Instead of directly feeding raw tokens into all-to-all global attention, AVSplat selects \(K\) assist views \(\mathcal{A}(s)\) (typically \(K=4\)) for each view \(s\). An initial candidate pool \(\mathcal{N}(s)\) is formed by selecting the Top-\(M\) nearest neighbors based on cosine similarity of mean view descriptors \(\mathbf{z}_s\). To prevent selecting near-duplicate views or leaving regions of the reference view uncovered, AVSplat defines a submodular coverage objective function:

\[ F(\mathcal{A}) = \frac{1}{P}\sum_{p=1}^{P}\max_{t\in\mathcal{A}} s_{t,p} + \beta\,\frac{1}{K}\sum_{t\in\mathcal{A}} q_t - \lambda\,\frac{1}{K(K-1)}\sum_{t,u\in\mathcal{A}, t\neq u} m_{tu} \]

where the first term measures the maximum projected token similarity support for each patch \(p\) in view \(s\), \(q_t\) evaluates candidate view reliability via token variance, and the third term penalizes pairwise redundancy among selected assist views. Because the token coverage term is monotone submodular, greedy forward selection \(\mathcal{A}_{k+1} = \mathcal{A}_k \cup \{\arg\max_{t} \Delta F(t \mid \mathcal{A}_k)\}\) yields an efficient near-optimal assist set. Each view then conducts a single cross-attention pass over its assist set tokens to obtain \(\widetilde{\mathbf{X}}_s = \operatorname{softmax}\bigl(\frac{1}{\sqrt{d}}\mathbf{Q}_s \mathbf{K}_s^\top\bigr)\mathbf{V}_s\), which is added via a training-step gated residual \(\mathbf{X}_s^{\text{out}} = \mathbf{X}_s + \gamma(g)\widetilde{\mathbf{X}}_s\). This shifts the burden of establishing local scene context prior to global aggregation, ensuring that global attention remains focused on plausible geometric matches.

2. Occupancy-guided Voxel Fusion: Preserving fine details via occupancy-aware adaptive temperatures

Standard voxel fusion averages all Gaussians within a voxel \(\mathcal{G}_v\), which causes severe over-smoothing when dense views generate high voxel occupancy \(n_v = |\mathcal{I}_v|\). AVSplat introduces an adaptive temperature \(\tau_v\) modulated by occupancy and spatial geometric consistency:

\[ \tau_v = \operatorname{clip}\left(\tau_{\min}, \; \tau_0 \left(\frac{n_v}{n_{\mathrm{ref}}}\right)^{-\alpha} \psi_v, \; \tau_{\max}\right) \]

where base temperature \(\tau_0=0.7\), occupancy exponent \(\alpha=0.5\), reference count \(n_{\mathrm{ref}}=8\), and bounds \([\tau_{\min}, \tau_{\max}]=[0.15, 2.0]\). The reliability factor \(\psi_v = \frac{1}{1 + \lambda \sigma_v^2}\) is derived from the spatial dispersion \(\sigma_v^2 = \frac{1}{n_v}\sum_{i\in\mathcal{I}_v}\|\mathbf{p}_i - \bar{\mathbf{p}}_v\|^2\). In high-occupancy voxels with strong geometric agreement, \(\tau_v\) decreases, sharpening the Softmax weights \(w_i = \frac{\exp(\tau_v c_i)}{\sum_j \exp(\tau_v c_j)}\) so that the most confident and geometrically consistent Gaussians dominate the fused attributes. In low-occupancy regions, a higher temperature ensures balanced, inclusive fusion. Candidates are sorted by weight and truncated at a cumulative top-mass threshold of 0.8 to discard outlier noise before computing the final fused attributes \(\mathbf{f}_v\) and positions \(\mathbf{p}_v\).

Loss & Training

The framework is trained end-to-end on DL3DV (over 10,000 real-world scenes). Optimization utilizes AdamW with an initial learning rate of \(1\times 10^{-4}\), weight decay of \(5\times 10^{-2}\), a cosine learning rate decay schedule, and 2,000 warmup steps over 40,000 total iterations. The model is trained on a single NVIDIA A100 (80GB) GPU with a batch size of 1, utilizing mixed precision and activation checkpointing. Training inputs randomly sample 4 to 24 views per batch. Gradients through the assist branch are stopped for the first 1,500 steps, and the gating factor \(\gamma(g)\) ramps linearly from 0 to 1 between step 2,000 and step 10,000 to ensure stable convergence.

Key Experimental Results

Main Results

The authors benchmark AVSplat against state-of-the-art feed-forward methods (including NoPoSplat, FLARE, and AnySplat) on Mip-NeRF 360, VR-NeRF, and DL3DV (140 test scenes). In dense input regimes (32, 48, and 64 views), baseline methods like NoPoSplat and FLARE fail due to Out-Of-Memory (OOM) errors, while AVSplat consistently demonstrates stable scaling and superior reconstruction fidelity.

Table 1: Quantitative results under dense-view inputs (extracted from original paper Table 1)

Dataset	Views	Metric	AnySplat (Baseline)	AVSplat (Ours)	Relative Gain / Trend
DL3DV	32 views	PSNR (dB) ↑ / SSIM ↑ / LPIPS ↓	21.05 / 0.680 / 0.284	21.54 / 0.710 / 0.266	+0.49 dB / SSIM +0.030
DL3DV	48 views	PSNR (dB) ↑ / SSIM ↑ / LPIPS ↓	21.04 / 0.678 / 0.281	21.60 / 0.713 / 0.265	+0.56 dB / Continuous gain
DL3DV	64 views	PSNR (dB) ↑ / SSIM ↑ / LPIPS ↓	21.14 / 0.684 / 0.282	21.67 / 0.717 / 0.265	+0.53 dB / No degradation
Mip-NeRF 360	32 views	PSNR (dB) ↑ / SSIM ↑ / LPIPS ↓	18.22 / 0.445 / 0.369	18.71 / 0.483 / 0.365	+0.49 dB / SSIM +0.038
Mip-NeRF 360	64 views	PSNR (dB) ↑ / SSIM ↑ / LPIPS ↓	19.23 / 0.500 / 0.365	20.11 / 0.528 / 0.366	+0.88 dB / SSIM +0.028
VR-NeRF	64 views	PSNR (dB) ↑ / SSIM ↑ / LPIPS ↓	21.26 / 0.719 / 0.350	21.70 / 0.723 / 0.363	+0.44 dB

Ablation Study

The ablation study conducts systematic evaluations on DL3DV under the 64-view dense input setting.

Table 2: Ablation study on DL3DV with 64 input views (extracted from original paper Table 2)

Configuration	PSNR (dB) ↑	SSIM ↑	LPIPS ↓	Key Contribution
Baseline (AnySplat base)	21.14	0.684	0.282	Diffuse attention and over-smoothed voxel averaging
+ Fusion only (OVF only)	21.57	0.692	0.272	Drives major single-point image quality gain (+0.43 dB)
+ Assist + Fusion (Full AVSplat)	21.67	0.717	0.265	Further stabilizes long-sequence global correspondence

Table 3: Dense-view attention diagnostics & overlap stress test (extracted from original paper Table 3)

Diagnostic Category	Metric / Condition	AnySplat (Baseline)	AVSplat (Ours)	Diagnostic Finding
Attention Diagnostics (16 views)	Normalized Entropy ↓	0.693	0.642	Sharper, more concentrated attention distribution
Attention Diagnostics (16 views)	Epipolar Mass (GeoMass@1patch) ↑	0.286	0.374	+30.8% relative gain on geometrically valid band
ACID Overlap Stress Test	PSNR @ 48 views (dB) ↑	23.12	23.15	Comparable baseline on heavy visual overlap
ACID Overlap Stress Test	PSNR @ 72 views (dB) ↑	22.45	23.24	Baseline drops 0.67 dB; AVSplat gains +0.09 dB (+0.79 dB gap)

Key Findings

• Complementary Roles of Dual Modules: OVF delivers the majority of single-point visual fidelity improvements at 64 views (PSNR climbs from 21.14 dB to 21.57 dB), whereas AVP resolves the underlying cause of dense-view scaling collapse by maintaining focused cross-view attention across long sequence lengths.
• Overcoming the Dense-View Degradation Curse: In the ACID heavy-overlap stress test, increasing input views from 48 to 72 causes AnySplat to drop sharply from 23.12 dB to 22.45 dB (a 0.67 dB loss), whereas AVSplat continues upward to 23.24 dB, opening an impressive 0.79 dB margin at 72 views.
• Measurably Sharper Epipolar Concentration: AVSplat increases GeoMass@1patch from 0.286 to 0.374 (a 30.8% relative gain), concentrating attention probability mass on epipolar bands to 3.7× that of a uniform random distribution (0.10).

Highlights & Insights

• Submodular Greedy View Selection: Formulating assist view selection as a submodular coverage objective elegantly balances token-level perceptual coverage with viewpoint diversity, obtaining high-quality complementary subsets in near-linear time.
• Adaptive Temperature Attribution: Grounding Gaussian voxel fusion in an occupancy-aware thermodynamic temperature framework gracefully reconciles the trade-off between retaining fine surface textures in dense clusters and preserving robustness in sparse areas.
• De-noising Long-Range Geometric Attention: Demonstrates that rather than relying solely on global multi-head self-attention to sift through unposed views, pre-conditioning tokens via targeted local interactions significantly mitigates attention dispersion in long-sequence transformers.

Limitations & Future Work

• Inactive in Sparse Settings: AVP requires an adequate pool of candidate viewpoints to formulate a diverse assist set; consequently, it must be disabled in extremely sparse configurations (3-4 views).
• Static Voxel Discretization: The fusion step still operates over uniform, predefined Cartesian voxel grids, which may struggle to adapt optimally across extreme scale disparities in unbounded outdoor environments.
• Future Directions: Exploring adaptive octree or graph-based hierarchical clustering for voxel-free Gaussian attribution, and unifying assist view selection into an end-to-end differentiable routing mechanism.

Related Work & Insights

• vs AnySplat: AnySplat established VGGT-based pose-free feed-forward 3DGS, but suffers from diffuse global attention and over-smoothed uniform voxel fusion under dense inputs. AVSplat introduces AVP and OVF to resolve these dual flaws, unlocking positive scaling without extra memory overhead.
• vs NoPoSplat / FLARE: NoPoSplat and FLARE cater primarily to sparse settings (3-16 views) and trigger out-of-memory errors beyond 32 views. AVSplat handles dense sequences up to 72+ views effortlessly.
• vs ZPressor: ZPressor relies on bottleneck latent compression to ingest 100+ views. In contrast, AVSplat preserves spatial detail by injecting context early and sharpening voxel fusion late, offering complementary benefits for extreme dense-view reconstruction.

Rating

• Novelty: ⭐⭐⭐⭐ [Well-motivated solutions targeting dense-view attention dispersion and voxel smoothing in feed-forward 3DGS]
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive evaluations across DL3DV, Mip-NeRF 360, VR-NeRF, attention entropy diagnostics, and ACID stress testing]
• Writing Quality: ⭐⭐⭐⭐⭐ [Clear problem formulation, rigorous mathematical definitions, and comprehensive ablation breakdowns]
• Value: ⭐⭐⭐⭐ [Provides crucial architectural insights for scaling feed-forward 3D Gaussian Splatting to photo-dense capture setups]

Related Papers

• [ECCV 2026] ViewSplat: View-Adaptive 3D Gaussian Splatting for Feed-Forward Synthesis
• [CVPR 2026] EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images
• [ECCV 2026] Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs
• [ECCV 2026] Sparse-View Surface Reconstruction using Gaussian Splatting through High-Confidence Depth Propagation with Normal Priors
• [ECCV 2026] Fast and Compact 3D Gaussian Splatting with Polarized Opacity Prior