EAGS: Error-Aware Gaussian Splatting with Dual-Confidence-Guided Modeling for Uncalibrated Driving Scenes

Conference: ECCV2026
Paper: ECCV Official Page
PDF: Full Paper
Area: Model Compression
Keywords: Gaussian pruning, dual confidence, error-aware generation, Gaussian distillation, driving scene reconstruction

TL;DR

EAGS reformulates feed-forward 3D Gaussian Splatting reconstruction from uncalibrated driving images as residual-driven generation and geometry-aware retention, using distillation to constrain the pruned representation and achieving 28.02 dB PSNR with 49,556 Gaussians per view on Waymo, versus DGGT's 27.41 dB and 152,292.

Background & Motivation

Adjacent driving frames overlap heavily, yet pixel-wise feed-forward Gaussian predictors often generate an entire representation for every frame. The same road surface or building is described repeatedly, while small depth and pose errors prevent those descriptions from aligning precisely. The result is floating artifacts, ghosting, and cross-view inconsistency. Methods such as DGGT already accommodate uncalibrated inputs, but not requiring external poses is different from knowing which geometry has already been reconstructed; the former does not automatically prevent duplicate generation.

Voxel-based methods reduce redundancy by grouping nearby observations in shared 3D cells, but face resolution and computational constraints. Another straightforward approach is to score and prune Gaussians. However, being needed by the current image does not imply being worth retaining: a Gaussian with unreliable depth may temporarily explain texture but become an artifact in another view, whereas reliable background geometry should be reused rather than regenerated. Moreover, rendering depends on a collective fit by the full set, so even removing low-scoring points can disrupt that cooperation.

EAGS therefore treats generation, lifetime, and post-compression quality together rather than merely reducing the final count. Core Idea: use reprojection errors to identify immediate generation needs, use depth reliability and temporal decay to distinguish temporary compensation from persistent backbone geometry, and train the pruned Gaussians to reproduce the full set's static rendering.

Method

Overall Architecture

The input is a sequence of driving images without supplied camera calibration parameters; the output is a compact Gaussian representation for novel-view rendering. EAGS follows VGGT's geometry encoding approach: DINOv2 extracts image tokens, alternating intra-frame and cross-frame attention fuses them, and shared features predict camera parameters, depth, and dynamic masks. Depth and camera estimates back-project pixels into Gaussian centers, using the first camera coordinate system as the common reference. Here, uncalibrated means that external camera poses are not required as input, not that training avoids depth or dynamic-region supervision.

The added management pipeline comprises Error-Aware Generation, Dual-Confidence Lifetime Management, and Confidence-Guided Pruning and Distillation. The first frame initializes the representation. Later frames project the historical static backbone into the current view and use residuals to select additional Gaussians. Static points are aggregated and pruned to update the reusable backbone, while dynamic points are aggregated only within a single frame and do not accumulate into persistent background geometry. During training, both full and pruned sets are rendered so that useful information can concentrate in the retained points.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Uncalibrated image sequence"] --> B["Geometry encoding and prediction<br/>Cameras, depth, dynamic masks"]
    B --> C["Error-Aware Generation"]
    H["Historical static backbone rendering"] --> C
    C --> D["Dual-Confidence Lifetime Management"]
    D --> E["Confidence-Guided Pruning and Distillation"]
    E --> F["Compact static backbone<br/>and single-frame dynamic Gaussians"]
    F -.-> H

Key Designs

1. Error-Aware Generation: inspect what the historical representation lacks before adding new points

A conventional pixel-wise generator sees the current image but cannot easily distinguish a richly textured region that is already reconstructed from a visually plain region that remains uncovered. EAGS first renders the current view with existing Gaussians and compares both image colors and Sobel gradients. Color residuals expose overall fitting errors, while gradient differences highlight missing edges and details. An error encoder with three convolutional layers transforms these discrepancies into features. A DPT Gaussian head combines them with DINO features and geometry-backbone features to jointly predict Gaussian attributes and generation confidence.

Generation confidence, denoted by \(C_g\), indicates whether adding a Gaussian at a pixel is worthwhile given the existing scene; it is not an ordinary object-presence probability. Binarization and sparsity constraints discourage ambiguous intermediate values and promote selective generation. Because the first frame has no history to compare against, its generation confidence is explicitly set to 1 and its validity mask to 0. Subsequent frames use a validity mask of 1 to tell the same prediction head to switch to residual-driven supplementation. This avoids a separate first-frame network and prevents initialization-wide missing coverage from being mistaken for a particular content feature.

2. Dual-Confidence Lifetime Management: a point needed now need not become persistent background

Retaining Gaussians solely by generation confidence would treat temporary appearance compensation as reliable 3D structure. EAGS introduces reuse confidence, first modulating generation confidence with depth confidence. The geometric modulation term explicitly given in the paper is

\[ C_g\bigl(\gamma+(1-\gamma)C_{\mathrm{depth}}\bigr). \]

Higher \(C_{\mathrm{depth}}\) indicates more reliable depth. The floor factor \(\gamma\) prevents currently useful but geometrically uncertain points from becoming immediately ineffective. Reuse confidence then decays with the time elapsed since creation: points with high initial reuse value decay slowly, while those with low value decay quickly. Rendering multiplies the original opacity by the current reuse confidence, gradually weakening temporary points instead of waiting for an abrupt hard deletion. The full lifetime equation is incompletely extracted in the cache, so only the verifiable modulation term is reproduced here, without guessing the precise decay-rate interpolation.

Together, the two confidences distinguish several roles. High generation demand with reliable depth suggests backbone geometry; high demand with unreliable depth suggests temporary compensation; low demand with reliable geometry usually indicates a redundant description of an already covered region; and points supported by neither are closer to noise. Dynamic masks also define the reuse boundary: static regions can accumulate across frames, whereas dynamic regions are compressed only within the current frame. This prevents old positions of moving objects from becoming persistent background, but also means the method does not establish cross-frame dynamic-object trajectories.

3. Confidence-Guided Pruning and Distillation: train retained points to take over information rather than only deleting points

On-demand generation cannot eliminate every duplicate description across frames. EAGS therefore groups spatially neighboring Gaussians into voxel clusters and aggregates their pre-activation features using normalized reuse-confidence weights. Voxel aggregation merges multiple descriptions of the same location, but averaging an entirely unreliable cluster can still preserve redundancy. Confidence-based gating addresses that remaining problem. The differentiable pruning mask in the paper is

\[ M_{\mathrm{prune}}=\operatorname{Sigmoid}\bigl(s(C_r^t-\tau)\bigr),\qquad s=5,\quad \tau=0.7. \]

Here, \(C_r^t\) is the current reuse confidence, \(s\) controls gate steepness, and \(\tau\) is the pruning threshold. Training uses continuous gating so confidence receives gradients; inference replaces it with hard pruning at a fixed threshold. Static regions undergo cross-frame aggregation and gating, whereas dynamic regions receive only single-frame voxel aggregation to avoid accumulating misalignment without reliable tracking.

Low confidence does not necessarily mean a point is dispensable. When colors, opacities, and confidences are jointly adapted to the full set, low-confidence points may still perform important rendering work. Gaussian distillation therefore encourages the pruned set's rendering to match the full set's rendering, with gradients stopped through the full-rendering branch. Supervision is restricted to the static mask, requiring the compact backbone to carry information previously distributed across more points. This is not a transfer of large-network parameters into a small network; it constrains functional consistency between full and sparse scene representations within the same model. The ablation discussion also explains why fitting the pruned result directly to the real image is not an equivalent substitute: the model tends to raise all confidence scores and revert to dense pixel-wise generation.

A Worked Example

Consider a roadside building becoming visible as a vehicle moves; this illustrates the mechanism rather than introducing an experiment. Frame 1 initializes Gaussians. Frame 2 renders the existing static backbone from the new camera position. Already explained wall surfaces have small residuals, while newly exposed corners and edges have larger residuals, so new points are concentrated in the latter. If a corner has reliable depth, its points can decay slowly and join the persistent backbone. If a point only compensates for uncertain depth, its influence fades more quickly.

Nearby points added to the same wall from multiple views are subsequently aggregated within voxels and compressed through gating. A foreground vehicle classified as dynamic retains only its current-frame representation, without writing its old position into the static backbone. Background revealed as it moves can be completed by later observations. Distillation requires the pruned set to preserve the static appearance already learned by the full set, rather than obtaining a low point count by discarding entire textured regions.

Loss & Training

Training uses four groups of objectives: L1 and LPIPS reconstruction between full renders and observed images; binary cross-entropy for dynamic masks and VGGT-style depth regression; binarization, sparsity constraints, and depth-guided regularization on generation confidence; and static-region rendering distillation from the full set to the pruned set. The depth-guided term stops gradients through depth confidence so the model cannot evade compression pressure by artificially reducing geometric reliability. The cached loss equations have missing symbols, and the prose also mentions a separate L1 sparsity term, so they do not support claiming an exactly recovered, reproduction-ready loss-weight configuration.

The model uses AdamW with a cosine learning-rate schedule; the reported prediction-head learning rate is \(5\times10^{-5}\). Training samples 4 images at intervals of 5 frames, with a resolution of \(518\times294\). SegFormer refines the dynamic supervision, and training runs for 4 epochs on two RTX 4090 GPUs. The geometric modulation parameter is \(\gamma=0.6\), and the decay-rate range endpoints are \(\sigma_{\min}=0.05\) and \(\sigma_{\max}=1.5\). Evaluation uses a single A100. Reconstruction is feed-forward after dataset-level training, not fresh optimization of every test scene.

Key Experimental Results

Main Results

Waymo evaluation uses 798 training scenes and 202 test scenes, reconstructing from 4 input frames and evaluating novel-view synthesis on interpolated intermediate frames. Higher PSNR and SSIM are better, while lower depth root mean square error, D-RMSE, is better. PGS denotes the average number of Gaussians per view; a lower value indicates a more compact per-view representation, not the total Gaussian count of a sequence. The following results come from Table 1. The time column reports inference time and must not be interpreted directly as rendering FPS.

Waymo Method	Pose-Free Input	PSNR ↑ / dB	SSIM ↑	D-RMSE ↓	PGS ↓	Inference Time ↓ / s
STORM	No	26.38	0.794	5.48	152292	0.18
UniSplat	No	27.82	0.830	Not reported	103984	0.18
DGGT	Yes	27.41	0.846	3.47	152292	0.39
VGGT++	Yes	22.50	0.749	3.80	152292	0.24
EAGS	Yes	28.02	0.849	3.44	49556	0.16

Relative to DGGT, which also requires no input poses, EAGS improves PSNR by 0.61 dB, reduces PGS by approximately 67.46%, and lowers reported inference time from 0.39 s to 0.16 s. Its PSNR advantage over UniSplat is only 0.20 dB, so the compactness gain should not be described as a large image-quality lead. Whether methods use input poses also remains relevant to the comparison.

All zero-shot experiments directly evaluate Waymo-trained models without fine-tuning on the target dataset. The following excerpt from Table 2 compares EAGS with the strong DGGT baseline. LPIPS measures perceptual distance, with lower values being better.

Zero-Shot Dataset	Method	PSNR ↑ / dB	SSIM ↑	LPIPS ↓
nuScenes	DGGT	25.31	0.794	0.152
nuScenes	EAGS	26.14	0.813	0.159
Argoverse2	DGGT	26.34	0.812	0.155
Argoverse2	EAGS	27.41	0.825	0.150

On nuScenes, EAGS improves PSNR by 0.83 dB but worsens LPIPS from 0.152 to 0.159. Only on Argoverse2 does it outperform DGGT on all three metrics. The paper's broad characterization of zero-shot superiority should therefore not be read as a lead on every metric.

Ablation Study

The following Waymo results come from Table 3, with 4 input frames for the full model. Each component removal should be evaluated jointly by quality and PGS: fewer points do not automatically indicate more effective compression.

Config	PSNR ↑ / dB	SSIM ↑	LPIPS ↓	PGS ↓
Without soft gating	27.43	0.812	0.169	89806
Without voxel aggregation	27.18	0.801	0.171	104434
Without depth-guided regularization	27.83	0.834	0.167	62121
Without Gaussian distillation	13.04	0.464	0.437	26201
Without error encoder	26.95	0.787	0.174	80229
Full model	28.02	0.849	0.157	49556

Key Findings

• Gaussian distillation is central to preserving quality. Removing it reduces PGS to 26,201 but lowers PSNR by 14.98 dB, showing that unconstrained pre-/post-pruning inconsistency can produce a small yet ineffective representation.
• Error feedback affects both quality and sparsity. Removing the error encoder lowers PSNR by 1.07 dB and increases PGS to 80,229, supporting the argument that current-image features alone cannot reliably identify existing coverage.
• Longer inputs do not guarantee monotonically better quality. The paper reports 28.32 dB and PGS 42,836 for 8 frames, versus 28.11 dB and PGS 38,794 for 16 frames. A declining per-view average does not establish that total memory stops growing.
• Randomly flipping dynamic masks with a probability of 20% reduces PSNR by only 0.12 dB, according to the paper. This supports tolerance to that synthetic perturbation, not to every form of real segmentation error.

Highlights & Insights

• Separating generation value from reuse value fits video reconstruction better than a single generic importance score. Residuals identify missing information now, while depth and time indicate whether that information should persist across viewpoints.
• Compression is part of training rather than an independent final operation. Distillation teaches the backbone to carry the full representation's rendering function before deletion, explaining why simple confidence-based removal fails.
• Different reuse scopes for static and dynamic regions provide an explicit engineering trade-off. They avoid erroneous long-term dynamic accumulation without presenting that robustness as complete dynamic tracking.

Limitations & Future Work

• The authors explicitly restrict dynamic-object redundancy reduction to individual frames and leave cross-frame spatiotemporal association to future work. The compact persistent backbone therefore primarily represents static scene content.
• Experiments mainly use low-resolution driving images, and sequence-length analysis covers only limited input lengths. They do not establish total memory or accumulated latency under full-resolution, continuously online operation.
• nuScenes LPIPS is worse than DGGT's, indicating that pixel-level and structural improvements do not fully translate into better perceptual distance. Results should be interpreted metric by metric.
• Several equations in the local full-text cache have extraction omissions, especially lifetime decay, sparsity loss, and distillation loss. Reproduction requires checking formulas and weights in the original PDF; conjectural completions are not presented here as paper definitions.

Related Work & Insights

• vs DGGT: Both address driving scene reconstruction without supplied poses. DGGT uses lifecycle modeling and motion interpolation, whereas EAGS emphasizes historical rendering residuals for generation control and training a compact backbone. Its advantage concerns the reported quality/count combination, not a claim that DGGT lacks temporal modeling.
• vs VolSplat / EVolSplat: Voxel-aligned or 3D-convolution methods use spatial aggregation as a primary modeling approach. EAGS still predicts Gaussians from 2D geometry features, using voxelization mainly for subsequent redundancy merging and further constraining compression through confidence and distillation.
• vs VGGT / AnySplat: Geometry backbones and Gaussian aggregation have precedents. The distinctive contribution is the combination of historical errors, dual-confidence lifetime management, and pruning consistency, rather than reinventing camera prediction or Gaussian rendering.

Rating

• Novelty: 4/5. Separating on-demand generation from persistent reuse and coupling both with representation distillation gives a more complete formulation than pruning alone.
• Experimental Thoroughness: 4/5. Main-dataset results, zero-shot transfer, multiple ablations, and mask perturbations are included, but longer-term resource curves and uncertainty statistics are missing.
• Writing Quality: 3/5. The overall mechanism is understandable, but cached equations are incomplete and broad zero-shot superiority claims require qualification from the tables.
• Value: 4/5. Useful for feed-forward driving reconstruction with controlled Gaussian counts, provided gains are interpreted alongside evaluation resolution and dynamic-modeling boundaries.