Free-Range Gaussians: Non-Grid-Aligned Generative 3D Gaussian Reconstruction

Conference: ECCV2026
Paper: ECCV paper
Area: 3D Vision
Keywords: Gaussian Splatting, sparse-view reconstruction, flow matching, hierarchical representation, generative completion

TL;DR

Free-Range Gaussians directly generates 8K Gaussians untied to pixel or voxel grids from four posed images, combining hierarchical pairing and reconstruction guidance to improve unseen-region completion, reaching 28.08 dB PSNR and 27.69 FID on partially observed GSO; its iterative generation is not single-pass reconstruction.

Background & Motivation

Sparse-view 3D reconstruction must represent not only visible surfaces but also the back of an object that was never photographed. Pixel-aligned methods such as LGM and GS-LRM anchor Gaussians to input pixels, potentially stacking redundant primitives on observed surfaces while leaving holes elsewhere. Generating additional views before reconstruction can improve coverage, but introduces camera-selection decisions and propagates inconsistencies between synthesized images into geometry.

Voxel-aligned methods such as LaRa can cover space outside the input views, yet remain constrained by grid capacity. With photometric regression alone, several plausible backsides can average into a blurry prediction. A generative model can instead sample a sharp completion, but weak image and pose conditioning may also alter the observed appearance or orientation. This paper therefore targets freely positioned Gaussians that remain faithful to the observations, rather than merely producing a plausible object.

Direct supervision of unordered Gaussian sets presents another difficulty: a predicted primitive has no obvious counterpart in the target set. Expensive set-matching losses increase computation and are imperfect proxies for rendering quality. Core Idea: use the elementwise pairing between target Gaussians and noise in flow matching to learn a conditional distribution, control token count with hierarchical local pairing, and feed input-view rendering errors back into generation.

Method

Overall Architecture

The input is a sparse set of images of one object with camera poses; the output is a parameter set directly usable for Gaussian Splatting. Before training, objects are fitted with Gaussians and organized into hierarchical trees. Training selects a level of detail and learns to denoise corrupted Gaussians. Inference starts from random noise and progressively generates the representation using image conditions and reconstruction feedback, without synthesizing additional images or passing through a voxel latent space.

The three central designs are parameter-space flow matching, hierarchical pairing, and reconstruction guidance. The tree organizes supervision: users do not supply a tree, and inference does not repeatedly spatially cluster the current noisy Gaussians.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Posed sparse images<br/>Training Gaussians / inference noise"] --> Flow["Parameter-space<br/>flow matching"]
    Flow --> Patching["Hierarchical pairing"]
    Patching --> Guidance["Reconstruction guidance"]
    Guidance --> Output["Non-grid-aligned Gaussians<br/>Novel-view rendering"]
    Guidance -->|Next denoising step| Flow

Key Designs

1. Parameter-space flow matching: establish supervision correspondence through the generative path

A Gaussian contains position, log-scale, quaternion rotation, logit-opacity, and RGB color, totaling 14 dimensions. Each parameter is independently standardized using training-set statistics so that differences in scale between position, size, and color do not dominate learning. Non-grid alignment means the final centers are not bound to image pixels or volume cells; it does not mean the training data has no organizational structure.

During training, each target Gaussian is paired with standard normal noise and interpolated along a linear path. The model predicts the clean sample rather than velocity directly. Its predicted endpoint and the current state determine the velocity used for Euler integration:

\[ z_t=(1-t)\epsilon+t z_1,\qquad \hat z_1=f_\theta(z_t,t,I),\qquad v_t=\frac{\hat z_1-z_t}{1-t}. \]

Here \(I\) denotes posed image conditions, \(\epsilon\sim\mathcal{N}(0,\mathbf{I})\), and \(t\in[0,1)\). The noise-target pairing fixes the supervision within each training example, avoiding a separate assignment between the predicted and target sets. It does not impose a cross-object semantic ordering. Fresh noise at inference produces a sample from the conditional reconstruction distribution rather than its average.

Image conditioning uses frozen DINOv2-Base features with 768 channels, concatenated with 32-channel positional embeddings, 3-channel downsampled RGB, and 6-channel Plucker ray coordinates, then projected to 1024 dimensions. Ray coordinates connect visual content to viewing geometry, allowing reconstruction in a shared coordinate frame rather than generation of an object with merely similar appearance.

Image tokens, paired Gaussian tokens, and 32 learnable register tokens enter the DiT together. It contains 18 Transformer blocks with 16 attention heads of 64 dimensions, timestep AdaLN, GEGLU, and RMSNorm-based QK-normalization for stability. Gaussian tokens receive no positional encoding, avoiding dependence on a prescribed global index.

2. Hierarchical pairing: preserve Gaussian capacity while shortening attention sequences

Assigning one token to every Gaussian makes self-attention expensive for large reconstructions. The authors first build a binary hierarchy over fitted target Gaussians, with each parent formed by a weighted combination of its two children. A tree depth therefore defines a uniform level-of-detail slice. Training starts at depth 11 with approximately 2K Gaussians and \(224\times224\) images, then fine-tunes at depth 13 with approximately 8K Gaussians and \(512\times512\) images. Only the selected level enters the model; parent nodes define the hierarchy.

The tree also supplies a substitute for neighboring patches on an irregular set: two sibling Gaussians are concatenated into one token. An \(N\times14\) parameter array becomes \((N/2)\times28\), is linearly projected into the Transformer, and is split back into individual Gaussians at the output. The default 8K Gaussians thus require approximately 4K Gaussian tokens, in addition to image and register tokens. Sequence length is reduced without halving output capacity.

Sibling pairing matters because the primitives are usually spatially close, letting the network jointly adjust local geometry and appearance. This differs both from random grouping and from keeping only 4K Gaussians. Inference uses the same paired representation interface, with local generative structure learned from training supervision. Neither the hierarchy curriculum nor pairing locks final centers to a fixed grid.

3. Reconstruction guidance: separate training supervision, network feedback, and sampling correction

Coarse level-of-detail slices lose detail, and parameter-space denoising alone does not guarantee faithful images. The model therefore renders its predicted clean Gaussians and measures L1 photometric error against input views and held-out training views. A timestep-dependent weight emphasizes this supervision late in denoising, when predictions are reliable enough to benefit from detailed image matching rather than noisy early gradients.

Another feedback path operates on the current noisy Gaussians, not the predicted endpoint. It renders the current state from the conditioning cameras and differentiates the seen-view photometric loss with respect to Gaussian parameters. These per-Gaussian gradients are concatenated to the network input during both training and inference, explicitly indicating how the present reconstruction should change. During inference, the same gradient also supplies external guidance in the Euler update, moving the state toward lower input rendering error with weight \(\lambda_{\mathrm{PG}}=50\).

These two uses must be distinguished: gradient concatenation lets the network learn how to use feedback, while external guidance directly adjusts the sampling trajectory. Both use conditioning views only, never held-out or evaluation views; held-out targets participate only in the training loss. Gradients are obtained directly from rendering the current Gaussians, unlike GSD's guidance that backpropagates through the denoiser to a noisy latent.

Finally, image features are zeroed with 10% probability during training to learn unconditional prediction. At inference, classifier-free guidance amplifies the difference between conditional and unconditional predictions:

\[ \hat z_1^{\mathrm{guided}}=\hat z_1^{\mathrm{uncond}}+s\left(\hat z_1^{\mathrm{cond}}-\hat z_1^{\mathrm{uncond}}\right),\qquad s>1. \]

The cached body does not specify the actual value of \(s\); the photometric guidance weight of 50 must not be substituted for it. The total-loss and external-guidance update equations in cached Section 3.3 are damaged by extraction. This note explains their mechanisms from the readable prose without reconstructing uncertain full equations.

Loss & Training

Training combines flow matching with timestep-weighted multi-view rendering supervision, using a cosine timestep schedule. Approximately 140K Objaverse training objects are each rendered from 40 views at \(512\times512\), fitted with 3DGS-MCMC, and organized into precomputed trees. The 3D targets are therefore fitted supervision rather than error-free direct measurements.

AdamW uses a learning rate decaying from \(10^{-4}\) to \(5\times10^{-6}\) with a cosine schedule and 2,000 warmup steps. The reported run uses 700K iterations on 64 NVIDIA H200 GPUs over 6 days. Inference performs 50 Euler steps with \(\Delta t=0.02\), applies the two guidance types at each step, and denormalizes the final parameters. One object takes approximately 26 seconds on a single H200.

Key Experimental Results

Main Results

Evaluation uses 500 held-out Objaverse objects and 1,030 GSO objects, the latter testing out-of-distribution transfer to scanned household items. Full observation clusters cameras into 4 groups using K-means and samples one image per group. Partial observation takes all four images from one group, concentrating input coverage on one side. Both settings use the same fixed evaluation-view set with broad angular coverage; its PSNR is not simply input-view reprojection accuracy.

The following selection from original Table 1 reports partial observation. PSNR is in dB and higher is better; FID is lower-is-better. ReconViaGen appears in qualitative comparisons, not in this quantitative table.

Method	Gaussian count	Objaverse PSNR	Objaverse FID	GSO PSNR	GSO FID
LGM	65K	22.88	88.43	19.65	54.96
LaRa	45K	27.79	93.19	25.64	69.18
GS-LRM	70K	27.90	65.06	27.57	34.13
Free-Range Gaussians	8K	29.92	43.58	28.08	27.69

Against GS-LRM, partial-observation PSNR improves by 2.02 dB on Objaverse and 0.51 dB on GSO; GSO FID decreases by 6.44. The Gaussian-count ratio is \(70/8=8.75\), describing representation size rather than inference speedup. GS-LRM is the authors' reimplementation, not an officially released implementation.

Original Table 2 exposes the full-observation trade-off. LPIPS below is multiplied by 1000 as in the source, with lower values better; these numbers must not be mixed with unscaled LPIPS.

Method	Objaverse PSNR (dB)	Objaverse LPIPS x1000	GSO PSNR (dB)	GSO LPIPS x1000
LaRa	31.91	43.60	29.15	60.70
GS-LRM	31.03	32.80	31.13	30.93
Free-Range Gaussians	31.66	53.15	31.49	77.25

The method slightly exceeds GS-LRM in GSO PSNR but has substantially worse LPIPS; its Objaverse PSNR also trails LaRa. The supported conclusion is competitive reconstruction with a compact representation and better partial-observation completion, not dominance across all conditions and metrics.

Ablation Study

Original Table 3 evaluates all 1,030 GSO objects and reports PSNR in dB. Absolute scores are reproduced below, avoiding the source's approximate parenthetical test-time drops that do not exactly match subtraction of the two-decimal scores.

Config	Full-observation PSNR	Partial-observation PSNR
Full model	31.49	28.08
No flow matching	28.40	26.30
No multi-view rendering loss	29.66	26.87
No LoD curriculum training	30.00	26.42
No pairing, 4K Gaussians	30.47	27.22
No pairing, 8K Gaussians	30.54	26.75
Random pairing	30.84	27.16
No rendering gradient concatenation	30.89	27.51
No classifier-free guidance at inference	31.30	27.50
No photometric gradient guidance at inference	31.10	27.90

Key Findings

• Removing flow matching gives the largest degradation: 3.09/1.78 dB for full/partial observation. This ablation also replaces the DiT with a vanilla Transformer using learnable latent tokens and photometric-only training; it is not simply switching off sampling.
• Removing the LoD curriculum reduces partial-observation PSNR by 1.66 dB; random pairing costs 0.92 dB. Local structure and training progression matter alongside token reduction.
• Original Table 4 evaluates one model fine-tuned with view counts sampled uniformly from 2 to 9: 2/4/6/9 inputs yield GSO full-observation PSNR of 28.22/31.49/32.04/32.39 dB. More views still help, but this is not zero-shot variable-view evaluation without adaptation.

Highlights & Insights

• The generative path supplies supervision correspondence as well as a prior. Noise-target pairing provides elementwise training signals for free Gaussian sets while avoiding deterministic averaging of unseen surfaces.
• The hierarchy serves both as a capacity curriculum and a local pairing rule. Packing two neighbors into one token retains output capacity better than deleting half the Gaussians, but does not imply a halving of total inference cost.
• Reconstruction gradients can be both learned inputs and external sampling constraints. The principle could transfer to other differentiable rendering representations, provided the feedback views are carefully separated from evaluation views.

Limitations & Future Work

• The authors acknowledge that 8K Gaussians limit fine geometry and texture, consistent with the full-observation LPIPS trade-off. Sparse or linear attention and Gaussian upsampling are proposed directions, not validated improvements here.
• Iterative sampling over 50 steps with rendering gradients takes approximately 26 seconds per object. Distillation or flow straightening may reduce the step count, but fewer Gaussians do not establish faster inference than single-pass baselines.
• Supervision quality depends on fitted Gaussians, which can omit geometry or contain artifacts under heavy occlusion. A plausible generated backside is not evidence of recovering the object's uniquely correct unseen structure.
• The current task requires camera poses and is evaluated on objects. These experiments do not establish pose-free, single-view, or large-scene capabilities; qualitative comparisons with pose-free ReconViaGen must retain this protocol distinction.
• The cached body omits the actual CFG scale and contains damaged equation extraction. Exact reproduction requires the intact paper and implementation rather than filling these gaps from the note.

Related Work & Insights

• vs GS-LRM / LGM: Pixel anchoring directly supports visible-region prediction but struggles with unseen coverage. Direct 3D set generation improves partial observations with fewer Gaussians at the cost of iterative inference.
• vs LaRa / TRELLIS: LaRa organizes reconstruction with voxels, while TRELLIS generates through structured latents. Learning directly in Gaussian parameter space removes fixed-grid anchoring without establishing superiority on every perceptual metric.
• vs GSD: Both use view guidance, but this method trains a dedicated multi-view conditional model and differentiates rendering of the current Gaussians without propagating that guidance through the denoiser.
• vs ReconViaGen: The latter combines a pretrained generator and VGGT features in a pose-free setting. The evidence here is primarily qualitative and should not be converted into quantitative cross-protocol claims.

Rating

• Novelty: 4/5. Parameter-space generation, hierarchical local pairing, and dual reconstruction feedback form a targeted solution.
• Experimental Thoroughness: 4/5. Two datasets, two coverage settings, and extensive ablations are useful, but large-scene and standardized runtime comparisons remain absent.
• Writing Quality: 4/5. Components map clearly to ablations; broad claims require qualification against full-observation perceptual results.
• Value: 4/5. A useful direction for compact Gaussian reconstruction with completion, still constrained by sampling latency and supervision cost.