PixGS: Pixel-Space Diffusion for Direct 3D Gaussian Splat Generation

Conference: ECCV 2026
Paper: ECCV
Area: 3D Vision
Keywords: 3D Gaussian Splatting, Pixel-Space Diffusion, Flow Matching, Text-to-3D, Image-to-3D

TL;DR

PixGS lays out every 3D Gaussian's attributes as a multi-view "attribute image" and denoises that attribute tensor directly with a pre-trained pixel-space diffusion model (PixNerd, ~45M-image priors) under a flow-matching objective, combining 2x super-resolution rendering supervision, depth/normal and multi-scale LoG regularization, and a three-phase training curriculum to produce a 3DGS asset in about one second on a single A100 for either text or image conditions — beating DiffSplat, TRELLIS, GaussianCube, LGM and DreamGaussian on both T3Bench and GSO.

Background & Motivation

Generating 3D content from text or a single image currently follows one of three routes, each with its own bottleneck. Optimization-based methods (SDS from DreamFusion, DreamGaussian) borrow the visual richness of 2D diffusion and optimize a 3DGS per scene, needing no ground-truth 3D data, but they re-optimize for every prompt, are extremely slow, and often produce over-smoothed geometry plus multi-view inconsistencies such as the "Janus" problem. 3D-native methods (TRELLIS, GaussianCube, DiffusionGS) learn 3D distributions straight from geometric data and are fast and geometrically consistent, but they are capped by the size of 3D datasets — orders of magnitude smaller than web-scale image corpora — so they must throw enormous model capacity at textures and topology, and they generalize poorly. Reconstruction-based two-stage methods (multi-view 2D diffusion plus a separate reconstructor, e.g. LGM, InstantMesh) first generate multi-view-consistent images with SyncDreamer/MVDream and then map them to 3D, but the reconstructor treats those generated images as fixed ground truth: even a slight discrepancy in perspective or texture between views accumulates into visible floaters. DiffSplat recently tried to skip the "generate images, then reconstruct" detour by arranging 3DGS as structured 2D attribute tensors and fine-tuning a pre-trained latent diffusion model to synthesize the splats directly, gaining both multi-view consistency and 2D priors. Its cost, however, is concrete: the pipeline needs three interdependent models — a 3DGS reconstructor, a VAE that compresses attributes into a latent space, and a latent diffusion model — that must converge in sequence, making training expensive and fragile with substantial engineering overhead; and to keep the memory cost of repeated Gaussian decoding tractable during diffusion training, DiffSplat trains an additional from-scratch lightweight decoder, adding yet another layer of system complexity.

The real tension is this: one wants the strong 2D diffusion priors, the view consistency of a structured representation, and the expressiveness of not going through lossy compression — but under the latent route these three are mutually exclusive. A latent is a compressed representation, so the model never sees the Gaussian attributes it will ultimately deliver; high-frequency detail is wiped out in the encode-decode round trip and comes back as artifacts. Cascaded training lets each stage's error accumulate downward, and final quality is locked by the product of per-stage capacities. Meanwhile, image generation itself has already shown another way: pixel-space diffusion models such as PixNerd, PixelFlow and PixelDiT demonstrate that high-fidelity generation without a VAE is possible by learning a velocity field on the raw pixel manifold, trading large patches for compute efficiency and recovering high-frequency detail with a patch-specific, coordinate-conditioned neural field. Since Gaussian attribute tensors already look like images — continuous values, arranged per pixel, multi-channel — that route transfers directly.

This paper performs exactly that transfer: drop the latent, treat the "4 views x 12 attributes per Gaussian" tensor as the pixel manifold itself, predict and regularize at the level of real splat attributes at every timestep, and demote the reconstructor to a source of pseudo-labels that only bootstraps convergence. Core idea: treat the Gaussian attribute tensor as a multi-channel pixel manifold and denoise it directly with flow matching, replacing the "reconstructor + VAE + latent diffusion" cascade with a single-stage model that inherits 2D image priors, then free generation quality from the pseudo-label ceiling using rendering-level multi-task supervision (2x super-resolution appearance, depth and normals, multi-scale LoG).

Method

Overall Architecture

PixGS is a single-stage model: the input is a text prompt or a single reference image (plus camera pose) and the output is a set of 3D Gaussians. There is no VAE, no latent, and no second generative stage on the inference path — the model denoises an "attribute image" in which each pixel happens to be one Gaussian. The representation follows Splatter Image / DiffSplat: a grid of \(V_{in}\) viewpoints at \(H\times W\) each, where every pixel stores 12 Gaussian parameters (3 color + 3 scale + 4 rotation quaternion + 1 opacity + 1 depth; this split is inferred from \(g_i\in\mathbb{R}^{12}\), ⚠️ refer to the original paper). A whole 3D asset is therefore a \(V_{in}\times g\times H\times W\) tensor — essentially a multi-view image set with the RGB channels replaced by Gaussian attribute channels. That \(g\)-dimensional map is not a rendered RGB image but an attribute canvas where each pixel carries its own Gaussian; the position is not regressed directly, it is unprojected from the pixel's depth \(d\) and pixel coordinate \(u\) through the camera intrinsics and extrinsics, \(x=R^\top K^{-1}[u\,|\,d]-t\), which is what keeps the network operating on a regular image-like grid.

The generative backbone adapts PixNerd and inherits 2D priors pre-trained on roughly 45 million images, learning a velocity field under the Flow Matching formulation that pushes a Gaussian noise tensor toward the data tensor. To give the model geometric cues, the noisy tensor additionally carries per-viewpoint Plücker coordinates (camera rays), so the channel count becomes \(c=g+d_{pl}\). Diffusion operates at patch level (16x16 patches flattened into tokens), and after the transformer each patch's own small MLP decodes the velocity pixel by pixel; cross-view consistency is handled by multi-view attention and multi-view RoPE2d. The training side adds rendering-level multi-task supervision — appearance loss rendered at twice the resolution, geometric loss from rendered depth and normals, high-frequency loss from multi-scale LoG — plus a three-phase curriculum that confines pseudo-labels to the bootstrapping role. At inference a single denoising pass yields the final Gaussians in about one second on a single A100.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["text prompt / reference image + pose"] --> B["direct pixel-space denoising<br/>Gaussian attribute tensor, no VAE"]
    B --> C["multi-view attention + multi-view RoPE2d<br/>global cross-view token interaction"]
    C --> D["3DGS asset (~1 s)"]
    C -.->|training-time supervision| E["rendering-level multi-task supervision<br/>super-res appearance + depth/normal + LoG"]
    E --> F["three-phase curriculum + rotation regularizer"]
    F -.->|updates the generative model| B

Key Designs

1. Direct pixel-space denoising of the Gaussian attribute tensor: trading the VAE and the cascade for 2D image priors

Methods like DiffSplat compress Gaussian attributes into a latent and diffuse there, which costs two hard things. The first is training organization: reconstructor, VAE and latent diffusion must converge in sequence, and a poorly trained earlier stage invalidates the later one, so the whole system is expensive and brittle. The second is more fundamental — a latent is a compressed representation, so the model never sees the Gaussian attributes it will actually deliver; high-frequency information is lost in the encode-decode round trip and returns as detail artifacts, while final quality is locked by the product of per-stage capacities. PixGS instead defines the probability path on the Gaussian attribute tensor itself: the data tensor \(G_1\) and the noise tensor \(G_0\) are linearly interpolated into \(G_t=tG_1+(1-t)G_0\), and the model learns the constant velocity \(G_1-G_0\) pointing from noise to data.

This definition buys three things. First, every channel of \(G_t\) is a genuine Gaussian attribute (opacity, scale, rotation, color, depth), so at every timestep the model is predicting the parameters it will actually deliver, and rendering- and geometry-level regularization can be applied to those parameters directly rather than to an intermediate latent. Second, with no VAE there is no encode-decode error and no compression capacity ceiling — fidelity of geometry and texture is bounded only by the model itself. Third, inputs and outputs are both image-like tensors, so weights pre-trained on large-scale image corpora can be reused directly, instead of having to learn basic visual semantics from scratch the way DiffusionGS does; this is precisely why PixGS still generalizes despite a small 3D corpus. Exploiting those 2D priors also requires solving conditioning: text conditioning reuses the original text encoder path, while for image conditioning the paper compares two options. Viewpoint concatenation projects the reference image's RGB channels to the attribute dimension \(g\) with a \(1\times1\) convolution, attaches the reference camera's Plücker coordinates, and concatenates the result along the view dimension as an extra "viewpoint" of the noisy tensor. Image prompt adapter extracts features with DINO-v2, injects viewpoint information through an AdaLN module that scales and shifts the embedding according to the reference Plücker coordinates, and prepends the image embedding — projected to the text-embedding dimension — to the text tokens. Experiments show the two are nearly tied (PSNR 30.36 vs 30.31), with viewpoint concatenation more parameter-efficient (1.2B vs 1.4B); the authors attribute this to view-axis concatenation preserving the reference image's 2D spatial structure, which gives denser and more direct cross-view attention. Recovering Gaussians from pixels is then straightforward: at \(t=1\) one obtains \(\hat G_1\), and the 12 numbers at each pixel are that Gaussian's attributes; depth and pixel coordinate unproject to the center, rotations are L2-normalized and the remaining attributes clamped into valid ranges during the forward pass, and the result is handed to 3DGS rasterization.

2. Multi-view attention and multi-view RoPE2d: letting patches from different views see each other in one coordinate frame

Although the attribute tensor packs multiple viewpoints into a single tensor, standard pixel diffusion computes attention inside a single view: after patchification the token sequence is \(X\in\mathbb{R}^{(B\cdot V_{in})\times L\times D}\), and if the view dimension is folded into the batch, the viewpoints become unrelated independent samples that each generate on their own — importing the very view-inconsistency problem of the "generate multi-view images, then reconstruct" route, which is the origin of floaters.

The paper attacks both attention and positional encoding. For attention, the token sequence is reshaped into \(\mathbb{R}^{B\times(V_{in}\cdot L)\times D}\) before the attention computation, so any patch can attend to all patches of all viewpoints of the same object, making cross-view information exchange explicit. For positional encoding, the original RoPE2d assigns coordinates only within one viewpoint's \(H\times W\) grid; here the \(V_{in}\) viewpoints are treated as spatially aligned segments of a high-resolution composite image, scaling the RoPE2d coordinate manifold to a virtual resolution of \((\frac{V_{in}}{2}H)\times(\frac{V_{in}}{2}W)\) (⚠️ this is how the paper writes it; given the "arranged into one composite image" semantics it more likely means laying the \(V_{in}\) viewpoints out on a 2D grid — \(2\times2\) when \(V_{in}=4\) — yielding the stated virtual resolution; refer to the original paper for the exact form), assigning unique positional embeddings across all \(V_{in}\cdot L\) tokens. The transformer thus learns relative spatial relationships that hold not only within one view but across the entire multi-view collection: the projections of the same 3D point in different views land at different coordinates of one shared coordinate frame at the positional-encoding level, giving geometric consistency an explicit inductive bias instead of relying on post-hoc alignment.

There is also an easily overlooked detail inherited from PixNerd: the transformer's output is not pixel values but the weights of a patch-specific MLP, \(\mathcal{W}^n_v=\text{Linear}(\text{SiLU}(X^n_v))\), which then decodes the velocity at each in-patch pixel coordinate \((v,i,j)\) from the concatenation of a coordinate encoding and the noisy input along the channel axis. The transformer thus handles global reasoning at patch granularity while fine per-pixel detail is filled in by a coordinate-conditioned neural field — the key to keeping high frequencies under large-patch pixel diffusion, and the reason it transfers directly to Gaussian attributes.

3. Rendering-level multi-task supervision: 2x super-resolution appearance loss plus depth/normal and multi-scale LoG high-frequency regularization

Flow matching alone amounts to fitting the pseudo-label distribution. Because the model's output is the Gaussian attributes themselves, small attribute errors — especially for near-zero-opacity background Gaussians — render as persistent floaters; and since the pseudo-labels come from a reconstructor of limited capacity, pure distribution matching caps final quality at that reconstructor. A more practical problem is numerical stability: in early iterations the attributes are very noisy, and applying rendering losses directly yields unreliable gradients because coordinates, color and opacity fall outside their valid ranges and rotations are not unit-norm, so training blows up into NaN (the "flow matching + appearance loss" row of the ablation table is exactly NaN).

The paper therefore adds three families of supervision that act on differentiably rendered results. The appearance loss renders the predicted Gaussian tensor at \(2H\times2W\) — twice the resolution of the attribute grid — computing MSE plus LPIPS on RGB and MSE on the silhouette mask, averaged over \(V\) supervision views. Rendering at twice the resolution rather than the native one is the crucial step of this design: the native grid's resolution ceiling is also the Gaussian-density ceiling, so comparing at 2x multiplies the spatial density of the supervision signal, provides denser gradients, and forces the model to concentrate Gaussian density on valid object surfaces and recover high-frequency detail beyond the base grid resolution, incidentally suppressing the low-opacity background pseudo-Gaussians. The geometry loss supervises with rendered depth and rendered surface normals, writing the implicit geometric constraint into the objective. The high-frequency loss uses a multi-scale Laplacian of Gaussian: rendered and GT images are blurred with Gaussian kernels at several standard deviations \(\sigma\), the Laplacian is taken, and the per-scale differences are compared, with scale weights \(\omega_\sigma\propto\sigma^2\) balancing sharp features against rendering noise. Its role is to make high-frequency alignment explicit in the loss and thereby mitigate the over-smoothing typical of diffusion models; the paper's visualizations show noticeably sharper geometric boundaries and textures — text and thin structures in particular — once LoG is added.

4. A three-phase training curriculum and a rotation-norm regularizer: learn the distribution first, then learn generation quality

Pseudo-labels are convenient — GSRecon's output already lies in the valid Gaussian attribute distribution and lets the model converge quickly — but they remain the reconstrutor's output, and depending on them throughout caps the ceiling. Ground-truth rendering supervision, on the other hand, corresponds directly to final quality but cannot be applied from the start: with the attribute distribution still wrong, rendering losses crash training outright. Both constraints point at the same remedy: split training into ordered phases.

Phase 1 spends 50K iterations on flow matching against pseudo-labels only, teaching the model what a valid Gaussian looks like. Phase 2 combines flow matching with GT rendering supervision as a smooth transition. Phase 3, the longest at 150K iterations, uses GT-only supervision and is what genuinely determines final quality. The paper stresses repeatedly that pseudo-labels only bootstrap convergence and improve stability rather than determining final quality, which the ablation confirms directly: replacing GSRecon with the weaker LGM pseudo-labels barely changes final GSO performance (PSNR 21.15 vs 21.21) at the cost of about 10% more training. The curriculum itself pays off clearly too — moving from phase 2 to phase 3 lifts PSNR from 30.36 to 31.15 and drops LPIPS from 0.022 to 0.019.

Beyond the curriculum, an explicit rotation-norm regularizer is required. The diffusion objective cannot by itself learn that a quaternion must have unit norm; although the forward pass L2-normalizes rotations and clamps the remaining attributes, those operations introduce discrepancies that make the rendering-loss gradients unreliable. Adding a penalty that pulls the rotation norm toward 1 removes the numerical instability — in the ablation it is precisely this regularizer that rescues the NaN row into 30-level convergence. The curriculum also brings an incidental efficiency gain: after phase 1 the background Gaussians have been suppressed to near-zero opacity, which the paper credits for the shorter iteration times in phases 2 and 3.

Loss & Training

⚠️ The PDF-extracted formulas in the cache are badly corrupted (operators and norms lost). The formulas below are reconstructed from the paper's prose and standard forms; the weight values are not given in the paper's main text and are subject to the original paper / appendix.

The flow-matching loss trains the model to predict the constant velocity from noise toward data:

\[\mathcal{L}_{\text{FM}}=\mathbb{E}_{t,\,G_0,\,G_1}\left\|v_\theta(G_t,t)-(G_1-G_0)\right\|^2,\qquad t\sim\mathcal{U}[0,1]\]

The appearance loss is computed after rendering at \(2H\times2W\) (\(I_v\) and \(M_v\) are the differentiably rendered RGB image and silhouette mask):

\[\mathcal{L}_{\text{app}}=\frac{1}{V}\sum_{v=1}^{V}\Big(\mathcal{L}_{\text{MSE}}(I_v,I_v^{\text{GT}})+\lambda\,\mathcal{L}_{\text{LPIPS}}(I_v,I_v^{\text{GT}})+\mathcal{L}_{\text{MSE}}(M_v,M_v^{\text{GT}})\Big)\]

The geometry loss supervises with rendered depth \(D_v\) and surface normals \(N_v\), using a cosine distance for the normal term:

\[\mathcal{L}_{\text{geo}}=\frac{1}{V}\sum_{v=1}^{V}\Big(\lambda_d\,\|D_v-D_v^{\text{GT}}\|+\lambda_n\,(1-\cos(N_v,N_v^{\text{GT}}))\Big)\]

The multi-scale LoG loss acts on rendered images, where \(\mathcal{S}\) is the set of Gaussian kernel scales, \(\Delta\) the Laplacian operator, \(G_\sigma\) a Gaussian kernel, and \(\omega_\sigma\propto\sigma^2\):

\[\mathcal{L}_{\text{LoG}}=\frac{1}{V}\sum_{v=1}^{V}\left\|\sum_{\sigma\in\mathcal{S}}\omega_\sigma\Big(\Delta(G_\sigma*I_v)-\Delta(G_\sigma*I_v^{\text{GT}})\Big)\right\|_1\]

The rotation-norm regularizer pulls the modulus of the predicted quaternion \(r\) toward 1:

\[\mathcal{L}_{\text{rot}}=\mathbb{E}\left[(1-\|r\|)^2\right]\]

The total objective is the weighted sum of four terms, \(\mathcal{L}_{\text{total}}=\lambda_f\mathcal{L}_{\text{FM}}+\lambda_a\mathcal{L}_{\text{app}}+\lambda_g\mathcal{L}_{\text{geo}}+\lambda_l\mathcal{L}_{\text{LoG}}\); note that the paper's total-objective equation does not explicitly include \(\mathcal{L}_{\text{rot}}\), which is introduced separately in the training-scheme section — whether it is folded in is subject to the original paper. For data, training uses G-Objaverse and G-Objaverse-XL Alignment (curated high-quality subsets derived from Objaverse / Objaverse-XL, further filtered by aesthetic score, for a final corpus of over 500K 3D objects, each rendered from 38 viewpoints with Cap3D text descriptions); pseudo-labels come from a fine-tuned GSRecon. Key hyper-parameters absent from the main text include the values of the \(\lambda\) weights, the number of diffusion sampling steps, \(V_{in}\), \(H\times W\) and the patchified training resolution, and the LoG scale set \(\mathcal{S}\) — ⚠️ all subject to the original paper / appendix.

Key Experimental Results

Evaluation runs two tracks: text conditioning uses 300 T3Bench prompts (single object, object with surroundings, multiple objects), measured by CLIP similarity, CLIP R-Precision (ViT-B/32) and ImageReward (human preference); image conditioning uses 300 randomly selected objects from GSO that were unseen during training, measured against GT renders with PSNR / SSIM / LPIPS. Ablations train on G-Objaverse (265K assets) and evaluate on 1,000 samples from G-Objaverse-XL Alignment.

Main Results

Text conditioning (T3Bench; ↑ higher is better, Latency ↓):

Method	Single Obj. CLIP Sim	Single Obj. R-Pre	Single Obj. ImgReward	w/ Surr. ImgReward	Multiple CLIP Sim	Multiple R-Pre	Latency (s)
DiffSplat (SD-1.5)	30.63	78.50	-0.490	-1.063	27.62	69.25	1
DiffSplat (SD-3.5)	30.99	84.75	-0.196	-0.447	29.44	74.00	3
TRELLIS-L	28.77	68.50	-1.033	-1.604	26.28	48.00	5
GaussianCube	27.35	50.75	-1.521	-2.063	23.93	26.25	3
LGM†	29.96	78.00	-0.720	-1.772	27.07	51.00	6
DreamGaussian	24.78	37.75	-1.635	-1.953	23.97	28.50	47
PixGS (Ours)	31.98	88.75	-0.171	-0.341	30.45	77.25	1

Image conditioning (GSO, compared against GT renders):

Method	PSNR↑	SSIM↑	LPIPS↓	Latency (s)
DiffSplat (SD-1.5)	19.57	0.810	0.159	1
DiffSplat (SD-3.5)	19.59	0.811	0.158	3
DiffusionGS	16.53	0.770	0.217	12
TRELLIS-L	17.70	0.797	0.172	6
LGM†	15.61	0.764	0.245	2
PixGS (Ours)	21.21	0.842	0.123	1

Ablation Study

Supervision losses and training curriculum (Tab. 4 and Tab. 5 merged; the first block stacks terms cumulatively, the second reports curriculum phases):

Config	PSNR↑	SSIM↑	LPIPS↓	Note
Only \(\mathcal{L}_{\text{FM}}\)	16.04	0.708	0.496	fits only the pseudo-label distribution, poor 3D consistency, heavy floaters
+ appearance loss	NaN	NaN	NaN	out-of-range attributes / non-unit rotations blow the gradients up to NaN
+ rotation-norm regularizer	30.11	0.969	0.027	numerical stability restored, appearance supervision becomes usable
+ geometry loss	30.24	0.967	0.025	depth/normal supervision, small gain
+ LoG loss	30.36	0.969	0.022	high-frequency detail recovered, LPIPS drops noticeably
Phase 2 (flow matching + GT rendering supervision)	30.36	0.969	0.022	numerically identical to the row above
Phase 3 (GT-only supervision)	31.15	0.987	0.019	full curriculum, determines final quality

Conditioning paradigm and pseudo-label source (two independent experiments, each evaluated under its own setting; the numbers are not comparable across the two):

Experiment	Config	Key metric	Params
Conditioning paradigm (PSNR/SSIM/LPIPS)	Viewpoint concatenation	30.36 / 0.969 / 0.022	1.2B
Conditioning paradigm	Image prompt adapter	30.31 / 0.977 / 0.023	1.4B
Pseudo-label source (GSO PSNR/SSIM/LPIPS)	LGM	21.15 / 0.844 / 0.125	415M
Pseudo-label source	GSRecon	21.21 / 0.842 / 0.123	42M

Key Findings

• The key finding is that flow matching alone is not enough. With only flow matching the training loss converges beautifully, yet PSNR is just 16.04 with LPIPS 0.496 and the visualizations are full of floaters; once GT rendering supervision is attached, PSNR jumps to the 30 level. The hard part of moving pixel-space diffusion onto the Gaussian attribute modality is therefore not distribution matching but the absence of visual-semantic constraints in attribute space — a small loss in attribute space does not imply that the render looks like an object.
• Rotations are the most fragile link. Dropping the rotation-norm regularizer and applying rendering losses directly yields NaN; adding it back converges immediately to 30.11/0.969/0.027. This also explains mechanistically why the curriculum is necessary: while attributes are out of range, rendering gradients are unreliable.
• Contributions of the supervision terms are diminishing: the rotation regularizer (rescuing NaN — the threshold for whether training works at all) > the appearance loss (the bulk of the jump from 16.04 to the 30 level) > the geometry loss (+0.13 PSNR) ≈ LoG (+0.12 PSNR, but pushing LPIPS from 0.027 down to 0.022). LoG's value lies mainly in perceptual metrics and high-frequency texture; judging it by PSNR alone understates it.
• Pseudo-labels should only bootstrap. Phase 2 → Phase 3 lifts PSNR from 30.36 to 31.15 and drops LPIPS from 0.022 to 0.019; switching to the weaker LGM pseudo-labels costs only 0.06 PSNR at the end (21.15 vs 21.21) for about 10% more training. The authors therefore chose GSRecon, which is an order of magnitude smaller (42M vs 415M).
• The two image-conditioning paradigms are nearly tied (PSNR differs by 0.05, SSIM by 0.008, LPIPS by 0.001), suggesting that recovering view-dependent appearance from the reference image is insensitive to the injection mechanism; the authors picked viewpoint concatenation on parameter efficiency.
• Speed is the other card. One second on T3Bench, on par with DiffSplat SD-1.5 and faster than SD-3.5 (3 s), TRELLIS (5-7 s), LGM (6 s) and DreamGaussian (47 s) — without trading quality for it.
• One exception worth noting: on the "object with surroundings" split of T3Bench, CLIP R-Precision is 87.75 for DiffSplat SD-3.5 versus 87.25 for PixGS, i.e. slightly lower here; every other entry and the overall trend on both datasets favor PixGS. Note also that the T3Bench and GSO tables are not comparable (alignment/preference metrics versus reconstruction metrics against GT), and the ablation PSNRs are measured on an in-distribution subset, so reaching 30+ there is not the same thing as the 21 on GSO.

Highlights & Insights

• The "attribute image = pixel manifold" representational isomorphism is the fulcrum of the whole method. Gaussian attributes are already continuous and already arranged per pixel, so the entire pixel-diffusion toolbox (patchification, the patch-wise neural field, RoPE2d, pre-trained weights) transfers with almost no modification. The transferable lesson is general: for any task that predicts a set of continuous primitive parameters laid out on a regular grid, first ask whether it is isomorphic to an image.
• Large patches plus a coordinate-conditioned neural field decoder: the transformer reasons only at patch granularity while per-pixel high frequencies are filled in by a small MLP parameterized by the token. This is the core trick that makes VAE-free pixel diffusion affordable, and the reason PixGS can port PixNerd straight to 3D.
• 2x super-resolution rendering supervision: computing the appearance loss at twice the attribute grid resolution is an almost free change that multiplies the spatial density of the supervision signal, turns "where should the Gaussians live" from an implicit decision into explicit gradients, and incidentally suppresses background pseudo-Gaussians. Any model that predicts structured 3D primitives and then renders them can adopt it directly.
• Using a classical edge operator as a regularizer: multi-scale LoG is the operator behind classic edge detection (Marr-Hildreth); used as a loss here, with \(\omega_\sigma\propto\sigma^2\) normalizing contributions across scales, it costs only a few Gaussian blurs and Laplacians yet shows up directly in LPIPS and high-frequency texture. An inexpensive "old tool, new use."
• A general "weak supervision bootstraps strong supervision" curriculum pattern: use a cheap, readily available signal that keeps the model inside the valid parameter domain (pseudo-labels) to push it there, then switch to the expensive but correct supervision (GT rendering). Any task exhibiting both "the valid parameter domain is hard to learn from scratch" and "correct supervision explodes outside the domain" can copy this three-phase pattern.

Limitations & Future Work

• The cached full text has no Limitations section, so the authors do not state limitations explicitly (⚠️ refer to the original paper / appendix). Judging from the ablation design, the model still needs a manually fine-tuned GSRecon to produce pseudo-labels for bootstrapping; although a weaker pseudo-label source barely changes the outcome, the more thorough ablation — no pseudo-labels at all, GT supervision from the start — is not run, so it cannot be ruled out that pseudo-labels remain irreplaceable in early convergence.
• Evaluation scope is narrow: all experiments are object-level (G-Objaverse family and GSO), image conditioning is single-image only, and text conditioning covers just the 300 T3Bench prompts; there is no quantitative analysis of scene-level or multi-object composition, and no comparison against pixel-space 3D methods newer than 2025.
• Key details are missing: the cache does not give \(V_{in}\), \(H\times W\), the patchified training resolution, the number of diffusion sampling steps, the loss weights \(\lambda\), or the LoG scale set \(\mathcal{S}\). The paper reports "about 1 second on a single A100" without stating the sampling steps (NFE) or batch setup, so cross-method latency comparisons should be read as rough orders of magnitude only.
• Representational ceiling: the \(V_{in}\)-view grid fixes both the total number of Gaussians and the coverage; regions outside the camera frustums can only be "splatted" out via predicted depth offsets. Whether this limits cases requiring full 360° coverage of complex topology — compared with sparse-voxel representations such as TRELLIS — is not analyzed. Nor is it clear whether 38 fixed rendering viewpoints provide enough coverage of arbitrary viewpoints in the wild for both pseudo-labels and GT supervision.
• Possible improvements: (i) replace pseudo-labels with self-supervised "valid-domain warm-up" (e.g. a few low-weight GT-rendering rounds), removing the reconstructor dependency entirely and ablating that premise away; (ii) turn the super-resolution supervision into an increasing-difficulty curriculum (\(1\times\to2\times\to4\times\)) to raise the high-frequency ceiling; (iii) the LoG scale weights \(\omega_\sigma\propto\sigma^2\) are currently fixed and could adapt to material or texture frequency; (iv) extend the same pixel-space framework to scene-level and multi-object generation.

Related Work & Insights

• vs DiffSplat: both use Splatter Image-style pixel-aligned Gaussians and both reuse image diffusion priors, but DiffSplat diffuses in a latent space and needs a three-level cascade of reconstructor + VAE + latent diffusion plus an extra lightweight decoder, so errors accumulate stage by stage and quality is capped by the compression capacity; PixGS diffuses on the attribute tensor directly — single-stage, no VAE — at the cost of fine-tuning GSRecon to make pseudo-labels. On quality, PixGS leads across essentially all T3Bench prompt categories on CLIP Sim / R-Precision / ImageReward (multiple objects: R-Precision 77.25 vs 74.00), at equal or better speed.
• vs DiffusionGS: also single-stage diffusion generation of 3DGS, but DiffusionGS uses a specialized architecture trained from scratch on synthetic 3D views and discards 2D pre-trained priors, so it converges slowly and generalizes poorly — PSNR only 16.53 with LPIPS 0.217 on GSO, and 12 seconds is the slowest latency in that table. PixGS's key difference is that its backbone is initialized directly from a pixel diffusion model pre-trained on 45M images.
• vs TRELLIS / GaussianCube: the 3D-native route, learning Gaussian distributions with a sparse voxel latent and a 3D UNet respectively — fast inference and good geometric consistency, but constrained by 3D data scale; more specifically, latent/voxel representations carry no explicit viewpoint-conditional supervision, so on the image-conditioned task they fail to reproduce the reference image's spatial orientation and perspective (TRELLIS-L GSO PSNR 17.70). PixGS's attribute tensor is inherently view-tagged, giving stronger reference-view alignment.
• vs LGM / DreamGaussian: one is a two-stage multi-view diffusion plus feed-forward reconstruction (requiring an additional conditioned multi-view generative model), the other is optimization-based (SDS with progressive densification). Both suffer because upstream 2D multi-view inconsistency is amplified into floaters downstream, and both are slow (on T3Bench, LGM 6 s and DreamGaussian 47 s). PixGS never passes through the intermediate "generated image" state and goes straight from noise to Gaussians, which is why it achieves speed and quality at once.

Rating

• Novelty: ⭐⭐⭐⭐ Porting pixel-space diffusion (PixNerd) onto the 3DGS attribute tensor with matching multi-view attention/RoPE and rendering-level supervision is a clean representational transfer; however, the "attribute image + 2D priors" route was pioneered by DiffSplat, so the core increment here is removing the latent together with the corresponding training scheme.
• Experimental Thoroughness: ⭐⭐⭐⭐ Two tracks (T3Bench text and GSO image) plus four ablation groups covering conditioning, losses, curriculum and pseudo-labels make a fairly complete study; the gaps are the absence of scene-level evaluation and the missing key hyper-parameters and sampling steps in the main text.
• Writing Quality: ⭐⭐⭐⭐ Motivation and method are explained clearly, ablations are well organized, and the conclusions are restrained; points are lost because the formulas in the cached extraction are badly corrupted and some evaluation settings must be inferred by the reader.
• Value: ⭐⭐⭐⭐ A single-stage, roughly one-second, VAE-free 3DGS generation pipeline is directly meaningful for deployment, and the three-phase curriculum plus super-resolution rendering supervision can be transferred to other 3D generation tasks at low cost.

Related Papers

• [ECCV 2026] Efficient and High-Quality Depth Estimation via Pixel-Space Diffusion with Linear Attention
• [ECCV 2026] DiGS-Avatar: Single-Image Animatable 3D Human Reconstruction via UV-Space Diffusion
• [CVPR 2025] Novel View Synthesis with Pixel-Space Diffusion Models
• [ECCV 2026] NanoGS: Training-Free and Lightweight Gaussian Splat Simplification
• [CVPR 2025] Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding Registration