PaD-GS: Leveraging Distortion Map for Panoramic Gaussian Splatting¶

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/CosyXu/PaD-GS
Area: 3D Vision
Keywords: 3D Gaussian Splatting, Panoramic Image Rendering, Distortion Map, Equirectangular Projection

TL;DR¶

PaD-GS introduces an analytical distortion map derived from panoramic imaging geometry and incorporates it into Gaussian learning via a whole-to-partial paradigm (distortion-aware decoding and adaptive opacity modulation), significantly eliminating rendering blur and distortion artifacts in 360-degree novel view synthesis.

Background & Motivation¶

Novel view rendering for panoramic images plays an indispensable role in virtual reality roaming, autonomous robot navigation, and large-scale 3D environment capture. Compared to Neural Radiance Fields (NeRF), which suffer from prohibitive computational footprints and sluggish rendering speeds due to dense volumetric sampling, 3D Gaussian Splatting (3DGS) has rapidly emerged as a promising alternative owing to its explicit scene representation and ultra-fast tile-based rasterization. Existing panoramic 3DGS extensions typically adapt the perspective rasterization pipeline by introducing equirectangular projection (ERP) functions or optimizing camera parameters to compensate for wide-angle distortion.

However, existing panoramic splatting methods remain heavily plagued by the inherent distortions of equirectangular imaging. When a spherical panoramic field of view is projected onto a 2D planar canvas, the mapping introduces severe, spatially non-uniform geometric stretching. Specifically, as the polar angle increases toward the poles, the ratio of the planar area element to the spherical surface element expands proportionally to $\frac{1}{\cos\theta}$, causing enormous geometric inflation. Prior panoramic 3DGS methods treat Gaussian optimization passively at the projection level or rely on brute-force Gaussian densification; their learned Gaussian representations remain unaware of the local distortion magnitude, which leads to erroneous opacity accumulation, broken slender structures, and blurred textures in highly distorted regions.

Addressing these non-uniform distortions requires more than passive rasterization adaptations—it demands explicitly informing Gaussian parameter generation and spatial accumulation about the underlying physical distortion. Core idea: construct an explicit distortion map directly from the panoramic projection geometry to quantify pixel-wise distortion, and inject this prior into Gaussian learning via a whole-to-partial strategy combining distortion-aware feature decoding with adaptive opacity modulation.

Method¶

Overall Architecture¶

PaD-GS takes omnidirectional panoramic images from a scene along with a sparse point cloud obtained via Structure-from-Motion (SfM) as inputs. The scene space is first partitioned into regular voxels, where voxel centers are initialized as learnable anchors. During forward synthesis, a precomputed physical distortion map provides pixel-level distortion indices without any runtime neural inference overhead. The framework is structured hierarchically from whole to partial: first, the Distortion-Aware Gaussian Decoding module generates complete Gaussian attributes (covariance, color, opacity) conditioned on anchor embeddings and distortion values; next, the Distortion-Aware Opacity Modulation module selectively suppresses the opacity of Gaussians falling in heavily distorted zones to avoid over-accumulation; finally, the refined Gaussians are rendered via a panoramic rasterizer supervised by an area-weighted loss.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["SfM Point Cloud + Panoramic Views"] --> B["Distortion Map Construction<br/>Polar angle surface area expansion"]
    B --> C["Distortion-Aware Gaussian Decoding<br/>Joint decoding of anchors and distortion map"]
    C --> D["Distortion-Aware Opacity Modulation<br/>Adaptive suppression of high-distortion regions"]
    D --> E["Panoramic 3DGS Rasterization<br/>Spherical area-weighted supervision"]

Key Designs¶

1. Physical Distortion Map Construction: Quantifying Pixel-Wise Area Distortion

Prior methods often treat panoramic projection distortion as an unmodeled nuisance. In panoramic imaging, a 3D camera-space point $(x, y, z)$ is projected onto the unit sphere with azimuth $\phi = \arctan(y, -x)$ and polar angle $\theta = \arctan(z, \sqrt{x^2+y^2})$. Under equirectangular projection to a $W \times H$ canvas, the spherical differential surface element $dA_{\text{sphere}} = \cos\theta d\phi d\theta$ maps to the planar pixel element $dA_{\text{erp}} = dudv = \frac{WH}{2\pi^2} d\phi d\theta$. The resulting area expansion ratio is: $$\frac{dA_{\text{erp}}}{dA_{\text{sphere}}} = \frac{WH}{2\pi^2} \cdot \frac{1}{\cos\theta}$$ This shows that local distortion is strictly proportional to $\frac{1}{\cos\theta}$, intensifying drastically toward the polar limits ($\theta \to \pm\frac{\pi}{2}$).

To feed this analytical property into neural representation learning, the polar angle $\theta$ is normalized into $[0, 1]$, where 0 indicates zero distortion at the equator and 1 indicates maximum distortion at the poles, yielding the distortion map $\Omega \in \mathbb{R}^{H \times W}$: $$\Omega(u, v) = 1 - \cos^2\theta = 1 - \cos^2\!\left(\frac{\pi}{H}v - \frac{\pi}{2}\right)$$ This closed-form distortion map requires zero neural parameters while providing an exact spatial distortion metric across the entire panoramic canvas.

2. Distortion-Aware Gaussian Decoding Module: Injecting Global Distortion Priors

Directly optimizing independent 3D Gaussians across large-scale panoramic scenes frequently leads to severe overfitting and needle-shaped artifacts in distorted regions. PaD-GS adopts an anchor-based representation where each voxel anchor $V^k$ holds a learnable feature vector $f^k \in \mathbb{R}^C$, a spatial scale $l^k$, and $N$ learnable offsets $\{P_i^k\}_{i=1}^N$ to spawn child Gaussians. When an anchor center projects onto the panorama, its corresponding distortion value $\Omega^k$ is retrieved from the distortion map.

The anchor feature $f^k$, distortion degree $\Omega^k$, camera distance $\delta^k$, and viewing direction $d^k$ are jointly fed into three dedicated MLPs: $$\{o_i^k, c_i^k, S_i^k, R_i^k\}_{i=1}^N = \mathcal{F}_{\{\text{opa}, \text{col}, \text{cov}\}}(f^k, \Omega^k, \delta^k, d^k)$$ The Gaussian centers are computed as $\mu_i^k = X^k + l^k \cdot P_i^k$. By conditioning the decoding network on $\Omega^k$, the model learns to synthesize adaptive Gaussian covariances and base opacities that inherently accommodate panoramic projection stretching.

3. Distortion-Aware Opacity Modulation Module: Mitigating Unreliable Accumulation

Even when initial Gaussians are decoded with distortion awareness, projecting large numbers of Gaussians into high-distortion polar regions creates massive overlap on the image plane. During $\alpha$-blending, these distorted Gaussians over-contribute to color accumulation, resulting in hazy artifacts and blurred boundaries.

To regulate this localized attribute without disturbing general feature learning, the opacity modulation module adaptively adjusts the opacity of each Gaussian according to its local distortion magnitude: $$\hat{o}_i^k = o_i^k \cdot (1 - \tau \cdot \Omega^k)$$ Here, $\tau$ is a modulation factor (set to 0.9). In low-distortion equatorial regions ($\Omega^k \to 0$), the original decoded opacity is fully retained to preserve sharp geometric textures. In severely distorted polar regions ($\Omega^k \to 1$), the opacity is attenuated, effectively suppressing noisy floaters and edge bleeding during volume accumulation.

Loss & Training¶

The training loss $\mathcal{L}$ combines an $\mathcal{L}_1$ pixel loss and an $\text{SSIM}$ structural similarity loss. To prevent the training gradients from being overwhelmingly dominated by stretched polar pixels, a distortion-aware weight map $M \in \mathbb{R}^{H \times W}$ rebalances pixel importance proportionally to true spherical surface area: $$\mathcal{L} = (1 - \lambda) \| M \odot (\hat{I} - I) \|_1 + \lambda (1 - \text{SSIM}(M \odot \hat{I}, M \odot I))$$ where $M(u, v) = \cos\theta = \cos\left(\frac{\pi}{H}v - \frac{\pi}{2}\right)$ and $\lambda = 0.8$. The entire network is optimized using Adam for 30,000 iterations on a single NVIDIA RTX 3090 Ti GPU, with adaptive densification and pruning performed every 100 iterations between steps 1,500 and 15,000.

Key Experimental Results¶

Main Results¶

PaD-GS is evaluated on two standard panoramic benchmarks: the 11 indoor scenes of 360Roam and the 12 outdoor scenes of Ricoh360, benchmarked against EgoNeRF, OP43DGS, ODGS, OmniGS, and SPaGS.

Dataset	Metric	PaD-GS (Ours)	SPaGS [18] (2nd Best)	OmniGS [19]	EgoNeRF [4]	Gain
360Roam (Indoor Average)	SSIM ↑	0.821	0.788	0.799	0.725	+0.022 (+2.7%)
	PSNR ↑	26.02	24.90	24.84	23.70	+1.12 dB
	LPIPS ↓	0.227	0.288	0.272	0.435	-0.045 (-16.5%)
Ricoh360 (Outdoor Average)	SSIM ↑	0.845	0.835	0.825	0.766	+0.010 (+1.2%)
	PSNR ↑	26.54	26.20	26.00	25.28	+0.34 dB
	LPIPS ↓	0.184	0.192	0.212	0.292	-0.008 (-4.2%)

(Note: Values transcribed from Tables 1 and 2 in the paper. OP43DGS encountered Out-Of-Memory errors on most Ricoh360 outdoor scenes due to excessive projection memory consumption.)

Ablation Study¶

Component ablations are conducted on the Bar scene of 360Roam and the Cat-tower scene of Ricoh360, isolating the distortion-aware Gaussian decoding (DAGD), opacity modulation (DAOM), and weighting map $M$.

Config	360Roam (Bar) SSIM / PSNR / LPIPS	Ricoh360 (Cat-tower) SSIM / PSNR / LPIPS	Note
Baseline (Panoramic 3DGS)	0.786 / 22.29 / 0.257	0.776 / 24.92 / 0.226	Standard panoramic 3DGS without distortion awareness
+ DAGD	0.801 / 23.13 / 0.240	0.797 / 25.36 / 0.208	Distortion map guides global anchor decoding (+0.84/+0.44 dB PSNR)
+ DAGD + DAOM (w/o M)	0.804 / 23.24 / 0.245	0.808 / 25.60 / 0.194	Adds adaptive opacity suppression in high-distortion zones
Full Model (PaD-GS)	0.808 / 23.40 / 0.230	0.812 / 25.75 / 0.187	Full model with spherical weighting $M$, achieving best overall fidelity

(Note: Values transcribed from Table 3 in the paper.)

Key Findings¶

Complementary Whole-to-Partial Design: The global decoding module (DAGD) accounts for the largest leap in structural fidelity (e.g. +0.84 dB on Bar), while the opacity modulation module (DAOM) refines fine details and eliminates perceptual noise, decreasing LPIPS significantly.
Superior Handling of Slender Geometry: Qualitative comparisons on chair legs, lamp posts, and tree branches demonstrate that OmniGS and SPaGS exhibit severe thinning, tearing, or ghosting due to projection stretching, whereas PaD-GS preserves crisp geometric boundaries and structural continuity.
Robustness in Unbounded Outdoor Scenes: While OP43DGS suffered fatal Out-Of-Memory (OOM) failures on large outdoor scenes, PaD-GS reliably converged across all 12 Ricoh360 outdoor scenes, training in ~1.66 hours and rendering at 23.12 FPS.

Highlights & Insights¶

Zero-Cost Physical Prior: Instead of hoping that deep neural networks implicitly figure out spherical distortions, PaD-GS analytically derives the exact area distortion ratio $\frac{1}{\cos\theta}$ from imaging geometry, establishing a lightweight, closed-form prior map $\Omega(u, v)$.
Whole-to-Partial Architectural Symmetry: By splitting anti-distortion regularization into global attribute decoding and local opacity suppression, the method avoids suboptimal trade-offs between expressive capacity and clean rendering.
Direct Practical Utility: Resolves the severe edge blurring that has historically limited panoramic splatting in commercial VR tours and robotic mapping, with official source code made public.

Limitations & Future Work¶

Inference Overhead from MLP Decoding: Unlike vanilla 3DGS where explicit Gaussian parameters are stored directly in memory, PaD-GS decodes Gaussians on the fly from anchors via MLPs, capping frame rates around 23–30 FPS. Future work could bake decoded attributes into static splats for >100 FPS viewing.
Sensitivity to Inaccurate Poses and Stitching Seams: The formulation relies heavily on SfM point cloud accuracy and assumes seamless equirectangular inputs. Significant camera stitching misalignment or rolling shutter effects could introduce discrepancies into the distortion mapping.

vs OmniGS [19]: OmniGS pioneered GPU-accelerated equirectangular rasterization for 3DGS. PaD-GS builds upon this rasterization foundation but introduces distortion-aware Gaussian representation learning, boosting indoor PSNR by 1.18 dB.
vs SPaGS [18]: SPaGS accelerates spherical splatting via ray-casting and AABB pruning to achieve 80+ FPS, but lacks explicit representation-level anti-distortion mechanics, resulting in noticeable blurring near polar latitudes where PaD-GS remains sharp.
vs SC-OmniGS [9] & Seam360GS [29]: These works focus on joint camera parameter calibration and seam boundary mitigation, whereas PaD-GS optimizes internal Gaussian attribute formation. The two paradigms are orthogonal and could be combined.

Rating¶

Novelty: ⭐⭐⭐⭐☆ (Derives an analytical distortion map from panoramic geometry and formulates an effective whole-to-partial anti-distortion learning pipeline)
Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Evaluated across 23 indoor and outdoor scenes with solid ablations and speed/memory metrics)
Writing Quality: ⭐⭐⭐⭐⭐ (Clear mathematical formulation and concise architectural motivation)
Value: ⭐⭐⭐⭐☆ (Highly practical for panoramic 3D reconstruction and VR view synthesis; open-sourced)