---

title: >- [Paper Note] Cube-Splat: High-Fidelity 360° Gaussian Splatting SLAM via Cubemap Factorization and Adjoint-Consistent Optimization description: >- [ECCV 2026][3d_vision][Gaussian Splatting] A panoramic 360° 3D Gaussian Splatting SLAM framework via cubemap factorization and adjoint-consistent optimization, eliminating wide-FoV distortion while achieving centimeter-to-millimeter localization and void-free dense reconstruction. tags: - ECCV 2026 - 3D Vision - SLAM - 3D Gaussian Splatting - Omnidirectional Vision date: 2026-09-19 content_hash: d626cdada2095835

Cube-Splat: High-Fidelity 360° Gaussian Splatting SLAM via Cubemap Factorization and Adjoint-Consistent Optimization

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/guoxf304/CubeSplat
Area: 3D Vision
Keywords: 3D Gaussian Splatting, Omnidirectional SLAM, Cubemap Factorization, Adjoint-Consistent Optimization, Panoramic Reconstruction

TL;DR

Addressing the narrow field-of-view, severe trajectory drift, and non-frontal rendering voids of conventional pinhole 3DGS-SLAM, Cube-Splat factorizes 360° panoramic imagery into four virtual pinhole cubemap faces sharing a single optical center, aggregating multi-face photometric gradients into a single primary pose via Lie-algebra adjoint mappings for robust tracking and omnidirectional high-fidelity reconstruction.

Background & Motivation

Dense visual SLAM systems driven by 3D Gaussian Splatting (3DGS) (e.g., MonoGS, Photo-SLAM, SplaTAM) have demonstrated impressive explicit geometry representation capabilities and real-time differentiable rendering speeds. However, the vast majority of current 3DGS-SLAM pipelines rely strictly on perspective pinhole camera models. Their inherently limited field of view (FoV) constrains each observation to a narrow frustum. When the camera undergoes rapid rotations, sudden viewpoint shifts, or navigates featureless spaces, pinhole SLAM suffers from severe cumulative drift. More critically, pinhole systems can only reconstruct the scene along the forward trajectory, leaving extensive rendering voids and unmapped blind spots in lateral and rearward directions.

Omnidirectional 360° imaging (such as Panoramic Annular Lens (PAL) or Equirectangular Projection (ERP)) provides seamless panoramic coverage that substantially alleviates occlusions and tracking dead zones, yet directly integrating panoramic frames into a 3DGS-SLAM front-end presents fundamental geometric challenges. Severe spherical distortions conflict with standard perspective Gaussian rasterization engines, making direct omnidirectional projection computationally burdensome while degrading the fidelity of anisotropic Gaussian ellipsoids. Conversely, simply decomposing panoramas into multiple uncoordinated views introduces gradient inconsistency and pose divergence in non-linear least squares tracking, creating a critical bottleneck in multi-view radiance field optimization.

This paper's core insight is that panoramic observations originate from continuous ray bundles passing through a single optical center, enabling distortion-free factorization into discrete orthogonal virtual pinholes that can be rigorously unified using Lie group adjoint representations. Core idea: decompose each 360° frame into a four-face cubemap of fixed-orientation virtual pinhole views sharing a single optical center, designate the front face as the primary pose state, and project multi-face photometric gradients back to this primary state via adjoint mappings for view-invariant, geometrically consistent panoramic tracking and mapping.

Method

Overall Architecture

Cube-Splat couples a camera tracking thread and a Gaussian mapping thread that execute cooperatively. Each incoming 360° panoramic frame (supporting PAL annular images or ERP panoramic projections) is unwrapped and factorized into four virtual pinhole views (Front, Left, Right, Back) that strictly share a single physical projection center. In the tracking thread, the 3D Gaussian map is frozen; multi-face photometric residuals are evaluated through differentiable rendering and mapped onto the primary pose via the Lie-algebra adjoint representation. In the mapping thread, the estimated camera pose is fixed, and depth priors extracted by Depth Anywhere guide Gaussian insertion and densification in under-observed regions, while multi-face photometric gradients jointly optimize the Gaussian attributes across all directions.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input 360° Panoramic Imagery<br/>PAL Annular / ERP Projections"] --> B["Cubemap Factorization & Virtual Pinholes<br/>Four orthogonal pinhole views sharing one optical center"]
    B --> C["Panoramic Geometry Prior & Keyframing<br/>Depth Anywhere dense depth aids Gaussian insertion"]
    C --> D["Adjoint-Consistent Multi-Face Tracking<br/>Freeze map / compute 4-face residuals / update primary pose"]
    D --> E["Multi-Face Gaussian Mapping & Densification<br/>Fix pose / aggregate multi-view rays / clone & split"]
    E --> F["High-Precision 6DoF Trajectory & Dense 360° Map"]

Key Designs

1. Cubemap Factorization and Virtual Pinhole Projection: Eliminating Distortion while Preserving Standard Rasterization To enable standard 3DGS rasterizers to seamlessly process omnidirectional observations without modifying low-level CUDA engines, the framework employs a unified cubemap factorization. For PAL industrial sensors, an omnidirectional polynomial model maps 2D pixels \((x_m, y_m)\) to unit rays and unwraps them into cubemap faces based on the dominant axis component. For ERP imagery, spherical coordinates are computed directly from normalized pixel grids: $$ x_c = \cos(\theta)\cos(\phi),\quad y_c = \sin(\theta),\quad z_c = \cos(\theta)\sin(\phi) $$ Four orthogonal virtual pinholes (Front, Left, Right, Back; excluding top and bottom to avoid feature-degenerate ground and sky poles) are established with fixed relative rotations \(\mathbf{R}_c^{(k)}\) relative to the front reference face. Because all virtual faces share the identical optical center, standard 2D covariance projection is directly applicable using the affine Jacobian \(\mathbf{J}\): $$ \mathbf{\Sigma}'i = \mathbf{J} \mathbf{R}}^{(k)} \mathbf{\Sigmai (\mathbf{R})^\top $$}^{(k)

2. Adjoint-Consistent Multi-Face Pose Optimization: Unbiased Panoramic Gradient Aggregation Treating each cubemap face as an independent pose variable inevitably induces geometric tearing and scale inconsistency. Cube-Splat designates the front camera pose \(\mathbf{T}_{cw}\) as the unique primary state, strictly constraining the \(k\)-th face via the known rigid body transform \(\mathbf{M}^{(k)} = [\mathbf{R}_c^{(k)} \mid \mathbf{0}]\) such that \(\mathbf{T}_{cw}^{(k)} = \mathbf{M}^{(k)} \mathbf{T}_{cw}\). For a Lie-algebra perturbation \(\boldsymbol{\xi} \in \mathfrak{se}(3)\) on the primary pose, the corresponding local perturbation \(\boldsymbol{\xi}_{(k)}\) satisfies: $$ \boldsymbol{\xi}{(k)} = \mathrm{Ad} $$ Consequently, the photometric gradients across all faces map rigorously back to the primary state through the transpose of the adjoint operator. Under the Gauss-Newton formulation, the global Hessian }^{(k)}} \boldsymbol{\xi\(\mathbf{H}\) and residual vector \(\mathbf{b}\) aggregate local Jacobians \(\mathbf{J}_{(k)}\) and Charbonnier weights \(\mathbf{W}_{(k)}\): $$ \mathbf{H} = \sum_k \left(\mathrm{Ad}{\mathbf{M}^{(k)}}\right)^\top \left(\mathbf{J}}^\top \mathbf{W{(k)} \mathbf{J}}\right) \mathrm{Ad{\mathbf{M}^{(k)}}, \quad \mathbf{b} = \sum_k \left(\mathrm{Ad}}^{(k)}}\right)^\top \mathbf{J{(k)}^\top \mathbf{W} $$ This formulation guarantees optimization invariance with respect to the chosen reference face and harnesses 360° visual constraints to prevent rotational drift.} \mathbf{r}^{(k)

3. Multi-Face Photometric Mapping and Dense Depth Prior Guidance In the mapping thread, primary camera poses are fixed, and Gaussian attributes \(\Theta = \{\boldsymbol{\mu}_i, \mathbf{\Sigma}_i, \alpha_i, \mathbf{C}_i\}\) are optimized by minimizing multi-face Charbonnier photometric loss \(\mathcal{L}_{\mathrm{map}} = \sum_k \sum_{\mathbf{u} \in \Omega^{(k)}} \rho(\hat{\mathbf{I}}^{(k)}(\mathbf{u}) - \mathcal{I}^{(k)}(\mathbf{u}))\). Gradients from the multiple viewing rays naturally coalesce onto the shared 3D Gaussians. To resolve monocular scale ambiguity during rapid exploration, a pre-trained monocular 360° depth estimator (Depth Anywhere) provides geometric depth priors exclusively at newly spawned keyframes for Gaussian point initialization and sky-mask filtering. Tracking remains solely driven by multi-face photometric consistency, keeping runtime overhead minimal.

4. Automated SynPano Benchmark Construction Due to the lack of challenging panoramic SLAM benchmarks featuring complex 6DoF motions, the authors engineered SynPano, an automated synthetic dataset using Blender's Cycles engine. Spanning 10 large-scale indoor and outdoor environments, SynPano parameterizes smooth 6DoF continuous trajectories via 3D Bézier curve interpolation across sampled keyposes (e.g., spirals and arbitrary roaming paths). It outputs photorealistic 1920×960 ERP frames alongside perfectly synchronized ground truths, including dense depth maps, 3D point clouds, and semantic segmentation masks.

Loss & Training

Both tracking and mapping minimize the robust Charbonnier photometric residual \(\rho(x) = \sqrt{x^2 + \epsilon^2}\). Thanks to the powerful omnidirectional geometric constraints, front-end Gauss-Newton pose tracking converges accurately within only ~35 iterations (compared to ~70 iterations for standard pinhole baselines). In mapping, keyframes are filtered via tracking loss and parallax thresholds; spatial density control periodically splits Gaussians with high positional gradients and clones under-reconstructed fine geometry, alongside periodic opacity resets to purge floaters.

Key Experimental Results

Main Results

Cube-Splat is evaluated against leading 3DGS-SLAM baselines (Photo-SLAM, MonoGS, S3PO-GS) across PALVIO (real-world PAL lens), OmniBlender (synthetic indoor/outdoor ERP), and SynPano (complex 6DoF ERP). Since baseline systems cannot ingest 360° panoramas, they are provided with cropped \(90^\circ \times 90^\circ\) front perspective views. Metrics include Absolute Trajectory Error (ATE, in meters), PSNR (dB), SSIM, and LPIPS.

Table 1: Quantitative comparisons on indoor and outdoor panoramic benchmarks (from Table 1 of the paper)

Environment	Method	ATE (m) ↓	PSNR (dB) ↑	SSIM ↑	LPIPS ↓
PALVIO (Real Indoor PAL)	Photo-SLAM	2.2299	24.74	0.8738	0.2537
	MonoGS	2.8881	23.67	0.8445	0.4575
	S3PO-GS	1.9317	27.70	0.8976	0.2902
	Cube-Splat (Ours)	0.0769	29.25	0.9307	0.2145
SynPano (Synthetic Indoor ERP)	Photo-SLAM	- (Failed)	-	-	-
	MonoGS	0.7112	28.92	0.8795	0.2276
	S3PO-GS	0.1877	32.27	0.9288	0.1454
	Cube-Splat (Ours)	0.0040	33.56	0.9356	0.1373
OmniBlender (Synthetic Indoor ERP)	Photo-SLAM	0.0095	28.97	0.8892	0.1314
	MonoGS	0.0740	29.68	0.8344	0.2277
	S3PO-GS	0.1221	32.18	0.8876	0.1714
	Cube-Splat (Ours)	0.0013	33.22	0.8958	0.1767
SynPano (Unbounded Outdoor ERP)	Photo-SLAM	- (Failed)	-	-	-
	MonoGS	6.5869	22.84	0.7087	0.3297
	S3PO-GS	- (Failed)	-	-	-
	Cube-Splat (Ours)	0.1058	23.44	0.7349	0.3211
OmniBlender (Unbounded Outdoor ERP)	Photo-SLAM	0.0337	24.73	0.7704	0.2205
	MonoGS	0.4269	18.86	0.5813	0.4299
	S3PO-GS	0.3334	23.91	0.7196	0.2327
	Cube-Splat (Ours)	0.0129	25.88	0.8109	0.2082

Ablation Study

To isolate the benefit of wide FoV from the algorithmic contribution of adjoint-consistent multi-face tracking, the authors progressively expand the tracking (T) and mapping (M) face count on SynPano.

Table 2: Controlled ablation of FoV and adjoint-consistent optimization on SynPano (from Table 2 of the paper)

Configuration	Tracking Views (T)	Mapping Views (M)	Adjoint-Consistent	ATE (m) ↓	PSNR (dB) ↑	SSIM ↑	LPIPS ↓	Note
MonoGS (Original)	1 (Front)	1 (Front)	✕ (N.A.)	0.7112	28.92	0.8795	0.2276	Single front view for both
MonoGS (1T-4M)	1 (Front)	4 Faces	✕ (N.A.)	0.1057	30.91	0.9095	0.1625	4-face mapping, front-only tracking
Cube-Splat (2T-4M)	2 Faces	4 Faces	✓	0.0526	32.18	0.9187	0.1539	2-face adjoint tracking
Cube-Splat (3T-4M)	3 Faces	4 Faces	✓	0.0057	32.88	0.9294	0.1476	3-face adjoint tracking
Cube-Splat (Full)	4 Faces	4 Faces	✓	0.0040	33.56	0.9356	0.1373	Full 4-face tracking & mapping

Table 3: Robustness to front-end optimization iterations (selected from Table 3 of the paper)

Iterations	barbershop (MonoGS)	barbershop (S3PO-GS)	barbershop (Cube-Splat Ours)	classroom (MonoGS)	classroom (S3PO-GS)	classroom (Cube-Splat Ours)
100 iters	0.1577 m	0.1056 m	0.0006 m	0.0076 m	0.1133 m	0.0012 m
50 iters	0.2828 m	0.1069 m	0.0006 m	0.0715 m	0.1145 m	0.0007 m
30 iters	0.3270 m	0.1062 m	0.0013 m	0.1783 m	0.1162 m	0.0013 m
20 iters	0.2256 m	0.1069 m	0.0409 m	0.3021 m	0.1061 m	0.0049 m

Table 4: Runtime efficiency on a single NVIDIA RTX 4070 Ti SUPER GPU (from Table 4 of the paper)

Method Configuration	Views	Overall FPS ↑	Time / Iter ↓	Tracking Iters	Time / Frame ↓
MonoGS	1 Face (Front)	~1.60	7.8 ms	~70 iters	0.47 s
Cube-Splat (w/o depth prior)	4 Faces (Cubemap)	~1.60	12.7 ms	~35 iters	0.49 s
Cube-Splat (Full System)	4 Faces (Cubemap)	~1.53	12.8 ms	~35 iters	0.49 s

Key Findings

• Essential Role of Adjoint Tracking: As evidenced in Table 2, simply augmenting mapping coverage to 4 faces (MonoGS 1T-4M) reduces ATE from 0.7112m to 0.1057m. However, enabling 4-face adjoint tracking further drives ATE down to 0.0040m (a >96% error reduction), proving that panoramic constraints must be tightly integrated into tracking.
• Robustness in Unbounded Outdoor Scenes: In outdoor SynPano, narrow-FoV baselines fail completely due to scale drift and lack of close-range landmarks (Photo-SLAM/S3PO-GS fail; MonoGS exhibits 6.58m drift). Cube-Splat maintains an ATE of 0.1058m by anchoring poses against stable peripheral visual cues.
• Convergence Speed Trade-off: Aggregating four orthogonal views produces stronger gradient directions, allowing Cube-Splat to converge within ~35 iterations rather than ~70. Even though per-iteration cost increases from 7.8ms to 12.8ms, the overall frame processing time remains identical (~0.49s, ~1.53 FPS).

Highlights & Insights

• Closed-Form Lie Algebra Adjoint Mapping: The use of rigid-body adjoint matrices to map multi-view Jacobians back to a single primary camera state provides a clean, mathematically exact solution to multi-camera/panoramic gradient aggregation without reference-face bias.
• Computational Offset via Accelerated Convergence: While rendering four views quadruples standard rasterization operations, the richer geometric constraints cut required optimization iterations in half, achieving panoramic coverage at virtually no net runtime penalty.
• SynPano Benchmark Contribution: By parameterizing 6DoF Bézier curves within Blender Cycles, SynPano establishes a reproducible, physically illuminated benchmark with flawless multi-modal ground truths for panoramic SLAM.

Limitations & Future Work

• Exclusion of Polar Views: Omitting the top and bottom faces avoids sky/ground distortion and reduces computation, but may reduce geometric constraints during steep aerial pitch or roll maneuvers.
• Monocular Scale Dependency: Without stereo baselines or IMU integration, outdoor metric scale relies on monocular depth priors. Fusing panoramic visual-inertial odometry (VIO) or LiDAR is an important future direction.
• Frame Rate and Dynamic Scenes: Operating at ~1.53 FPS places the system in the near-real-time regime, requiring further acceleration for high-speed robotics (20+ FPS), and dynamic objects are not currently filtered or modeled.

Related Work & Insights

• vs MonoGS / SplaTAM: Conventional monocular GS-SLAM assumes perspective pinhole geometry, suffering catastrophic failure during rapid rotations and leaving blind spots. Cube-Splat unlocks omnidirectional coverage while retaining standard rasterizer efficiency.
• vs OmniGS / ODGS / ErpGS: Prior omnidirectional 3DGS methods focus primarily on offline novel view synthesis with known poses. Cube-Splat bridges the gap by tackling real-time joint online tracking and mapping.
• vs 360ORB-SLAM / RPV-SLAM: Classical feature-based panoramic SLAM produces only sparse map points without photorealistic view synthesis. Cube-Splat unifies high-accuracy trajectory estimation with dense, photorealistic Gaussian scene modeling.

Rating

• Novelty: ⭐⭐⭐⭐⭐ [Pioneering cubemap factorization and adjoint-consistent optimization for 360° 3DGS-SLAM]
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Comprehensive evaluations across real-world PAL sensors, OmniBlender, and the novel 6DoF SynPano benchmark]
• Writing Quality: ⭐⭐⭐⭐⭐ [Rigorous mathematical formulation, clear structural exposition, and informative figures]
• Value: ⭐⭐⭐⭐⭐ [Provides an open-source, robust foundation for omnidirectional dense SLAM in robotics and immersive spatial computing]

Related Papers

• [ECCV 2026] MAC-Splat: Multi-Attribute Consistency for High-Fidelity Sparse-View Reconstruction
• [ECCV 2026] VIGS-SLAM: Visual Inertial Gaussian Splatting SLAM
• [ECCV 2026] Multi4D: High-Fidelity Dynamic Gaussian Splatting via Multi-Level Competitive Allocation
• [CVPR 2026] HyperGaussians: High-Dimensional Gaussian Splatting for High-Fidelity Animatable Face Avatars
• [CVPR 2026] Pose-Free Omnidirectional Gaussian Splatting for 360-Degree Videos with Consistent Depth Priors