Real-Time LiDAR Gaussian Splatting SLAM via Geometry-Aware Covariance Coupling

Conference: ECCV 2026
Paper: ECCV Official Link
Code: https://github.com/Lab-of-AI-and-Robotics/LiDAR-GS-SLAM
Area: Autonomous Driving
Keywords: LiDAR SLAM, Gaussian Splatting, Dense Mapping, Covariance Reuse, Real-Time SLAM

TL;DR

Addressing GPU memory explosion and error persistence in large-scale online LiDAR SLAM, this work establishes a bidirectional covariance coupling between fast G-ICP tracking and spherical 2D Gaussian Splatting mapping, delivering continuous high-fidelity surface reconstruction at >20 FPS real-time speed.

Background & Motivation

In large-scale outdoor scenes, LiDAR-based SLAM systems are fundamental for high-accuracy pose estimation in autonomous driving and mobile robotics. However, classical front-end architectures such as LOAM, FAST-LIO2, and KISS-SLAM depend almost exclusively on sparse feature points or discrete voxel grids. While these representations offer computationally lightweight tracking, they lack explicit continuous surface representations and visibility reasoning, falling short of the fidelity demanded by autonomous driving simulation, HD map construction, and physical interaction tasks. Recently, neural implicit representations like PIN-SLAM and NeRF-LOAM have attempted to model continuous geometry via neural distance fields. Yet on long online sequences, frequent global backpropagation and heavy compute burdens hinder true real-time operation, while accumulated drift often degrades global map consistency.

3D Gaussian Splatting (3DGS) and surface-oriented 2DGS provide an appealing alternative due to their explicit geometric parameters and high-speed rasterization. Nonetheless, directly deploying Gaussian Splatting into purely LiDAR-driven online SLAM encounters two major bottlenecks. First is the unbounded map growth: as vehicles travel long trajectories, new primitives continuously accumulate with incoming keyframes, causing GPU memory and rasterization backpropagation costs to escalate until real-time performance collapses. Second is the error persistence dilemma: unlike image-based pipelines driven by dense RGB photometric loss, LiDAR observations provide only sparse ranges, and corrupted or degenerated primitives persist as noisy targets in subsequent scan-to-map registrations, biasing long-term odometry. Moreover, LiDAR scans exhibit beam divergence where point density falls quadratically with range, combined with severe sensitivity to surface incidence angles, rendering traditional camera-centric 3DGS initialization unsuitable.

The core angle of attack in this paper is that front-end registration (such as Generalized ICP) and back-end Gaussian surface optimization inherently operate on the exact same local second-order surface statistics. Instead of introducing heavy neural decoders or separate geometric estimation pipelines, one can bidirectionally reuse the local covariances computed naturally inside G-ICP: tracking covariances initialize Gaussian scale, surface normal, and sensor-aware confidence with zero overhead, while optimized Gaussian geometry provides analytic, noise-suppressed registration targets for tracking. Core idea: tightly couple fast G-ICP tracking with spherical 2D Gaussian Splatting mapping by bidirectionally reusing local covariance statistics with zero overhead, and steer map budget via covariance-derived geometric control scores for planar pruning and structure-adaptive splitting to achieve real-time, drift-resilient LiDAR dense SLAM.

Method

Overall Architecture

The system adopts an anisotropic 2D Gaussian representation in the spherical coordinate domain matching the native LiDAR beam geometry. Each primitive is formulated as \(g_i = (\mu_i, q_i, \tilde{s}_i, \alpha_i)\), where \(\mu_i \in \mathbb{R}^3\) denotes the spatial center, quaternion \(q_i\) defines the tangent plane orientation, \(\tilde{s}_i \in \mathbb{R}^2\) represents the in-plane anisotropic scales, and \(\alpha_i\) is the opacity. The overall pipeline operates across four interdependent stages: tracking with trackable Gaussian selection, tracking-to-mapping prior transfer, spherical rasterization map optimization, and geometry-aware map budget management.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Incoming LiDAR Point Cloud<br/>Motion deskewing & voxel downsampling"] --> B["Bidirectional Covariance Reuse & Tracking<br/>Trackable subset selection + O(1) target covariance + curvature-weighted G-ICP"]
    B --> C["Range-Adaptive Scale & Opacity Initialization<br/>Tangent covariance extraction + range scaling + incidence confidence"]
    C --> D["Spherical Rasterization Map Optimization<br/>Depth L1 + valid-mask BCE + dual normal alignment + scale regularization"]
    D --> E["Control-Score-Driven Map Budget Management<br/>Multi-pass orthogonal planar pruning + structure-adaptive splitting"]
    E -->|Refined Gaussians & Target Covariances| B

Key Designs

1. Bidirectional Covariance Reuse & Geometry-Weighted Tracking: O(1) Target Covariance Construction and Drift Mitigation

In large-scale continuous SLAM, computing k-NN neighborhoods per frame across a growing map to build G-ICP covariances creates a severe runtime bottleneck. The authors design a trackable Gaussian subset selection and an \(O(1)\) target covariance synthesis scheme. The tracker first queries active map elements satisfying reliability masks, recent keyframe updates, and minimum opacity \(\mathcal{G}_{\text{trk}} = \{g_i \in \mathcal{G} \mid m_i = 1, k_i \ge k_{\min}, \alpha_i \ge \alpha_{\min}\}\). For each target Gaussian \(g_i\), rather than searching for spatial neighbors, its optimized orientation \(q_i\) and in-plane scales \((s_{i,0}, s_{i,1})\) are used directly to construct an analytic \(3 \times 3\) surface covariance matrix with a small normal thickness \(\epsilon\). In the registration phase, G-ICP pairs downsampled source points with Gaussian centers and derives a normalized surface curvature \(\bar{\kappa}_i \in [0, 1]\) from the scale ratio, assigning geometric weights: $\(w(\bar{\kappa}_i) = \min\left(\frac{1}{\bar{\kappa}_i + \epsilon}, \omega_{\max}\right)\)$ This objective upweights stable planar structures (such as road surfaces and building facades), serving as geometric anchors that suppress rotational drift. In return, as the mapper refines the Gaussian geometric parameters over time, the target covariances become more accurate, directly lowering the number of tracking iterations required for convergence.

2. Range-Adaptive Scale and Physical Confidence Initialization: Eliminating Long-Range Holes and Suppressing Grazing Noise

Gaussian Splatting optimization is notoriously sensitive to initialization, where arbitrary scale or orientation defaults easily fall into poor local minima in sparse LiDAR point clouds. This method reuses the local covariance eigendecomposition \((R_i^s, \sigma_i)\) already calculated during G-ICP tracking of the source scan. Due to the angular resolution of spinning LiDARs, spatial beam spacing expands linearly with range \(r_i\). Normalizing covariance eigenvalues to unit scale would leave distant primitives unable to span the gap between laser scan lines, causing surface tearing. The initial in-plane scale is therefore scaled proportionally to range \(r_i = \max(\|p_i\|, 1)\): $\(\mathbf{s}_i^{(0)} = \text{clip}\left(\kappa r_i \boldsymbol{\sigma}_i^{\text{tan}}, s_{\max}\right)\)$ Furthermore, recognizing that LiDAR return quality depends on physical beam mechanics, the system calculates a sensor-specific physical confidence \(u_i = \exp\left(-(r/r_0)^2\right) \cdot \text{clip}\left(\frac{c - c_0}{1 - c_0 + \epsilon}, 0, 1\right)\) based on range attenuation and the beam incidence cosine \(c = |\mathbf{n}^\top \hat{\mathbf{r}}|\). Primitives are initialized with opacity \(\alpha_i^{(0)} = \alpha_{\min} + (\alpha_{\max} - \alpha_{\min}) u_i\), preventing grazing-angle edge noise and distant uncertain returns from polluting the initial map.

3. Control-Score-Driven Map Budget Management: Aggressive Planar Pruning and Structure-Adaptive Splitting

Unchecked primitive accumulation across long sequences inevitably leads to out-of-memory errors and sluggish rendering. The framework derives a continuous geometric control score \(c_i\) by linearly fusing the tracking covariance eigenvalues (\(\lambda_{i,0} \ge \lambda_{i,1} \ge \lambda_{i,2}\) yielding linearity \(\text{linear}_i\) and curvature \(\text{curv}_i\)) with the normalized G-ICP registration residual \(\text{res}_i\): $\(c_i = \text{clip}\left(w_l \widetilde{\text{linear}}_i + w_c \widetilde{\text{curv}}_i + w_r \widetilde{\text{res}}_i, 0, 1\right)\)$ with parameters \(w_l = 0.55, w_c = 0.30, w_r = 0.15\). A low \(c_i\) characterizes planar, redundant terrain, whereas a high \(c_i\) denotes sharp geometric features, corners, and object edges. Using \(c_i\), map management enforces a dual strategy: in planar regions, a multi-pass orthogonal plane projection groups primitives into 2D voxel cells, picks the representative primitive with highest opacity, extends its coverage radius, and prunes redundant enclosed planar Gaussians. In complex regions, primitives with high control scores undergo adaptive splitting along their major axis \(\mathbf{a}_i = R(q_i)\mathbf{e}_0\), creating two child Gaussians with small opposite displacements. This symmetry-breaking initialization enables subsequent gradient steps to specialize each child to subtle structural variations, capturing intricate geometry at minimal memory cost.

Loss & Training

The mapping back-end optimizes primitive parameters \(\Theta = \{\mu, q, \tilde{s}, \alpha\}\) on valid depth pixels \(\mathbf{M}_D = \mathbb{I}[D > 0]\) via gradient descent under a unified objective: $\(\mathcal{L} = \mathcal{L}_{\text{depth}} + \lambda_{\alpha}\mathcal{L}_{\alpha} + \lambda_{n}\mathcal{L}_{n} + \lambda_{s}\mathcal{L}_{\text{scale}}\)$ The depth loss utilizes masked L1, and opacity loss uses positive-only binary cross-entropy \(\mathcal{L}_{\alpha} = \frac{1}{|\Omega_D|}\sum_{i\in\Omega_D} -\log(\hat{A}_i + \epsilon)\). The normal objective \(\mathcal{L}_n\) integrates geometric normals \(\mathbf{N}_{\text{g-icp}}\) from tracking with surface normals \(\mathbf{N}_{\text{surf}}\) computed from rendered depth using cosine distance, where \(\mathbf{N}_{\text{surf}}\) is activated only after the splitting stage to ensure stability. Scale regularization penalizes primitives exceeding maximum threshold \(s_{\max}\). Weights are set to \(\lambda_{\alpha} = 0.1, \lambda_n = 1, \lambda_s = 10, \lambda_{n,g} = 0.05, \lambda_{n,s} = 0.01\). Upon loop detection, pose-graph optimization computes keyframe delta poses, applying rigid piecewise updates to associated Gaussians without expensive global re-optimization.

Key Experimental Results

Main Results

The framework is evaluated on handheld benchmarks (Newer College, Oxford Spires) with ground-truth meshes and large-scale autonomous driving trajectories (KITTI Odometry). Metrics include Absolute Trajectory Error (ATE RMSE in meters), surface reconstruction fidelity (Accuracy, Completeness, Chamfer-L1 in centimeters; F-score in % with threshold \(\tau = 0.2\text{ m}\)), and end-to-end system frame rate (FPS).

Table 1: Trajectory Tracking Evaluation (ATE RMSE [m] ↓)

Method	Type	Newer-Quad	Newer-Math	Oxford-Obs	Oxford-Keb	KITTI-00	KITTI-01	KITTI-07	KITTI-08	KITTI-09
KISS-SLAM [8]	Sparse Point SLAM	0.100	0.151	0.494	0.367	1.699	19.632	0.435	3.982	1.687
PIN-SLAM [23]	Neural Field SLAM	0.097	0.071	0.150	12.196	0.913	3.670	0.281	1.778	1.246
SuMa [1]	Surfel SLAM	0.249	0.210	5.145	16.803	1.200	Fail	0.405	2.210	3.884
Splat-LOAM [7]	Gaussian Splatting	0.138	0.818	17.494	0.770	Fail	Fail	Fail	Fail	Fail
Ours	Covariance-Coupled GS	0.080	0.160	0.336	0.177	1.080	2.260	0.338	2.900	1.260

Table 2: Mesh Reconstruction Quality and Runtime Speed (Newer College & Oxford Spires)

Dataset	Sequence	Metric	Voxblox (GT Pose)	VDBFusion (GT Pose)	PIN-SLAM (Est. Pose)	Splat-LOAM (Est. Pose)	Ours (Est. Pose)
Newer College	Quad	Acc (cm) ↓ / Com (cm) ↓	16.58 / 19.00	14.88 / 14.09	17.11 / 22.83	12.23 / 18.91	7.62 / 15.02
		C-L1 (cm) ↓ / F-score (%) ↑	17.79 / 64.00	14.49 / 76.27	19.97 / 60.59	15.57 / 72.06	11.32 / 86.78
		Processing Speed FPS ↑	66.10 (Map only)	26.44 (Map only)	7.62 (Full system)	3.91 (Full system)	22.15 (Full system)
Newer College	Math	C-L1 (cm) ↓ / F-score (%) ↑	14.00 / 77.23	10.61 / 84.59	18.18 / 63.78	15.02 / 73.66	16.48 / 70.35
		Processing Speed FPS ↑	82.70 (Map only)	45.40 (Map only)	11.04 (Full system)	2.54 (Full system)	25.25 (Full system)
Oxford Spires	Obs	C-L1 (cm) ↓ / F-score (%) ↑	15.86 / 71.73	10.93 / 83.68	24.42 / 44.15	36.34 / 6.14	20.83 / 56.89
		Processing Speed FPS ↑	143.00 (Map only)	48.16 (Map only)	8.20 (Full system)	1.52 (Full system)	17.99 (Full system)
Oxford Spires	Keb	C-L1 (cm) ↓ / F-score (%) ↑	13.83 / 77.29	10.20 / 84.99	29.19 / 33.93	28.26 / 33.97	18.64 / 64.02
		Processing Speed FPS ↑	140.30 (Map only)	43.69 (Map only)	8.62 (Full system)	5.77 (Full system)	19.58 (Full system)

Ablation Study

Table 3: Ablation on Tracking-to-Mapping Information Transfer (Quad & Keble College)

Variant Configuration	Quad: ATE (m) ↓	Quad: C-L1 (cm) ↓	Quad: F-score (%) ↑	Keb: ATE (m) ↓	Keb: C-L1 (cm) ↓	Keb: F-score (%) ↑	Impact & Mechanism
A. w/o Covariance Init (identity)	0.088	22.190	48.688	0.283	24.020	44.123	Without orientation/scale priors, optimization destabilizes
B. w/o Range-Adaptive Scale	0.087	11.635	85.996	0.479	22.558	51.582	Long-range scan gaps form significant surface holes
C. w/o Opacity Init	0.085	11.450	86.020	0.451	22.210	53.173	Unreliable grazing measurements are not suppressed
D. w/o G-ICP Normal Loss	0.099	11.600	85.899	0.325	21.428	55.522	Tangent planes degrade, reducing registration constraints
E. w/o Feature-Aware Mgmt	0.089	11.729	86.237	0.222	19.379	63.217	Slight short-term drop, but causes unbounded map bloat
F. Full System	0.079	11.320	86.776	0.212	18.640	64.024	Optimal balance of mapping fidelity and tracking accuracy

Table 4: Map Management Trade-Off on Map Fidelity and Resources

Management Variant	Quad: F-score (%) ↑	Quad: #GS Primitives ↓	Quad: Map Size ↓	Quad: Peak VRAM ↓	Keb: F-score (%) ↑	Keb: #GS Primitives ↓	Keb: Map Size ↓	Keb: Peak VRAM ↓
Baseline (opacity prune only)	87.179	239,175	9.6 MB	2.9 GB	64.863	546,400	21.9 MB	4.1 GB
Split only	88.040	248,060	9.9 MB	4.8 GB	56.681	591,567	23.7 MB	4.2 GB
Plane Prune only	86.521	119,285	4.8 MB	4.5 GB	59.327	341,435	13.7 MB	4.1 GB
Split + Plane Prune (Full)	86.776	129,885	5.0 MB	4.5 GB	64.024	344,473	13.8 MB	4.3 GB

Table 5: Effect of Mapping-to-Tracking Feedback on Tracking Efficiency

Configuration Variant	ATE RMSE (m) ↓	Tracking FPS ↑	Systematic Function
w/o GS Opt. (frozen / no feedback)	0.099	17.69	Lack of optimized surface targets increases G-ICP iterations
w/o Geometric Weight	0.088	22.66	Planar constraints weaken, leading to larger rotational drift
w/o Confidence Filtering	0.085	20.47	Unstable noise primitives degrade registration speed
Full (w/ feedback)	0.079	23.70	Refined target covariances accelerate convergence

Key Findings

• Crucial Role of Covariance Initialization: Initializing Gaussians with identity rotations and unaligned scales (Table 3-A) causes a catastrophic 38.09% drop in F-score on Newer College Quad (plunging from 86.78% to 48.69%), confirming that explicit physical geometric priors are indispensable for sparse LiDAR optimization.
• Resource Efficiency via Planar Pruning: Combining planar pruning with structural splitting (Table 4) cuts the primitive count on Quad by 45.7% (from 239k to 129k) and halves map storage from 9.6 MB to 5.0 MB with under 0.4% change in F-score. On KITTI sequence 00, persistent storage drops from 61.3 MB to 42.1 MB.
• Mutual Acceleration via Closed-Loop Coupling: Mapping does not create a computational drag on tracking; instead, it accelerates it (Table 5). Because optimized Gaussians provide precise surface orientations and scales, G-ICP builds target covariances in \(O(1)\) time and converges in fewer steps, pushing tracking speed from 17.69 FPS to 23.70 FPS.

Highlights & Insights

• Zero-Overhead Bidirectional Coupling: Seamlessly linking front-end G-ICP covariance statistics with 2D Gaussian tangent parameters eliminates redundant k-NN searches and dispenses with auxiliary neural feature extractors.
• Curvature Anisotropy as a Drift Anchor: Formulating scan-to-map registration weights via in-plane scale ratios dynamically turns flat terrain and walls into geometric anchors, effectively replacing hand-crafted point-to-plane heuristics.
• Symmetry-Breaking Splitting: Displacing child Gaussians along the major tangent axis breaks optimization deadlocks, allowing backpropagated gradients to guide primitives toward fine structural contours.

Limitations & Future Work

• Degenerated Geometry Sensitivity: The pipeline relies on reliable G-ICP covariance estimates. In long, featureless tunnels or open fields filled with sparse foliage, covariances can become ill-conditioned, leading to noisy initial surface normals.
• Dynamic Obstacle Artifacts: While incidence and range confidence attenuate fleeting returns, the absence of an explicit dynamic object segmentation network can leave ghosting trails in busy urban intersections.
• Piecewise Rigid Loop Closure: Loop corrections apply keyframe-wise rigid transformations rather than full non-rigid Gaussian re-optimization, which may leave slight local boundary seams after massive loop closures.

Related Work & Insights

• vs Splat-LOAM [7]: While Splat-LOAM also targets Gaussian Splatting LiDAR SLAM, it uses unconstrained spherical projection, leading to memory explosion and tracking failure across all KITTI sequences; this work maintains robust tracking and >20 FPS across long sequences through covariance coupling.
• vs PIN-SLAM [23]: PIN-SLAM adopts a point-based neural implicit field, achieving competitive odometry but struggling with low frame rates (7-11 FPS) and heavy VRAM overhead; this method achieves +18.9% higher F-score reconstruction while running over 2.4× faster.
• vs KISS-SLAM [8]: KISS-SLAM produces only sparse point clouds; this framework preserves similar front-end robustness while generating continuous, compact 2D Gaussian surface reconstructions with negligible additional compute.

Rating

• Novelty: ⭐⭐⭐⭐☆ Elegant bidirectional coupling between classical G-ICP covariances and modern 2DGS primitives
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive benchmarks across handheld and driving datasets with detailed ablations
• Writing Quality: ⭐⭐⭐⭐⭐ Clean formulation, clear technical narrative, and faithful qualitative-quantitative alignment
• Value: ⭐⭐⭐⭐⭐ Establishes a highly practical benchmark for real-time, dense LiDAR Gaussian Splatting SLAM