MAGICIAN: Efficient Long-Term Planning with Imagined Gaussians for Active Mapping¶

Conference: CVPR 2026 arXiv: 2603.22650 Code: https://shiyao-li.github.io/magician/ Area: 3D Vision Keywords: Active Mapping, Long-Term Planning, 3D Gaussian Splatting, Scene Reconstruction, Viewpoint Selection

TL;DR¶

This paper proposes MAGICIAN, a framework that leverages a pretrained occupancy network to generate "Imagined Gaussians" for efficiently estimating surface coverage gain. Combined with beam search, MAGICIAN enables long-term trajectory planning for active mapping, achieving state-of-the-art performance in both indoor and outdoor scenes with coverage improvements exceeding 10%.

Background & Motivation¶

Background: Active Mapping requires agents to autonomously select optimal viewpoints for efficient reconstruction of unknown environments. Current mainstream methods adopt greedy Next-Best-View (NBV) strategies, selecting the next pose based on information gain, Fisher information, or surface coverage gain.
Limitations of Prior Work: Greedy NBV methods optimize only single-step local gain, causing agents to fall into dead ends, backtrack inefficiently, and explore suboptimally. Although some methods attempt longer-horizon planning (e.g., FisherRF selects frontier targets; NextBestPath predicts path-level gain), they either still rely on frontier heuristics or are sensitive to training data quality.
Key Challenge: Long-term planning faces a chicken-and-egg problem — optimal trajectory planning requires knowledge of the environment map, yet that map is precisely what planning aims to construct. Meanwhile, combinatorial explosion of the trajectory space and high computational cost make long-term planning extremely challenging.
Goal: (1) Efficiently estimate surface coverage gain in unobserved regions; (2) Search for optimal long-term paths in the combinatorially explosive trajectory space; (3) Achieve scalable closed-loop planning.
Key Insight: Inspired by the human ability to rapidly infer the structure of unfamiliar environments and plan exploration accordingly, the paper proposes "imagining" unseen regions via a pretrained occupancy network.
Core Idea: Occupancy predictions are converted into a 3D Gaussian representation, enabling fast volumetric rendering to compute coverage gain and making beam-search-based long-term planning feasible.

Method¶

Overall Architecture¶

MAGICIAN executes a perceive–plan–act loop at each step: (1) a pretrained occupancy model predicts a probabilistic occupancy field for the current environment, including unobserved regions; (2) the occupancy field is converted into Imagined Gaussians — a collection of 3D Gaussians with occupancy probability as opacity and novelty as color; (3) fast volumetric rendering estimates coverage gain for arbitrary candidate viewpoints; (4) beam search plans long-term trajectories; (5) the first \(N_f\) steps of the optimal trajectory are executed before replanning.

Key Designs¶

Imagined Gaussians Representation:
- Function: An efficiently renderable representation of scene uncertainty that supports rapid coverage gain computation.
- Mechanism: Isotropic Gaussian spheres are placed at proxy points of the occupancy network \(\hat{\sigma}(\mathbf{x}|\mathbf{C}_t)\), with opacity encoding occupancy probability and color encoding binary novelty \(\hat{\gamma} \in \{0,1\}\). By exploiting the structural correspondence of the volumetric rendering equation (density ↔ occupancy, transmittance ↔ occlusion, color ↔ novelty), the coverage gain integral is reformulated as standard GPU-accelerated Gaussian rendering. For any candidate pose, a "novelty map" is rendered and summed to obtain the coverage gain.
- Design Motivation: Traditional Monte Carlo sampling requires repeated queries to two neural networks at dense 3D points, which is computationally expensive (0.05 s/viewpoint). Imagined Gaussians exploit GPU rasterization, reducing per-viewpoint cost to 0.002 s — a 25× speedup.
Beam Search Long-Term Planning:
- Function: Efficiently searches for optimal long-term paths in the combinatorially explosive trajectory space.
- Mechanism: A beam of \(N_b\) candidate trajectories is maintained, each independently tracking its own Imagined Gaussians state. At each expansion step, all reachable poses for each beam are enumerated, coverage gains are computed, and only the top-\(N_b\) beams are retained. During search, Gaussian parameters are frozen; only the novelty values of observed Gaussians are updated (from 1 to 0), ensuring already-observed regions are automatically excluded in subsequent rendering. The trajectory maximizing cumulative coverage gain \(\sum_{i=1}^{N_d} G(\mathbf{c}_i)\) is selected.
- Design Motivation: Greedy NBV is a degenerate case with \(N_b=1, N_d=1\). Increasing beam width and lookahead steps systematically improves coverage efficiency (AUC +6.3%, coverage +9.3%).
Pretrained Occupancy Network as World Model:
- Function: Predicts the geometric structure of unobserved regions, providing prior knowledge for planning.
- Mechanism: A multi-layer Transformer network \(\hat{\sigma}(\mathbf{x}|\mathbf{C}_t)\) takes query points, the reconstructed point cloud, and historical poses as input, and outputs \([0,1]\) occupancy probabilities. Pretrained on ShapeNet and fine-tuned on 3D scenes, it encodes strong structural priors. It is also used for collision-free trajectory planning.
- Design Motivation: Long-term planning requires the ability to "imagine" the structure of unseen regions; otherwise, the future value of candidate viewpoints cannot be estimated. Ablation results show that performance is nearly unchanged even without fine-tuning on the target scene domain.

Loss & Training¶

The occupancy network is pretrained using a standard occupancy prediction loss. The exploration process itself involves no gradient updates — Imagined Gaussians are generated via forward inference and novelty values are updated by rule, making the closed-loop planning entirely training-free.

Key Experimental Results¶

Main Results¶

Dataset	Metric	MAGICIAN	MACARONS	FisherRF	SCONE
Macarons++	AUC↑	0.721	0.647	0.546	0.534
Macarons++	Final Coverage↑	0.919	0.819	0.786	0.670
MP3D (Wheeled)	Comp.(%)↑	85.45	-	-	-
MP3D (Wheeled)	Comp.(cm)↓	4.93	-	-	-
MP3D (UAV)	Comp.(%)↑	96.83	-	90.18 (NARUTO)	-
MP3D (UAV)	Comp.(cm)↓	2.11	-	3.00 (NARUTO)	-

Rendering/Reconstruction Quality (large-scale real scan scenes):

Method	SSIM↑	PSNR↑	LPIPS↓	Acc.(%)↑
FisherRF	0.55	13.95	0.38	79.15
MACARONS	0.61	15.68	0.34	86.42
MAGICIAN	0.64	17.12	0.30	94.20

Ablation Study¶

Configuration	AUC↑	Final Cov.↑	Notes
\(N_b=1, N_d=1\) (Greedy)	~0.66	~0.83	Degenerates to NBV; still outperforms MACARONS
\(N_b=10, N_d=10\) (Full)	0.721	0.919	+6.3% AUC, +9.3% Coverage
Pretrained occupancy model	0.652	0.888	Strong generalization
Fine-tuned occupancy model	0.646	0.893	Fine-tuning yields no clear benefit

Key Findings¶

Even when degenerated to greedy NBV, Imagined Gaussians-based rendering outperforms MACARONS's Monte Carlo approach: AUC +5.2%, coverage +10.9%. The 25× speedup in per-viewpoint gain estimation is the key enabler.
The value of long-term planning grows substantially with lookahead horizon: Extending lookahead from 1 to 10 steps improves coverage from ~82% to ~92%, demonstrating the necessity of long-term planning.
High replanning frequency is not required: Replanning every 6 steps suffices to achieve state-of-the-art performance, indicating that planned trajectories are reasonably robust.
The occupancy model transfers well across domains: A model pretrained only on outdoor scenes can be directly applied to indoor scenes with negligible performance degradation.

Highlights & Insights¶

The formal correspondence between coverage gain and volumetric rendering is the paper's most elegant insight: the surface coverage gain integral (occupancy × occlusion × novelty) is mathematically equivalent to the volumetric rendering equation (density × transmittance × color). This allows direct reuse of the highly optimized Gaussian rendering pipeline for coverage gain computation, effectively recasting exploration planning as a "rendering problem."
Each beam independently maintaining its own Gaussian state is a clever design — different candidate trajectories have different observation histories, and independent novelty states enable correct cumulative gain computation while preserving parallelism.
The framework naturally extends to other exploration criteria: only the semantics encoded in the "color channel" need to change (e.g., uncertainty, reconstruction error); the rendering framework remains unchanged.

Limitations & Future Work¶

A pretrained occupancy network is required, which may need retraining or fine-tuning for entirely novel domains (e.g., underwater, space).
Beam search still incurs computational overhead; evaluating large numbers of candidate viewpoints at \(N_b=10, N_d=10\) is non-trivial.
Experiments assume accurate known poses; the impact of localization error on planning is not addressed.
Future directions: (1) Lighter-weight occupancy estimation (e.g., 2D feature lifting to 3D) to reduce pretraining dependency; (2) Semantically guided active mapping via LLM/VLM integration; (3) Uncertainty-aware replanning strategies.

vs. MACARONS: MACARONS uses the same occupancy network but with greedy NBV and Monte Carlo gain estimation. MAGICIAN achieves long-term planning via Imagined Gaussians and beam search, improving coverage from 0.819 to 0.919.
vs. FisherRF: FisherRF decouples frontier-based path planning from Fisher information gain computation, causing path-level gain to be ignored. MAGICIAN's beam search optimizes cumulative gain at the trajectory level.
vs. ActiveGAMER: ActiveGAMER performs strongly on MP3D (95.32%); MAGICIAN further improves to 96.83% without relying on any traditional planner or navigation model.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ The formal correspondence between coverage gain and volumetric rendering is exceptionally elegant; this is the first method to achieve long-term planning for active mapping.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Indoor/outdoor multi-benchmark evaluation, multiple action spaces, dual assessment of rendering and reconstruction quality, and comprehensive ablations — the experimental design is thorough.
Writing Quality: ⭐⭐⭐⭐⭐ Mathematical derivations are clear, the logical flow from problem formulation to method design is coherent, and the paper is richly illustrated.
Value: ⭐⭐⭐⭐⭐ Addresses the long-standing open problem of long-term planning in active mapping with high practical utility.