Active Viewpoint Selection for Tomography via Perturbed Gaussian Ensemble¶

Conference: ECCV2026
arXiv: 2603.06852
Project Page: perturbed-gaussian-ensemble.cvmlgroup.web.illinois.edu
Code: None
Area: Medical Imaging
Keywords: Sparse CT Reconstruction, Active Viewpoint Selection, Radiative 3D Gaussian Splatting, Uncertainty Quantification, Perturbed Ensemble

TL;DR¶

To address the active viewpoint selection challenge for X-ray in sparse CT, this paper proposes a perturbed Gaussian ensemble framework based on radiative 3DGS. By randomly perturbing the density parameters of low-density Gaussian primitives, a lightweight model ensemble is constructed. Epistemic uncertainty is quantified using the structural similarity (SSIM) variance of predictions under candidate viewpoints, thereby selecting the viewpoint that best exposes geometric artifacts as the next optimal acquisition angle.

Background & Motivation¶

X-ray Computed Tomography (CT) is an indispensable non-destructive imaging modality in medical diagnostics and industrial inspection. Sparse-view CT reduces ionizing radiation exposure to patients by significantly decreasing the number of projection acquisition angles. However, this transforms tomographic reconstruction into a highly ill-posed inverse problem, where existing algorithms often suffer from severe streak artifacts and structural distortions under extremely sparse conditions. Recently, the emergence of Radiative Gaussian Splatting (Radiative 3DGS) has brought breakthrough progress to this challenge. It adapts the classic 3DGS framework to X-ray transmission imaging, modeling the 3D density field of an object with a set of explicit Gaussian primitives, and achieves fast and accurate CT reconstruction through differentiable rendering (subsequent works like R2-Gaussian push radiative GS towards unbiased reconstruction directly facing voxels). Nonetheless, despite the substantial enhancement in 3D modeling capabilities, the ultimate bottleneck of reconstruction quality shifts to the source of data—under a limited viewpoint budget, which angles should projections be acquired from to both cover the global envelope and accurately capture fine local structural details? This is the under-explored problem of "active viewpoint selection" in CT.

Existing active viewpoint selection methods are mainly tailored for natural light scenes, where representative methods like FisherRF estimate the expected information gain of candidate viewpoints via diagonal approximation of the Fisher Information Matrix (FIM). However, the physical properties of X-rays differ fundamentally from those of natural light: X-rays follow the Beer-Lambert law of absorption, where projections are pure linear integrals of the density field along the ray paths, without occlusion or surface reflection. Moreover, X-ray attenuation is isotropic, meaning Gaussian primitives lack viewpoint-dependent spherical harmonics. Consequently, the assumptions of "front-surface dominant rendering + viewpoint-dependent gradients" that natural light methods rely on fail completely in CT. The diagonal approximation of the FIM yields severe bias, preventing the method from distinguishing between streak artifacts and true high-density structures, often selecting redundant viewpoints rather than those that truly resolve geometric ambiguities.

The key insight of this paper is that under sparse-view constraints, geometric ambiguities typically manifest as fragile structures—such as uncertain boundaries and streak artifacts formed by overfitting—whose projections exhibit significant instability under different observation angles. An effective next-best-view (NBV) should maximize the exposure of this latent structural instability. Core Idea: Low-density Gaussian primitives are leveraged as proxies for uncertain regions. Their density parameters are subjected to random perturbations to construct a model ensemble. The structural variance (SSIM variance) of the projections rendered by this ensemble is calculated under candidate viewpoints, where the viewpoint with the largest variance is selected as the optimal view that maximizes the exposure of geometric artifacts.

Method¶

Overall Architecture¶

The proposed method is built upon the R2-Gaussian radiative Gaussian splatting framework, and the core pipeline is an iterative closed loop. In the initial phase, given a few sparse projections (e.g., 2 initial views), a radiative GS model is trained to model the 3D density field. Subsequently, the active selection loop is entered: first, a batch of low-density Gaussian primitives is identified from the trained model as uncertain primitives, and their density parameters are subjected to random scaling to generate $N$ perturbed models, forming a Perturbed Gaussian Ensemble. Next, for each viewpoint in the candidate viewpoint pool, a projection is rendered using the unperturbed base model and each of the $N$ perturbed models. The $N$ SSIM scores are computed, and their variance is taken as the uncertainty score for that viewpoint. The viewpoint with the highest score is selected as the next best view. The ground-truth projection of this viewpoint is then acquired and added to the training set. After re-optimizing the GS model, the pipeline proceeds to the next iteration until the total number of viewpoints reaches the target budget.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Initial Sparse Projection Set<br/>N_init Viewpoints"] --> B["Train Radiative GS Model<br/>R2-Gaussian"]
    B --> C["Identify Low-Density Gaussians<br/>Take Low-Density α=10%"]
    C --> D["Random Scaling Perturbation of Density<br/>→ Generate N=10 Perturbed Models"]
    D --> E["For each candidate view v:<br/>Render N+1 projections<br/>→ Compute SSIM variance u(v)"]
    E --> F["Select v*=argmax u(v)<br/>Acquire real projection → Add to training set"]
    F --> B

Key Designs¶

1. Density-Guided Random Perturbation: Low-Density Gaussian Primitives as Uncertainty Probes

In radiative GS, the density parameter $\rho$ of each Gaussian primitive determines its contribution to X-ray attenuation. High-density primitives correspond to well-defined structures such as bones; even under sparse views, their projection constraints along multiple rays are sufficiently abundant, making the model highly confident in these parameters. Conversely, low-density primitives typically reside near tissue boundaries, interfaces between soft tissues and air, or the "tails" of streak artifacts derived from overfitting. Due to the extremely limited training viewpoints, the geometry in these regions exhibits severe ambiguity—the model can stretch a long sequence of low-density Gaussians along the ray direction to fit known projections, which manifests as obvious artifacts under unobserved viewpoints.

Based on this observation, the authors select the lowest density $\alpha\%$ (experimentally $\alpha=10\%$) primitives as the fragile subset, and perform random scaling on the density of each primitive in this subset: $$\rho_{i,j} = \begin{cases} \rho_j \cdot (1 + \epsilon_{i,j}), & \text{if } G_j \in \mathcal{G}_{\text{low}} \\ \rho_j, & \text{otherwise} \end{cases}, \quad \epsilon_{i,j} \sim \text{Uniform}(-\beta, \beta)$$ High-density primitives remain completely unchanged. Each perturbed model essentially represents a plausible but distinct hypothesis—how the reconstructed structure would behave if the density of those uncertain regions fluctuated slightly. The ensemble of $N=10$ perturbed models naturally exhibits divergent interpretations of geometric ambiguities in the projection space, and to identify which interpretation is more correct requires observation from a new viewpoint—which is precisely the information needed for active selection.

2. Projection Space Structural Variance: Locating Epistemic Uncertainty with SSIM Variance

With the perturbed ensemble, how is the "information content" of a candidate viewpoint defined? The core idea is: if a viewpoint can amplify the discrepancy in the rendered structures among the different density hypotheses, it can provide information to distinguish these hypotheses. Specifically, for a candidate viewpoint $v$, a reference projection $I$ is rendered using the unperturbed base model, and another $N$ projections $I_i$ are rendered using the $N$ perturbed models. The SSIM score of each projection against the reference projection is computed as $s_i = \text{SSIM}(I, I_i)$. The uncertainty score of the candidate viewpoint is modeled as the sample variance of these $N$ SSIM scores: $u(v) = \text{Var}[s_1, \dots, s_N]$.

Why choose SSIM instead of $L_1$ error or PSNR? This is a subtle but critical engineering insight. The linear integration characteristic of X-ray projections dictates that perturbing the Gaussian densities inevitably leads to a shift in the overall brightness of the projection, as the contributions of all primitives along the ray are accumulated. $L_1$ and PSNR are absolute or mean squared errors calculated pixel-by-pixel, which are extremely sensitive to such global brightness drifts; hence, low-frequency brightness changes can easily overwhelm structural change signals arising from geometric ambiguities. SSIM, however, incorporates built-in luminance and contrast normalization, successfully decoupling absolute brightness shifts from genuine structural information. Ablation studies explicitly confirm that replacing the uncertainty metric with $L_1$ variance or PSNR variance leads to a decline in reconstruction $\text{PSNR}_3$ by $0.43\text{ dB}$ and $0.69\text{ dB}$, respectively, fully demonstrating that the structure-aware nature of SSIM is key to the success of this method.

3. Single Training with Multiple Perturbations: Lightweight Ensemble Simulation instead of Brute-Force Ensembling

The most intuitive uncertainty estimation method is to train $N$ independent GS models with different random seeds and compare their rendering discrepancies. However, the cost of such brute-force ensembling scales linearly with $N$. Training a single GS already requires tens of thousands of iterations, making the training of $N$ models practically infeasible. The key innovation of this work is the realization that the source of uncertainty primarily stems from parameter variations of low-density primitives. Therefore, one only needs to train a single base model and repeatedly perturb the fragile subset during evaluation, simulating multi-model divergence with $N$ forward passes. This is mathematically equivalent to a Monte Carlo approximation of the posterior distribution $p(I(v)|D)$, where the perturbations sample $N$ plausible posterior locations in the parameter space. The cost of a single forward pass is far lower than a full optimization, running the entire active selection pipeline highly efficiently on a single A40 GPU. In the ablation study, $N=10$ is identified as the optimal choice: $N=5$ suffers from insufficient sample size and unstable uncertainty estimation, while $N=40$ dilutes the contribution of extreme structural failures with a large number of samples, thereby reducing the contrast of the uncertainty scores.

Loss & Training¶

The optimization objective of the base model includes projection-level $L_1$ loss, D-SSIM loss, and a 3D total variation (TV) regularization loss. Active selection and model optimization are executed alternately: whenever an optimal viewpoint is selected, the real projection of this viewpoint is added to the training set for incremental optimization of the GS model. The trigger timing of selection follows the configuration of FisherRF—occurring before the densification phase of GS ends—to ensure that the model parameters are sufficiently stable.

Key Experimental Results¶

Main Results¶

Evaluation is conducted on a synthetic dataset (15 volumes from multiple public CT datasets) and a real-world dataset (3 cases from FIPS, with full-sampled FDK as the pseudo-GT), employing a hemispherical scanning trajectory. The candidate viewpoint pool contains 448 uniformly sampled scanning poses. Baselines include heuristic rules (Random/Uniform/FPS), 2D no-reference IQA methods (MUSIQ/MANIQA/TOPIQ), and 3D uncertainty methods (FisherRF).

Dataset	Protocol	Metric	Random	Uniform	FPS	FisherRF	Ours
Synthetic	24-view	$\text{PSNR}_3 \uparrow$	32.629	33.562	33.508	33.347	34.078
Synthetic	24-view	$\text{SSIM}_3 \uparrow$	0.881	0.890	0.891	0.887	0.896
Synthetic	36-view	$\text{PSNR}_3 \uparrow$	34.823	35.877	35.367	35.551	36.226
Synthetic	36-view	$\text{SSIM}_3 \uparrow$	0.915	0.921	0.919	0.919	0.926
Real	24-view	$\text{PSNR}_3 \uparrow$	36.112	—	36.134	36.205	36.399
Real	36-view	$\text{PSNR}_3 \uparrow$	36.765	—	36.898	37.258	37.480

In even sparser settings (6/8/12/16-view), the proposed method still consistently outperforms all baselines. At the lowest 6-view setting, the $\text{PSNR}_3$ reaches $25.93$ (compared to $25.80$ of the second-best, FisherRF), validating the applicability of the framework under extremely sparse acquisition conditions.

Ablation Study¶

Configuration	24-view $\text{PSNR}_3$	Description
Full (SSIM Variance)	34.078	Default configuration
Replaced with L1 variance	33.644	↓0.43dB, luminance shift pollutes structural signals
Replaced with PSNR variance	33.390	↓0.69dB, similarly dominated by global brightness variation
N=5	33.952	Ensemble size too small, unstable uncertainty estimation
N=10	34.078	Optimal trade-off
N=40	33.670	Too many samples smooth out the contrast of extreme structural failures
α=5%	33.680	Perturbation range too small, missing primitives that form artifacts
α=10%	34.078	Optimal
α=20%	33.589	Perturbations affect high-confidence structures, introducing non-informative variations
β=0.1	34.040	Perturbation too weak to break the equilibrium of artifacts
β=0.5	34.078	Optimal
β=1.0	33.338	Perturbation too strong, global structural distortion, loss of variance discriminability

Key Findings¶

SSIM variance vs. $L_1$/PSNR variance is the most critical ablation finding of this work. It not only verifies the necessity of SSIM's luminance normalization in X-ray scenarios but also reveals a design principle for active viewpoint selection in CT: one must isolate global luminance shifts caused by X-ray linear integration and focus on structural changes.
Density-guided perturbation outperforms sparse dropout and positional jitter: Directly applying dropout to remove low-density Gaussians is equivalent to completely losing signals in these regions rather than detecting their sensitivity. Positional jitter alters spatial locations instead of density parameters, which does not align with the physical model of Beer-Lambert attenuation.
FisherRF even underperforms the FPS heuristic under the synthetic 24-view setting, indicating that the mathematical bias of its diagonal FIM approximation not only yields no gains but also might introduce misleading information estimation in highly coupled X-ray scenarios.
On real-world data, the performance gaps between methods narrow compared to synthetic data, yet the proposed method still maintains its leading edge, demonstrating robustness of the framework against different noise levels.

Highlights & Insights¶

Ingenious reformulation of uncertainty estimation from impractical brute-force ensembling to "single training + multiple perturbations", directly utilizing the explicit physical interpretation of radiative GS density parameters to locate uncertain regions. Replacing $N$ full optimizations with $N$ forward passes reduces the computational overhead of the scheme by an order of magnitude.
The choice of SSIM variance over $L_1$/PSNR variance is a very nuanced insight—recognizing that the luminance drift caused by X-ray linear integration contaminates point-wise metrics, and that SSIM's luminance/contrast normalization precisely circumvents this pitfall. This choice is derived from the underlying physics model of transmission imaging rather than being purely empirical.
The "low density = high uncertainty" heuristic is simple yet powerful, bypassing the need for complex Bayesian networks or MC Dropout, and directly capitalizing on the semantic meaning of radiative GS density parameters. This design paradigm of coupling model parameter semantics with uncertainty proxies offers general reference value for explicit particle/voxel-based rendering frameworks.
The paper adapts and re-implements baselines such as FisherRF under the radiative GS framework for evaluation, establishing a standardized evaluation protocol and benchmark for active viewpoint selection in CT.

Limitations & Future Work¶

The experiments are only evaluated on a hemispherical scanning trajectory (simulating C-arm CT), without validation under geometric constraints such as circular or helical scans common in traditional medical CTs. Differences in the geometric distribution of potential viewpoints across different trajectories may affect the efficacy of the selection strategy.
The assumption "low density = high uncertainty" holds in most cases, but there could be counterexamples in extreme scenarios—such as very thin but structurally well-defined membrane materials, which have low density but lack geometric ambiguity. The boundary of this assumption needs more systematic stress testing.
The ensemble perturbation only targets density parameters, leaving the randomness of coordinates (positions) and covariances unexplored. Although ablation shows that positional jitter is less effective than density scaling, whether the joint randomization of position and covariance yields further gains remains an open question.
Real-world experiments are based on pseudo-GT from fully sampled FDK reconstructions, rather than raw clinical CT data. Performance under real noise environments (e.g., quantum noise, scattering, beam hardening) still requires clinical validation.

vs. FisherRF: FisherRF estimates information gain based on the diagonal approximation of the FIM, which is effective in front-surface-dominated natural light rendering but fails in X-ray scenarios due to the severe mathematical bias of the diagonal approximation under strong coupling along the ray path. Here, starting from a forward perturbation path, the proposed method directly simulates the impact of parameter changes on projections, avoiding gradient approximation bias while preserving real-time efficiency.
vs. ActiveNeRF / NAF and other NeRF-based methods: NeRF uses implicit MLPs to model density fields, making training and rendering significantly slower than 3DGS, and lacks the specialized adaptation to the physical interpretation of X-ray transmission found in R2-Gaussian (e.g., covariance scaling factor correction), failing to satisfy the concurrent requirements of rapid reconstruction and real-time active selection.
vs. 2D No-Reference IQA Methods (MUSIQ/MANIQA/TOPIQ): These methods directly use projection quality as the basis for viewpoint selection but lack the capability to model 3D structural uncertainty. The viewpoints they select tend to favor "locally clear projections" rather than those "reflecting the uncertainty of the 3D structure."

Rating¶

Novelty: ⭐⭐⭐⭐ Integrating perturbed ensembles with the physical characteristics of X-ray 3DGS is highly original, and using SSIM variance as an uncertainty metric is a clever fine-grained contribution.
Experimental Thoroughness: ⭐⭐⭐⭐ The ablation study is comprehensive (covering uncertainty metrics, perturbation ratios, ensemble sizes, and alternative strategies), though real-world data only uses pseudo-GT rather than authentic clinical CT.
Writing Quality: ⭐⭐⭐⭐ The motivation is clearly articulated, the logical chain across the methodology is intact, and the figure/table qualities are high.
Value: ⭐⭐⭐⭐ Provides a feasible technical solution for intelligent scan planning in low-dose CT, possessing clear potential for clinical translation.

Dataset	Protocol	Metric	Random	Uniform	FPS	FisherRF	Ours
Synthetic	24-view	\(\text{PSNR}_3 \uparrow\)	32.629	33.562	33.508	33.347	34.078
Synthetic	24-view	\(\text{SSIM}_3 \uparrow\)	0.881	0.890	0.891	0.887	0.896
Synthetic	36-view	\(\text{PSNR}_3 \uparrow\)	34.823	35.877	35.367	35.551	36.226
Synthetic	36-view	\(\text{SSIM}_3 \uparrow\)	0.915	0.921	0.919	0.919	0.926
Real	24-view	\(\text{PSNR}_3 \uparrow\)	36.112	—	36.134	36.205	36.399
Real	36-view	\(\text{PSNR}_3 \uparrow\)	36.765	—	36.898	37.258	37.480