FaCT-GS: Fast and Scalable CT Reconstruction with Gaussian Splatting

Conference: ECCV2026
Paper: ECCV Paper
Project: https://papieta.github.io/fact-gs/
Area: Medical Imaging
Keywords: Gaussian Splatting, sparse-view CT, CUDA acceleration, volumetric prior, volume compression

TL;DR

Rather than introducing another reconstruction network, FaCT-GS redesigns initialization and GPU computation for Gaussian CT reconstruction, reducing R2-Gaussian's 9 minutes to 1 minute 52 seconds while improving SSIM from 0.946 to 0.950 in the 75-view setting of Table 1.

Background & Motivation

CT reconstructs an object's internal attenuation from X-ray projections acquired at different angles. Reducing the projection count can shorten acquisition or lower radiation dose, but introduces substantial artifacts in direct FDK reconstruction. Traditional iterative algorithms, including SIRT and total-variation-regularized FISTA, remain competitive: they require no training dataset and deliver useful reconstructions at relatively low computational cost.

Neural fields and Gaussian Splatting offer alternative representations. Neural fields repeatedly query a network along each ray, whereas Gaussians admit analytical projection and avoid much of that sampling. Nevertheless, R2-Gaussian still takes several minutes at standard resolution. Increasing detector resolution from \(512^2\) to \(2048^2\) pixels amplifies wasted coverage, gradient write conflicts, and memory traffic inherited from general-purpose rendering code.

The authors therefore ask whether the computation truly matches CT, rather than simply increasing model capacity. X-ray projections accumulate attenuation without requiring visible-light occlusion ordering; faster voxelization could also make fitting an existing volume into Gaussians a practical initialization step. Core Idea: exploit CT's additive structure to redesign Gaussian kernels, while jointly optimizing initialization and loss computation so that the representation supports both fast reconstruction and direct volumetric priors.

Method

Overall Architecture

Inputs are sparse-view projections and acquisition geometry, optionally accompanied by a prior volume. The output is a Gaussian representation of the three-dimensional attenuation field, which can be voxelized into a regular grid when needed. Each Gaussian has a position, scale, rotation, and density. Attenuation at a spatial location is the sum of relevant Gaussian density contributions, with no view-dependent color branch.

The pipeline starts with Gradient and Prior Initialization, uses CT-Specific Rasterization and Voxelization to produce predicted projections and local voxel blocks, and evaluates them through Fused Loss Kernels. Gradients update Gaussian parameters while the inherited adaptive-control mechanism prunes, clones, or splits primitives. This loop does not generate the full volume at every step.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    input["Projections and geometry<br/>Optional volume prior"] --> init["Gradient and Prior<br/>Initialization"]
    init --> compute["CT-Specific Rasterization<br/>and Voxelization"]
    compute --> loss["Fused Loss Kernels"]
    loss --> update["Parameter updates<br/>and adaptive control"]
    update --> compute
    update --> output["Gaussian representation<br/>Final volume"]

Key Designs

1. Gradient and Prior Initialization: allocate limited Gaussians to structural changes

A cold start first reconstructs an FDK volume from the current projections and removes background through thresholding. R2-Gaussian samples Gaussian centers in proportion to reconstructed intensity, potentially spending primitives on bright but relatively uniform regions. FaCT-GS instead samples in proportion to gradient magnitude, emphasizing boundaries and high-frequency structures. Initial density is taken from reconstructed intensity at the sampled position and multiplied by \(k=0.15\); rotation starts at zero, and scale is set to the nearest-neighbor distance. This reallocates initial capacity rather than removing FDK artifacts outright.

When a similar object or an earlier volume of the same object is available, initialization need not detour through synthetic projections. The method samples Gaussians from the prior's gradients, voxelizes the resulting representation, and briefly fits it directly to the prior volume. This stage is supervised by the prior itself, whereas subsequent reconstruction is constrained by the currently acquired projections. The two information sources must remain distinct.

Rapid fitting in Figure 5 uses only 50 iterations and adds approximately 2 seconds. Its purpose is to provide a useful starting point, not to reproduce every detail of the prior. The authors explicitly acknowledge that higher initial SSIM does not automatically imply faster convergence to a better final solution in every task.

2. CT-Specific Rasterization and Voxelization: eliminate wasted work and gradient write conflicts

Rasterization transforms each three-dimensional Gaussian into ray/detector coordinates and analytically produces its two-dimensional projection contribution, retaining R2-Gaussian's integration factor for X-ray line integrals. Voxelization instead sums Gaussian contributions directly at three-dimensional grid locations without detector projection. Both operate on the same state but serve different roles: projection matching versus local regularization, prior fitting, and final volume output.

The original kernels cover elliptical Gaussians with square bounding boxes, visiting many negligible-contribution locations for elongated or low-density primitives. FaCT-GS uses density-aware, axis-aligned rectangular bounds to tighten the active region. It also reorganizes the backward pass around Gaussians instead of having neighboring pixels repeatedly compete to write gradients into the same primitive, reducing write conflicts and warp stalls.

A further change removes ray-wise sorting entirely. Visible-light alpha compositing depends on occlusion order, but CT attenuation integration is additive: reordering contributions does not change the signal. The authors note that R2-Gaussian recognized this property while retaining sorting overhead in its implementation. FaCT-GS additionally packs position and radius, or projected covariance parameters and the integration factor, into compact float4 structures to reduce memory traffic.

Voxelization is particularly important because a complete three-dimensional grid grows cubically with side length. During optimization, only a random local block is voxelized, with the total-variation block enlarged from \(8^3\) to \(32^3\). Full voxelization can be postponed until the end, so high-resolution output does not force every update to traverse the entire three-dimensional grid.

3. Fused Loss Kernels: prevent the bottleneck from moving into loss evaluation

Once rasterization and voxelization become fast, numerous small loss operations and their backward passes account for substantial runtime. FaCT-GS adopts fused-ssim for two-dimensional SSIM, reporting a 5–8-fold acceleration of that component. It also implements fused CUDA kernels for three-dimensional SSIM and total variation, reducing kernel launches and intermediate-memory traffic.

The three-dimensional SSIM kernel cannot simply extend the two-dimensional implementation without excessive thread-block shared-memory use. Instead, a ring buffer processes depth while each thread operates on one Z-axis row. The paper reports an 11-fold acceleration for three-dimensional SSIM and a 3-fold acceleration for total variation on a \(32^3\) block. These are component-level measurements, not end-to-end reconstruction speedups.

A Worked Example

Consider the 75-view setting in Table 1. An initial FDK volume is reconstructed from the current projections, and its gradients guide the placement of 50,000 Gaussians. Reconstruction compares predicted and acquired projections, constrains local smoothness on random \(32^3\) voxel blocks, and then updates and adaptively adjusts the Gaussian collection.

At 150 iterations, Table 1 reports 43 seconds, 38.33 dB PSNR, and 0.946 SSIM. Continuing to 400 iterations gives 112 seconds, 39.05 dB, and 0.950. These are stopping points within the same reported setting, not a claim that each iteration processes only one view, and they do not mix optional prior-fitting time into the cold-start results.

Loss & Training

Reconstruction combines projection L1 loss, two-dimensional SSIM loss, and three-dimensional total variation on a random local volume. The following restates Equation (5) with consistent notation, where \(I_r\) is the predicted projection and \(I_{gt}\) is acquired data:

\[ \mathcal{L}_{\mathrm{recon}}=\mathcal{L}_1(I_r,I_{gt})+\alpha_{\mathrm{SSIM}}\mathcal{L}_{\mathrm{SSIM}}(I_r,I_{gt})+\alpha_{\mathrm{TV}}\mathcal{L}_{\mathrm{TV}}(V_{32^3}). \]

Prior fitting directly compares voxelized output \(V_r\) with \(V_{\mathrm{prior}}\) using the L1 and three-dimensional SSIM combination from Equation (6). The original target-volume symbol is renamed to clarify that subsequent reconstruction does not gain access to target ground truth:

\[ \mathcal{L}_{\mathrm{fit}}=\mathcal{L}_1(V_r,V_{\mathrm{prior}})+\alpha_{\mathrm{SSIM}}\mathcal{L}_{\mathrm{SSIM}}^{3D}(V_r,V_{\mathrm{prior}}). \]

The main text does not specify numerical values for these two weights, and they cannot be inferred from speedups or quality tables. Reconstruction is per-scan self-supervised optimization rather than training a general network on paired CT data; standalone volume fitting and prior-based warm starts explicitly require an input volume.

Key Experimental Results

Main Results

Section 4.1 evaluates synthetic projections generated from 15 real CT volumes and 3 real-projection cases from FIPS. Output volumes are \(256^3\), synthetic projections are \(512^2\) pixels, and real projections are \(560^2\) pixels; Table 1 abbreviates this as the \(512^2\) dataset. The machine uses an NVIDIA H100 with CUDA 12.8.1.

The following selects results from Table 1, converting all times to seconds. PSNR is in dB and SSIM is dimensionless, with higher values better. R2-Gaussian and full FaCT-GS both initialize 50,000 Gaussians and run 400 iterations. FISTA runs 100 full-projection iterations; iteration counts across these algorithm families are not interchangeable.

Views	Method	PSNR (dB)	SSIM	Time (s)
75	FISTA	37.07	0.934	55
75	R2-Gaussian	39.04	0.946	540
75	FaCT-GS	39.05	0.950	112
75	FaCT-GS, 150 iterations	38.33	0.946	43
50	R2-Gaussian	38.09	0.939	346
50	FaCT-GS	37.97	0.943	75
25	R2-Gaussian	35.23	0.912	164
25	FaCT-GS	35.33	0.920	38

With 75 views, the speedup over R2-Gaussian is \(540/112\approx4.82\), with an absolute SSIM gain of 0.004. With 50 views, PSNR is actually 0.12 dB lower, so the method does not lead on every metric. The 25-view speedup is \(164/38\approx4.32\); the prose summary of "more than 4.5" cannot be applied mechanically to every column.

Ablation Study

The following reorganizes the initialization analysis in Figure 5, not Table 2 and not a final-reconstruction module-removal ablation. The four volume pairs contain brain MRI, pancreas CT, walnut CT, and cricket CT. Each target provides 50 synthetic projections; the metric is three-dimensional SSIM between initialization output and the target volume.

Object	FDK intensity sampling	FDK gradient sampling	Prior applied directly	FaCT-GS rapid prior fit
Brain	0.413	0.439	0.871	0.880
Pancreas	0.391	0.397	0.622	0.652
Walnut	0.198	0.215	0.417	0.533
Cricket	0.488	0.521	0.809	0.830

Gradient sampling produces small but consistent improvements. A prior changes the available information, so its larger gains cannot be attributed entirely to the initialization algorithm. Rapid fitting takes only 50 iterations, approximately 2 seconds; this figure assesses the starting state rather than establishing improved subsequent convergence time.

Key Findings

• Figure 4 scales a single coral volume using 50 projections, from \(256^3\) to \(1024^3\) voxels and from \(512^2\) to \(2048^2\) projection pixels. Initial Gaussian count doubles whenever side length doubles. At a fixed 100 iterations, the largest setting is 31.5 times faster than FISTA and 13 times faster than R2-Gaussian; this is not a matched-quality comparison.
• Figure 4 also measures time to SSIM 0.9. FISTA retains an advantage at the smallest volume. As resolution increases, FaCT-GS moves from 0.8 times to approximately 16 times FISTA's speed, showing that its advantage is strongest at larger scales.
• Figure 3 uses synthetic kernel tests on an RTX 4070 Ti SUPER, not the main experiments' H100. The text reports 2K-image rasterization decreasing from 210 ms to 9.3 ms, approximately 22-fold. This excludes the complete optimization pipeline.
• Standalone volume fitting in Section 5 uses 40,000 Gaussians and 500 iterations, taking 19 seconds on average. Table 2 reports 0.84 MB, 0.976 SSIM, and 44.2 dB for Gaussians, versus 0.92 MB, 0.981, and 44.5 dB for a JPEG stack. This is lossy fitting of a known volume, not sparse-projection reconstruction performance.

Highlights & Insights

• CT physics directly supports implementation simplification. Removing occlusion sorting exploits additive attenuation rather than introducing a rendering approximation.
• Fast voxelization expands the usefulness of the representation. It makes local three-dimensional regularization affordable and allows existing volumes to enter reconstruction through direct fitting instead of synthetic projections.
• End-to-end acceleration requires tracking bottleneck migration. Fusing SSIM and TV after accelerating rasterization prevents impressive local kernel gains from being lost elsewhere in reconstruction.

Limitations & Future Work

• The authors acknowledge that the initial Gaussian count remains manually selected. Adaptive control does not provide a general selection rule; structure complexity and explicit error budgets could guide future initialization strategies.
• The authors distinguish a better starting point from faster final convergence. Additional warm-start results are referenced in supplementary material but are not included in this cache, so no numerical claims are added from them.
• This note observes that scaling curves mainly use one coral volume, while initialization analysis contains only four volume pairs. Broader real-scan testing, mismatched priors, and changes in pathology are needed before general deployment claims are justified.
• Compression SSIM comparable to JPEG does not establish diagnostic equivalence. Each Gaussian stores 11 float16 parameters totaling 22 bytes, but structural-detail loss still requires downstream-task and clinical evaluation.

Related Work & Insights

• vs R2-Gaussian: FaCT-GS inherits the Gaussian attenuation field, analytical projection, and adaptive control. Its main contributions are initialization and CT-specific computation, making it valuable systems optimization rather than a wholly new reconstruction paradigm.
• vs FISTA / SIRT: Traditional methods update voxel grids directly and naturally accept priors, but their computational cost grows rapidly at high resolution. FaCT-GS primarily updates projections and a Gaussian collection, although FISTA still wins the smallest matched-quality case.
• vs NAF / IntraTomo: Neural fields obtain projections through repeated ray-wise network queries; analytical Gaussian projection avoids much of that work. The transferable lesson is to examine the imaging operator before choosing a more complex neural representation.

Rating

• Novelty: 4/5. CT adaptation, initialization, and complete-pipeline optimization are meaningful, while the basic representation follows prior work.
• Experimental Thoroughness: 4/5. Quality, runtime, resolution scaling, initialization, and compression are evaluated, but real deployment and final warm-start outcomes remain less established.
• Writing Quality: 4/5. Bottlenecks and engineering changes are explained clearly; some speed summaries require column-by-column checking against the table.
• Value: 4/5. The method materially improves high-resolution sparse-view CT practicality, with diagnostic safety still requiring independent validation.