Resolution-Agnostic Neural Operators for Multi-Rate Sparse-View CT¶
Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/neuraloperator/sparse_ct
Area: Physics / Medical Imaging
Keywords: Sparse-view CT, Neural Operator, DISCO Convolution, Resolution-Agnostic, Rotational Equivariance
TL;DR¶
This paper introduces Computed Tomography neural Operator (CTO), the first neural operator framework for sparse-view CT reconstruction that utilizes DISCO continuous convolutions and dual-domain spatial-frequency learning to achieve high-fidelity zero-shot generalization across measurement sampling rates and grid resolutions.
Background & Motivation¶
Computed Tomography (CT) reconstructs cross-sectional anatomical slices from multi-angle X-ray projection measurements via the inverse Radon transform. In routine clinical workflows, sparse-view CT (SVCT) serves as a primary paradigm to minimize patient ionizing radiation exposure and reduce scan acquisition time. However, severely subsampling projection view angles turns image reconstruction into a severely ill-posed inverse problem. Classical filtered back projection (FBP) suffers from pronounced streak artifacts under sparse sampling, while traditional compressed sensing schemes relying on total variation (TV) minimization demand lengthy iterative optimization and tend to over-smooth fine anatomical structures.
Recent advances in deep learning, particularly convolutional neural networks (CNNs) and unrolled model-based networks (e.g., Learned Primal-Dual, LEARN, and RegFormer), have dramatically elevated reconstruction fidelity. Nonetheless, existing deep learning architectures remain fundamentally constrained to discrete fixed-resolution grids and predetermined view-sampling rates. Standard CNNs employ discrete kernels with fixed support; when measurement sampling rates shift or detector resolutions change, the kernel's physical receptive field is distorted, causing catastrophic performance degradation on unseen sampling configurations. In clinical practice, acquisition protocols and target resolutions routinely vary across organs, diagnostic tasks, and scanner hardware. To accommodate multiple rates, current approaches train separate models for each specific view count, incurring prohibitive training costs and lacking cross-resolution generalization. Diffusion models exhibit empirical flexibility across sampling rates, but their multi-step iterative reverse sampling (often dozens of seconds per slice) prevents real-time clinical deployment.
To overcome the grid-binding bottleneck of discrete networks, CT reconstruction can be reformulated as operator learning between infinite-dimensional continuous function spaces. Because both projection measurements and anatomical slices represent underlying continuous physical functions, a neural network should map directly between function spaces. Core idea: Introduce neural operators into sparse-view CT via discrete-continuous (DISCO) convolutions parameterized in function space across dual sinogram and image domains, incorporating dual-branch spatial-frequency learning and rotation equivariance to achieve zero-shot generalization across arbitrary sampling rates and resolutions within a single unified model.
Method¶
Overall Architecture¶
CTO (Computed Tomography neural Operator) adopts an end-to-end unrolled network backbone, operating directly in continuous function spaces to alternate between sinogram-domain completion, physical projection consistency enforcement, and image-domain texture refinement.
The input is a sparse-view sinogram measurement \(\tilde{p}(\omega, r) = \mathcal{M}p(\omega, r)\) at an arbitrary subsampling rate, where \(\omega\) denotes the projection angle, \(r\) is the detector coordinate, and \(\mathcal{M}\) represents the angular sampling mask. The subsampled sinogram is first processed by the sinogram-space neural operator (\(\text{NO}_s\)), which comprises two parallel U-shaped discrete-continuous convolution blocks (UDNOs) processing spatial geometry and detector radial frequency representations respectively. The completed sinogram is backprojected into image space via the inverse Radon transform \(\mathcal{R}^{-1}\) to yield an initial image estimate. Subsequently, features undergo 3 cascades of unrolled iterative updates, where each cascade couples a physical forward-projection data-consistency update with an image-space neural operator (\(\text{NO}_i\)) for fine structural denoising, producing the final high-resolution CT image.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
Input["Input: multi-rate sparse sinogram<br/>p(ω, r) ∈ S¹ × ℝ"] --> DISCO["Continuous function-space DISCO operator<br/>Basis-parameterized continuous kernel, adaptive discretization"]
DISCO --> SinogramNO["Rotation-equivariant dual-branch sinogram operator<br/>Spatial UDNO + radial 1D-FFT frequency UDNO"]
SinogramNO --> Radon["Inverse Radon transform<br/>Map sinogram space to image space"]
Radon --> UnrolledDC["Dual-domain unrolled cascades with physical data consistency<br/>Image-space NO_i denoising + adjoint data consistency update"]
UnrolledDC --> Output["Output: resolution-agnostic CT image<br/>Zero-shot generalization across view rates & grid sizes"]
Key Designs¶
1. Continuous function-space DISCO operator: Decoupling grid discretization from local continuous convolutions
Standard discrete convolution kernels operate on discrete integer grids; when the grid resolution increases, fixed discrete support causes convolutional layers to collapse into pointwise linear mappings, losing their interpretation as local integral operators in the continuous limit. To achieve true resolution-agnostic learning, CTO adopts Discrete-Continuous (DISCO) convolutions as its core building block. DISCO defines kernel \(\phi\) as a continuous function parameterized over a compact continuous domain via a linear combination of continuous basis functions \(\phi(v) = \sum_{l=1}^L w_l \phi_l(v)\). When evaluated on a discretized grid, DISCO approximates the continuous convolution integral via quadrature: $\((\phi * g)(v_i) \approx \sum_{j=1}^m \phi(u_j - v_i) g(u_j) q_j\)$ where \(q_j\) are quadrature weights. CTO employs a piecewise-linear basis composed of 1 isotropic basis and 5 anisotropic concentric rings (each containing 7 basis functions) with a compact radius cutoff. When the grid resolution scales, DISCO preserves a consistent physical receptive field across resolutions and exhibits discretization convergence, ensuring the approximation error converges asymptotically to zero as resolution increases.
2. Rotation-equivariant dual-branch sinogram operator: Physical priors and Fourier slice-guided measurement completion
Tomographic sinograms exhibit exact geometric symmetries: rotating the physical object by \(\theta\) induces a translation along the angular axis \(\omega\); furthermore, parallel-beam acquisition satisfies the identity \(p(\omega + \pi, r) = p(\omega, -r)\). To strictly enforce this physical continuity, \(\text{NO}_s\) implements circular padding with a detector flip: boundary data along \(\omega\) is flipped along the detector axis \(r\) before wrapping around the angular dimension, eliminating boundary discontinuities and endowing the operator with rotational equivariance. In addition, inspired by the Fourier Slice Theorem and the classical ramp filter in FBP, \(\text{NO}_s\) features a dual-branch architecture combining spatial and frequency domain processing: $\(\hat{p} = \frac{1}{2} \left( \text{NO}_{s,\text{spatial}}(\tilde{p}) + \mathcal{F}_r^{-1}\left( \text{NO}_{s,\text{freq}}\left( \mathcal{F}_r(\tilde{p}) \right) \right) \right)\)$ where \(\mathcal{F}_r\) denotes a 1D Fourier transform along the detector position axis \(r\). By the Fourier Slice Theorem, the 1D spectrum of a projection along \(r\) corresponds to a radial slice of the object's 2D Fourier spectrum. Operating in this transformed domain via \(\text{NO}_{s,\text{freq}}\) allows the network to learn global frequency-dependent priors directly from data, effectively eliminating low-frequency blur while \(\text{NO}_{s,\text{spatial}}\) recovers local geometric structures.
3. Dual-domain unrolled cascades with physical data consistency: Joint iterative refinement across operator and measurement spaces
Unrolled model-based networks translate classical proximal gradient descent iterations into an end-to-end trainable architecture. CTO constructs 3 unrolled cascades in continuous function space. At cascade stage \(t\), the intermediate image estimate \(x_t\) is refined by the image-space operator \(\text{NO}_i^t\). \(\text{NO}_i\) uses a U-shaped DISCO block (UDNO) defined in 2D Cartesian coordinates to extract multi-scale spatial features and remove residual artifacts. The estimate is then regularized by a physical data consistency update: $\(x_{t+1} = x_t - \varrho_t \mathcal{A}^* (\mathcal{A}(x_t) - \tilde{p}) + \vartheta_t \text{NO}_i^t(x_t)\)$ where \(\mathcal{A} = \mathcal{M}\mathcal{R}\) denotes the forward Radon transform combined with the view-sampling mask, \(\mathcal{A}^*\) is its adjoint backprojection operator, and \(\varrho_t, \vartheta_t\) are learnable scalar weights. This architecture guarantees that the reconstructed image honors observed measurement constraints while benefiting from the continuous inductive bias of the neural operator.
Loss & Training¶
CTO is trained end-to-end using mean squared error (MSE) loss optimized via Adam with an initial learning rate of \(10^{-3}\). To ensure robust generalization across sparse configurations, the framework utilizes a multi-rate co-training strategy: within each training batch, a sampling rate is sampled uniformly at random from a discrete set of view counts (e.g., \(N_v \in \{9, 18, 36, 72\}\)). Consequently, a single model learns to process extreme undersampling and denser views simultaneously without overfitting to any specific acquisition rate.
Key Experimental Results¶
Main Results¶
On the AAPM Low-Dose Abdominal CT dataset (5,936 axial slices, 526 slices held-out for testing), all learning-based baselines and CTO are trained using the same multi-rate co-training regime across 9, 18, 36, and 72 views. Evaluation reports RMSE in Hounsfield Units (HU, lower is better), PSNR in dB (higher is better), and SSIM (\(\times 10^{-2}\), higher is better).
| Dataset | Category | Method | 18-view PSNR (dB) | 18-view RMSE (HU) | 36-view PSNR (dB) | 36-view RMSE (HU) | 72-view PSNR (dB) | 72-view RMSE (HU) |
|---|---|---|---|---|---|---|---|---|
| AAPM Abdomen | Classical / Free | FBP | 13.14 ± 1.63 | 632.58 ± 42.17 | 13.36 ± 1.63 | 616.66 ± 41.07 | 13.82 ± 1.63 | 584.72 ± 38.87 |
| AAPM Abdomen | Algebraic | SART | 24.97 ± 1.46 | 161.91 ± 5.93 | 25.91 ± 1.41 | 145.15 ± 2.59 | 26.66 ± 1.40 | 133.24 ± 1.81 |
| AAPM Abdomen | Diffusion | DPS | 25.68 ± 1.77 | 149.97 ± 17.34 | 28.33 ± 1.46 | 110.19 ± 8.78 | 31.64 ± 1.43 | 85.17 ± 4.71 |
| AAPM Abdomen | Diffusion | ALD | 25.78 ± 1.71 | 148.13 ± 14.63 | 29.13 ± 1.51 | 100.50 ± 7.33 | 30.56 ± 1.41 | 75.19 ± 5.35 |
| AAPM Abdomen | Unrolled Net | LEARN | 25.51 ± 1.94 | 153.39 ± 22.22 | 27.53 ± 1.81 | 121.31 ± 16.15 | 29.90 ± 1.59 | 92.49 ± 12.91 |
| AAPM Abdomen | Unrolled Net | RegFormer | 25.67 ± 1.91 | 150.57 ± 21.34 | 28.00 ± 1.82 | 114.93 ± 15.18 | 30.43 ± 1.70 | 86.83 ± 10.46 |
| AAPM Abdomen | Unrolled Net | Unrolled CNN | 29.14 ± 1.84 | 100.65 ± 11.36 | 31.76 ± 1.80 | 74.44 ± 8.32 | 33.35 ± 1.81 | 62.38 ± 11.16 |
| AAPM Abdomen | Neural Operator | CTO (Ours) | 31.57 ± 1.73 | 75.97 ± 7.32 | 35.06 ± 1.64 | 50.79 ± 4.27 | 37.88 ± 1.56 | 36.64 ± 2.44 |
Under the challenging 18-view regime, CTO outperforms the Unrolled CNN baseline by 2.43 dB PSNR and reduces RMSE by 24.68 HU. At 72 views, CTO surpasses Unrolled CNN by 4.53 dB and exceeds diffusion-based DPS by over 6.2 dB.
Ablation Study¶
The table below contrasts single-rate overfitting versus multi-rate training (Table 2a) and highlights performance drops when isolating architectural components (Sec. 5.4):
| Config / Model | 18-view PSNR (dB) | 36-view PSNR (dB) | 72-view PSNR (dB) | Note |
|---|---|---|---|---|
| LEARN (72-view trained) | 4.16 | 22.40 | 32.32 | Severe overfitting to seen sampling rate; collapses on 18-view |
| RegFormer (72-view trained) | 5.04 | 24.63 | 35.11 | Single-rate training fails to generalize across views |
| LEARN (multi-rate co-trained) | 25.51 | 27.53 | 29.90 | Co-training mitigates catastrophic failure across rates |
| RegFormer (multi-rate co-trained) | 25.67 | 28.00 | 30.43 | Broadens generalizability across sparse regimes |
| CTO (full model, multi-rate) | 31.57 | 35.06 | 37.88 | Superior performance across all view settings |
| CTO w/o image-space \(\text{NO}_i\) | - | - | - | Average PSNR drops by 3.86 dB across views |
| CTO w/o sinogram-space \(\text{NO}_s\) | - | - | - | Average PSNR drops by 4.82 dB across views |
| CTO w/o sinogram freq-branch \(\text{NO}_{s,\text{freq}}\) | - | - | - | Removing frequency branch incurs 2.70 dB drop |
| CTO w/o sinogram spatial-branch \(\text{NO}_{s,\text{spatial}}\) | - | - | - | Removing spatial branch incurs 3.10 dB drop |
| CTO w/o circular padding (24-view) | 32.17 (24-view) | - | - | Removing rotation-equivariant padding drops 0.83 dB |
Benchmarking on an NVIDIA A100 GPU reveals that CTO requires only 0.065s per slice during inference with zero sampling-specific tuning. In contrast, diffusion baselines DPS and ALD take 51.58s and 32.72s respectively, demonstrating that CTO achieves a >500× inference speedup over diffusion models.
Key Findings¶
- Dual-domain neural operators are complementary: Omitting the sinogram-space operator \(\text{NO}_s\) results in a 4.82 dB drop, underscoring the critical necessity of restoring missing projection rays directly in measurement space; removing \(\text{NO}_i\) drops 3.86 dB, showing the importance of image-space multi-scale refinement.
- Frequency-domain sinogram processing suppresses blur: Incorporating a 1D Fourier transformed UDNO allows the network to learn data-driven filtering akin to classical ramp filters, yielding a 2.7 dB improvement.
- Zero-shot super-resolution capability: When trained strictly on \(256 \times 256\) grids from 18-view data and evaluated zero-shot at \(512 \times 512\), CTO achieves a 3.0 dB gain over the CNN baseline. When doubling both detector counts and view angles (144 views × 1344 detectors), CTO surpasses Unrolled CNN by 3.7 dB, validating the fidelity of function-space continuous kernels.
Highlights & Insights¶
- Harmonizing physical geometry and function space: By recognizing the \(\pi\)-periodicity with detector flipping in parallel-beam sinograms, CTO enforces boundary continuity via modified circular padding, rendering the operator rotation-equivariant and resilient to object orientation shifts.
- Overcoming discretization dependence: Traditional computational imaging mandates retraining specialized networks for each protocol; CTO demonstrates that a single continuous operator can span extreme sparse-view to dense sampling and arbitrary image resolutions without retraining.
- Dual-space local-to-global transformation: By leveraging the Radon transform and Fourier Slice Theorem, local continuous convolutions in the sinogram domain correspond to direction-selective global modifications in image space, achieving global receptive fields at local computational complexity.
Limitations & Future Work¶
- Restriction to parallel-beam 2D geometry: Current validations focus on 2D parallel-beam geometry, whereas contemporary clinical scanners frequently employ helical fan-beam or 3D cone-beam geometries with non-trivial projection trajectories.
- Handling highly anisotropic detector sampling: Disproportionate aspect ratios between angular sampling intervals and detector pixel sizes may require adaptive kernel basis scaling.
- Future directions: Extending CTO to 3D dynamic volumetric CT and photoacoustic tomography (PACT), while incorporating Bayesian uncertainty quantification into the operator predictions.
Related Work & Insights¶
- vs Learned Primal-Dual / LEARN / RegFormer: Prior unrolled networks utilize discrete convolutions or discrete Vision Transformers tied to fixed grid sizes, experiencing severe degradation across different view samplings; CTO replaces discrete layers with continuous DISCO operators to deliver true discretization agnosticism.
- vs DuDoNet / DuDoTrans: While dual-domain models process both measurement and image domains, their feature extractors remain resolution-bound; CTO introduces rotation-equivariant and Fourier-slice-guided continuous operators with superior generalization.
- vs Diffusion Models (DPS / ALD): Diffusion models provide empirical flexibility across sampling rates but require hundreds of iterative sampling steps (~30-50s per slice) and extensive per-rate hyperparameter tuning; CTO performs single-pass cascaded inference in 0.065s (>500× faster) with no hyperparameter recalibration.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ [Pioneers neural operators for sparse-view CT; introduces dual-branch spatial-frequency learning and rotation-equivariant continuous convolutions]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Evaluated on AAPM and Kidney benchmarks with extensive comparisons against classical, unrolled, and diffusion models across multi-rate, super-resolution, and OOD settings]
- Writing Quality: ⭐⭐⭐⭐⭐ [Clear mathematical formulation in function space with rigorous physical motivations and well-structured empirical ablations]
- Value: ⭐⭐⭐⭐⭐ [Directly resolves the persistent clinical bottleneck of model retraining across acquisition protocols, providing major practical utility]