NGPS: Structure-Preserving Self-Supervised Denoising via Neighbor-Guided Patch Sampling

Conference: ECCV 2026
arXiv: 2606.23200
Code: https://github.com/cv-cho/NGPS
Area: Medical Image / Self-Supervised Learning
Keywords: Self-Supervised Denoising, Neighbor-Slice Supervision, Medical Imaging, Misalignment Handling, Patch Matching

TL;DR

NGPS proposes a lightweight neighbor-slice self-supervised denoising framework: for inter-slice misaligned regions, local patch matching is performed on a noise-attenuated guidance map, while the training target retrieves pixel values from the corresponding matched coordinates in the original noisy neighbor-slices. This decouples "structural matching" from "signal retrieval" without requiring dense deformation field estimation, consistently improving fidelity and structure-sensitive metrics on low-dose CT and MRI.

Background & Motivation

Self-supervised denoising is highly attractive for volumetric medical imaging (CT, MRI) as it avoids the need for paired clean-noisy images. Neighboring-slice self-supervised learning (neighboring-slice SSL) leverages adjacent slices as mutual supervision targets, based on the assumption of independent noise between slices. However, spatial misalignment of inter-slice anatomical structures disrupts co-located correspondences: clean signals shift by \(\delta\) in adjacent slices, resulting in the overlay of \(\frac{1}{2}\big(x_z(p) + x_z(p-\delta)\big)\), which causes ghosting artifacts and edge blurring.

Existing solutions fall into two categories, each with its pros and cons. Masking-based methods (e.g., Noise2Sim, NS-N2N) directly discard pixels with excessive differences, offering stable training but at the cost of losing a significant amount of supervision signals. Quantitative analysis in the paper shows that approximately 20% of ROI neighbor pixels in the LIDC-IDRI dataset are discarded by masking, and the discarded regions are highly concentrated at high-gradient (high-frequency) anatomical boundaries. This creates a paradox where critical boundaries that require the most delicate restoration lack supervision. Registration-based methods (e.g., Deformed2Self) explicitly estimate inter-slice correspondences and warp neighboring slices. However, the reliability of registration degrades under heavy noise or large slice spacing, and spatial resampling tends to smooth edges while introducing additional computational overhead.

Hence, the core challenge is: are the misaligned pixels in neighbor-slice supervision actually "unusable" or simply "misplaced"? This work argues that misalignment does not equal absence—the shifted anatomical evidence still exists within the local neighborhood of the adjacent slice, just not at the identical coordinates. The core idea is to decouple correspondence searching (structural matching) from supervision value retrieval (signal retrieval): patch similarity search is performed on the noise-attenuated guidance map to locate corresponding coordinates, whereas the regression target always retrieves the raw pixel values from the original noisy adjacent slice. This avoids selection bias driven by noise when using original noisy patches for matching, and avoids altering noise statistical properties through warping or interpolation.

Method

Overall Architecture

NGPS is a training-time supervision construction framework that does not modify the inference architecture. The input is a triplet of noisy slices \(\{y_{z-1}, y_z, y_{z+1}\}\), and the backbone network \(f_\theta\) is a standard 2D denoiser (NAFNet), requiring only a single noisy slice during inference. The supervision construction consists of three stages: (1) generating a noise-attenuated guidance map using bilateral filtering + median filtering, and computing a direction-aware misalignment mask on this guidance map; (2) performing patch SSD (Sum of Squared Differences) matching on the guidance map within a local window of the neighboring slice for the masked misaligned pixels to locate the top-\(K\) candidate coordinates; (3) retrieving the center pixel values from the original noisy adjacent slices at these coordinates and averaging them to serve as the retrieved pseudo-targets. During training, static pixels are supervised by the standard co-located neighbor-slice loss (\(\mathcal{L}_{N2N}\)), while misaligned pixels are supervised by the retrieved targets (\(\mathcal{L}_{NGPS}\)). These are summed as the reconstruction loss, supplemented by a regional consistency regularization term \(\mathcal{L}_{RC}\).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Noisy Slice Triplet<br/>y_{z-1}, y_z, y_{z+1}"] --> B["Guidance Map Generation<br/>BF+MF → Noise-attenuated Guidance Map ỹ"]
    B --> C["Direction-aware Misalignment Mask<br/>|ỹ_z - ỹ_z'| > τ → M_{z→z'}=1"]
    C --> D{"Pixel State?"}
    D -->|"Static M=0"| E["Co-located Neighbor Supervision<br/>L_N2N"]
    D -->|"Misaligned M=1"| F["Local Guided Matching<br/>Guidance Map SSD Search Top-K"]
    F --> G["Raw Value Retrieval & Aggregation<br/>Average of K Center Pixels in Original Neighbor Slices"]
    G --> H["L_NGPS"]
    E --> I["L_recon + λ L_RC"]
    H --> I
    I --> J["Train NAFNet Denoiser f_θ"]

Key Designs

1. Noise-Attenuated Guidance Map and Direction-Aware Misalignment Mask: Driving "Where to Retrieve"

Traditional masking methods compute inter-slice differences directly on raw noisy images, where high noise levels interfere with mask estimation—noisy regions are falsely detected as misaligned, while genuine misalignments at structural boundaries are obscured by noise. NGPS first applies lightweight edge-preserving filtering to the inputs: a 2D bilateral filter (\(d=5, \sigma_{color}=35, \sigma_{space}=50\)) followed by a \(5\times5\) median filter to obtain the noise-attenuated guidance map \(\tilde{y}_z\). This guidance map suppresses quantum noise while preserving anatomical boundaries, offering a cleaner reference base for subsequent matching.

The mask is direction-aware rather than a single global mask: for each neighbor slice direction \(z' \in \{z-1, z+1\}\), it individually computes \(\mathcal{M}_{z\to z'}(p) = \mathbb{1}\big(|\tilde{y}_z(p) - \tilde{y}_{z'}(p)| > \tau\big)\), where \(\tau = 0.05\). This means a pixel might be static relative to the preceding slice (co-located supervision suffices) but misaligned relative to the succeeding slice (requiring retrieval), avoiding unnecessary computational overhead or supervision waste caused by global, one-size-fits-all thresholding.

2. Decoupled Guided Matching and Raw Value Retrieval: Core Insight of NGPS

This is the most fundamental difference between NGPS and PixelBank or traditional patch matching methods. For a masked misaligned pixel \(p\), within a local search window \(\Omega_p\) (\(15\times15\)) in the neighboring slice \(z'\), the SSD similarity is calculated using a \(k\times k\) patch (\(k=7\)) on the guidance map:

\[\mathcal{D}(p, q; z') = \|\mathcal{P}_k(\tilde{y}_z, p) - \mathcal{P}_k(\tilde{y}_{z'}, q)\|_2^2, \quad q \in \Omega_p\]

Key operation: matching is performed on the guidance maps, but retrieval extracts values from the original noisy images. The supervision signal corresponding to the matched coordinate \(q\) is \(y_{z'}(q)\) (a single scalar, i.e., the center pixel value) instead of \(\mathcal{P}_k(y_{z'}, q)\) (the entire patch).

The motivation behind this design is twofold: (1) Performing matching directly on original noisy patches leads to a selection bias where noise patterns guide the coordinate selection to match the input noise, violating the zero-mean noise assumption of Noise2Noise. (2) Retrieving the average or weighted average of the entire matched patch essentially introduces spatial smoothing on neighboring slices, blurring high-frequency structures. Retrieving only the center pixel preserves the statistical properties of the original noise (zero-mean, constant variance), allowing the network to learn an unbiased estimator for point-wise noise.

3. Top-K Ensembled Target: Reducing Single Match Variance

Retrieving only the center pixel from a single best match results in high variance—a single match might coincidentally land on noise peaks or valleys. NGPS retrieves the top-\(K\) candidate coordinates with the smallest SSD, \(\{q^{(1)}, \dots, q^{(K)}\}\) (\(K=4\)), and averages their corresponding center pixel values from the original noisy neighbors:

\[t_{z\to z'}(p) = \frac{1}{K}\sum_{k=1}^{K} y_{z'}\big(q^{(k)}\big)\]

While this shares sampling concepts with PixelBank, three key differences exist: (1) the search space is restricted to a local window of the adjacent slice rather than the entire image, ensuring high efficiency and preventing noise coupling from self-retrieval; (2) it is triggered only at misaligned pixels instead of dense global sampling; (3) \(K=4\) is the sweet spot for the noise-detail trade-off—ablation studies reveal that \(K=1\) or \(2\) leaves prominent residual noise, while \(K=8\) or \(16\) incorporates poor-quality matches, resulting in blurred details.

A Complete Example

Taking a chest CT slice with \(2.5\text{mm}\) thickness from LIDC-IDRI as an example: for a pixel \(p\) at a blood vessel boundary on current slice \(z\), the difference between its guidance value and the co-located value in neighbor-slice \(z+1\) is \(0.12 > \tau=0.05\), marking it as misaligned. Within a \(15\times15\) search window (225 candidate coordinates) on slice \(z+1\), NGPS performs SSD matching using \(7\times7\) patches on the guidance maps, selecting the top-4 candidate coordinates (shifted by \((-2,+1), (-1,+1), (-3,0), (-2,+2)\) pixels, respectively) with ascending SSD costs. The final target value \(t_{z\to z+1}(p)\) is the average (0.431) of the center pixel values at these 4 coordinates on the original noisy neighboring slice (e.g., 0.423, 0.438, 0.415, 0.448). This target replaces the direct co-located neighbor value of 0.387 (which is erroneously low due to structural shift), allowing the network to recover the actual vessel signal at the boundary pixel rather than a blurred average.

Loss & Training

The total loss is \(\mathcal{L}_{total} = \mathcal{L}_{recon} + \lambda \mathcal{L}_{RC}\), where \(\lambda=0.5\).

Hybrid Reconstruction Loss: The supervision source adaptively switches based on the mask:

\[\mathcal{L}_{recon} = \mathcal{L}_{N2N} + \mathcal{L}_{NGPS}\]

\[\mathcal{L}_{N2N} = \frac{1}{|\mathcal{N}(z)|}\sum_{z'}\frac{\sum_p (1-\mathcal{M}_{z\to z'}(p))(f_\theta(y_z)(p) - y_{z'}(p))^2}{\sum_p (1-\mathcal{M}_{z\to z'}(p)) + \epsilon}\]

\[\mathcal{L}_{NGPS} = \frac{1}{|\mathcal{N}(z)|}\sum_{z'}\frac{\sum_p \mathcal{M}_{z\to z'}(p)(f_\theta(y_z)(p) - t_{z\to z'}(p))^2}{\sum_p \mathcal{M}_{z\to z'}(p) + \epsilon}\]

The two terms are normalized individually by the number of pixels in their respective regions to avoid gradient bias caused by imbalanced pixel counts.

Regional Consistency Regularization: Applied only to the static regions (\(1-\mathcal{M}\)), this term encourages consistent predictions across adjacent slices in static areas, reducing inter-slice flicker:

\[\mathcal{L}_{RC} = \frac{1}{|\mathcal{N}(z)|}\sum_{z'} \|(1-\mathcal{M}_{z\to z'}) \odot (f_\theta(y_z) - f_\theta(y_{z'}))\|_2^2\]

Note that NGPS does not use the Inter-slice Continuity (\(\mathcal{L}_{IC}\)) term from NS-N2N. Ablation studies show that adding \(\mathcal{L}_{IC}\) to NGPS decreases PSNR by 0.10 dB, as \(\mathcal{L}_{IC}\) enforces a local linear constraint (\(f_\theta(\frac{y_z+y_{z+1}}{2}) \approx \frac{f_\theta(y_z)+f_\theta(y_{z+1})}{2}\)) that conflicts empirically with the retrieval targets provided by NGPS in misaligned regions.

Training configuration: AdamW optimizer, initial learning rate \(2\times10^{-4}\), weight decay \(10^{-5}\), batch size 4, trained for 10 epochs with input randomly cropped to \(256\times256\). The NAFNet backbone uses a base width of 32, with encoder blocks [2,2,4,8], 8 middle blocks, and decoder blocks [2,2,2,2]. Hyperparameters for NGPS are fixed across all datasets at \(p=7, W=15, K=4, \tau=0.05\).

Key Experimental Results

Main Results

NGPS is compared against 12 baseline methods on three datasets, including a classic method (BM3D), single-image self-supervised methods (Noise2Void, NB2NB, Pixel2Pixel, etc.), neighbor-slice masking methods (Noise2Sim, NS-N2N), and registration-based methods (Deformed2Self).

Method	AAPM-Mayo QD CT					LIDC-IDRI ULD CT
	PSNR↑	SSIM↑	FSIM↑	HFEN↓	GMSD↓	PSNR↑	SSIM↑	FSIM↑	HFEN↓	GMSD↓
Noisy	30.30	0.7222	0.8772	0.3363	0.0874	22.14	0.4213	0.5920	0.8429	0.1959
BM3D	34.67	0.7764	0.8547	0.3258	0.0900	25.07	0.5825	0.7768	0.6010	0.1449
Noise2Void	32.57	0.7786	0.8977	0.3286	0.0739	26.10	0.5995	0.8295	0.4948	0.1078
Pixel2Pixel	33.71	0.8357	0.9117	0.3183	0.0739	25.58	0.5078	0.7277	0.6196	0.1237
Deformed2Self	35.85	0.8662	0.9144	0.2299	0.0377	28.94	0.6703	0.8170	0.5134	0.0808
Noise2Sim	35.49	0.8639	0.9420	0.2406	0.0390	28.81	0.7817	0.8749	0.5003	0.1040
NS-N2N	35.91	0.8584	0.9235	0.2325	0.0396	30.62	0.8080	0.8944	0.4406	0.0777
NGPS	36.68	0.8986	0.9470	0.2056	0.0362	31.03	0.8102	0.9168	0.4161	0.0788

On the AAPM dataset, NGPS achieves the best performance across all five metrics, with PSNR leading NS-N2N by 0.77 dB. On LIDC-IDRI, improvements in PSNR, FSIM, and HFEN are significant (PSNR improves by 0.41 dB, with the volume-level 95% CI [0.23, 0.59] excluding zero), while the SSIM improvement is marginal (+0.0022, with CI overlapping zero), and GMSD is slightly worse than NS-N2N (0.0788 vs 0.0777). Across three levels of synthetic Rician noise (5%/7%/9%) on IXI MRI, NGPS consistently achieves the highest PSNR and SSIM (e.g., PSNR of 24.21 dB under 9% noise vs. 24.11 dB for the second-best Deformed2Self).

Qualitatively, under AAPM quantum noise and LIDC streak artifact scenarios, masking methods show smoother boundaries in zoomed-in ROI regions (consistent with lost supervision), registration methods present local artifacts, while NGPS preserves sharper anatomical boundaries while reducing residual noise. Under 9% Rician noise on IXI, masking-based approaches soften cortical boundaries, whereas NGPS maintains sharper local structures.

Ablation Study

Configuration	Recon	RC	IC	PSNR (dB)	SSIM
NS-N2N	✓			35.43	0.8481
NS-N2N	✓	✓		35.88	0.8578
NS-N2N (full)	✓	✓	✓	35.91	0.8584
NGPS	✓			36.50	0.8894
NGPS (Final)	✓	✓		36.68	0.8986
NGPS	✓		✓	36.39	0.8938
NGPS	✓	✓	✓	36.58	0.8981

The reconstruction-only NGPS (without RC/IC) already outperforms the full NS-N2N by 0.59 dB, validation the high quality of the retrieved targets. RC contributes +0.18 dB PSNR and +0.0092 SSIM to NGPS, as enforcing consistency in static regions stabilizes training. Adding IC drops performance by 0.10 dB because the local linear constraint of IC conflicts with the NGPS retrieval targets.

In terms of robustness to slice thickness, switching from \(1.25\text{mm}\) to \(2.5\text{mm}\) on LIDC-IDRI results in a decay of only 0.16 dB for NGPS, compared to drops of 0.63 dB for NS-N2N and 2.83 dB for Noise2Sim, indicating that local retrieval is highly robust to increased slice spacing. Ablation on guidance filters shows that the BF+MF combination achieves the best trade-off between quality and efficiency: while NLM+Median yields a slightly higher score (36.74 dB on AAPM), the CPU generation time per triplet surges from 0.14s to 12.71s (~89x slow down). Regarding computational efficiency, target generation for a 100-slice volume takes only ~0.72s with NGPS, which is ~19x faster than NS-N2N and ~7.5x faster than Pixel2Pixel.

Key Findings

• Retrieval Target as Primary Contributor: Reconstruction-only NGPS (without regularization) already outperforms the full version of NS-N2N, demonstrating that pseudo-targets constructed via decoupled matching-retrieval are of much higher quality than those discarded by masking, serving as the main driver of PSNR gains.
• \(K=4\) as the Sweet Spot: PSNR rises monotonically as \(K\) increases from 1 to 4, but drops when \(K > 4\), as excessive candidates invite low-quality targets from mismatched coordinates, blurring details.
• Robustness to Threshold at \(\tau=0.05\): Unlike NS-N2N, whose performance drops sharply if the threshold is too strict or too relaxed, the retrieval mechanism of NGPS provides a buffer—even if the mask is inaccurate, the retrieved neighbor values are still preferable to direct discarding.
• Degradation below NS-N2N Under Extremely Large Slice Spacing: When slice spacing gap \(\ge 4\text{mm}\) on AAPM, NGPS underperforms NS-N2N (34.48 vs 35.12 dB at \(5\text{mm}\)), exposing the limit of the local homology assumption within a fixed window.

Highlights & Insights

• Decoupled Matching and Retrieval as the Core Contribution: This seemingly simple design—matching on the filtered map and retrieving values from the raw map—simultaneously addresses two issues: noise-driven matching bias (if matching were performed on the raw map) and detail degradation from spatial smoothing (if values were retrieved from the filtered map). This serves as a versatile design pattern that can be transferred to other tasks where 'finding correspondences under noise without suffering from noise bias' is needed.
• Direction-Aware Masking over Global Binary Masking: Neighbor-slice misalignment is direction-dependent—one adjacent slice might be aligned while the other is not. Generating masks directionally avoids supervision waste from one-size-fits-all thresholding, offering valuable insights for multi-frame/multi-view denoising.
• Engineering Rationality of the \(15\times15\) Local Window: Cumulative Distribution Function (CDF) analysis of matching offsets across multiple geometries proves that a \(15\times15\) window covers \(>80\%\) of effective matching offsets, avoiding quadratic candidate costs of larger windows. This methodology of 'analyzing statistics before determining window sizes' is highly generic.
• Replacing Inference Post-processing with Training-Time Supervision Construction: NGPS preserves the network architecture and inference pipeline intact, modifying only how 'supervision signals are constructed'. This non-intrusive refinement strategy can be easily plugged into any Noise2Noise-style training pipeline.

Limitations & Future Work

• Failure of Local Homology Assumption Under Extreme Slice Spacing: When slice spacing exceeds the coverage of the local window (e.g., \(5\text{mm}\) on AAPM, \(6.25\text{mm}\) on LIDC), the matched 'nearest neighbor' no longer corresponds to the same anatomical structure, leading to retrieval degradation. Although the authors propose a Calibrated Match-Cost Gate in the supplement as a mitigation (recovering 0.72-0.96 dB under large gaps), this does not fundamentally eliminate the assumption.
• Verification Limited to Zero-Mean, Slice-Independent Noise: Whether cross-slice correlated artifacts (such as z-directional reconstruction artifacts in helical CT) satisfy the Noise2Noise conditions remains unverified.
• Limitations in MRI Evaluation: Only the IXI dataset with synthetic Rician noise is utilized, lacking evaluations on clinically acquired low-field MRI or on different protocols/contrasts.
• Hyperparameters Tuned to Pragmatic Defaults Rather Than Optimal Values: The parameters \(p=7, W=15, K=4, \tau=0.05\) are fixed across all datasets. Although their robustness is validated through ablation, adaptive adjustment mechanisms (e.g., dynamically adjusting \(W\) and \(K\) based on slice spacing) could potentially elevate performance further.
• Unexplored Downstream Performance: The performance of denoised images on downstream clinical tasks (such as lesion detection or segmentation) is not evaluated. Reviewers or clinical practitioners would often emphasize that 'better metrics do not necessarily translate to improved clinical diagnosis'.

Related Work & Insights

• vs NS-N2N (Neighboring Slice Noise2Noise): Both share the neighboring slice Noise2Noise training framework and NAFNet backbone. However, NS-N2N employs NLM denoising + median masking to actively exclude misaligned pixels from supervision, while NGPS actively retrieves them. The two represent the philosophies of 'discarding' vs. 'retrieving'—they are complementary rather than mutually exclusive.
• vs Pixel2Pixel: Both share patch-level searching and Top-K ensembling. However, Pixel2Pixel performs global search within the same image, carrying a high risk of noise coupling. NGPS restricts the search space to a local window in the adjacent slice, and decouples the matching space (guidance map) from the retrieval values (raw noisy map).
• vs Deformed2Self: Both are neighbor-slice SSL methods designed to handle inter-slice misalignment. Deformed2Self explicitly estimates deformation fields and warps, while NGPS bypasses warping in favor of discrete coordinate retrieval. This demands far less computational cost and shows higher robustness under heavy noise—though warping may extract finer details when registration accuracy is high.
• vs Noise2Sim: Both utilize similarity concepts. However, Noise2Sim uses similarity for soft-weighting (giving higher weight to co-located pixels with high similarity) without actively shifting coordinates, whereas NGPS employs similarity for explicit coordinate search, which is fundamentally different.

Rating

• Novelty: ⭐⭐⭐⭐ The idea of decoupling matching and retrieval is elegant. It concrete-izes the perspective of "misalignment = displacement rather than absence" into an actionable training-time supervision construction scheme. While inspired by PixelBank, it makes crucial adaptations specifically for neighboring-slice SSL.
• Experimental Thoroughness: ⭐⭐⭐⭐ Evaluated on three datasets (real-dose CT, simulated ultra-low-dose CT, and synthetic Rician MRI) against 12 baselines across five metrics. Comprehensive ablations cover loss components, slice spacing, guidance filters, spatial hyperparameters, K values, and large-gap stress tests. Cross-seed variance and volume-level CI analyses add further credibility.
• Writing Quality: ⭐⭐⭐⭐ Clear problem formulation (the discarded pixel distribution analysis in Fig.4 quantifies the cost of masking methods). Methods, formulations, and algorithms correspond seamlessly. The experimental analysis demonstrates nuance (e.g., transparently reporting the non-significant SSIM improvement on LIDC and slightly degraded GMSD).
• Value: ⭐⭐⭐ Clear practical utility for the medical image denoising community (lightweight, zero changes to inference architecture, plug-and-play). Generality is restricted to volumetric data satisfying the independent inter-slice noise assumption, rendering it less applicable for standard 2D natural image denoising.

Related Papers

• [ICLR 2026] Prior-aware and Context-guided Group Sampling for Active Probabilistic Subsampling
• [NeurIPS 2025] Self-Supervised Learning via Flow-Guided Neural Operator on Time-Series Data
• [CVPR 2026] Active Inference for Micro-Gesture Recognition: EFE-Guided Temporal Sampling and Adaptive Learning
• [CVPR 2025] Multiscale Structure-Guided Latent Diffusion for Multimodal MRI Translation
• [AAAI 2026] Self-supervised Multiplex Consensus Mamba for General Image Fusion