CLDefocus: Physically Grounded Compound-Lens Defocus Blur Synthesis

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/lykelee/CLDefocus
Area: Image Restoration
Keywords: Defocus Deblurring / Compound Lens / Debye CZT / Point Spread Function / Synthetic Dataset

TL;DR

Addressing the lack of optical diversity and physical realism in existing defocus deblurring datasets, this paper introduces a scalable synthesis pipeline combining Debye CZT wave-optics propagation, depth-aware occlusion handling, and linear-space ISP simulation to produce CLDefocus, a benchmark of 700 compound lenses that substantially improves cross-device deblurring generalization.

Background & Motivation

Defocus blur naturally emerges in optical systems with wide apertures whenever light rays emanating from a scene point fail to converge on the sensor plane, forming an expanded circle of confusion (CoC). While deliberate defocus is prized for aesthetic background bokeh in artistic photography, it is decidedly detrimental in computer vision systems, as severe blur obliterates high-frequency boundaries and structural textures, degrading downstream vision tasks such as object detection, facial recognition, and semantic segmentation. Deep learning architectures like NRKNet and Restormer have significantly outperformed conventional non-blind deconvolution methods; however, their generalization capability is strictly bounded by the diversity, scale, and physical validity of available training datasets.

Acquiring high-fidelity real-world defocus datasets faces formidable physical barriers. Collecting paired sharp-blurred scenes requires static setups and sequential shooting across multiple aperture settings. This capture paradigm scales poorly across diverse camera bodies and lenses, severely restricting optical diversity. Moreover, physical aperture switches inevitably introduce minor spatial displacements, exposure shifts, and chromatic variations, while small-aperture ground-truth images frequently retain residual defocus blur due to non-zero depth-of-field. On the synthetic front, early datasets relied on simplified thin-lens models with isotropic Gaussian or disk kernels, which completely omit compound lens aberrations and pupil vignetting. Physics-based wave-optics alternatives typically calculate diffraction via Rayleigh-Sommerfeld integrals, which suffer from quadratic computational complexity, lack explicit anti-aliasing sampling criteria, and struggle to scale across diverse compound lenses and depth levels. Furthermore, applying blur directly in non-linear sRGB space neglects the radiometrically linear nature of optical superposition and sensor image signal processing (ISP).

Confronting the tension between the physical alignment imperfections of real captures and the oversimplifications of existing synthetic datasets, this paper develops a scalable and physically faithful compound-lens defocus synthesis pipeline. Core Idea: Accelerate wave-optics PSF diffraction calculations across diverse compound lenses via the Debye Chirp Z-Transform (CZT) with explicit sampling bounds, and execute depth-layered, occlusion-aware rendering in radiometrically linear space coupled with camera ISP simulation to establish a photorealistic benchmark dataset with zero spatial drift.

Method

Overall Architecture

The proposed synthesis pipeline takes sharp RAW image patches and optical prescriptions of diverse compound lenses as inputs to synthesize physically grounded sharp-blurred training pairs. The pipeline proceeds across three primary modules: first, wave-optics point spread functions (PSFs) are efficiently computed for varying object depths, field positions, and lens designs via ray-traced exit pupil sampling, Zernike polynomial fitting, and Debye CZT propagation; second, metric depth maps estimated from sharp inputs are adaptively partitioned into discrete layers using signed circle of confusion (CoC) thresholds alongside foreground inpainting for occlusion awareness; third, depth-layered convolutions are composited back-to-front in radiometrically linear RGB space, followed by sensor noise injection and forward camera ISP processing into non-linear sRGB space.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Sharp RAW Patch Input"] --> B["Stage 1: Debye CZT Wave-Optics PSF Computation<br/>Exit pupil ray tracing + Zernike fitting + CZT propagation"]
    B --> C["Stage 2: Depth-Aware Occlusion-Consistent Rendering<br/>Metric depth estimation + Signed CoC discretization + Inpainting"]
    C --> D["Stage 3: Linear-Space Blur & Camera ISP Simulation<br/>Linear back-to-front compositing + Inverse/Forward ISP + Noise"]
    D --> E["Photorealistic Blurred-Sharp Pair (sRGB)"]

Key Designs

1. Efficient Wave-Optics PSF Computation via Debye CZT: Explicit Sampling Bounds and Sensor Resolution Matching

Conventional wave-optics simulations based on the Rayleigh-Sommerfeld diffraction integral scale as \(\mathcal{O}(N^2 M^2)\) and lack rigorous analytical sampling bounds, leading to severe aliasing or prohibitive runtimes when simulating large defocus. The authors formulate wave propagation converging near focus using the Debye representation, rewriting the exit pupil sphere integral into a Fourier transform:

\[\mathbf{E}(x, y, z) = \mathcal{F}\left\{ E_t \exp\left(j k_z z\right) \right\}\]

where \(E_t\) represents the continuous wavefront over the exit pupil (fitted from ray-traced optical path differences via Zernike polynomials), \(k_z\) is the axial wave vector component, and \(z\) denotes sensor plane defocus displacement. By analyzing the phase variation factor \(\exp(j k_z z)\), the paper derives an explicit anti-aliasing sampling lower bound:

\[N_{\text{inf}} = \frac{2 \cdot \text{NA} \cdot |z|}{\lambda \cdot n_t}\]

To suppress high-order wavefront oscillations induced by lens aberrations, the input grid is set to \(N = 2 N_{\text{inf}}\). Standard FFT restricts the output sampling pitch to the reciprocal of the input pupil grid, precluding direct alignment with physical sensor pixels. The framework therefore integrates the Chirp Z-Transform (CZT) via Bluestein's algorithm, decoupling output evaluation from input sampling and lowering complexity to \(\mathcal{O}((N+M)^2 \log(N+M))\). The PSF is computed on a \(u=5\times\) supersampled grid before downsampling to sensor pixel pitch, completely eliminating numerical artifacts.

2. Depth-Aware Occlusion-Consistent Rendering: Signed CoC Quantization and Background Inpainting

Evaluating continuously varying PSFs across all scene depths per pixel is computationally intractable, while uniform whole-image convolution introduces severe depth-boundary artifacts. The authors discretize metric depth maps into \(K\) discrete layers using signed circle of confusion (Signed CoC), which distinguishes whether light converges in front of or behind the focal plane. Pixels whose signed CoC differs by less than one sensor pixel are grouped into the same layer, minimizing discretization banding.

To account for optical occlusion—where blurred foregrounds physically expand and partially reveal background structures behind them—the pipeline inpaints regions of each background layer \(C_i\) that are occluded by foreground geometry. Each layer is convolved with its respective depth-dependent PSF \(P_i\) and composited back-to-front using layered alpha blending:

\[C_i^{\text{blur}} = C_{i-1}^{\text{blur}} \cdot (1 - A_i * P_i) + (C_i \cdot A_i) * P_i\]

where \(A_i\) represents the opacity mask, yielding smooth and physically authentic boundary transitions without black borders or phantom halo artifacts.

3. Linear-Space Blur Synthesis & Camera ISP Simulation: Photometrically Consistent Photon Modeling

In optical imaging systems, photons accumulate linearly on sensor photodiodes, whereas standard display images undergo non-linear gamma curves, color matrix adjustments, and tone mapping inside camera Image Signal Processors (ISPs). Synthesizing blur directly in sRGB space violates radiometric linearity and yields dimmed, distorted bokeh highlights. The proposed pipeline converts input RAW patches to radiometrically linear space \(I_{\text{lin}}^s\) via a partial inverse ISP (demosaicing, white balancing, color correction). The depth-aware blur convolution is performed strictly in this linear domain, producing \(I_{\text{lin}}^b\). Subsequently, \(I_{\text{lin}}^b\) is mapped back into Bayer space where calibrated heteroscedastic Poisson-Gaussian sensor noise (shot and read noise) is injected. Passing both the degraded image and the unaltered linear sharp image through the forward ISP generates photometrically consistent and visually realistic blurred-sharp pairs \((I_{\text{sRGB}}^s, I_{\text{sRGB}}^b)\).

A Worked Example

Consider capturing an indoor scene with a foreground figure at \(0.8\,\text{m}\) and a background shelf at \(2.5\,\text{m}\): 1. Patch Extraction and Metric Depth: A \(384 \times 384\) RAW patch is extracted and converted into linear RGB; monocular depth network Depth Pro estimates absolute metric depths for all pixels. 2. Optics Sampling & Wavefront Evaluation: A \(50\,\text{mm}\) \(F/1.8\) prime lens is randomly selected from the filtered library of 700 designs with focus set to \(0.8\,\text{m}\). Ray tracing projects exit pupil wavefronts; Debye CZT evaluates sharp diffraction at \(0.8\,\text{m}\) and complex aberrated blur kernels at \(2.5\,\text{m}\) within \(18\,\text{ms}\). 3. Discretization & Layered Compositing: Depth is partitioned into \(K=12\) layers based on signed CoC; background shelf regions occluded by the figure silhouette are inpainted. Back-to-front convolutions smoothly merge background blur behind the sharp subject. 4. ISP Simulation & Noise Injection: The composite linear image receives sensor noise in Bayer space and passes through tone mapping, yielding a perfectly aligned, realistically blurred sRGB pair with authentic optical bokeh.

Key Experimental Results

Main Results

To demonstrate dataset effectiveness, NRKNet deblurring networks were trained under identical settings (350,000 iterations) on DPDD (real captures), SYNDOF (thin-lens synthetic), and the proposed CLDefocus dataset. Models were benchmarked across the CLDefocus test split, RTF, RealDOF, and DPDD on both full-reference metrics (PSNR, SSIM, LPIPS) and no-reference image quality metrics (NIQE, MUSIQ, TOPIQ).

Table 1: Quantitative Full-Reference Comparison (PSNR ↑ / SSIM ↑ / LPIPS ↓)

Test Set	Train Set	PSNR (dB) ↑	SSIM ↑	LPIPS ↓
CLDefocus (Ours)	CLDefocus (Ours)	32.16	0.865	0.201
	SYNDOF	27.58	0.790	0.296
	DPDD	28.27	0.825	0.263
RTF	CLDefocus (Ours)	26.62	0.845	0.242
	SYNDOF	24.55	0.743	0.261
	DPDD	25.95	0.833	0.207
RealDOF	CLDefocus (Ours)	24.74	0.764	0.303
	SYNDOF	21.64	0.654	0.474
	DPDD	25.03	0.771	0.335
DPDD	CLDefocus (Ours)	24.73	0.776	0.265
	SYNDOF	23.77	0.743	0.315
	DPDD	26.11	0.817	0.223

Table 2: Quantitative No-Reference Quality Assessment (NIQE ↓ / MUSIQ ↑ / TOPIQ ↑)

Test Set	Train Set	NIQE ↓	MUSIQ ↑	TOPIQ ↑
CLDefocus (Ours)	CLDefocus (Ours)	7.724	39.850	0.348
	SYNDOF	12.048	36.904	0.316
	DPDD	7.508	38.961	0.345
RTF	CLDefocus (Ours)	4.263	59.144	0.533
	SYNDOF	3.855	57.279	0.539
	DPDD	4.196	58.822	0.531
RealDOF	CLDefocus (Ours)	5.029	35.787	0.303
	SYNDOF	6.285	23.905	0.241
	DPDD	6.030	31.350	0.270
DPDD	CLDefocus (Ours)	4.657	56.696	0.487
	SYNDOF	5.067	55.819	0.493
	DPDD	4.746	59.473	0.507

Ablation Study

The contribution of each pipeline component was systematically evaluated by training NRKNet models on ablated datasets and testing on RealDOF.

Table 3: Ablation Results on the RealDOF Benchmark

Configuration	PSNR ↑	SSIM ↑	LPIPS ↓	NIQE ↓	MUSIQ ↑	TOPIQ ↑	Note
w/o Lens	22.34	0.667	0.521	7.733	25.162	0.238	Replaced lens PSFs with isotropic Gaussian kernels
w/o ISP	23.49	0.746	0.349	5.191	32.045	0.285	Synthesized blur directly in sRGB without RAW noise
w/o Depth	24.54	0.762	0.317	5.292	34.999	0.300	Single-layer uniform convolution ignoring scene depth
Full Pipeline	24.74	0.764	0.303	5.029	35.787	0.303	Full wave-optics, depth-aware, and linear ISP pipeline

Key Findings

1. Realistic Lens Aberrations are Essential for Generalization: Replacing wave-optics PSFs with isotropic Gaussian kernels (w/o Lens) causes a severe performance drop of \(2.40\,\text{dB}\) in PSNR and degrades LPIPS from 0.303 to 0.521. Models trained on symmetric Gaussian kernels fail to deconvolve realistic asymmetric blur patterns, coma, and spherical aberrations.
2. Real-Capture Benchmarks Suffer from Ground-Truth Imperfection Bias: While the DPDD-trained model scores higher PSNR on its own real test split, the CLDefocus-trained model achieves superior perceptual sharpness and higher no-reference scores (e.g., MUSIQ of 35.787 vs 31.350 on RealDOF). Pixel-wise metrics penalize sharp reconstructions when real ground-truth images contain subtle misalignments or residual defocus, artificially rewarding conservative, blur-preserving predictions.
3. Debye CZT Delivers Over \(2500\times\) Speedup Over Rayleigh-Sommerfeld: Evaluating a \(64 \times 64\) PSF via Rayleigh-Sommerfeld took \(45.1\,\text{s}\) at \(N=256\), and smaller sampling produced distorted, aliased kernels. In contrast, Debye CZT with its explicit anti-aliasing criterion (\(N=732\)) completed in only \(0.018\,\text{s}\), eliminating the computational bottleneck for large-scale optical simulation.

Highlights & Insights

• Millisecond Wave-Optics via Debye CZT: Reformulating focused wave diffraction into Fourier domain CZT operations decouples output sensor resolution from input pupil sampling. This allows millisecond-level evaluation of physically accurate diffraction PSFs for arbitrary photographic lenses.
• Critical Expose of Real-World Evaluation Pitfalls: The paper exposes how aperture-switching physical capture in real datasets inevitably introduces geometric shifts, photometric discrepancies, and residual defocus. Demonstrating that PSNR/SSIM reward under-deblurred results establishes a crucial foundation for physics-based synthetic evaluation.
• Programmable Optical Pipeline: Because the entire optical and camera pipeline is explicitly parameterized (lens designs, aperture sizes, subject depths, sensor pixel sizes), it provides an extensible foundation for controllable data generation, curriculum learning, and inverse depth-from-defocus tasks.

Limitations & Future Work

• Approximation Decay at Extreme Off-Axis Fields: The scalar Debye formulation and paraxial exit pupil assumptions lose fidelity at extreme wide-angle field positions where severe mechanical vignetting and steep incident angles produce asymmetric pupil truncation.
• Dependence on Monocular Depth Estimation Accuracy: Synthesis quality depends heavily on monocular metric depth estimators like Depth Pro; boundary errors in depth estimation can manifest as localized blur halo artifacts.
• Future Directions: Integrating vector diffraction models for high-NA lenses, combining neural radiance fields for continuous volumetric defocus without discrete depth layers, and leveraging controllable blur levels for curriculum training.

Related Work & Insights

• vs SYNDOF (Lee et al., CVPR 2019): SYNDOF relies on thin-lens disk or Gaussian approximations. In contrast, CLDefocus models complex wave diffraction, spherical aberrations, and optical vignetting from 700 real-world compound lens designs.
• vs DPDD (Abuolaim & Brown, ECCV 2020): DPDD gathers real scenes from a single Canon DSLR camera with inherent aperture-switching alignment errors. CLDefocus provides perfectly aligned ground truths across 700 virtual lens systems, exhibiting superior cross-device generalization on no-reference perceptual metrics.
• vs RSBlur (Rim et al., ECCV 2022): RSBlur demonstrated the necessity of linear-space convolution and camera ISP simulation for motion blur. This work extends ISP-aware principles to wave-optics compound-lens defocus with depth-dependent occlusion compositing.

Rating

• Novelty: ⭐⭐⭐⭐⭐ [Pioneers the application of Debye CZT with explicit anti-aliasing bounds to compound photographic lenses for millisecond wave-optics PSF synthesis.]
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive cross-dataset evaluations across four benchmark test sets, thorough ablations, and insightful analysis of real-capture ground-truth bias.]
• Writing Quality: ⭐⭐⭐⭐⭐ [Exceptionally clear mathematical formulation, rigorous optical analysis, and intuitive visual explanations.]
• Value: ⭐⭐⭐⭐⭐ [Provides open-source code and a photorealistic 42,000-pair benchmark, unlocking scalable optical data generation for computational imaging and deblurring.]