Broadband Wide Field of View Imaging with Computational Mirrors¶
Conference: ECCV 2026
Paper: ECCV Official
Code: https://codelab-ucr.github.io/computational-mirrors/
Area: Others
Keywords: Computational Imaging, Reflective Optics, VIS-SWIR Broadband, Focal Stacking, Point Spread Function
TL;DR¶
Addressing severe chromatic aberrations in refractive lenses across the 400–1700nm VIS-SWIR spectrum alongside the debilitating Petzval field curvature in reflective mirrors, this work introduces Computational Mirrors—a hardware-software system pairing sparse focal stacks (2–4 exposures) with a physics-inspired affine-deformed PSF model (SeidelConv) and Plug-and-Play deconvolution to achieve full-FoV achromatic imaging on a single sensor.
Background & Motivation¶
With the advent of broadband image sensors capable of capturing visible to short-wave infrared wavelengths (VIS-SWIR, 400–1700nm), multi-band imaging has shown remarkable utility across forensic authentication, material differentiation, and seeing through challenging scatterers such as smoke or fog. However, conventional refractive optics (glass lenses) present a severe fundamental bottleneck: glass dispersion causes different spectral wavelengths to focus at distinct axial depths. Correcting chromatic blur across a wide wavelength octave requires bulky, heavy, and prohibitively expensive multi-element lens assemblies. Even when optimized, refractive lenses typically require narrow operating bands or manual per-wavelength refocusing, substantially restricting light throughput and portability.
In stark contrast, reflective optics—such as concave mirrors—are inherently achromatic from deep ultraviolet to far-infrared wavelengths. Mirrors are substantially lighter, more compact, and naturally support high apertures (e.g., F/1, F/2), making them standard in astronomical observatories and reflective microscopes. Nevertheless, simple mirror systems with few elements remain largely unusable for general macro or medium focal-length imaging (50–100mm). The limiting barrier lies in severe geometric aberrations, most notably Petzval field curvature and off-axis distortions: the focused image surface is non-planar, meaning a flat sensor focused at the optical axis experiences massive blur toward the periphery. Prior computational restoration strategies relied on capturing dense focal stacks (10–30 frames) or enforced radial symmetry assumptions, which either introduce high acquisition latency or collapse when handling practical, misaligned wide-aperture optics.
Navigating the trade-off between broadband achromaticity and severe off-axis geometric blur, this paper presents the Computational Mirrors framework. Core idea: exploit the depth-invariant and wavelength-independent geometry of Petzval curvature in reflective optics by capturing a sparse focal stack (2–4 frames), and model spatially varying off-axis aberrations using SeidelConv—a physics-inspired PSF formulation combining affine coordinate warping with learned local blur kernels—to reconstruct a sharp, all-in-focus VIS-SWIR image via Plug-and-Play deconvolution.
Method¶
Overall Architecture¶
The Computational Mirrors system couples a compact reflective objective with a computational reconstruction backend. The hardware objective employs a drop-in form factor compatible with standard camera mounts: light reflects off a primary concave mirror and is folded by a small planar secondary mirror to exit through a central aperture, focusing onto a flat broadband sensor (Sony IMX990, 400–1700nm). The sensor is mounted onto a precision motorized linear translation stage (Thorlabs ELL17k) aligned with the optical axis. By shifting the sensor across \(N\) discrete axial positions (empirically \(N=2\) to \(4\)), the system captures a sparse focal stack that intercepts the curved Petzval surface at distinct radial radii, ensuring that every point across the full field of view (FoV) is sampled near peak focus in at least one exposure.
In the computational backend, a physics-inspired forward operator named SeidelConv is calibrated offline using randomized dot targets displayed on an OLED monitor. SeidelConv models the severely aberrated, spatially varying point spread function (PSF) at each focal stack slice. Finally, the forward operator is integrated into a multi-image Plug-and-Play (PnP) restoration framework using a pre-trained deep denoiser prior (DPIR), jointly inverting the degraded focal stack into a single, high-fidelity, all-in-focus broadband image.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Broadband Scene Radiation (400-1700nm)"] --> B["Sparse Focal Stack Acquisition<br/>Linear stage scans Petzval surface (N=2~4)"]
B --> C["SeidelConv Forward Degradation Model<br/>Affine coordinate warping + basis blur + per-pixel weights"]
C --> D["Multi-Image PnP Deconvolution<br/>Data fidelity gradient update + DPIR denoiser prior"]
D --> E["Sharp Full-FoV All-In-Focus Output"]
Key Designs¶
1. Sparse Focal Stack Acquisition: Capturing Petzval Curvature with Minimal Exposures
Traditional field-curvature compensation techniques require capturing dense focal sweeps of 20 to 30 exposures or fabricating custom curved sensors, which either introduces severe motion blur in practical scenes or lacks hardware flexibility. The authors observe that Petzval curvature in single-element mirrors exhibits a monotonic, smooth depth trajectory. By setting the initial sensor position \(v_0\) to align peak focus at the center of the FoV and capturing \(N\) frames separated by a fixed axial increment \(\Delta v\) (\(200\,\mu\mathrm{m}\) for the 50mm objective and \(100\,\mu\mathrm{m}\) for 100mm), a sparse set of \(N=3\) exposures suffices. Specifically, the stack samples the center, intermediate zone, and periphery near their optimal focal depths, reformulating full-FoV reconstruction into a well-conditioned multi-image inverse problem.
2. SeidelConv: Physics-Inspired Spatially Varying PSF Modeling via Affine Warping
At large apertures (F/1), even focal slices close to the Petzval surface suffer from massive off-axis aberrations including coma, astigmatism, and spherical blur, which span tens of pixels and break rotational symmetry due to mechanical mounting misalignments. Existing models that assume circular symmetry (e.g., Ring Deconvolution) or purely coordinate-gated weighting (CoordGate) degrade severely off-axis. Inspired by classical third-order Seidel aberration theory and Spatial Transformer Networks, SeidelConv parameterizes the forward blur operator for the \(k\)-th focal slice as a mixture of \(Q\) components:
where \(\phi_q^{(k)}(\mathbf{p}) = \mathbf{R}_q^{(k)}\mathbf{p} + \mathbf{t}_q^{(k)}\) represents an affine coordinate warping operator, \(h_q^{(k)}\) is a compact spatial blur kernel (\(11\times 11\)), and \(w_q^{(k)}(\mathbf{p})\) denotes a learned per-pixel blending weight map. By decoupling large-scale directional geometric stretching from high-frequency local blur through affine transformations, SeidelConv captures severe asymmetric off-axis aberrations using only \(Q=31\) components with high parameter efficiency.
3. Multi-Image Plug-and-Play Deconvolution: Joint Data Fidelity and Deep Priors
Rather than averaging focal stacks into a single blurred frame or evaluating a mosaic along the Petzval manifold, the latent sharp image \(I_0\) is recovered by solving a multi-image regularized inverse problem:
where \(\mathcal{A}^{(k)}\) is the calibrated SeidelConv forward operator for exposure \(k\), and \(\mathcal{R}(\cdot)\) is an image regularizer. The optimization is implemented within a Plug-and-Play (PnP) Half-Quadratic Splitting (HQS) framework. The data fidelity subproblem is updated via the adjoint gradient of SeidelConv, while the proximal regularization step invokes a pre-trained deep denoiser (grayscale DPIR). This setup exploits natural image manifold priors without requiring end-to-end task-specific supervised training, suppressing high-frequency deconvolution artifacts across the field.
Loss & Training¶
The parameters \(\theta_k = \{\mathbf{R}_q^{(k)}, \mathbf{t}_q^{(k)}, h_q^{(k)}, w_q^{(k)}\}\) of the SeidelConv operators are calibrated using a monitor-based pipeline. A set of \(L=10\) calibration images displaying randomized sparse point patterns is rendered on an OLED screen. The optical prototype records the corresponding \(N=3\) focal stack responses. Parameter optimization minimizes measurement error regularized by an \(\ell_1\) kernel sparsity penalty:
The model is optimized using Adam with a learning rate of \(5 \times 10^{-3}\). Because reflective optics exhibit zero chromatic dispersion, the SeidelConv model calibrated once under broadband visible light generalizes directly across the full VIS-SWIR range (400–1700nm) without retraining or per-band tuning.
Key Experimental Results¶
Main Results¶
Quantitative evaluations were performed on 20 test images from the Kodak benchmark displayed on a radiometrically calibrated OLED display (evaluating the green channel). Performance was measured in PSNR (dB) and SSIM across the On-Axis zone (center \(50\%\) area) and the Off-Axis zone (outer periphery):
| Focal Stack Inputs | PSF Degradation Model | On-Axis (PSNR / SSIM) | Off-Axis (PSNR / SSIM) |
|---|---|---|---|
| 3 frames (Proposed Computational Mirrors) | SeidelConv (\(11\times 11\) kern, 31 terms) | 27.6 / 0.84 | 27.1 / 0.74 |
| 3 frames (Baseline) | CoordGate (\(11\times 11\) kern, no warping) | 27.6 / 0.80 | 24.8 / 0.63 |
| 3 frames (Baseline) | Patch-wise (\(11\times 11\) kern) | 27.6 / 0.80 | 24.8 / 0.63 |
| 3 frames (Baseline) | Patch-wise (\(21\times 21\) kern) | 27.9 / 0.80 | 21.6 / 0.65 |
| 20 frames averaged [Yokoya et al.] | SeidelConv | 27.3 / 0.74 | 23.1 / 0.64 |
| 20 frames on Petzval [Matsunaga et al.] | SeidelConv | 27.6 / 0.80 | 23.9 / 0.67 |
| 1 frame single-focus [Heide et al.] | SeidelConv | 25.6 / 0.66 | 20.9 / 0.49 |
| 1 frame single-focus [Ring Deconvolution] | Ring Deconvolution [Kohli et al.] | 22.7 / 0.52 | 18.7 / 0.29 |
Ablation Study¶
The impact of the number of focal stack frames \(N\) was systematically evaluated on the 50mm F/1 objective:
| Focal Stack Size \(N\) | Off-Axis Resolution & Restoration Quality | Note |
|---|---|---|
| \(N = 1\) | Severe peripheral defocus blur; deconvolution cannot recover high frequencies | Fails to span Petzval curvature depth |
| \(N = 2\) | Intermediate field recovered; peripheral edges retain noticeable artifacts | Undersampled at extreme periphery |
| \(N = 3\) | Uniform sharp resolution across full FoV; \(>6\,\text{dB}\) gain over \(N=1\) | Optimal performance/efficiency trade-off |
| \(N = 4\) | Marginal numerical improvement (\(<0.2\,\text{dB}\)) | Information saturation along axial depth |
| \(N = 10\) | Visually identical to \(N=3\); acquisition latency and motion risk multiply | Redundant sampling without tangible gain |
Key Findings¶
- Affine warping is essential for off-axis aberration correction: On-axis performance is nearly uniform across methods (27.6–27.9 dB). However, in off-axis regions with severe coma and astigmatism, SeidelConv achieves 27.1 dB, outperforming CoordGate and patch-wise convolution by more than 2.3 dB. Expanding patch-wise kernel sizes to \(21\times 21\) causes overfitting artifacts, dropping off-axis PSNR to 21.6 dB.
- Joint sparse inversion vastly outperforms focal averaging: Collapsing 20 images into an average blur yields only 23.1 dB off-axis, whereas multi-image deconvolution with just 3 raw exposures achieves 27.1 dB, highlighting the necessity of preserving complementary focal slice measurements.
- Wavelength invariance enables cross-band generalization: SeidelConv calibrated solely in visible light generalizes across all five tested bands (Blue, Green, Red, NIR, SWIR). The 100mm F/2 mirror prototype weighs only 60g, whereas a comparable 100mm Nikon refractive lens weighs 405g and experiences severe defocus when extended into SWIR.
Highlights & Insights¶
- Synergy of physical aberration theory and learnable coordinate transforms: SeidelConv grounds spatial coordinate transformations in classical Seidel aberration kinematics, providing an efficient parameterized bridge between continuous optical physics and differentiable learning.
- Achromatic imaging without per-band refocusing: Eliminates the need for complex achromatic glass groupings or wavelength-specific motorized recalibration, enabling simultaneous multi-band macro inspection and long-range surveillance with a single hardware calibration.
- Material classification and non-destructive inspection: Real captures demonstrate penetrating opaque packaging in SWIR, distinguishing chlorophyll reflectance between live and artificial foliage in NIR, and verifying currency security threads.
Limitations & Future Work¶
- Motion sensitivity across sequential exposures: Capturing 2–4 exposures sequentially makes dynamic scenes susceptible to motion artifacts. Integrating optical flow alignment (such as SEA-RAFT) across exposures represents a viable mitigation strategy.
- Iterative reconstruction compute overhead: The iterative PnP deconvolution requires multiple gradient steps, hindering real-time video framerates. Using SeidelConv as a differentiable forward model to generate synthetic training data for end-to-end feed-forward restoration networks offers an attractive future direction.
Related Work & Insights¶
- vs Dense Focal Sweep Deconvolution (Matsunaga & Nayar, 2015): Prior methods required 20–30 exposures and assumed shift-invariant blur on the curved surface. Computational Mirrors reduces the acquisition to 2–4 frames while accurately handling severe spatially varying aberrations via SeidelConv.
- vs Ring Deconvolution (Kohli et al., 2025): Ring Deconvolution relies strictly on radial symmetry, failing in practical mirror assemblies with mechanical alignment tolerances. SeidelConv learns generalized affine deformations, achieving an 8.4 dB improvement off-axis.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Elegant integration of simple reflective optics, sparse Petzval focal stacking, and physics-grounded affine PSF modeling.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Validated across custom 50mm F/1 and 100mm F/2 prototypes with quantitative bench experiments and rich multi-spectral real-world applications.
- Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical formulation, coherent narrative, and well-structured comparative tables.
- Value: ⭐⭐⭐⭐⭐ Establishes a practical, lightweight paradigm for high-throughput, ultra-broadband imaging on conventional flat sensors.