Exploiting Deblurring Networks for Radiance Fields

Conference: CVPR 2025
arXiv: 2502.14454
Code: None
Area: 3D Vision
Keywords: Radiance Field Deblurring, Deep Learning Deblurring, Novel View Synthesis, 3D Gaussian Splatting, Iterative Optimization

TL;DR

This paper proposes DeepDeblurRF, which introduces DNN deblurring networks into the radiance field reconstruction pipeline for the first time. By designing an RF-guided deblurring mechanism and an iterative alternating framework, it achieves high-quality novel view synthesis under blurry image inputs. The training speed is 10-100 times faster than existing methods, while supporting multiple 3D representations such as voxel grids and 3D Gaussian Splatting.

Background & Motivation

Background: Constructing radiance fields (NeRF/3DGS) from multi-view images is the mainstream method for novel view synthesis. However, real-captured images often suffer from motion blur or defocus blur, and using them directly for radiance field training leads to blurry 3D representations. Existing radiance field deblurring methods (Deblur-NeRF, DP-NeRF, BAD-NeRF, BAGS, etc.) address this issue by jointly optimizing the blur kernels and the radiance field.

Limitations of Prior Work: (1) Existing methods rely on linear blur models, which describe blurry pixels as a linear combination of sharp pixels, failing to handle non-linear degradations like saturated pixels and noise; (2) They lack sharp image priors and solely rely on complementary information across different views to estimate blur kernels, failing completely when all input views share similar blur directions; (3) Ray-based methods require sampling multiple rays per pixel to simulate blur effects, leading to extremely long training times (DP-NeRF takes 30+ hours and requires dual GPUs).

Key Challenge: DNN deblurring methods exhibit clear advantages in handling non-linear blur, utilizing sharp image priors, and inference speed. However, directly deblurring blurry images individually and then training radiance fields yields poor results, as the information in a single image is insufficient to fully restore sharp details. How to effectively combine the advantages of DNN deblurring with the multi-view information aggregation capability of radiance fields?

Goal: (1) Overcome the limitations of linear blur models to handle real-world non-linear degradations; (2) Utilize sharp image priors from DNNs to compensate for the lack of complementary information; (3) Significantly reduce training time.

Key Insight: During the radiance field reconstruction process, information from multiple deblurred images is "aggregated" into a unified 3D representation, rendering images that are sharper than any single deblurring result. In turn, these sharper rendered images can serve as guidance signals, helping the deblurring network produce better results in the next round.

Core Idea: Gradually improve the quality of both deblurring and radiance fields through an iterative alternating pipeline of "DNN deblurring \(\rightarrow\) radiance field reconstruction \(\rightarrow\) rendered image-guided redeblurring".

Method

Overall Architecture

DeepDeblurRF takes \(M\) blurry images as input and outputs a sharp radiance field. The pipeline is iteratively executed in three steps: (1) Initial deblurring—single-image deblurring is performed on each blurry image using off-the-shelf NAFNet; (2) Radiance field reconstruction—the deblurred images are used to estimate camera poses (COLMAP) and train the radiance field (supporting both Plenoxels and 3DGS representations); (3) RF-guided deblurring—corresponding view images are rendered from the radiance field as guidance, combined with the original blurry images, and input into a modified deblurring network to obtain sharper deblurred results. This iteration is repeated \(N=5\) times.

Key Designs

1. RF-guided Deblurring:

   • Function: Utilize the aggregated information rendered from radiance fields to guide the DNN deblurring network to produce higher-quality results.
   • Mechanism: In each iteration, the image \(R_m^i\) corresponding to each input view is rendered from the current radiance field \(V^i\). Although the deblurred image \(D_m^{i-1}\) may contain residual blur and artifacts, the rendered image \(R_m^i\) obtains more accurate details by aggregating multi-view information. Concatenating the rendered image \(R_m^i\) and the original blurry image \(B_m\) as input to a modified NAFNet (with the first layer adapted to receive a 6-channel input), the deblurring process is guided by the rendered image, yielding a higher-quality deblurred image \(D_m^i\).
   • Design Motivation: Single-image deblurring is limited by the amount of information in a single image, whereas radiance fields naturally generate higher-quality images via "cross-view information aggregation." Feeding this aggregated information back into the deblurring network creates a virtuous cycle.

2. BlurRF-Synth Dataset:

   • Function: Provide large-scale blurry-sharp paired data for training the radiance field deblurring framework.
   • Mechanism: 100 3D model scenes are rendered using Blender to generate training/testing sets for motion blur and defocus blur respectively. Motion blur is generated by sampling 6DOF camera trajectories via Bézier interpolation and averaging renders at 51 intermediate poses. Defocus blur is generated by adjusting Blender's depth-of-field parameters. A key innovation is the introduction of RSBlur's realistic degradation simulation pipeline—adding saturated pixels after averaging in linear sRGB space, converting to RAW space to add shot/read noise, and finally converting back to sRGB space. A total of 4,350 blurry-sharp pairs are generated.
   • Design Motivation: Previously, all radiance field deblurring methods did not require large-scale training sets (since they employ scene-specific optimization instead of learning-based methods), leading to a lack of suitable training data. Existing evaluation datasets (e.g., Deblur-NeRF) are small in scale and neglect non-linear factors such as real camera noise.

3. Iterative Alternating Optimization Framework:

   • Function: Gradually improve the quality of both radiance fields and deblurring.
   • Mechanism: Set the number of iterations to \(N=5\). Each iteration consists of two steps: (1) Reconstructing the radiance field using the deblurred images from the previous round; (2) Rendering guidance images from the radiance field to perform RF-guided deblurring. Different iterations use independently trained deblurring network weights (each iteration corresponds to fixed noise and blur characteristics). The final iteration only performs radiance field reconstruction. Although deblurred images may contain residual blur and artifacts, the radiance field reconstruction step naturally suppresses erroneous information in individual images by aggregating multiple overlapping deblurred images.
   • Design Motivation: A one-off deblurring + radiance field reconstruction pipeline performs poorly (experimentally verified PSNR of only 28.40). However, via iteration, the rendered images from the radiance field in each round become sharper than the previous one, rendering increasingly better guidance signals for the next round of deblurring, achieving a gradual climb in quality.

Loss & Training

The deblurring network is optimized using the Adam optimizer (\(\beta_1=0.9\), \(\beta_2=0.9\)), with the learning rate decayed from \(10^{-3}\) to \(10^{-7}\) via cosine annealing. Radiance field reconstruction employs standard training strategies of their respective frameworks (Plenoxels/3DGS). For DeepDeblurRF-P, voxel pruning and upsampling are performed every 38,400 steps; for DeepDeblurRF-G, depth pruning and extra point strategies from Deblurring-3DGS are used to handle sparse point clouds.

Key Experimental Results

Main Results

Method	3D Representation	PSNR↑ (Motion Blur)	PSNR↑ (Defocus Blur)	Training Time
Deblur-NeRF	MLP	27.67	30.03	31.33h
DP-NeRF	MLP	28.03	30.15	30.00h*
PDRF-10	Voxel	28.33	30.03	4.26h
DeepDeblurRF-P	Voxel	29.81	32.51	1.14h
Deblurring-3DGS	3DGS	26.30	29.37	0.33h
BAGS	3DGS	27.41	29.90	1.25h
DeepDeblurRF-G	3DGS	29.94	32.58	0.28h

On the BlurRF-Synth test set. DeepDeblurRF-G achieves the highest PSNR and fastest training speed among all methods. Compared to BAGS, it gains +2.53dB on motion blur and +2.68dB on defocus blur. *DP-NeRF requires dual GPUs.

Ablation Study

Configuration	PSNR (Motion Blur)	PSNR (Defocus Blur)	Description
N=1 (No iteration)	28.40	30.74	Single deblurring + reconstruction only
N=3	29.63	32.15	3 iterations
N=5 (Full)	29.94	32.58	Optimal with 5 iterations

Key Findings

• DeepDeblurRF-G takes only 0.28 hours to train, which is 107 times faster than DP-NeRF (30h) and 15 times faster than PDRF-10 (4.26h).
• On BlurRF-SB (where all views share the same blur direction), existing methods completely fail due to the lack of complementary information, whereas DeepDeblurRF-G achieves a PSNR of 29.59 (vs 24.49 for BAGS), demonstrating the value of DNN-based sharp image priors.
• On real-world low-light noisy scenes (BlurRF-Real), DeepDeblurRF-G achieves a non-reference image quality metric of NIQE=5.423 (lower is better), substantially outperforming other methods and validating its capability to handle non-linear degradations.
• During the iteration process, the PSNR continues to rise: \(N=1\rightarrow3\rightarrow5\) corresponds to motion blur PSNR of \(28.40\rightarrow29.63\rightarrow29.94\).

Highlights & Insights

• Clever shift in perspective: Existing methods attempt to model blur kernels within the radiance field framework. This paper completely breaks this paradigm by leveraging mature DNN deblurring networks, compensating for the lack of information in single-image deblurring through an ingenious iterative framework. This "standing on the shoulders of giants" strategy avoids reinventing the wheel.
• RF-guided deblurring leverages the unique information-aggregation property of radiance fields: Radiance fields are essentially 3D fusions of multi-view information, and their rendered results naturally possess a "denoising" effect. Using this characteristic as a feedback signal for deblurring is a creative utilization of radiance field capabilities.
• BlurRF-Synth dataset holds long-term value: It is the first large-scale training and evaluation dataset designed specifically for radiance field deblurring, simulating the non-linear degradation process of real cameras, which will benefit subsequent research.

Limitations & Future Work

• It requires training an additional deblurring network (though reusable after being trained once), which increases the initial deployment cost of the method.
• Re-running COLMAP to estimate camera poses is required in each iteration, which may lead to inaccurate pose estimation in early iterations when the deblurring quality is poor.
• Independent deblurring network weights are used in each of the 5 iterations, requiring a total of 9 networks to be trained (1 initial + 4 × 2 blur types), resulting in a large number of parameters.
• The ability to handle extreme blur (e.g., ultra-long exposure) is limited by the upper bound of the pre-trained deblurring network's capability.

Related Work & Insights

• vs Deblur-NeRF/DP-NeRF: These methods model blur kernels using MLPs, limited by linear blur assumptions and the high computational overhead of multi-ray rendering. DeepDeblurRF handles blur through an external DNN, completely avoiding these limitations, resulting in higher quality and faster speeds.
• vs BAGS: BAGS handles blur by optimizing dense pixel-wise blur kernels, which is computationally expensive. DeepDeblurRF leverages large-scale pre-trained deblurring priors instead of scene-specific optimization, yielding better generalization and efficiency.
• vs Single-Image Deblurring (NAFNet): Directly using NAFNet for frame-by-frame deblurring followed by radiance field training yields poor results. DeepDeblurRF compensates for the lack of single-image information through an RF-guided iterative framework, transforming the advantages of single-image deblurring into advantages for radiance field reconstruction.

Rating

• Novelty: ⭐⭐⭐⭐ First to introduce DNN deblurring into radiance fields, with an ingeniously designed RF-guided iterative framework.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Covers both motion and defocus blur, multiple synthetic/real datasets, and two 3D representations.
• Writing Quality: ⭐⭐⭐⭐ Clear structure with well-elaborated motivation.
• Value: ⭐⭐⭐⭐⭐ Significantly advances the practicality of radiance field deblurring, with the BlurRF-Synth dataset offering long-lasting value.

Related Papers

• [CVPR 2025] RelationField: Relate Anything in Radiance Fields
• [CVPR 2025] DiET-GS: Diffusion Prior and Event Stream-Assisted Motion Deblurring 3D Gaussian Splatting
• [CVPR 2025] FFaceNeRF: Few-Shot Face Editing in Neural Radiance Fields
• [CVPR 2025] Joint Optimization of Neural Radiance Fields and Continuous Camera Motion from a Monocular Video
• [CVPR 2026] Evidential Neural Radiance Fields