HNDiff: Haze-Noise Diffusion for Image Dehazing

Conference: ECCV 2026
Paper: ECCV 2026 Poster
Code: https://jin-ting-he.github.io/HNDiff/
Area: Image Restoration
Keywords: image dehazing / diffusion models / atmospheric scattering model / latent priors / feature gating

TL;DR

Addressing detail loss in regression-based dehazing under dense haze and fidelity degradation in standard Gaussian diffusion, HNDiff embeds the atmospheric scattering model into the diffusion process via haze-aware noise scheduling and dual-estimator reverse sampling, serving as an effective plug-and-play latent prior generator.

Background & Motivation

Single image dehazing is a classic ill-posed inverse problem in computer vision. Atmospheric aerosols scatter and absorb ambient light, significantly degrading image contrast, causing noticeable color cast, and obscuring fine scene structures, which severely hinders downstream tasks such as object detection, semantic segmentation, and autonomous navigation. In recent years, deep learning approaches—ranging from CNN backbones (e.g., MSBDN, FocalNet) to vision Transformers (e.g., Dehamer, ConvIR) and state space models (Mamba)—have continuously advanced the state of the art. However, these methods are predominantly trained as deterministic direct regressors mapping hazy inputs to clean targets. When confronted with dense haze or distant backgrounds where high-frequency textures are heavily attenuated, regression models struggle to synthesize lost structures, often producing overly smoothed outputs or leaving conspicuous residual haze.

Concurrently, generative diffusion models have emerged as a powerful paradigm for low-level image restoration owing to their strong generative priors and texture synthesis capabilities. Nonetheless, directly adapting conventional diffusion frameworks (such as DDPM) to image dehazing introduces a fundamental physical mismatch. Standard diffusion models initiate reverse sampling from pure isotropic Gaussian noise, disregarding the physical principles governing haze formation. Natural hazy images follow the Atmospheric Scattering Model (ASM), where degradation is non-uniform and structured by spatial scene depth and atmospheric attenuation. Unconstrained stochastic diffusion not only struggles to preserve structural fidelity but also tends to hallucinate inconsistent artifacts and color shifts; moreover, uniform Gaussian noise injection fails to accommodate the conflicting requirements of hallucination in dense haze versus content preservation in clear regions.

To overcome these limitations, this paper bridges the physical degradation mechanism and the diffusion trajectory. Core idea: embed the atmospheric scattering model as an inductive bias into the diffusion process by coupling physics-driven haze addition with haze-aware adaptive noise scheduling in the forward process, performing dual-estimator reverse restoration, and injecting the recovered latent priors into arbitrary dehazing backbones via feature gating.

Method

Overall Architecture

The overall architecture of HNDiff consists of three tightly coupled components: the forward haze-noise diffusion process, the reverse dehazing-denoising process, and the plug-and-play Latent HNDiff framework. Given a hazy input image, a lightweight image encoder extracts compact latent feature representations. The forward process models a joint haze-noise degradation trajectory guided by the ASM mean shift, with a Haze-Aware Noise Scheduler (HANS) adaptively adjusting noise variance according to spatial haze density. In the reverse process, two dedicated neural estimators decouple and predict the atmospheric noise and residual transmission, progressively removing both perturbations to reconstruct a clean latent prior. Finally, the restored prior is injected into a dehazing backbone across multiple scales through the Feature Gating Module (FGM) via affine modulation, achieving high restoration fidelity and rich detail synthesis.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input hazy image IH & Image Encoder IE"] --> B["Haze-Noise Forward Diffusion<br/>ASM-driven physical mean shift"]
    B --> C["Haze-Aware Noise Scheduler (HANS)<br/>Spatial haze-density-dependent noise scaling"]
    C --> D["Physically Consistent Reverse Process<br/>Dual estimators for haze residual & noise"]
    D --> E["Latent Prior Generation & FGM Fusion<br/>Affine modulation injected into dehazing backbone"]
    E --> F["Output high-fidelity dehazed image Idehz"]

Key Designs

1. Haze-Noise Forward Diffusion: embedding atmospheric scattering into Gaussian mean shifts

Conventional diffusion models perturb images with zero-mean isotropic Gaussian noise until the representation collapses into pure noise, breaking the intrinsic structure of hazy scenes. HNDiff grounds diffusion in the Atmospheric Scattering Model \(I_H(x) = I_0(x)\tau(x) + A(1-\tau(x))\) (where \(\tau(x)=e^{-\sigma(x)d(x)}\) is the transmission map, \(A\) is the global atmospheric light, and \(\sigma\) and \(d\) denote the scattering coefficient and scene depth, respectively). The single-step transition at time step \(t\) is formulated as:

\[I_t(x) = I_{t-1}(x)e^{-\alpha_t \sigma(x)d(x)} + A\left(1 - e^{-\alpha_t \sigma(x)d(x)}\right) + \beta_t(x)\epsilon_t(x)\]

where \(\alpha_t = 1/T\) is the uniform time step and \(\epsilon_t \sim \mathcal{N}(0, I)\). In this formulation, each degradation step attenuates scene radiance and accumulates atmospheric light according to the ASM, shifting the Gaussian mean toward the natural hazy distribution. By leveraging the reparameterization trick, the full \(T\)-step forward process yields a closed-form marginal distribution \(I_T(x) = I_H(x) + \bar{\beta}_T(x)\epsilon(x)\), which ensures that the terminal noisy state aligns with real hazy observations while preserving stochastic perturbations.

2. Haze-Aware Noise Scheduler (HANS): dynamically allocating generative capacity by spatial haze density

In natural hazy scenes, haze density varies dramatically across spatial coordinates: foreground objects typically experience mild attenuation, while distant background regions suffer from severe information loss. Applying uniform noise injection across all pixels would over-corrupt clear regions while under-perturbing dense haze regions. To address this issue, the Haze-Aware Noise Scheduler (HANS) explicitly binds the pixel-wise noise scaling factor \(\beta_t(x)\) to the local scattering attenuation:

\[\beta_t(x) = 1 - e^{-\alpha_t \sigma(x)d(x)}\]

In heavily hazy areas where \(\sigma(x)d(x)\) is large, \(\beta_t(x)\) increases, injecting stronger stochastic perturbations that empower the diffusion model to hallucinate missing high-frequency structural details; conversely, in light haze or clear regions, \(\beta_t(x)\) drops near zero, preserving structural fidelity through deterministic regression. Over \(T\) steps, the accumulated noise scaling coefficient \(\bar{\beta}_T(x) = \sqrt{\frac{(1-e^{-\frac{1}{T}\sigma(x)d(x)})(1-e^{-2\sigma(x)d(x)})}{1+e^{-\frac{1}{T}\sigma(x)d(x)}}}\) inherently maintains this adaptive property.

3. Physically Consistent Reverse Dehazing-Denoising: decoupling atmospheric noise and haze residual

During reverse restoration, the model must simultaneously remove coupled additive noise and atmospheric haze. Drawing inspiration from DDIM deterministic sampling, the paper derives the reverse transition from the variational lower bound, showing that the single-step reconstruction subtracts atmospheric noise before compensating for transmission attenuation:

\[I_{t-1}(x) = \left( I_t(x) - N_t(x)\left(1 - e^{-\alpha_t \sigma(x)d(x)}\right) \right) e^{\alpha_t \sigma(x)d(x)}\]

where \(N_t(x) = A + \epsilon_t(x)\) represents the composite atmospheric noise comprising atmospheric light and Gaussian perturbation. Because ground-truth transmission maps and atmospheric parameters are unavailable during inference, HNDiff employs two dedicated lightweight U-Net estimators: the noise estimator \(N_t^\theta(I_t, I_H, t)\) approximates composite noise \(N_t\), while the haze estimator \(1 - e^{-\alpha_t o_\theta(I_t, I_H, t)}\) estimates the residual transmission factor, with \(o_\theta\) predicting the local scattering-depth product \(\sigma d\). Decoupling the physical scattering component from stochastic noise guarantees strict physical consistency with the forward process.

4. Latent Prior Generation and Feature Gating Fusion: lightweight plug-and-play backbone enhancement

Executing multi-step diffusion directly in high-resolution image space incurs prohibitive computational complexity and may introduce spatial instability. Latent HNDiff addresses this bottleneck by executing the haze-noise diffusion process in a compact latent space extracted by an Image Encoder (IE). Once clean latent priors \(Z_0\) are reconstructed after \(T\) reverse steps, the Feature Gating Module (FGM) fuses them into arbitrary dehazing backbones. The FGM compresses spatial dimensions via pixel unshuffle and global average pooling, followed by an MLP mapping to generate affine modulation parameters \((z^{\alpha_i}, z^{\beta_i})\):

\[F_i^{out} = F_i^{in} \times z^{\alpha_i} + z^{\beta_i}\]

This feature modulation adaptively injects generative geometric and contextual priors into multi-scale backbone representations with negligible computational overhead.

Loss & Training

Latent HNDiff is optimized using a progressive three-stage strategy: 1. Backbone and Prior Pretraining: Given paired data \((I_H, I_0)\), the ground-truth prior \(Z_{gt} = \text{IE}(\text{Concat}(I_H, I_0))\) is extracted to pretrain the Image Encoder, Feature Gating Module, and dehazing backbone under standard reconstruction losses; 2. Diffusion Prior Training: The image encoder is frozen, and the haze-noise forward process generates perturbed latent representations \(Z_T\). The dual estimators are trained in latent space to reconstruct clean priors \(Z_0\), supervised by the L1 prior loss \(\mathcal{L}_{prior} = \|Z_0 - Z_{gt}\|_1\); 3. End-to-End Joint Fine-tuning: The recovered prior \(Z_0\) is routed through the FGM to the dehazing backbone, and all modules are jointly fine-tuned end-to-end under the backbone's native loss functions. In practice, setting \(T = 4\) provides optimal restoration performance with high sampling speed.

Key Experimental Results

Main Results

Quantitative evaluations across two synthetic benchmarks (SOTS-Indoor, SOTS-Outdoor) and four real-world datasets (NH-HAZE, O-HAZE, Dense-HAZE, RW2AH) demonstrate substantial gains when integrating HNDiff into leading baselines (FocalNet, ConvIR, SGDN):

Dataset	Metric	FocalNet Baseline	+ HNDiff (Ours)	ConvIR Baseline	+ HNDiff (Ours)	SGDN Baseline	+ HNDiff (Ours)	Average Gain
NH-HAZE (real non-homogeneous)	PSNR (dB) SSIM	20.36 0.696	20.89 (+0.53) 0.697 (+0.001)	20.65 0.692	21.23 (+0.58) 0.701 (+0.009)	20.13 0.680	20.64 (+0.51) 0.686 (+0.006)	+0.54 +0.005
O-HAZE (real outdoor)	PSNR (dB) SSIM	25.46 0.791	26.32 (+0.86) 0.801 (+0.010)	25.25 0.784	26.20 (+0.95) 0.799 (+0.015)	24.59 0.778	25.40 (+0.81) 0.782 (+0.004)	+0.87 +0.010
Dense-HAZE (real dense haze)	PSNR (dB) SSIM	16.95 0.597	17.29 (+0.34) 0.599 (+0.002)	16.86 0.600	17.18 (+0.32) 0.623 (+0.023)	16.60 0.571	17.17 (+0.57) 0.611 (+0.040)	+0.41 +0.022
RW2AH (real-world adverse)	PSNR (dB) SSIM	21.93 0.635	22.29 (+0.36) 0.647 (+0.012)	21.99 0.640	22.25 (+0.26) 0.646 (+0.006)	22.24 0.631	22.81 (+0.57) 0.653 (+0.022)	+0.40 +0.013
SOTS-Indoor (synthetic indoor)	PSNR (dB) SSIM	40.82 0.992	41.19 (+0.37) 0.994 (+0.002)	41.53 0.994	42.10 (+0.57) 0.995 (+0.001)	41.01 0.992	41.47 (+0.46) 0.995 (+0.003)	+0.47 +0.002
SOTS-Outdoor (synthetic outdoor)	PSNR (dB) SSIM	37.71 0.995	38.10 (+0.39) 0.996 (+0.001)	37.95 0.994	38.83 (+0.88) 0.995 (+0.001)	36.22 0.986	37.10 (+0.88) 0.991 (+0.005)	+0.72 +0.002

On the unreferenced real-world RTTS benchmark with RIDCP backbone: - Hazy input: FADE 2.484, NIMA 4.33, BRISQUE 37.01 - RIDCP baseline: FADE 0.944, NIMA 4.43, BRISQUE 18.78 - RIDCP + HNDiff: FADE 0.417, NIMA 5.08, BRISQUE 16.09 (achieving superior perceptual quality and haze removal).

Ablation Study

Ablations conducted with FocalNet on the challenging NH-HAZE dataset validate the contribution of each module (Tables 3 and 4):

Config	Noise Diffusion	Haze Diffusion	HANS	PSNR (dB)	SSIM	Note
Net1 (Baseline)	✗	✗	✗	20.36	0.696	Standard FocalNet baseline
Net2	✓	✗	✗	20.46	0.695	Standard DDPM Gaussian noise diffusion
Net3	✗	✓	✗	20.61	0.696	ASM haze diffusion only (no noise)
Net4	✓	✓	✗	20.68	0.696	Joint haze-noise diffusion without HANS
Net5 (HNDiff)	✓	✓	✓	20.89	0.697	Full HNDiff with adaptive HANS scheduler

Comparison with alternative prior generation mechanisms (Table 4): - Net1 (No prior): 20.36 dB / 0.696 - Net2 (Plain U-Net direct prior): 20.41 dB / 0.696 - Net3 (DDPM prior): 20.46 dB / 0.695 - Net4 (RDDM residual diffusion prior): 20.43 dB / 0.690 - Net5 (HNDiff physical diffusion prior): 20.89 dB / 0.697

Comparison against model capacity scaling under comparable parameter counts (Table 7): - FocalNet baseline: 3.74M parameters, 30.53G FLOPs, 20.36 dB PSNR - FocalNet+ (scaled width 32→48): 8.40M parameters, 68.54G FLOPs, 20.37 dB PSNR (+0.01 dB gain) - FocalNet* (scaled depth 4→10 blocks): 8.28M parameters, 64.05G FLOPs, 20.51 dB PSNR (+0.15 dB gain) - FocalNet + HNDiff (Ours): 7.82M parameters, 36.38G FLOPs, 20.89 dB PSNR (+0.53 dB gain with roughly half the FLOPs of capacity scaling)

Image space versus latent space formulation on RW2AH (Table 6): - FocalNet baseline: 21.18 dB / 0.5970, 30.53G FLOPs - HNDiff (Image Space): 21.37 dB / 0.6166, 65.59G FLOPs - HNDiff (Latent Space): 21.52 dB / 0.6254, 36.38G FLOPs (higher fidelity and lower computation)

Key Findings

• Complementary synergy between physics and stochastic generation: Introducing physical haze diffusion alone (+0.25 dB) outpaces standard Gaussian diffusion (+0.10 dB), while integrating both via HANS yields a +0.53 dB leap, demonstrating that physical inductive bias effectively channels generative capacity toward true scene radiance.
• Structured priors outperform brute-force capacity expansion: Scaling network channels or block depths more than doubles FLOPs (up to 68.54G) while yielding negligible performance gains (+0.01 to +0.15 dB); in contrast, HNDiff achieves 20.89 dB at only 36.38G FLOPs, underscoring the vital role of diffusion priors over raw parameter scaling.
• Fast convergence within few diffusion steps: Step-sensitivity curves reveal that performance peaks at \(T = 4\), with negligible returns from larger step budgets, enabling rapid inference while maintaining superior restoration quality.

Highlights & Insights

• Atmospheric Scattering Model as an Inductive Bias: Replaces unguided Gaussian corruption with a physically consistent clean-to-hazy transition trajectory, embedding scene radiance attenuation directly into diffusion mean shifts.
• Haze-Aware Noise Scheduling (HANS): Solves the tension between hallucinating missing details in heavy haze and preserving high fidelity in clear regions through continuous, transmission-dependent noise scaling.
• Modular Latent Prior Integration: Restricts multi-step iterative sampling to a compact latent space and injects the learned prior via affine feature gating, delivering consistent SOTA improvements across multiple backbones with minimal overhead.

Limitations & Future Work

• Strict adherence to the single atmospheric scattering model: Degradations involving non-uniform night-time artificial illumination, raindrop occlusion, or dynamic motion blur deviate from the standard ASM assumption and cannot be directly addressed without task-specific physical priors.
• Dual-estimator memory footprint: Evaluating both the haze estimator and the noise estimator during reverse sampling elevates peak GPU training memory (e.g., up to 23.9GB on SGDN).
• Future directions: Exploring unified multi-head estimators to reduce peak memory and extending the physical diffusion principle to underwater optical imaging and adverse weather restoration.

Related Work & Insights

• vs Standard Diffusion Dehazing (e.g., DDPM, DiffIR): Traditional diffusion relies on pure Gaussian noise and unguided sampling, leading to hallucinations; HNDiff enforces physical ASM consistency along the entire diffusion trajectory.
• vs Residual Diffusion (e.g., RDDM): RDDM diffuses over a linear difference space without modeling depth-dependent exponential light attenuation; HNDiff explicitly models physical transmission and scattering, yielding superior restoration (20.89 dB vs. 20.43 dB).
• vs Classical ASM Inversion (e.g., PSD, RIDCP): Pure ASM inversion methods suffer from ill-posed transmission map estimation in dense haze; HNDiff integrates ASM as a generative diffusion prior to robustly synthesize severely occluded structures.

Rating

• Novelty: ⭐⭐⭐⭐⭐ [Pioneering integration of atmospheric scattering physics into diffusion forward mean shifts and spatial noise scheduling]
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive verification across 7 synthetic and real benchmarks, 4 distinct dehazing backbones, and rigorous parameter-matched ablations]
• Writing Quality: ⭐⭐⭐⭐⭐ [Clear mathematical derivations, strong physical motivation, and well-structured empirical analyses]
• Value: ⭐⭐⭐⭐⭐ [Provides an elegant template for integrating physical imaging models into generative diffusion architectures for low-level vision]