content_hash: cbd6b484af8a6cb7

SplitHDR: Saturation-Aware HDR Recovery and Denoising for Real-Time Detection¶

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/snu-srml/SplitHDR
Area: Object Detection
Keywords: High Dynamic Range Imaging, Object Detection, Autonomous Driving, Vision for Edge Devices, Adaptive Inference

TL;DR¶

Addressing the stringent latency constraints and abrupt illumination variations of autonomous driving edge platforms, SplitHDR proposes an input-luminance-adaptive dual-mode architecture that decouples short-exposure highlight recovery in saturated regions from lightweight denoising in unsaturated dominant regions, achieving state-of-the-art downstream detection accuracy (38.38% AP / 58.75% AP50) with merely 2.47k parameters and 3.63 GMACs.

Background & Motivation¶

Autonomous driving systems rely heavily on real-time visual perception, such as vehicle, pedestrian, and traffic-sign detection, to ensure safe planning and vehicle control. However, under extreme illumination transitions—such as exiting tunnels, driving at night facing blinding high-beam headlight glare, or navigating high-contrast scenes with simultaneous deep shadows and bright sun glints—conventional low-dynamic-range (LDR) imaging inevitably causes extensive highlight clipping and under-exposed shadow degradation, discarding radiometric cues critical for downstream recognition. Existing deep learning-based multi-exposure high-dynamic-range (HDR) reconstruction networks heavily depend on dense attention alignment modules or generative diffusion priors, incurring millions of parameters and thousands of GMACs per frame, rendering them impractical for real-time edge deployment on automotive embedded platforms like FPGAs or ASICs.

The core tension behind this computational bottleneck lies in an architectural misalignment: conventional HDR fusion networks treat every pixel uniformly across the entire image grid with heavy multi-frame alignment and deep feature extraction, whereas real-world driving frames exhibit an extreme spatial asymmetry in information density. Empirical driving datasets reveal that unsaturated pixels occupy the vast majority of scene areas (averaging 79.80% in daytime, 88.08% at night, and 83.94% overall), where long- and mid-exposure frames naturally provide superior signal-to-noise ratios (SNR) and structural fidelity. Conversely, highlight clipping is confined to sparse, localized regions (accounting for only ~16% of pixels). Applying dense multi-frame fusion across the entire frame squanders precious computing budgets on well-exposed flat areas and risks propagating severe sensor shot noise from underexposed short-exposure frames into clean regions.

To resolve this disparity between computational allocation and physical signal statistics, this paper embraces a divide-and-conquer strategy inspired by advanced automotive CMOS image sensors that physically eliminate temporal motion offsets via multi-gain or split-pixel readouts. Core idea: adaptively construct a soft saturation mask from mid-exposure luminance to explicitly decompose the HDR task into a full-scale multi-exposure fusion mode for saturated highlights and a lightweight denoising mode for unsaturated dominant areas, executing heavy multi-frame fusion strictly on saturated tiles during inference.

Method¶

Overall Architecture¶

The SplitHDR pipeline is designed around a three-exposure input interface (short exposure $X_s$, mid exposure $X_m$, long exposure $X_l$), operating across four coordinated stages: soft saturation mask generation, dual-mode dynamic residual backbone processing, exclusive masked supervision, and tile-wise conditional execution with soft blending. The system first computes a continuous soft mask $\alpha \in [0, 1]$ from the luminance of $X_m$. It then evaluates non-overlapping $128 \times 128$ image tiles: if the average saturation ratio within a tile exceeds $\gamma = 0.01$, the full-capacity fusion mode (utilizing $X_s, X_m, X_l$ with full channel width) is activated; otherwise, a lightweight denoising mode (bypassing $X_s$ and halving intermediate channels) is invoked. Finally, transition boundaries are smoothly interpolated via $\alpha$ to predict the HDR residual $\Delta$ on top of $X_m$.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    In["Multi-Exposure Input Stack<br/>(Short Xs, Mid Xm, Long Xl)"] --> S1["Soft Saturation Mask Routing<br/>Mid luminance extraction & threshold mapping α"]
    S1 --> S2{"Tile-wise Saturation Check<br/>128×128 Tile-wise Routing"}
    S2 -->|Saturated Tile α=1| M1["Full-Scale Multi-Exposure Fusion Mode<br/>Full-width GlobalNet + DetailNet (Xs+Xm+Xl)"]
    S2 -->|Unsaturated Tile α=0| M2["Unsaturated Lightweight Denoising Mode<br/>Halved-width GlobalNet + DetailNet (Xm+Xl)"]
    S2 -->|Transition Tile 0<α<1| M3["Dual-Mode Concurrent Execution"]
    M1 --> S3["Conditional Inference Soft Blending<br/>Pixel-wise alpha-weighted blending"]
    M2 --> S3
    M3 --> S3
    S3 --> Out["Residual Addition & Final HDR Output<br/>Y = Xm + Δ"]

Key Designs¶

1. Soft Saturation Mask Routing: Zero-Overhead Region Partitioning via Exposure Scaling

Traditional hard binary thresholding introduces severe boundary blocking and spatial stitching artifacts across regions. SplitHDR designs an analytical, continuous soft saturation mask computed directly from the mid-exposure frame, eliminating any auxiliary neural network latency. Mid-exposure intensities are represented in linear space and normalized against the short-exposure reference scale. Consequently, the maximum representable value of the mid-exposure input is bounded by the exposure ratio $r = t_s / t_m$. To align with network feature space, mid-exposure luminance is extracted after range compression $\mathcal{T}(\cdot)$: $$Y_m = \frac{R_m + 2G_m + B_m}{4}$$ To establish a stable transition band around highlight edges, an upper threshold is set at the saturation ceiling $\tau_{upper} = \mathcal{T}(r)$ and a lower threshold at the onset of saturation $\tau_{lower} = 0.9 \cdot \mathcal{T}(r)$. The continuous soft mask is generated via linear clamping: $$\alpha = \mathrm{clamp}\left(\frac{Y_m - \tau_{lower}}{\tau_{upper} - \tau_{lower} + \varepsilon}, 0, 1\right)$$ Here, $\alpha = 0$ designates confident unsaturated pixels, $\alpha = 1$ denotes fully saturated highlights, and $0 < \alpha < 1$ forms a smooth transition ramp, providing continuous gradient routing during training and artifact-free blending at test time.

2. Task-Specialized Dual-Mode Backbone: Asymmetric Channels and Input Pruning

Targeting the distinct degradation physics of the two regions, SplitHDR adopts a dual-branch residual backbone comprising GlobalNet (a lightweight depthwise-separable U-Net) and DetailNet (a cascade of pointwise $1 \times 1$ convolutions operating at full resolution), anchored by a global skip connection around mid-exposure: $\hat{Y} = X_m + \Delta(X_s, X_m, X_l)$. In unsaturated regions (occupying over 80% of pixels), the short-exposure frame $X_s$ inherently suffers from severe Poisson shot and readout noise due to minimal photon collection. Propagating $X_s$ into these clean regions degrades downstream perceptual representations. Therefore, the unsaturated mode explicitly excludes $X_s$, feeding only $X_m$ and $X_l$. Furthermore, GlobalNet input channels are pruned from 9 to 6, and DetailNet intermediate channels are halved from 32 to 16, slashing active parameters from 1.80k down to only 0.67k. In saturated highlights, the full backbone is activated with $X_s$ to restore clipped physical irradiance.

3. Exclusive Masked Supervision: Disentangled Gradient Specialization

Forcing a single unified network to simultaneously denoise dark regions and reconstruct clipped highlights creates conflicting gradient objectives that destabilize convergence. SplitHDR introduces exclusive masked supervision, spatially weighting the $\ell_1$ loss of the two mode predictions ($\hat{Y}_{unsat}$ and $\hat{Y}_{sat}$) using the continuous soft mask $\alpha$: $$\mathcal{L}_{unsat} = \frac{\sum_p (1 - \alpha_p) \|\hat{Y}_{unsat, p} - Y_p\|_1}{\sum_p (1 - \alpha_p) + \varepsilon}, \quad \mathcal{L}_{sat} = \frac{\sum_p \alpha_p \|\hat{Y}_{sat, p} - Y_p\|_1}{\sum_p \alpha_p + \varepsilon}$$ The composite loss is optimized as $\mathcal{L} = \lambda_{unsat} \mathcal{L}_{unsat} + \lambda_{sat} \mathcal{L}_{sat}$. This formulation directs gradients from unsaturated pixels almost exclusively into denoising and tone fidelity, while gradients from saturated pixels optimize highlight recovery. Pixels in the transition band ($0 < \alpha_p < 1$) softly share supervision, preventing boundary discontinuity artifacts.

4. Tile-Wise Conditional Inference and Soft Blending: Hardware-Friendly Edge Efficiency

Pixel-wise branching induces severe thread divergence and memory bandwidth thrashing on embedded hardware accelerators. SplitHDR implements a practical tile-wise conditional execution scheme. The input image is tiled into $128 \times 128$ non-overlapping patches, and the mean saturation $\bar{\alpha} = \frac{1}{|T|}\sum_{p \in T} \alpha_p$ is computed per patch: - If $\bar{\alpha} < 0.01$, the tile is scheduled purely on the lightweight denoising mode (0.67k active params, bypassed $X_s$ memory fetch); - If the tile contains strong saturation, the full fusion mode is scheduled; - For transition tiles bridging exposure regimes, both configurations are evaluated and seamlessly blended pixel-wise: $\hat{Y} = (1 - \alpha) \odot \hat{Y}_{unsat} + \alpha \odot \hat{Y}_{sat}$. On average, 214 out of 345 tiles in each frame are pure unsaturated tiles, avoiding expensive fusion and reducing total frame computation from 10.23 GMACs to 3.63 GMACs, achieving a real-time latency of 15.0 ms on an RTX 3090 GPU.

Loss & Training¶

The network is trained end-to-end on synthetic multi-exposure stacks derived from the ROD benchmark. The dataset comprises 16,089 training stacks ($2880 \times 1856$ resolution) divided into $256 \times 256$ non-overlapping patches (1,238,853 samples in total). Weights are initialized via He initialization and optimized with Adam ($\beta_1 = 0.9, \beta_2 = 0.999, \epsilon = 10^{-8}$) using a fixed learning rate of $1 \times 10^{-3}$ and batch size of 4 for 309,714 optimization steps. Geometric augmentations (random horizontal/vertical flips and rotations) are applied during training. Loss weights are set to $\lambda_{unsat} = 1.0, \lambda_{sat} = 1.0$. Downstream object detection is evaluated using a YOLOX-Tiny detector trained from scratch for 300 epochs on tone-mapped ground-truth HDR images at $1280 \times 1280$ resolution, ensuring fair and unbiased perceptual benchmarking across all HDR methods.

Key Experimental Results¶

Main Results¶

On the ROD autonomous driving benchmark (4,000 test images spanning daytime and nighttime conditions, annotated with 6 traffic categories), SplitHDR was comprehensively evaluated against single-image HDR (HDRUNet), lightweight dual-exposure fusion (LightFuse), and heavy multi-exposure networks (AHDRNet, SCTNet, SAFNet):

Method	Params (k)	GMACs	PSNR-$\mu$ (dB)	SSIM-$\mu$	Detection AP (%)	Detection AP50 (%)	Detection AP75 (%)	Detection AR (%)
HDR (Ground Truth)	-	-	-	-	38.34	58.77	40.18	47.83
LDR (Mid exposure)	-	-	30.41	0.865	34.24	53.35	36.13	43.67
Exposure Fusion [Mertens 07]	-	-	25.01	0.631	33.94	52.04	38.82	43.27
AHDRNet [Yan et al. 19]	1441.28	7733.12	47.37	0.994	38.38	58.72	40.19	47.84
HDRUNet [Chen et al. 21]	1651.49	1898.31	37.19	0.961	36.36	56.38	38.14	45.78
SCTNet [Tel et al. 23]	961.48	5256.57	48.28	0.995	38.33	58.61	40.12	47.78
SAFNet [Kong et al. 24]	1120.85	3473.70	41.23	0.950	38.16	58.51	39.98	47.63
LightFuse [Liu et al. 21]	1.57	7.13	40.67	0.958	37.88	58.24	39.75	47.20
Ours (SplitHDR)	2.47	3.63	44.96	0.985	38.38	58.75	40.31	47.87

In scene-specific evaluations, SplitHDR attains the top overall performance in daytime scenes (38.67% AP, 48.62% AR) and remains competitive at night (38.27% AP), trailing AHDRNet (38.28% AP) by only 0.01% despite using nearly $600\times$ fewer parameters and $2,130\times$ fewer computations.

Ablation Study¶

The authors conducted progressive component-wise ablations evaluating input exposure count, dual-mode architectures, exclusive masked supervision, and tile-wise conditional execution:

Config	Input Exposures	# Modes	Masked Supervision	Tile-wise Exec.	Params (k)	GMACs	PSNR-$\mu$ (dB)	Detection AP (%)
C1 (Dual-exposure baseline)	2	1	-	-	1.57	7.13	40.67	37.88
C2 (Triple-exposure input)	3	1	-	-	1.80	7.84	40.93	37.68
C3 (Dual-mode without mask loss)	3	2	✗	✗	2.47	10.23	44.01	37.80
C4 (+ Exclusive masked supervision)	3	2	✓	✗	2.47	10.23	44.96	38.38
C5 (Ours full model)	3	2	✓	✓	2.47	3.63	44.96	38.38

Tile-size profiling (Table 5) confirmed that $128 \times 128$ yields the optimal trade-off between mask scheduling overhead (1.46 ms) and network execution (13.78 ms), yielding 15.24 ms total latency (measured 15.0 ms wall-clock time), outperforming $32 \times 32$ (21.24 ms) and $256 \times 256$ (17.10 ms).

Key Findings¶

Exclusive masked supervision unlocks perceptual accuracy: Moving from C3 to C4 with identical parameter counts and FLOPs (10.23 GMACs), introducing masked loss boosts detection AP significantly from 37.80% to 38.38% (+0.58% AP), proving that gradient disentanglement prevents noise interference between exposure regimes.
Conditional execution decouples capacity from compute: C5 reduces computational burden by 64.5% compared to C4 (10.23 down to 3.63 GMACs) without any drop in reconstruction PSNR (44.96 dB) or detection AP (38.38%), while operating 49.1% cheaper than the baseline C1.
Continuous soft transition eliminates boundary artifacts: A 90% threshold soft mask surpasses binary hard masking in both PSNR (+0.09 dB) and detection AP, completely suppressing spatial transition seams.

Highlights & Insights¶

Leveraging Spatial Information Asymmetry: The paper capitalizes on the physical insight that highlight saturation in autonomous driving is inherently sparse (~16%), whereas unsaturated regions (~84%) already possess ample SNR, breaking the convention of running uniform heavy networks across all pixels.
Pruning Short Exposure Avoids Noise Pollution: Excluding the photon-starved, noisy short-exposure frame from unsaturated regions simultaneously slashes computational overhead and prevents noise propagation, demonstrating that selective input omission can improve fidelity.
Unrivaled Computational Efficiency Ratio: SplitHDR achieves a $\Delta\text{PSNR}/\text{GMACs}$ efficiency of 4.008 ($2.8\times$ higher than LightFuse at 1.439) and a detection efficiency $\Delta\text{AP}/\text{GMACs}$ of 1.140, setting a new benchmark for resource-constrained edge perception.

Limitations & Future Work¶

Reliance on Motion-Free or Hardware-Aligned Sensor Inputs: SplitHDR presupposes modern automotive CMOS sensors (e.g., LOFIC, multi-gain, split-pixel) where multi-exposure frames are captured within an identical integration period. Under traditional sequential multi-exposure capture with rapid ego-motion, lack of explicit deghosting or optical flow alignment could cause ghosting artifacts.
Sensitivity to Fixed Exposure Ratios: The analytical saturation thresholding assumes a known exposure ratio $r = t_s / t_m$. Deploying across third-party ISP pipelines with dynamic exposure bracketing or non-linear tone curves may require continuous parameter re-calibration.
Future Directions: Integrating adaptive tile routing directly into on-sensor readout circuitry or embedded ISP accelerator units could enable sub-milliwatt, event-driven intelligent HDR perception.

vs LightFuse [Liu et al. 2021]: LightFuse utilizes a static dual-exposure architecture lacking unsaturated channel pruning and mid-exposure residual reference, leading to short-exposure noise leakage. SplitHDR delivers a $2.8\times$ efficiency gain and superior detection accuracy (38.38% vs 37.88% AP) via its dual-mode routing.
vs AHDRNet / SCTNet [Yan et al. 2019 / Tel et al. 2023]: Heavyweight multi-exposure architectures rely on dense self-attention or Transformer blocks (>1M parameters, >5,000 GMACs) to handle dynamic misalignment. SplitHDR demonstrates that for modern motion-free automotive sensors, an ultra-compact 2.47k-parameter network matches or exceeds their detection performance at a fraction of the cost.

Rating¶

Novelty: ⭐⭐⭐⭐ [Insightful formulation of saturation sparsity and physical exposure statistics into a dynamic dual-mode decoupled framework]
Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive quantitative reconstruction and detection evaluations, tile-size profiling, hardware latency measurements, and perturbation robustness analyses]
Writing Quality: ⭐⭐⭐⭐⭐ [Clear motivation, rigorous mathematical formulation, and well-designed diagrams and tables]
Value: ⭐⭐⭐⭐⭐ [Exceptional practical utility for edge device deployment, autonomous driving perception, and real-time computational imaging]