LIIFusion: Coarse-to-fine Framework for Generative MEF via Implicit Neural Representation¶

Conference: ECCV 2026
Paper: ECCV Official
Project Page: https://sangmin213.github.io/LIIFusion/
Area: Others (Multi-Exposure Fusion / Image Restoration)
Keywords: Multi-Exposure Fusion, Implicit Neural Representation, Diffusion Prior, Coarse-to-Fine, Adaptive Exposure Correction

TL;DR¶

LIIFusion decouples generative multi-exposure fusion into a low-resolution global diffusion stage with adaptive exposure correction and a high-resolution continuous implicit neural refinement stage, eliminating costly patch-wise sampling to achieve a 3.5× speed-up while improving structural fidelity and perceptual quality.

Background & Motivation¶

Capturing scenes with high dynamic range (HDR) poses a fundamental challenge for standard image sensors, as single exposures frequently suffer from blown-out highlights or underexposed shadows. Multi-exposure fusion (MEF) synthesizes complementary details from multiple low dynamic range (LDR) exposures into a single high-quality image without requiring explicit camera response function inversion or complex tone mapping. However, traditional multi-scale pyramid approaches rely heavily on heuristic pixel-level measures, easily producing ghosting artifacts in dynamic scenes and failing in saturated regions where structural details are completely missing. Modern deep learning regressions using CNNs or Transformers improve deghosting robustness, but they fundamentally lack the generative capability to hallucinate plausible textures in heavily saturated areas.

Recent generative MEF approaches, such as UltraFusion, reformulate fusion as guided inpainting within the latent space of diffusion models, effectively suppressing ghosting and synthesizing realistic missing content. However, these models operate at fixed resolutions (e.g., $512 \times 512$) and require expensive patch-wise sliding-window sampling when processing high-resolution images, requiring dozens of denoising passes per image and taking hours across a full benchmark. Furthermore, when over-exposed inputs are severely saturated, local guidance becomes fragile, causing independent patch sampling to produce boundary artifacts, structural collapse, or unstable color tones.

The key to overcoming this dilemma is recognizing that global tone harmonization and ghost removal do not need expensive diffusion sampling at full resolution, nor does high-frequency detail restoration require rigid discrete pixel super-resolution. Core idea: decouple generative MEF into a coarse-to-fine pipeline where low-resolution global fusion is executed with a diffusion prior (stabilized by an adaptive exposure correction module) in minimal sampling passes, while a multi-exposure continuous implicit neural representation (LIIF) queries arbitrary target coordinates conditioned directly on high-resolution source evidence to reconstruct crisp, full-resolution textures.

Method¶

Overall Architecture¶

LIIFusion operates in two complementary stages: coarse generative fusion and fine-scale continuous implicit refinement. The inputs consist of a high-resolution over-exposed image $I_{oe} \in \mathbb{R}^{H \times W \times 3}$ and an under-exposed image $I_{ue} \in \mathbb{R}^{H \times W \times 3}$. Both inputs are downsampled via bicubic interpolation to yield low-resolution counterparts $\{I_{oe}^{LR}, I_{ue}^{LR}\} \subset \mathbb{R}^{h \times w \times 3}$, and RAFT is used to estimate optical flow and align the under-exposed frames to the over-exposed reference. In the coarse stage, an adaptive exposure correction (AEC) module stabilizes the luminance of the over-exposed input before feeding the pair into the diffusion model $g$, producing a coarse fused image $I_{mef}^{LR}$ in just 1–2 forward passes. In the fine stage, a SwinIR encoder extracts coarse representations from $I_{mef}^{LR}$, while a lightweight 4-layer CNN encodes concatenated HR exposure patches from $\{I_{oe}, \tilde{I}_{ue}\}$. Local features are sampled at continuous query coordinates and concatenated for a compact MLP decoder to predict full-resolution RGB values $I_{mef}^{HR}$.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["High-Resolution Exposure Pair<br/>I_oe, I_ue (HR)"] --> B["Bicubic Downsampling & Motion Alignment<br/>RAFT flow alignment yields aligned LR/HR pairs"]
    B --> C["1. Adaptive Exposure Correction<br/>Luminance difference in LAB space computes weight W"]
    C --> D["2. Low-Resolution Generative Fusion<br/>Diffusion prior performs global deghosting to get I_mef(LR)"]
    D --> E["3. Exposure-Conditioned Implicit Refinement<br/>SwinIR(coarse) + 4-Layer CNN(HR sources) extract features"]
    E --> F["Continuous Coordinate Query & MLP Decoding<br/>Bilinear sampling over continuous grid + MLP RGB prediction"]
    F --> G["High-Resolution Fused Image<br/>I_mef(HR) (Arbitrary Target Resolution)"]

Key Designs¶

1. Adaptive Exposure Correction: stabilizing over-exposed structures for diffusion guidance To address the failure of diffusion fidelity guidance when over-exposed inputs are severely saturated, this module adaptively attenuates extreme highlights in the low-resolution over-exposed image prior to coarse fusion. Because diffusion models in UltraFusion rely on the over-exposed reference to guide the VAE decoder, complete white clipping degrades this guidance, leading to structural blur or hallucinated distortions in neon signs, text, and map boundaries. The module first converts the LR pair into the LAB color space to extract luminance channels $L_{oe}^{LR}, L_{ue}^{LR} \in [0, 1]^{h \times w \times 1}$, and computes a clipped exposure disparity map: $$L_{diff}^{LR} = \min(\max(L_{oe}^{LR} - L_{ue}^{LR}, 0), 1)$$ A per-pixel weight map $\mathbf{W}$ is then constructed with a perceptual gamma factor ($1/2.2$) to ensure smooth tonal transitions: $$\mathbf{W} = (1 - \alpha \cdot L_{diff}^{LR})^{1/2.2}$$ The modulated over-exposed image $I_{oe}^{'LR} = I_{oe}^{LR} \odot \mathbf{W}$ replaces the original input during diffusion fidelity guidance. By suppressing highlight blowout ahead of the generative model, essential edge contours and geometric structures are preserved at the coarse level, providing dependable anchors for the subsequent refinement stage.

2. Low-Resolution Generative Fusion: efficient global deghosting and semantic synthesis To overcome the extreme computational burden of patch-wise sliding-window diffusion (which typically requires around 30 sampling iterations and creates potential spatial seams), LIIFusion decouples generative modeling from output resolution. The diffusion backbone operates strictly on downsampled whole images near its native pre-training scale. This allows the diffusion network to capture full-scene context in a single view, executing global exposure harmonization, non-rigid motion deghosting, and generative hallucination across missing regions in only 1 to 2 forward passes. This design slashes total diffusion sampling time by more than 70%, eliminating the primary computational bottleneck of generative MEF.

3. Exposure-Conditioned Implicit Refinement: resolution-agnostic multi-exposure evidence fusion To transcend the limitation of standard single-image implicit super-resolution (such as vanilla LIIF), which can only interpolate from a single low-resolution input, this module repurposes the local implicit image function into a multi-exposure conditional fusion operator. A SwinIR coarse encoder extracts deep semantic features $f_{coarse} = E_{coarse}(I_{mef}^{LR})$, while a lightweight 4-layer CNN fine encoder extracts high-frequency spatial features $f_{fine} = E_{fine}(I_{oe}, \tilde{I}_{ue})$ from $19 \times 19$ patches of the aligned full-resolution exposures. For any target continuous coordinate $x \in [-1, 1]^2$ with cell size $c=[c_h, c_w]$, local feature vectors $z_{coarse}$ and $z_{fine}$ are obtained via bilinear interpolation, concatenated, and decoded by a lightweight MLP $f_\theta$: $$s = f_\theta([z_{coarse}, z_{fine}], [x, c])$$ This enables the network to directly query uncompressed high-frequency details from the original HR exposures, synthesizing crisp details, sharp alphanumeric characters, and fine textures without any resolution constraints.

Loss & Training¶

During training, the coarse diffusion model parameters are frozen or pre-trained, and the implicit refinement network is optimized end-to-end. Training pairs are sampled from the SICE dataset, augmented with synthetic dynamic occlusion masks extracted from Vimeo-90K to simulate object motion. With the ground-truth high-resolution image denoted as $I_{GT}$, the entire network is supervised using an $L_1$ reconstruction loss: $$\mathcal{L}_{fusion} = \| I_{mef}^{HR} - I_{GT} \|_1$$ The network is optimized using the Adam optimizer with a learning rate of $1 \times 10^{-4}$ and a batch size of 16. Training runs for 500 epochs on a single NVIDIA H100 GPU, converging fully in approximately 22 hours.

Key Experimental Results¶

Main Results¶

Quantitative evaluations are conducted on the dynamic RealHDRV benchmark (50 dynamic scenes with complex motion) and the UltraFusion Benchmark (100 real-world exposure pairs with extreme dynamic range). Evaluated metrics include four non-reference image quality assessment (NRIQA) scores (MUSIQ, DeQA-Score, PAQ2PIQ, HyperIQA) and the structure-aware MEF-SSIM. Total inference time is benchmarked on a single NVIDIA RTX A5000 GPU.

Dataset / Scenario	Model	MUSIQ ↑	DeQA-Score ↑	PAQ2PIQ ↑	HyperIQA ↑	Total Time ↓
RealHDRV (50 scenes)	Defusion (ECCV22)	56.38	3.2867	68.31	0.4838	2 min
	MEF-LUT (ICCV23)	62.42	3.2864	70.04	0.5020	4 sec
	HSDS-MEF (2024)	61.82	3.6045	71.14	0.5055	18 min
	UltraFusion (CVPR25)	67.54	3.8998	73.39	0.5834	101 min
	Ours (LIIFusion)	69.52	3.8908	74.06	0.6175	27 min
UltraFusion (100 scenes)	Defusion (ECCV22)	60.11	3.3529	71.83	0.5440	6 min
	MEF-LUT (ICCV23)	64.06	3.2859	71.80	0.5103	8 sec
	HSDS-MEF (2024)	65.23	3.6662	73.77	0.5786	46 min
	UltraFusion (CVPR25)	68.40	4.0123	75.18	0.6214	203 min
	Ours (LIIFusion)	70.19	3.9807	75.59	0.6467	59 min

On a single $1988 \times 1326$ image from the UltraFusion Benchmark, LIIFusion slashes computational complexity from 944.4 TFLOPs to 282.1 TFLOPs (a 70.1% reduction) compared to UltraFusion, while maintaining a comparable parameter footprint (1.872 B vs. 1.860 B).

Ablation Study¶

1. Exposure Inputs for Fine Fusion (evaluated on UltraFusion Benchmark) To verify that the fine stage performs genuine multi-exposure conditional fusion rather than generic super-resolution, the contribution of each exposure source was systematically evaluated:

Fine Stage Input Config	MUSIQ ↑	DeQA-Score ↑	PAQ2PIQ ↑	HyperIQA ↑	Note
w/o OE, UE (coarse only)	54.65	3.7961	74.09	0.4277	Degrades to single-image SR; severe blur
w/ UE only	62.73	3.8412	74.59	0.5773	Recovers dark regions but amplifies noise
w/ OE only	68.38	3.8438	75.58	0.6141	Restores bright textures but misses saturated gaps
w/ OE, UE (Full Model)	70.19	3.9807	75.59	0.6467	Optimal fusion integrating complementary exposures

2. Adaptive Exposure Correction (AEC) Module - Ours (w/o AEC): MUSIQ 70.07, PAQ2PIQ 75.64, HyperIQA 0.6451, DeQA-Score 3.9870 - Ours (w/ AEC): MUSIQ 70.19, PAQ2PIQ 75.59, HyperIQA 0.6467, DeQA-Score 3.9807 Insight: While metric improvements appear moderate, AEC provides indispensable qualitative value: without AEC, severe over-exposure causes neon lettering and display board maps to collapse during low-resolution diffusion, producing unrecoverable distortions; with AEC, bright outlines remain stable throughout both stages.

Key Findings¶

Multi-exposure conditioning vs. standard SR: Pre-trained LIIF applied purely as super-resolution on coarse outputs achieves an MEF-SSIM of 0.8942 and HyperIQA of 0.6389 on MEFB, lagging substantially behind LIIFusion (MEF-SSIM 0.9201, HyperIQA 0.6418). This demonstrates that direct conditioning on high-resolution exposure sources is essential for restoring fine structural fidelity.
Overwhelming human preference: In a blind user study with 22 participants across 4 anonymized methods, LIIFusion achieved a 61.36% best-choice rate and an average rank of 1.52, decisively outperforming UltraFusion (21.21% best-choice, 2.02 average rank) and HSDS-MEF (13.64% best-choice, 2.89 average rank).
Viability of synthetic pseudo-labels: Training LIIFusion on pseudo-labels synthesized by UltraFusion yielded competitive results (MUSIQ 69.34, HyperIQA 0.6323) matching ground-truth supervision (70.19 / 0.6467), proving that generative MEF models can act as scalable pseudo-label generators to expand training data without manual tuning.

Highlights & Insights¶

Continuous implicit formulation for multi-exposure fusion: This paper presents the first application of Implicit Neural Representations (INR) to multi-exposure fusion, reframing INR not as single-image super-resolution but as a continuous coordinate-level multi-source fusion function.
Synergistic decoupling of global priors and local evidence: Confining stochastic diffusion to global low-resolution tone mapping and delegating high-frequency detail restoration to a deterministic MLP decoder solves both the patch-boundary inconsistency and runtime bottleneck of generative vision models.
AEC as a generalizable highlight stabilization technique: Modulating over-exposed luminance using LAB disparity and gamma weighting is an elegant, lightweight strategy to prevent generative models from hallucinating ungrounded artifacts in extreme highlights.

Limitations & Future Work¶

Structural dependency on coarse diffusion: The final high-resolution output remains bounded by the structural and topological correctness of the low-resolution coarse diffusion stage. Severe geometric distortions in the coarse stage cannot be entirely rectified by the fine stage.
Latency gap to real-time execution: Although 3.5× faster than UltraFusion, total runtimes (27–59 minutes per benchmark) remain far slower than lightweight regression baselines like MEF-LUT, precluding real-time mobile viewfinder deployment.
Residual artifacts under extreme motion: In scenarios featuring rapid non-rigid motion or severe occlusions, optical flow misalignments can occasionally introduce subtle residual ghosting at fine boundaries.

vs. UltraFusion (CVPR 2025): UltraFusion performs patch-wise sliding-window diffusion across full-resolution images, incurring 944.4 TFLOPs and hours of computation with potential patch-seam artifacts; LIIFusion restricts diffusion to low-resolution whole images and employs an implicit decoder, slashing TFLOPs by 70% and accelerating inference by 3.5× with superior fidelity.
vs. LIIF (CVPR 2021): Vanilla LIIF is a single-image continuous super-resolution model; LIIFusion extends this into a dual-branch multi-exposure fusion architecture that simultaneously queries coarse generative embeddings and local high-resolution exposure patches.
vs. HSDS-MEF / MEF-LUT: Conventional and lightweight deep learning MEF methods lack generative hallucination priors, causing ghosting around dynamic objects and dull, low-contrast tones in blown-out highlights; LIIFusion leverages generative diffusion priors to eliminate ghosting and recover natural colors.

Rating¶

Novelty: ⭐⭐⭐⭐☆ [Pioneering adaptation of implicit neural representations for multi-exposure fusion with a well-motivated coarse-to-fine decoupling]
Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Thorough benchmarking across static and dynamic datasets, extensive ablations on exposure conditioning, and insightful user studies]
Writing Quality: ⭐⭐⭐⭐⭐ [Clear motivation, well-structured methodology, and insightful analysis of runtime-fidelity trade-offs]
Value: ⭐⭐⭐⭐☆ [Substantially lowers the barrier for deploying diffusion models in high-resolution low-level vision tasks]