NUN: Nested Unfolding Network for Real-World Concealed Object Segmentation¶

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/ChunmingHe/NUN
Area: Segmentation
Keywords: Deep Unfolding Network, Concealed Object Segmentation, Real-World Degradation, Bi-Level Optimization, Bi-Directional Unfolding Interaction

TL;DR¶

To resolve the fundamental optimization conflict between low-level texture restoration and high-level semantic segmentation in degraded real-world concealed object scenarios, NUN introduces the first DUN-in-DUN nested unfolding network, decoupling parameter spaces via bi-level proximal gradient optimization and controlled bi-directional interaction across 12 diverse benchmarks.

Background & Motivation¶

Concealed object segmentation (COS) aims to identify visual targets that blend imperceptibly into their background environments, spanning camouflaged creatures, clinical polyps, transparent glassware, and subtle industrial defects. Under ideal laboratory imagery, modern segmentation models exploit subtle edge and textural cues with notable success. However, real-world deployment frequently encounters severe optical degradations—including low-illumination noise, atmospheric haze, and resolution loss—which severely corrupt discriminative boundary cues and lead to catastrophic performance collapses for conventional COS algorithms.

A natural countermeasure is to combine image restoration with segmentation, yet joint formulation exposes a fundamental objective conflict: low-level restoration prioritizes fine-grained texture fidelity and pixel reconstruction, whereas high-level segmentation demands invariant semantic contrast and boundary sharpening. Existing two-stage pipelines (restoring first, then segmenting) discard inter-task feedback, preventing the restoration phase from benefiting from semantic priors. Conversely, coupled unfolding architectures interweave both objectives within shared parameter spaces, inducing severe gradient interference where conflicting objective vectors compromise updates for both tasks.

To reconcile this core tension, the system requires a principled architecture that isolates task optimization within dedicated parameter subspaces while enabling disciplined, mutual cross-task refinement. Core idea: propose a nested deep unfolding architecture (DUN-in-DUN), embedding an inner degradation-resistant unfolding network (DeRUN) inside each outer stage of a segmentation-oriented unfolding network (SODUN) to mathematically decouple parameter gradients while facilitating bi-directional feedback via IQA quality selection and cross-stage consistency.

Method¶

Overall Architecture¶

NUN is structured under a bi-level optimization (BLO) paradigm: the outer loop handles reversible foreground-background estimation and mask prediction, while the inner loop unrolls blind degradation estimation and proximal restoration. Macroscopically, NUN comprises $K$ outer stages (Stage $k \in [1, K]$, default $K=4$). Each outer stage executes one SODUN update step, inside of which is nested an inner DeRUN module executing $N$ proximal gradient iterations (with stage-wise iterations configured as $N \in \{4, 3, 3, 2\}$). The two systems interact through Bi-directional Unfolding Interaction (BUI): SODUN injects semantic guidance into DeRUN's proximal operator, while an Image Quality Assessment (IQA) module selects the top perceptual restoration $\mathbf{X}_{k-1}^{T1}$ to steer subsequent stages.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Degraded Input Image Y"] --> B["Outer Stage SODUN<br/>Reversible FG-BG Estimation"]
    B --> C["Nested Inner DeRUN<br/>Learnable Operator & Proximal Unfolding"]
    C --> D["Bi-Directional Interaction BUI<br/>IQA Filtering + Semantic Injection"]
    D --> E["Cross-Stage Consistency Constraint<br/>Align Top-1 & Top-2 Predictions"]
    E --> F["Output: Optimal Mask MK + Restored Image XTK1"]

Key Designs¶

1. SODUN: Reversible Foreground-Background Optimization with Visual State Space Modeling To suppress false positives and resolve ambiguous boundaries in concealed environments, SODUN formulates segmentation as an explicit, reversible decomposition in both mask and RGB domains. In outer stage $k$, gradient descent first computes an intermediate mask $\hat{\mathbf{M}}_k$ using the prior background estimate $\mathbf{B}_{k-1}$ and the previous stage's selected restoration $\mathbf{X}_{k-1}^{T1}$. The proximal step refines $\hat{\mathbf{M}}_k$ using a Visual State Space (VSS) module capable of modeling global context across sparse discriminative cues, fed with concatenated backbone features from both the restored image and the raw degraded input $E(\text{con}(\mathbf{X}_{k-1}^{T1}, \mathbf{Y}))$. Concurrently, the background estimator $\mathcal{B}$ computes residual gradient corrections in the RGB domain conditioned on $\mathbf{M}_k$, followed by a lightweight U-shaped proximal refinement, establishing a closed-loop mutual verification between foreground contours and background textures.

2. DeRUN: Blind Degradation Modeling via Learnable Proximal Gradient Unfolding Given that real-world degradation matrices $\mathbf{D}$ are unknown and multi-faceted, DeRUN avoids handcrafted physical degradation assumptions by introducing learnable residual convolution operators $RC_{k,n}^\mathbf{D}(\cdot)$ and $RC_{k,n}^{\mathbf{D}T}(\cdot)$ to implicitly approximate $\mathbf{D}$ and its transpose: $$RC_{k,n}^\mathbf{D}(\mathbf{X}) = \boldsymbol{\sigma}_{k,n} \odot (CRC(\mathbf{X})) + \boldsymbol{\mu}_{k,n}$$ where $CRC(\cdot)$ denotes a Conv-ReLU-Conv block, and modulation parameters $\{\boldsymbol{\sigma}_{k,n}, \boldsymbol{\mu}_{k,n}\}$ are conditioned on degradation descriptors from a frozen DA-CLIP encoder (or learned purely implicitly). In the proximal step, DeRUN pairs a standard restoration branch $\mathcal{X}_1$ with a semantic complementarity term $\mathcal{X}_2(\hat{\mathbf{X}}_{k,n}, \mathbf{B}_k, \mathbf{M}_k, \mathbf{Y})$. By feeding SODUN's intermediate masks and background confidence into the restoration update, DeRUN directs its restoration capacity toward highly uncertain, ambiguous concealed regions.

3. Bi-Directional Unfolding Interaction (BUI) and Theoretical Conflict Alleviation To eliminate the gradient interference typical of coupled models without sacrificing cross-task feedback, BUI establishes controlled bilateral bridges. Forwardly, intermediate outputs from inner iterations are scored by a composite Image Quality Assessment metric (synthesizing TOPIQ, Q-Align, and MUSIQ), and the highest-fidelity image $\mathbf{X}_{k-1}^{T1}$ is chosen as the reference input for subsequent segmentation. Reverse error propagation is prevented by a cross-stage consistency loss $\mathcal{L}_{csc}$ that aligns predictions derived from Top-1 and Top-2 restorations. Theoretically, because DeRUN parameters $\theta_{res}$ and SODUN parameters $\theta_{seg}$ form disjoint sets, parameter-level gradient interference $\langle \nabla \mathcal{L}_{seg}, \nabla \mathcal{L}_{res} \rangle < 0$ is eliminated. Furthermore, under local strong convexity and Lipschitz smoothness, degradation affects segmentation strictly through a bounded error $\mathcal{O}(\delta_k + \delta_{res})$, where inner-loop optimization error $\delta_k$ decays exponentially with iteration depth $N$.

Loss & Training¶

The framework is optimized end-to-end using a joint objective composed of multi-stage basic supervision $\mathcal{L}_{basic}$ and the cross-stage consistency regularization $\mathcal{L}_{csc}$: $$\mathcal{L}_{total} = \mathcal{L}_{basic} + \epsilon \mathcal{L}_{csc}$$ where the basic loss accumulates geometrically discounted stage-wise penalties on both mask accuracy and image fidelity: $$\mathcal{L}_{basic} = \sum_{k=1}^K \frac{1}{2^{K-k}} \left[ \mathcal{L}_{BCE}^w(\mathbf{M}_k, GT_s) + \mathcal{L}_{IoU}^w(\mathbf{M}_k, GT_s) + \|\mathbf{X}_k - \mathbf{X}\|_1 \right]$$ with $\mathcal{L}_{BCE}^w$ and $\mathcal{L}_{IoU}^w$ denoting weighted binary cross-entropy and weighted IoU losses, and $\mathbf{X}$ representing the ground-truth clean image. $\mathcal{L}_{csc}$ applies symmetric consistency between Top-1 and Top-2 mask hypotheses $\mathbf{M}_k$ and $\mathbf{M}_k^{T2}$. The network is trained on two NVIDIA RTX 4090 GPUs with Adam optimizer, batch size 36, image dimensions $352 \times 352$, and an initial learning rate of $1 \times 10^{-4}$ decayed every 80 epochs.

Key Experimental Results¶

Main Results¶

Evaluated across 12 diverse benchmarks—including 4 clean COD benchmarks, synthetic degradations (low-light, haze, low-resolution, and their combination), real-world degraded subsets PCOD-LQ / MCOD-LQ, polyp segmentation (PIS), transparent object detection (TOD), and concealed defect detection (CDD)—NUN consistently surpasses existing state-of-the-art methods. The table below presents results on the challenging combined synthetic degradation benchmark from original Table 5.

Excerpt from original Table 5 (Performance on Combined Synthetic Degradation):

Method	Backbone	CAMO (M ↓)	CAMO (Fβ ↑)	COD10K (M ↓)	COD10K (Fβ ↑)	NC4K (M ↓)	NC4K (Fβ ↑)
FEDER	ResNet50	0.169	0.391	0.104	0.322	0.147	0.408
FSEL	ResNet50	0.173	0.394	0.110	0.317	0.150	0.397
RUN	ResNet50	0.169	0.394	0.100	0.345	0.143	0.547
NUN (Ours)	ResNet50	0.137	0.463	0.068	0.448	0.096	0.572
RUN	PVT V2	0.142	0.496	0.085	0.429	0.115	0.543
NUN (Ours)	PVT V2	0.077	0.770	0.035	0.732	0.048	0.815

Ablation Study¶

Ablations conducted with ResNet50 on COD10K under combined synthetic degradations demonstrate the specific efficacy of each architectural component, summarizing Tables 9 and 11 from the original paper.

Summary of Ablation Studies (Tables 9 & 11):

Configuration / Variant	M ↓	Fβ ↑	Eϕ ↑	Sα ↑	Note
SODUN- (w/o background branch)	0.102	0.328	0.603	0.583	Degrades to single-branch mask unfolding
SODUN full	0.093	0.362	0.619	0.592	Reversible dual-domain foreground-background estimation
SODUN + DeRUN (w/o BUI)	0.076	0.406	0.646	0.618	Inner blind degradation proximal unfolding added
Full NUN (Ours)	0.068	0.448	0.686	0.643	Complete nested unfolding with bilateral interaction
w/o $X_2(\cdot)$ (w/o semantic guidance)	0.071	0.430	0.671	0.633	Restoration lacks high-level segmentation priors
w/o IQA (w/o quality selection)	0.072	0.421	0.655	0.626	Suboptimal intermediate frames degrade convergence
w/o $\mathcal{L}_{csc}$ (w/o consistency loss)	0.071	0.428	0.665	0.633	Reduced robustness against inter-stage perturbation

Key Findings¶

Decoupled Optimization Eliminates Negative Transfer: Under combined degradation on COD10K, the coupled unfolding baseline RUN achieves an $F_\beta$ of 0.345, whereas NUN reaches 0.448 (+29.8% relative gain on ResNet50); scaling to PVT-V2 surges $F_\beta$ to 0.732 vs. 0.429, showing that disjoint parameter spaces prevent gradient cancellation under heavy corruption.
Mutual Synergy in Image Restoration: In restoration quality assessments (Table 12), coupled RUN achieves only 15.21 dB PSNR—worse than standalone DiffIR (18.63 dB)—due to task contention. NUN attains 22.37 dB PSNR, lowest FID (73.15), and highest user preference (3.92/5.0), proving that segmentation guidance genuinely assists low-level artifact removal.
Robust Real-World Generalization: Evaluated zero-shot on real-world low-quality subsets PCOD-LQ and MCOD-LQ with out-of-distribution artifacts (e.g., severe flare, snow), NUN achieves $F_\beta^{max}$ scores of 0.674 and 0.556, markedly outperforming FEDER (0.585 / 0.440) and RUN (0.653 / 0.490).

Highlights & Insights¶

First DUN-in-DUN Architecture: NUN establishes the nested deep unfolding paradigm in computer vision, mapping bi-level mathematical optimization into hierarchical network topologies to naturally disentangle distinct visual abstraction layers.
Bilateral Controlled Interaction: The BUI mechanism pairs IQA-driven top-frame forward selection with semantic mask backward conditioning, allowing each task to inform the other without mutual degradation.
Rigorous Error Bounds: The paper provides a formal degradation-sensitivity bound $\mathcal{O}(\delta_k + \delta_{res})$, proving that nested unfolding reduces inner optimization error exponentially with iterations while shielding downstream segmentation from direct corrupted feature exposure.

Limitations & Future Work¶

Theoretical Surrogate Formulation: The present mathematical guarantee relies on treating intermediate top restorations as fixed inputs per stage; establishing a fully rigorous convergence rate over the unconstrained joint bi-level mapping $\mathbf{X}^*(\mathbf{M}, \mathbf{B})$ remains an open problem.
Inference Latency: Executing multiple proximal gradient iterations within each stage ($4+3+3+2 = 12$ total inner forward passes) increases inference latency to 38–44 ms compared to 22 ms for single-stage forward models, posing hurdles for strictly real-time edge devices.
Future Directions: Exploring dynamic early-stopping criteria for the inner iteration count $N_k$ based on input degradation severity, and distilling large degradation priors (e.g., DA-CLIP) into compact convolutional operators.

vs RUN (ICML 2025): RUN pioneered reversible unfolding for COS but coupled restoration and segmentation within a shared parameter space and relied on pre-specified degradation matrices; NUN advances this to nested unfolding with disjoint parameter sets and learnable blind operators, improving degraded COD10K performance by over 20%.
vs Conventional COS (FEDER, FSEL): Existing feature-decomposition models implicitly assume clean imagery and collapse when ambient noise or fog obscures edges; NUN illustrates that integrating low-level physical unfolding is indispensable for real-world robustness.
vs Foundation Models (RobustSAM, SAM-Adapter): Despite heavy ViT-H parameterization, SAM variants struggle under heavy combined degradation (RobustSAM $F_\beta$ drops to 0.669); NUN combined with ViT-H reaches 0.806, highlighting that principled degradation unfolding provides orthogonal benefits to pre-trained foundation scales.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ Pioneering DUN-in-DUN nested formulation with elegant bi-level optimization mapping and theoretical degradation bounds.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Extensive evaluation across 12 clean and degraded benchmarks, synthetic and real OOD subsets, multiple vision domains, and backbones.
Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear narrative, disciplined mathematical formulation, tight architectural coupling, and comprehensive ablation evidence.
Value: ⭐⭐⭐⭐⭐ Establishes a foundational paradigm for degradation-robust vision applicable across autonomous driving in adverse weather, medical endoscopy, and concealed defect inspection.