Denoising-Enhanced Coarse-to-Fine Infrared Small Target Detection with Attention Prior-Guided Knowledge Distillation

Conference: ECCV2026
arXiv: 2606.21956
Code: None
Area: Object Detection
Keywords: Infrared Small Target Detection, Coarse-to-Fine Detection, Denoising-Assisted Training, Knowledge Distillation, Region Binary Classification

TL;DR

This paper proposes ECFNet, a coarse-to-fine infrared small target detection framework. In the coarse stage, RBCN reduces the full-image dense prediction to grid binary classification and introduces Denoising-Assisted Training (DAT) to force the network to learn target-background context. In the fine stage, a lightweight detector is utilized with Attention Prior-Guided Knowledge Distillation (APKD) to enable the student model to focus on key target regions, achieving SOTA performance on three infrared datasets with only 31.7G FLOPs.

Background & Motivation

Infrared small target detection is a critical technology in scenarios such as UAV surveillance and remote sensing. However, targets in infrared images contain very few pixels (typically only a few to dozens of pixels), possess a low signal-to-clutter ratio, and are accompanied by complex dynamic backgrounds (clouds, building edges, thermal clutter, etc.) that generate a large number of target-like distractors, resulting in frequent missed detections and false alarms in traditional methods. Existing methods are roughly divided into two categories: single-stage methods (such as UIU-Net, MSHNet) perform dense prediction on the entire image, but small targets only occupy a tiny fraction of the image, wasting significant computational resources on target-free background areas and limiting real-time processing of high-resolution images; two-stage methods (such as ESOD, QueryDet) first generate target mask proposals through segmentation and then perform sparse detection, but the dense segmentation stage is still computationally heavy, and mask-level proposal cropping discards critical contextual information. For infrared small targets, context is precisely the key clue to distinguish targets from background halos; discarding context leads to a significant increase in false alarm rates.

The key challenge lies in the fact that infrared small targets require context to assist in discrimination (due to their extremely weak intrinsic features) while simultaneously needing to avoid redundant computations across the entire image. Meanwhile, the lightweight detector in the two-stage framework has limited capability, and conventional knowledge distillation methods only perform alignment at the global feature level without imposing explicit attention guidance on small target regions, leading to limited benefits from distillation.

The proposed ECFNet addresses these problems from two aspects simultaneously. In the coarse stage, the full-image dense pixel-level classification problem is downsampled to grid-level region binary classification (RBCN), which only determines whether each grid contains a target, completely avoiding the overhead of dense convolution. Concurrently, a Denoising-Assisted Training strategy (DAT) is designed: during training, Gaussian target-like noise is injected into the GT mask, merging noisy masks into RBCN's feature space, forcing the network to reconstruct the original clean mask through a denoising task. Only by truly understanding the relationship between the target and its surrounding background can the network correctly isolate the real target from noise. In the fine stage, ExpSlicer extracts local region crops retaining full context from the multi-scale feature maps of RBCN and feeds them into a lightweight detector. Core Idea: Decompose infrared small target detection into a two-stage process of "coarse region selection -> fine localization". The coarse stage replaces full-image dense prediction with grid binary classification to reduce computation and utilizes denoising-assisted training to force the network to model target-background context to improve proposal quality. The fine stage uses cross-attention knowledge distillation to focus the lightweight detector on key target regions.

Method

Overall Architecture

ECFNet is a coarse-to-fine two-stage infrared small target detection framework. The core of the coarse stage is RBCN (Region Binary Classification Network), which divides the input image into 20×20 non-overlapping grids and outputs a probability value for each grid representing whether it contains a target. During training, RBCN is additionally connected to the DAT module: Gaussian target-like noise is injected into the GT mask and concatenated with the multi-scale feature maps of RBCN, forcing the network to reconstruct the original GT mask through a denoising task, thereby embedding target-background context information into the feature learning stage. The binary probability map output by RBCN is binarized, and ExpSlicer crops 3×3 grid-range region blocks centered on each positive grid from the multi-scale feature maps (using fixed size and padding to handle boundary targets), preserving complete context. The fine stage inputs these region blocks into a lightweight detector (a student model distilled from a YOLOv12-L teacher model via APKD) for precise box-level or pixel-level localization, mapping the results back to the original image.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Infrared Image Input"] --> B["RBCN: Grid Binary Classification<br/>20×20 Grids → Probability Map"]
    B --> C["ExpSlicer:<br/>Center on Positive Grids<br/>Crop 3×3 Region Blocks"]
    C --> D["Lightweight Detector<br/>(YOLOv12 Student)"]
    D --> E["Local Localization Results<br/>Mapped Back to Original Image"]
    B -.->|Connected during training| F["DAT: Inject Target-like Noise<br/>Reconstruct GT Mask<br/>Learn Target-Background Context"]
    D -.->|Distillation Stage| G["APKD: Cross-Attention<br/>Teacher Focus Region → Student Feature Modulation"]

Key Designs

1. RBCN: Downsampling Full-Image Dense Pixel Classification to Grid-Level Region Binary Classification

Conventional two-stage methods usually perform pixel-level semantic segmentation in the first stage to generate target masks before conducting sparse detection. Pixel-level classification requires making predictions at every single pixel, which leads to huge computational overhead, and the segmentation mask itself discards context—the mask only represents "target/non-target" without containing any background information surrounding the target. RBCN reformulates this problem by partitioning the input image into \(N \times N\) non-overlapping grids (experimentally set to \(N=20\)) and outputting a binary prediction for each grid indicating whether it contains a target region. This transforms the classification problem of \(H \times W\) pixels into that of \(N^2\) grids, reducing the computational cost by several orders of magnitude. RBCN employs a 4-stage hierarchical backbone, where stages 2-4 utilize C2fP modules (a fusion of C2f-style dual 3×3 convolutional residual paths and PConv residual paths), balancing directional awareness and spatial modeling capability. The output probability map is binarized by a threshold (0.5) to produce region proposals. The supervision signal of RBCN is jointly optimized by BCE Loss + IoU Loss.

2. DAT: Injecting Target-like Noise into Feature Space to Force Contextual Learning via Denoising

Infrared small targets and background clutter are highly similar spectrally, making them extremely difficult to distinguish solely based on the weak features of the target itself. The core concept of DAT is to adversarially increase the training difficulty: on the multi-scale feature maps of RBCN, a set of target-like noises synthesized using a 2D Gaussian function \(N_i^l(x,y)=\exp(-((x-x_i)^2+(y-y_i)^2)/2\sigma^2)\) is element-wise superimposed on the GT mask. The noisy mask is then concatenated along the channel dimension with the RBCN feature maps and fed into an auxiliary branch to reconstruct the original GT mask. The strength (where \(\sigma\) controls the spatial spread) and amount of noise are designed to simulate target-like clutter in real infrared images. Since real targets and synthesized noise are highly similar in local features, the network will inevitably misjudge if it only relies on isolated pixel-level responses (i.e., treating any high brightness as a target). It must learn to utilize the surrounding larger context to judge "whether this highlight is a target or noise"—which is precisely the capability most needed for infrared small target detection. The DAT loss is computed in parallel with the main RBCN loss (summing BCE + IoU Loss across three scales) and is only used during training, incurring zero overhead during inference. Notably, simply concatenating the GT mask onto the feature maps degrades performance (weakening the backbone's capacity), whereas adding noise significantly improves the CRP and CRR in the coarse stage.

3. ExpSlicer: Dynamic Region Slicer Preserving Complete Context

After RBCN outputs the binary grid, corresponding regions must be extracted from the original feature maps and passed to the fine-stage detector. A naive approach is to crop only the positive grid itself, but this can truncate incomplete targets (especially when a target spans multiple grids) and discards surrounding context. ExpSlicer's strategy is to crop a \(k \times k\) (experimentally set to \(k=3\), i.e., a 3×3 grid region) rectangular area centered on the corresponding location of each positive grid on the multi-scale feature maps. Zero padding is applied to boundary targets to maintain consistent dimensions. All cropped region blocks are stacked along the batch dimension and fed into the fine detector, while global non-maximum suppression is employed to eliminate redundant predictions arising from overlapping areas. ExpSlicer offers two key advantages: first, the 3×3 region guarantees that the target is completely enclosed (even if the target is covered by a positive grid of RBCN, the surrounding 8 neighbor grids provide complete context and tolerance for over-segmentation); second, \(k\) is dynamically adjustable, providing much higher flexibility than fixed-mask cropping. Additionally, to keep the batch size constant during training despite varying numbers of positive grids per image, the proposal batch size is set to the average number of positive grids +1, ensuring over 50% of images participate fully in training.

4. APKD: Transferring Teacher's Focus on Small Target Regions via Cross-Attention in Feature Space

The fine stage uses a lightweight detector (distilled from a YOLOv12-L teacher), but lightweight models have limited representation capability, and conventional global feature distillation methods (e.g., directly aligning student and teacher feature maps) fail to pay sufficient attention to small target regions. The breakthrough of APKD is that it avoids 1D token flattening (which destroys spatial locality) and instead computes teacher-student cross-attention directly on multi-dimensional spatial features. Specifically, the student feature maps are grouped and downsampled via 3×3 convolutions to serve as the PixelQuery (retaining local spatial details), and the teacher feature maps are processed as the AreaKey (retaining global scene understanding). The cross-attention weights \(A = \text{Softmax}(\text{PixelQuery}^T \cdot \text{AreaKey})\) are calculated and used to modulate student features, producing a dynamic convolution kernel \(W_{\text{dyn}}\). This modulation process amplifies student features in regions corresponding to teacher focus while suppressing background areas. Finally, the modulated features are fused and output via RepConv residual gating. The distillation loss is a weighted sum of cosine similarity between teacher and student features across various scales. The key insight of APKD is: the teacher's biggest strength is not telling the student how to "align globally", but rather telling the student "where to look"—cross-attention naturally achieves this spatial prior transfer, and during inference, the teacher is frozen or discarded, adding no extra latency.

Loss & Training

Coarse-stage joint loss: \(\mathcal{L}_{\text{coarse}} = \mathcal{L}_{\text{BCE}}(M_{\text{pred}}, M_{\text{GT}}) + \lambda_1 \cdot \mathcal{L}_{\text{IoU}}(M_{\text{pred}}, M_{\text{GT}})\), where \(\lambda_1\) is a learnable weight (initially 1.0).
DAT loss: \(\mathcal{L}_{\text{DAT}} = \sum_{l=1}^3 (\mathcal{L}_{\text{BCE}}(M_{\text{DAT}}^l, M_{\text{GT}}^l) + \lambda_2 \cdot \mathcal{L}_{\text{IoU}}(M_{\text{DAT}}^l, M_{\text{GT}}^l))\).
Distillation loss: \(\mathcal{L}_{\text{KD}} = \sum_{i=1}^L w_i \cdot \mathcal{D}_{\cos}(F_s^i, F_t^i)\), where \(w_i = 1/i\).
The coarse and fine stages are trained separately for 100 epochs using the Adam optimizer with \(\text{lr} = 0.01\), weight decay = 0.1, and batch size = 12. Data augmentations include random cropping, horizontal flipping, and brightness adjustments. The teacher model (YOLOv12-L) is pre-trained using the same data and strategy, and is frozen during distillation to only provide supervision signals.

Key Experimental Results

Main Results

Dataset	Metric	Ours	Prev. SOTA	Gain
UAV (Object)	AP50	95.87	95.39 (YOLO11-L)	+0.48
Car (Object)	AP50	96.06	93.68 (D-FINE-L)	+2.38
IRSTD-1k (Object)	AP50	89.23	84.32 (EFLNet)	+4.91
UAV (Pixel)	nIoU	56.11	54.23 (PConv)	+1.88
Car (Pixel)	nIoU	58.21	55.98 (PConv)	+2.23
IRSTD-1k (Pixel)	nIoU	69.67	67.93 (PConv)	+1.74

With FLOPs = 31.7G, Params = 22.7M, and FPS = 90, the proposed method significantly outperforms existing SOTA methods across all three metrics. Compared to ESOD-L (38.1G, 86 FPS), it has faster inference speed and higher accuracy; compared to D-FINE-L (90.7G, 40 FPS), the computational cost is reduced by approximately 65%. Notably, FLOPs do not directly correspond to FPS—general detectors (such as D-FINE), although having low FLOPs, incorporate components like multi-scale fusion that lead to high actual latency, whereas infrared-specific detectors are optimized for hardware-friendliness.

Ablation Study

Configuration	AP50	Description
Baseline	87.26	Without DAT, without APKD
+APKD	89.30	+2.10, adding distillation alone
+DAT	92.71	+5.45, adding denoising training alone
+DAT+APKD	95.87	+8.61, mutual complementarity achieves optimal performance

Configuration	CRP	CRR	Description
W/o DAT	89.38	93.45	Coarse stage baseline
+Loss	92.80	94.35	Computing Loss only on feature maps
+GT	92.62	94.11	Directly concatenating GT mask
+Noise (DAT)	94.13	95.69	Reconstructing after noise injection

Distillation Method	P	R	AP50
W/o Distillation	93.45	90.32	92.71
Plain KD	95.04	85.72	91.15
CrossKD	93.82	84.14	87.70
DCSF	95.82	86.73	89.29
APKD (Ours)	95.96	91.63	95.87

Key Findings

• DAT is the largest contributor to performance (+5.45 AP50). Its core lies in the injected noise forcing the network to learn target-background context, rather than simple feature augmentation. Comparative experiments show that directly concatenating the GT mask degrades the results instead, as it weakens the learning pressure on the backbone.
• APKD shows significant effectiveness on the lightweight detector and is robust to the choice of teacher model—a teacher (PConv vs YOLOv12-L) with a 7-fold difference in complexity still injects over 2.15 AP50 gain. A stronger teacher (YOLOv12-L) yields greater benefits.
• The 20×20 grid resolution of RBCN is the optimal balance point: 10×10 resolution is too low and leads to insufficient context, while 40×40 resolution is too high, causing individual grids to contain too little information, thereby losing the context required for discrimination.
• The C2fP module achieves the highest CRR (95.69%) in the coarse stage. Even if its CRP is slightly lower than SwinTransformer, CRR is the key metric in the coarse stage—it is preferable to allow a few false alarms (which the fine stage can filter out) than to miss any target regions.

Highlights & Insights

• The design of Denoising-Assisted Training (DAT) is incredibly clever. It does not introduce noise in the image domain to perform denoising; instead, it concatenates target-like noise onto the feature maps in the feature domain and drives contextual learning through the denoising objective. This "adversarial learning pressure" concept is more elegant than simply scaling up models or using more data to boost discriminative capacity, and it introduces absolutely zero inference overhead once training is complete.
• Replacing pixel-level segmentation with "grid binary classification" in RBCN is an excellent design fit for infrared small target detection. Small targets are inherently sparse, making full-image dense prediction highly inefficient; reducing the problem to the grid level makes the computation almost negligible while naturally preserving the context information within each grid.
• APKD's cross-attention distillation avoids the common 1D token flattening, preserving spatial locality—which is paramount for position-sensitive tasks like infrared small target detection. "Telling the student where to look" matches the small target scenario much better than "making the student replicate the teacher's feature maps", because global feature map alignment exerts almost zero bias on small target regions.
• The overall coarse-to-fine two-stage design is highly transferable to other small target or large-resolution detection tasks (e.g., aerial image detection, remote sensing target detection, microscopic image detection), as long as the targets occupy a very small ratio of the image compared to the background.

Limitations & Future Work

• The coarse stage relies on a fixed grid resolution (20×20), which may lack flexibility in scenarios with massive target scale variations. Although ExpSlicer utilizes multi-scale feature maps, the grid resolution itself is a fixed hyperparameter, and different scenarios may require different values of \(N\).
• The proposed method strongly depends on the specific target-like noise distribution of infrared data (2D Gaussian synthesized noise). When transferring to visible light or other bands, the noise modeling method needs to be redesigned and cannot be directly applied.
• DAT is only introduced during training, making it highly dependent on the diversity of the training data. If the target-background relationships in the training set are overly simplistic (e.g., all targets are against a sky background), the contextual discriminative power learned by DAT may lack generalization.
• The design of the lightweight detector in the fine stage depends on the YOLOv12 architecture. Future work could explore adapting more general detection heads (such as DETR-based lightweight variants) to enhance compatibility with different detection paradigms.

Related Work & Insights

• vs ESOD (Two-Stage Small Target Detection): ESOD uses depthwise separable convolutions for dense prediction to generate masks and then performs sparse detection. ECFNet replaces dense prediction with grid binary classification, yielding superior computational efficiency (38.1G → 31.7G) and comprehensive performance gains (AP50 78.33 → 89.23 on IRSTD-1k). The contextual modeling powered by DAT is the key differentiator.
• vs DN-DETR (Denoising Queries): DN-DETR introduces denoising queries into Transformer detectors to stabilize training bipartite matching. In contrast, DAT injects target-like noise in the feature domain of the coarse stage in a two-stage framework to force the network to learn target-background discrimination, which fundamentally addresses a different problem.
• vs Conventional KD (e.g., FitNets, CrossKD): Conventional KD aligns global feature maps or detector head outputs, paying insufficient attention to small targets. APKD's cross-attention mechanism explicitly passes the teacher's attention prior on critical regions, producing much greater gains on small targets than traditional distillation methods.

Rating

• Novelty: ⭐⭐⭐⭐☆ The combination of DAT and grid binary classification is highly novel in infrared small target detection, though coarse-to-fine frameworks themselves are not entirely new.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Evaluated on three real infrared datasets with extensive ablation studies (DAT variants, grid resolutions, distillation methods, different teachers, etc.). Both the main text and appendix provide complete and logical ablation analyses.
• Writing Quality: ⭐⭐⭐⭐☆ The method is clearly described with complete charts and figures, although the Introduction is slightly long and some mathematical notations could be streamlined.
• Value: ⭐⭐⭐⭐⭐ Both DAT and APKD modules have clear motivations and reusable design concepts. The method is highly efficient (31.7G, 90 FPS), ideal for embedded deployment, and practically advances the field of infrared/small target detection.

Related Papers

• [CVPR 2026] Target-Aware Invertible Encoder with Reconstruction Guidance for Infrared Small Target Detection
• [CVPR 2025] Small Target Detection Based on Mask-Enhanced Attention Fusion of Visible and Infrared Remote Sensing Images
• [NeurIPS 2025] Rethinking Evaluation of Infrared Small Target Detection
• [CVPR 2026] CHAL: Causal-guided Hierarchical Anomaly-aware Learning for Moving Infrared Small Target Detection
• [CVPR 2026] Seeing Through the Noise: Improving Infrared Small Target Detection and Segmentation from Noise Suppression Perspective