Rethinking IRSTD: Single-Point Supervision Guided Encoder-only Framework is Enough for Infrared Small Target Detection

Conference: ECCV2026
Paper: ECCV Paper
Project: https://nirixiang.github.io/SPIRE-IRSTD/
Code: https://github.com/NIRIXIANG/SPIRE-IRSTD
Area: Object Detection
Keywords: infrared small target detection, single-point supervision, probabilistic response modeling, high-resolution encoder, peak localization

TL;DR

SPIRE replaces contour segmentation with centroid response regression, using single points to construct structured local supervision, a lightweight encoder-only network to predict heatmaps, and direct peak extraction to achieve 97.05% F1 on SIRST-UAVB with approximately 0.29M parameters.

Background & Motivation

Infrared small targets typically occupy only a few pixels, lack recognizable texture and category semantics, and are affected by the point spread function, sensor sampling, and clutter. A target can consequently have a relatively clear intensity center without an equally clear pixel boundary. Mainstream methods nevertheless treat it as segmentation: humans annotate masks, an encoder compresses the image, a decoder reconstructs spatial detail, and another step derives positions from predicted regions. When the required output is a centroid, this pipeline asks annotators to delineate uncertain boundaries and asks the network to reconstruct contours that may not be useful downstream. The paper cites MCLC's annotation study, which reports 11.4 seconds for pixel-wise annotation versus 1.4 seconds for a single point; this is a prior result, not a new timing experiment by SPIRE.

Replacing mask labels with points does not automatically solve the learning problem. An isolated positive pixel in a large image supplies sparse gradients that can be overwhelmed by background pixels. LESPS and MCLC expand supervision through point-derived pseudo masks, but their networks still learn segmentation regions and retain encoder-decoder complexity. Meanwhile, aggressive downsampling can erase weak local peaks, and subsequent multi-scale fusion or attention cannot necessarily restore the lost localization cues. The paper therefore changes both the supervision target and the architecture instead of simply compressing an existing segmentation model.

Its starting point is local energy diffusion: an infrared point target has a stronger center and decaying neighboring responses, with the decay also shaped by observed local contrast. Expanding a point into this response allows neighboring pixels to contribute to learning while keeping the prediction focused on localization rather than contour recovery. This shares a foundation with keypoint heatmap regression but additionally incorporates local radiometric structure into target construction. The relevant assumption is that centroid localization is the objective; the paper does not establish that segmentation is unnecessary for every infrared task. Core Idea: expand sparse centroid annotations into learnable local heatmaps through point-response priors, then preserve and predict their peaks with a high-resolution encoder-only network, aligning supervision, architecture, and localization.

Method

Overall Architecture

SPIRE takes a single-channel infrared image; training additionally requires one centroid annotation per target, whereas inference requires only the image. Point-Response Prior Supervision (PRPS) constructs training response maps offline, and the High-Resolution Probabilistic Encoder (HRPE) predicts a single-channel response on the same lattice. Lightweight Peak Inference converts that response into a set of centroid coordinates in the original image, rather than boxes or segmentation masks. The default output stride is \(s=4\), and intermediate features retain a single spatial resolution after initial downsampling. PRPS constructs training targets; it is not an additional detector requiring annotated centroids at inference time.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Training image and centroids"] --> P["Point-Response Prior Supervision<br/>PRPS"]
    I["Infrared image"] --> E["High-Resolution Probabilistic Encoder<br/>HRPE"]
    P -.->|Training only: response supervision| R["Predicted response map"]
    E --> R
    R --> D["Lightweight Peak Inference"]
    D --> O["Original-image centroid set"]

The dashed edge denotes supervision, not the delivery of ground-truth centers or response maps to the inference network. The overall pipeline appears in Figure 3 on page 6; the three designs correspond to Sections 3.2, 3.3, and 3.4.

Key Designs

1. Point-Response Prior Supervision: turn one centroid into spatially structured learning signals

PRPS first maps annotated centroids to the heatmap lattice according to the output stride and further aligns them to a radiometric peak using the maximum-intensity position in a local original-image neighborhood. This adjustment addresses a possible offset between the annotated center and the strongest observed response, rather than estimating the target boundary. It then constructs an isotropic Gaussian prior with unit peak around the center and truncates it outside a finite neighborhood. The default settings are \(\sigma=2\) and \(r=3\sigma=6\), giving supervision compact support rather than extending it across the background. The kernel is not normalized to sum to one; the term probabilistic response is closer to a localization confidence heatmap than a distribution whose probabilities sum to one across pixels. The Gaussian encodes a center-enhanced, outward-decaying imaging prior, not a claim that every target has an exactly Gaussian shape.

A pure Gaussian assigns equal weight to different pixels at the same radius, ignoring observed local intensity variations. PRPS therefore also extracts a local \((2r+1)\times(2r+1)\) image patch and normalizes its contrast using its minimum and maximum intensities, producing a modulation term \(C_k\). For target \(k\), it multiplies the Gaussian term \(G_k\) and the contrast term elementwise, then normalizes the result to \([0,1]\). The prose accompanying Equation (5) supports the following expression, where \(\odot\) denotes elementwise multiplication:

\[ H_k=\operatorname{Norm}(G_k\odot C_k). \]

Locations close to the center and consistent with local radiometric structure receive stronger supervision, rather than simply labeling a fixed disk as foreground. Individual responses are combined into the full supervision map through deterministic aggregation; the readable text does not specify whether this is a maximum, sum, or another operator, so that choice cannot be supplied on the authors' behalf. Overlapping support regions do not necessarily merge targets, but the aggregated response must still preserve separable local maxima. The design thus addresses sparse gradients and mismatched responses rather than mathematically guaranteeing separation at arbitrary target distances. The impulse, pure-Gaussian, and PRPS comparisons in Table 3 test the effects of expanding effective supervision and incorporating local contrast, respectively.

2. High-Resolution Probabilistic Encoder: retain weak peaks at one scale instead of losing and reconstructing them

HRPE starts with two \(3\times3\) stride-2 convolutions, each followed by batch normalization and ReLU, giving an overall stride of 4. Channels initially expand to 64, pass through a bottleneck residual block and lightweight channel reorganization units, and are reduced to 32 through a transition layer. Subsequent processing cascades local feature operations on this lattice without adding lower-resolution branches or a decoder. For targets spanning only a few pixels, retaining spatial continuity reduces damage to centers and neighboring responses from repeated downsampling. High resolution is relative to heavily downsampled deep backbones, not computation at the original image resolution throughout.

Each channel reorganization unit splits channels evenly: one half remains an identity branch, while the other receives lightweight processing such as depthwise convolution and channel reweighting. The processed branch uses squeeze-and-excitation-style reweighting, so the network is not entirely attention-free. The branches are concatenated and channel-shuffled to exchange information across groups in subsequent units; some units add another depthwise convolution for local spatial refinement. This preserves a direct signal path while allocating limited computation to useful local response changes rather than repeated cross-scale fusion. Encoder-only means no segmentation decoder or cross-scale skip pathways, not the absence of residual or identity connections inside individual units.

A final \(1\times1\) convolution maps the 32 channels to one response channel, with no output activation. This supports direct regression, but predictions are not inherently constrained to \([0,1]\) and should not be described as calibrated probabilities. Table 4 shows that increasing stride from 4 to 8 substantially reduces recall, while decreasing it to 2 raises computation without improving F1. This supports the stride choice for the evaluated task and configuration, not a universal rule that higher resolution is always better.

3. Lightweight Peak Inference: convert response maxima directly into centroid coordinates

Inference first applies \(3\times3\) stride-1 max pooling to the predicted heatmap and retains positions equal to their neighborhood maximum, implementing local non-maximum suppression. A confidence threshold \(\tau\) removes low responses, and response-ranked selection retains at most \(N_{\max}\) candidates. The authors set \(N_{\max}\) to each dataset's maximum target count and report that increasing it severalfold leaves metrics unchanged because the threshold removes additional low-scoring peaks. This is not an unconditional guarantee for dense scenes: deployment on an unknown target distribution still requires a suitable candidate limit. The text claims a broad stable threshold range but provides neither a complete threshold sweep in the readable experimental tables nor an explicit default value.

Integer-grid peaks are then refined to subpixel positions using the response map before NMS. The correction uses the signs of finite differences between horizontal and vertical neighbors, moving toward the stronger response; the text specifies a correction scale of \(1/s\), or \(1/4\) by default. An inverse affine transformation finally maps refined heatmap positions to original-image coordinates. Equations (11) and (12) are damaged in the extracted text, so this note explains the readable mechanism without presenting guessed complete expressions as the authors' equations. The procedure requires neither mask connected-component clustering nor another learned localization head, but it still includes peak filtering and coordinate transforms and is not post-processing-free.

A Worked Example

For the experimental \(640\times640\) single-channel input, stride 4 corresponds to a \(160\times160\) response lattice. During training, PRPS turns a target point into a local decaying response; HRPE predicts the complete heatmap from the image and compares it with that supervision. With the default \(r=6\), the local window has side length 13; this is a response-construction neighborhood, not the target's physical width or a predicted box size. At test time, the annotated point is absent and annotation-dependent PRPS is not run. Predicted peaks undergo local-maximum selection, thresholding, subpixel refinement, and coordinate restoration. Two neighboring targets can each be returned if they retain separable peaks; when they produce one merged peak, the inference procedure has no explicit secondary splitting stage. This example illustrates dimensions and data flow, not an additional per-image detection result reported by the paper.

Loss & Training

Section 3.2 specifies standard pixel-wise regression between predicted responses and PRPS targets but does not identify its exact norm or weighting formula in the readable text. MSE, BCE, and focal loss therefore cannot be presented as confirmed implementation choices. The authors argue that structured local supervision supplies sufficient gradients without a specialized imbalance-aware loss; this is a methodological explanation, not an experiment measuring gradient statistics. The implementation uses PyTorch on an RTX 4090, Adam with an initial learning rate of 0.005, batch size 10, and 500 epochs. The scheduler is ReduceLROnPlateau with factor 0.01 and patience 3; competing methods follow their original schedules rather than a common epoch budget. Input resolution is \(640\times640\) for SIRST-UAVB and \(512\times512\) for SIRST4. Coordinate-mapping and Gaussian equations are also damaged in the extraction, so exact sampling alignment between original-image patches and heatmap coordinates requires code verification rather than extrapolation from the prose.

Key Experimental Results

Main Results

SIRST-UAVB contains 3000 images with a 2400/600 train/test split; SIRST4 contains 3352 images with a 2285/1067 split. Evaluation is unified at the centroid level: segmentation outputs use 8-connected clustering, box detectors use box centers, and SPIRE uses response peaks. The default matching tolerance is \(\delta=5\) original-image pixels; Recall equals probability of detection \(P_d\), and F1 is the harmonic mean of Precision and Recall. The paper defines false alarm rate as \(F_a=\mathrm{FP}/N_{\mathrm{pixels}}\), where FP counts incorrect centroid detections, not incorrectly segmented foreground pixels.

The following selection comes from Table 1 on page 13; Precision, Recall, and F1 are percentages, while \(F_a\) values are reported in units of \(10^{-8}\). Parameters and FLOPs are computed at \(640\times640\), so the FLOPs column should not be interpreted as SIRST4's actual \(512\times512\) computation.

Dataset	Method	Precision	Recall	F1	\(F_a\)	FLOPs (G)	Params (M)
SIRST-UAVB	DNANet	94.48	89.54	91.95	15.77	89.13	4.70
SIRST-UAVB	SCTransNet	98.27	95.95	97.09	5.09	63.22	11.19
SIRST-UAVB	SPIRE	99.82	94.44	97.05	1.02	7.68	0.29
SIRST4	DNANet	93.99	81.20	87.13	29.82	89.13	4.70
SIRST4	SCTransNet	81.20	86.39	83.72	114.98	63.22	11.19
SIRST4	SPIRE	95.00	94.21	94.60	28.53	7.68	0.29

On SIRST-UAVB, SPIRE's main strengths are precision and false-alarm control; its F1 is 0.04 percentage points below SCTransNet, so it does not lead every metric on that dataset. Its 94.60% F1 on SIRST4 is the highest among methods in Table 1, but this is a centroid-level result and does not establish superior contour segmentation. Table 2 on page 13 additionally reports 261.2 FPS for SPIRE and 80.86 FPS for DNANet under the paper's same-input, same-hardware setting; these are reported measurements, not end-to-end throughput remeasured for this note.

Ablation Study

The following selection from Table 3 on page 14 uses SIRST-UAVB; the first three rows compare supervision forms, and the remaining rows compare scales against default PRPS. Precision, Recall, and F1 are percentages, and \(F_a\) remains in units of \(10^{-8}\).

Supervision configuration	Precision	Recall	F1	\(F_a\)
PRPS, \(\sigma=2,r=6\)	99.82	94.44	97.05	1.02
Single-point impulse	98.71	90.22	94.27	2.85
Unconstrained Gaussian	98.93	93.76	96.28	2.44
PRPS, \(\sigma=1,r=3\)	97.60	88.60	92.87	5.26
PRPS, \(\sigma=3,r=9\)	99.70	92.50	95.99	0.67

PRPS raises F1 by 2.78 percentage points over impulse supervision and by 0.77 percentage points over unconstrained Gaussian supervision. Increasing \(\sigma\) to 3 reduces false alarms to 0.67 but sacrifices recall, showing that broader supervision is not uniformly beneficial.

The next selection comes from Table 4 on page 14, again on SIRST-UAVB, with FLOPs measured for \(640\times640\) inputs. The parameter column is omitted to avoid unit confusion: the source uses \(10^{-2}\)M in Table 4 but M in Table 1.

Encoder configuration	Recall (%)	F1 (%)	\(F_a\) (\(10^{-8}\))	FLOPs (G)
HRPE, \(s=4\)	94.44	97.05	1.02	7.68
HRPE, \(s=2\)	91.23	95.25	0.81	26.77
HRPE, \(s=8\)	83.98	86.91	22.38	3.05
Without channel reorganization	93.90	95.76	5.33	7.70
Without reweighting	93.71	95.09	8.15	7.72

Key Findings

Increasing stride from 4 to 8 lowers F1 from 97.05% to 86.91%, showing the cost of reducing computation by damaging small-target responses. Removing channel reorganization or reweighting changes recall relatively little but substantially increases false alarms, supporting their role in clutter discrimination. In Table 2, relaxing the matching tolerance from 3 to 5 pixels raises SPIRE Recall from 92.93% to 94.21%, but also raises \(F_a\) from 18.61 to 28.53 and lowers F1 from 94.74% to 94.60%. For fixed predictions and conventional matching, false alarms increasing with a relaxed tolerance require further explanation; this note preserves the source values without correcting them or treating the table as unambiguous robustness evidence.

Highlights & Insights

The most useful change is to establish what the downstream task requires: when only localization matters, supervision can provide dense learning signals without requiring dense semantic-region outputs. PRPS combines a spatial prior with image contrast, illustrating that sparse labels need not imply isolated-impulse supervision. HRPE simplification and supervision redesign work together; the results cannot be attributed solely to removing the decoder or generalized to arbitrary IRSTD networks with their decoders deleted. Other point-localization tasks may benefit from compact response supervision and direct peak outputs, but their response kernels and intensity priors need independent validation rather than automatic reuse of infrared assumptions.

Limitations & Future Work

The paper has no dedicated limitations section; the following points are primarily reader assessments of the method and experimental scope. Experiments cover only two centroid-localization benchmarks and do not establish contour, size, or category recovery, cross-sensor transfer, or latency on real edge hardware. The main table lacks direct results for LESPS, MCLC, FromEasy2Hard, and P2P-HDNet, leaving the comparison with existing single-point and point-to-point approaches incomplete. Local maximum-intensity alignment may be attracted to strong nearby clutter, while fixed Gaussian scale and isotropy may poorly fit elongated or severely degraded targets; controlled tests are needed for these conditions. Closely spaced targets still require separable peaks, and the paper provides no separation curve as a function of centroid distance. Aggregation, the exact regression loss, the default threshold, and coordinate sampling require code checks; the matching statistics in Table 2 also need original predictions or evaluation-code evidence. Useful next tests include annotation perturbation, target spacing, sensor changes, threshold sweeps, and repeated-run variance to distinguish modeling benefits from setting sensitivity.

Related Work & Insights

Relation to LESPS, MCLC, and FromEasy2Hard: these methods also reduce annotation cost but largely retain segmentation through pseudo masks or progressively refined supervision; SPIRE changes the prediction target itself to centroid responses. Relation to P2P-HDNet: point-to-point heatmap localization is not introduced here; the paper differentiates its local imaging prior, reduced feature fusion, and encoder-only implementation, so novelty should be assessed at that combination. Relation to HRNet and HigherHRNet: high-resolution keypoint representations provide background, but SPIRE uses a single fixed-stride branch rather than directly adopting their multi-resolution architectures. Relation to ShuffleNet: grouped processing and channel shuffle are established efficient-network ideas, used here to preserve and exchange local response information rather than introduced as new operators.

Rating

• Novelty: 4/5. Supervision, architecture, and centroid localization are closely aligned, although heatmap regression and channel reorganization have clear precedents.
• Experimental Thoroughness: 3/5. Two benchmarks and supervision, scale, and architecture ablations are useful, but direct single-point comparisons and some reproducibility details are missing.
• Writing Quality: 3/5. The motivation is clear, but Table 2 raises statistical questions and the available text extraction contains damaged equations.
• Value: 4/5. Useful for low-compute infrared detection requiring only centroids, without implying applicability to every segmentation requirement.