Remedying Target-Domain Astigmatism for Cross-Domain Few-Shot Object Detection¶

Conference: CVPR 2026 arXiv: 2603.18541 Code: N/A Area: Object Detection Keywords: cross-domain few-shot detection, attention astigmatism, bio-inspired foveal vision, prototype learning, negative context modeling

TL;DR¶

This paper is the first to identify an "astigmatism" phenomenon in cross-domain few-shot object detection (CD-FSOD), wherein model attention remains persistently diffuse in the target domain. Inspired by the human foveal visual system, the authors design three complementary modules — Positive Pattern Refinement (PPR), Negative Context Modulation (NCM), and Text Semantic Alignment (TSA) — to reshape attention, achieving state-of-the-art performance with significant margins across six cross-domain benchmarks.

Background & Motivation¶

Background: CD-FSOD aims to adapt source-domain pre-trained detectors to target domains with scarce annotations, addressing critical demands in practical applications such as medical diagnosis and industrial inspection. Existing methods such as CD-ViTO have established multi-domain benchmarks, yet performance remains unsatisfactory.

Limitations of Prior Work: Through in-depth analysis of attention patterns across Transformer layers, the authors identify a previously overlooked phenomenon: in the source domain, attention progressively concentrates on foreground objects as network depth increases, whereas in the target domain, attention remains consistently diffuse and unfocused, leading to oversized bounding boxes and abundant redundant predictions. This mirrors human astigmatism — the model fails to focus on key objects in the target domain.

Key Challenge: By measuring attention distance \(\bar{d} = \frac{1}{N}\sum_{i,j} A_{ij} \cdot \|p_i - p_j\|\), the authors find that attention distance in the target domain consistently exceeds that in the source domain. Although standard fine-tuning shows a trend toward reducing defocus (negative difference in attention distance), the effect is far insufficient — target-domain attention dispersion after fine-tuning still greatly exceeds that of the source domain.

Goal: To enhance the model's intrinsic tendency to remedy astigmatism, shifting attention from a diffuse pattern to a focused, object-centric pattern, thereby enabling precise detection in the target domain.

Key Insight: Inspired by the foveal structure of the human visual system — where the central region captures high-detail foreground information and the peripheral region captures low-detail background information, with a center-surround contrast mechanism maintaining attentional focus — the authors design three modules to respectively enhance central region representations, peripheral region representations, and the discriminability between the two.

Core Idea: Foreground-region attention is enhanced via class-specific foreground prototypes (increasing \(A_2, A_3\)); spurious foreground responses in background regions are suppressed via a unified background prototype (decreasing \(A_1\)); and foreground-background separation is further reinforced from a cross-modal perspective using "not [class]" text cues. Together, these three strategies convert diffuse attention into a focused pattern.

Method¶

Overall Architecture¶

Built upon the GLIP detector with a Swin Transformer visual backbone for multi-scale feature extraction. During fine-tuning, class-specific foreground prototypes and a unified background prototype are extracted from support samples and stored in a prototype memory. The model is jointly optimized with detection and cross-modal alignment losses. At inference, prototypes are retrieved from the memory; PPR enhances foreground features and NCM enhances background features, and their complementary outputs are fused before being passed to the detection head.

Key Designs¶

Positive Pattern Refinement (PPR) Module:
- Function: Enhances foreground region feature representations using class-specific prototypes, simulating the central perception region of human foveal vision.
- Mechanism: Class prototypes \(\mathbf{p}_{fg}^c\) are obtained via mean pooling over foreground regions of support samples. During inference, cosine similarity between each position of the feature map and all class prototypes is computed; a foreground mask \(\mathbf{M}_{fg}\) is generated via threshold \(\tau_{fg}\), and features in high-similarity regions are enhanced using temperature-scaled softmax-weighted prototypes: \(\mathbf{f}_v^{pos}(x,y) = \mathbf{f}_v \cdot \mathbf{M}_{fg} + \gamma_{fg} \sum_c w_c \mathbf{p}_{fg}^c \cdot \mathbf{M}_{fg}\)
- Design Motivation: The core issue of astigmatism is the low intra-object attention weights \(A_2, A_3\). By identifying target regions via prototype similarity and injecting prototype information, PPR improves feature consistency within objects and increases attention weights among patches belonging to the same object.
Negative Context Modulation (NCM) Module:
- Function: Constructs a unified background prototype to enhance background region representations, simulating the peripheral perception region of human foveal vision.
- Mechanism: A class-agnostic background prototype \(\mathbf{p}_{bg}\) is obtained via mean pooling over regions outside annotated bounding boxes in support samples. At inference, background regions are identified analogously to PPR, and background prototype information is injected to enhance their representations: \(\mathbf{f}_v^{neg}(x,y) = \mathbf{f}_v \cdot \mathbf{M}_{bg} + \gamma_{bg} \mathbf{p}_{bg} \cdot \mathbf{M}_{bg}\)
- Design Motivation: The counterpart of defocus is the excessively high attention weight \(A_1\) on background patches. By enhancing background region features to become more separable from foreground features, erroneous foreground attention toward background is reduced. The background is treated as a unified concept because "non-target" is a universal notion.
Text Semantic Alignment (TSA) Module:
- Function: Leverages cross-modal knowledge to reinforce foreground-background discriminability, realizing center-surround contrast enhancement.
- Mechanism: Negative text descriptions of the form "not [class]" (e.g., "not sofa, not dog") are constructed and encoded via BERT to obtain text features \(\mathbf{F}_t^{bg}\). The background prototype from NCM serves as the visual feature \(\mathbf{F}_v^{bg}\). Both are projected into a shared semantic space via learnable projection layers and aligned using a contrastive loss \(\mathcal{L}_{ctr}\): \(\mathcal{L}_{ctr} = -\log \frac{\exp(\text{diag}(\mathcal{S})/\tau)}{\sum_{i,j}\exp(\mathcal{S}_{i,j}/\tau)}\)
- Design Motivation: Pure visual prototypes may lack robustness when domain gaps are large; introducing the language modality provides an additional semantic supervision signal. The "not [class]" negative prompt design is elegant, directly defining the semantic meaning of background and helping the model establish clear foreground/background boundaries at the semantic level.

Loss & Training¶

The total loss is \(\mathcal{L}_{total} = \mathcal{L}_{detection} + \lambda_{bg} \cdot \mathcal{L}_{ctr}\), where the detection loss includes classification and localization losses. The optimal TSA loss weight is \(\lambda_{bg} = 10^3\). During inference, the enhanced features from PPR and NCM are complementarily fused as \(\mathbf{F}_v^{enhanced} = \mathbf{f}_v^{pos} + \mathbf{f}_v^{neg}\) before being passed to the detection head.

Key Experimental Results¶

Main Results¶

Method	1-shot Avg mAP	5-shot Avg mAP	10-shot Avg mAP
GLIP	18.98	29.36	33.93
CD-ViTO*	15.96	28.30	33.47
Domain-RAG*	16.52	27.98	32.81
VFM-MoE*	17.59	28.45	33.94
Ours	23.81	33.73	39.17

Evaluated across six datasets (ArTaxOr, Clipart1k, DIOR, DeepFish, NEU-DET, UODD) spanning insects, cartoons, remote sensing, underwater, and other domains, the proposed method achieves state-of-the-art results under all 1/5/10-shot settings. The 5-shot average mAP exceeds the strongest baseline by 5.28 points (33.73 vs. 28.45).

Ablation Study¶

Configuration	ArTaxOr	Clipart1k	DeepFish	NEU-DET	UODD	Avg. Gain
Baseline (GLIP)	48.11	39.28	28.40	19.55	11.40	-
+NCM	49.95	40.52	29.58	20.48	12.21	+1.06
+NCM+PPR	52.68	42.95	31.96	22.38	14.26	+2.06
+NCM+PPR+TSA	54.98	44.83	33.87	23.64	15.66	+1.59

All three modules contribute incrementally; PPR yields the largest individual gain (+2.06%), confirming that foreground enhancement is the central mechanism for correcting astigmatism.

Key Findings¶

Attention distance analysis quantitatively confirms that standard fine-tuning reduces attention dispersion by only 0.25%–1.30%, whereas the proposed method achieves reductions of 0.47%–1.72%.
Background ratio analysis shows that NCM yields greater advantages when the background ratio is high (0.7–1.0), validating the effectiveness of negative context utilization.
Using 200 background text descriptions achieves the optimal cost-effectiveness trade-off (+1.39% AP with only 84 MB additional memory).
The method substantially reduces redundant detection boxes (e.g., from 100 to 9 in a maritime scene), with highly compelling qualitative results.

Highlights & Insights¶

The discovery of the "astigmatism" phenomenon is itself a significant scientific contribution; the quantitative attention distance analysis reveals a previously neglected core problem in CD-FSOD.
The biologically-inspired design analogy is remarkably natural: fovea → PPR, peripheral vision → NCM, center-surround contrast → TSA, with a high degree of coherence between biological inspiration and technical solutions.
The "not [class]" negative text prompt is an innovative and practical design that defines background semantics from a negative perspective to reinforce foreground/background separation.
The method introduces minimal additional parameters and computational overhead, exhibiting good deployment efficiency.

Limitations & Future Work¶

Prototype quality is heavily dependent on the limited number of support samples; under extreme few-shot settings (e.g., 1-shot), prototypes may be inaccurate.
Validation is conducted only on the Swin-Tiny backbone; performance on larger models or alternative architectures (e.g., ViT-L) remains unknown.
Treating the background prototype as a unified concept may be an oversimplification in complex scenes; hierarchical background modeling could be more effective.
The "not [class]" prompts require prior knowledge of target-domain class names, necessitating adaptation for open-domain scenarios where class identities are unknown.

vs. CD-ViTO: CD-ViTO preserves source-domain priors via knowledge distillation but does not address attention dispersion. The proposed method surpasses it by 5.43 points in 5-shot average mAP (33.73 vs. 28.30).
vs. Distill-CDFSOD: Distillation-based methods are limited when domain gaps are large; the proposed method directly improves feature quality at the attention mechanism level.
vs. IPNet: IPNet designs independent domain alignment paths for foreground and background but does not exploit cross-modal textual information; the TSA module provides additional semantic-level supervision.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ — The discovery of the astigmatism phenomenon and the bio-inspired foveal vision design paradigm are both highly original.
Experimental Thoroughness: ⭐⭐⭐⭐ — Six datasets, three shot settings, multi-module ablations, and attention visualizations constitute a comprehensive evaluation.
Writing Quality: ⭐⭐⭐⭐⭐ — The narrative logic from problem discovery → quantitative analysis → biological inspiration → technical solution is exceptionally clear.
Value: ⭐⭐⭐⭐ — The discovery of the astigmatism phenomenon and the proposed remediation approach offer important insights for the CD-FSOD community, with methods generalizable to other cross-domain tasks.