DA-F2F: Domain-Adaptive Object Detection with Feature-to-Feature Modulation and Alignment

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/ohhotaek/DA-F2F
Area: Object Detection
Keywords: domain adaptive object detection, feature modulation, adversarial learning, soft-weighted alignment, Mean Teacher

TL;DR

Addressing the computational burden, pixel artifacts, and pseudo-label instability of traditional image-to-image translation in domain adaptive object detection, DA-F2F dynamically modulates source features with target style statistics (SFM) in latent space and applies continuous soft-weighted proposal adversarial alignment (SPA), achieving robust cross-domain alignment and new state-of-the-art detection accuracy.

Background & Motivation

Object detection serves as a fundamental perception module in downstream real-world applications including autonomous driving, intelligent surveillance, and embodied robotics. While modern deep detectors deliver outstanding precision under standard supervised training on large annotated datasets, their performance degrades sharply when deployed in out-of-distribution environments. Real-world distribution shifts—such as adverse weather conditions (e.g., dense fog), changing illumination, or varying camera optics—drastically impair detector generalizability. Because collecting and labeling extensive bounding boxes for every newly encountered target environment is prohibitive and labor-intensive, unsupervised domain adaptive object detection (DAOD) has emerged as an essential paradigm to generalize detectors from an annotated source domain to an unannotated target domain.

In existing DAOD research, the Mean Teacher self-training paradigm has achieved remarkable prominence. In this paradigm, an exponential moving average (EMA) teacher model produces pseudo-labels on weakly augmented target images to supervise a student model learning under strong augmentations. However, driven by the supervised loss on the source domain, the student's parameters inevitably carry a pronounced source-domain bias. The EMA-updated teacher inherits this inductive bias, which causes it to generate unreliable and noisy pseudo-labels under severe cross-domain visual discrepancies, triggering confirmation bias and compounding optimization errors. To alleviate this dependency, prior works adopt image-to-image (I2I) translation networks (such as CycleGAN or CUT) to translate source images into target-like appearances, providing pseudo-annotated training samples. Nevertheless, high-dimensional pixel synthesis introduces substantial computational overhead and often suffers from mode collapse or structural artifacts, corrupting the fine-grained geometry and semantic contours that object detection relies upon. Meanwhile, existing adversarial alignment approaches attempt to separate foreground and background representations, but relying on discrete, hard-thresholded pseudo-labels in the unannotated target domain introduces significant assignment errors and destabilizes adversarial training.

Overcoming these challenges requires abandoning explicit pixel-level image reconstruction in favor of lightweight, controllable style injection and smoothly decoupled region alignment directly in the feature space. Core idea: bypass pixel-level translation by introducing a feature-to-feature modulation framework (DA-F2F), which dynamically reshapes source representations via target style statistics (SFM) in latent space and stabilizes foreground-background adversarial learning through continuous soft-weighted proposal alignment (SPA).

Method

Overall Architecture

DA-F2F builds upon the Mean Teacher self-training architecture, where the student detector and teacher detector share identical backbone and detection head structures. During training, the framework takes paired inputs consisting of labeled source images and unlabeled target images, extracting multi-scale deep features through the backbone network. To eliminate domain discrepancies without synthesis artifacts, the framework integrates two complementary modules: the Style-Aware Feature Modulation (SFM) module extracts channel-wise style statistics from target features to dynamically modulate source features via affine transformations, aligning source representations with the target distribution while preserving source ground-truth semantics; the Soft-Weighted Proposal Alignment (SPA) module constructs a domain discriminator and employs continuous soft confidence weights instead of discrete hard thresholds, suppressing background noise and stabilizing foreground adversarial alignment in the target domain.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    In["Input: Labeled source image Xs + Unlabeled target image Xt"] --> Ext["Backbone & FPN feature extraction<br/>Yields source features Fs and target features Ft"]
    Ext --> SFM["Style-Aware Feature Modulation<br/>Channel stats [μ, σ] → Affine params (γ, β) applied to Fs"]
    Ext --> SPA["Soft-Weighted Proposal Alignment<br/>Continuous soft-weight map W smooths foreground & background"]
    SFM --> Det["Detection heads & Mean Teacher mutual learning<br/>Supervised loss Lsup + Teacher pseudo-label consistency Lunsup"]
    SPA --> Det
    Det --> Out["Output: Domain-invariant, robust object detector"]

Key Designs

1. Style-Aware Feature Modulation: Latent statistics-driven feature-level transfer The primary drawback of pixel-level image translation lies in the heavy computational cost of pixel manifold reconstruction and its vulnerability to visual artifacts that distort object boundaries. Observing that channel-wise feature statistics in deep neural networks inherently encode domain style attributes such as illumination, weather, and texture, the proposed SFM module performs domain transformation entirely in latent space. For target features \(F_t \in \mathbb{R}^{H \times W \times C}\), the module calculates the spatial mean \(\mu_c\) and standard deviation \(\sigma_c\) across each channel: $\(\mu_c = \frac{1}{HW}\sum_{h=1}^H\sum_{w=1}^W F_{t, h, w, c}, \quad \sigma_c = \sqrt{\frac{1}{HW}\sum_{h=1}^H\sum_{w=1}^W (F_{t, h, w, c} - \mu_c)^2 + \epsilon}\)$ The concatenated vector \([\mu, \sigma] \in \mathbb{R}^{2C}\) is passed through a two-layer MLP with ReLU activation to predict a channel-wise scaling offset \(\Delta \gamma \in \mathbb{R}^C\) and translation parameter \(\beta \in \mathbb{R}^C\), yielding the affine scale parameter \(\gamma = 1 + \Delta \gamma\). The source features \(F_s\) are then modulated into target-aligned features \(\tilde{F}_{s \to t, c} = \gamma_c F_{s, c} + \beta_c\). To prevent excessive feature deviation that could degrade source contextual semantics, a blending factor \(w\) balances the modulated representation with the original feature: $\(F_{s \to t, c} = (1 - w) F_{s, c} + w \tilde{F}_{s \to t, c}\)$ This implicit feature-level style transfer circumvents the artifacts of image synthesis and equips the student detector with realistic target style cues at virtually zero computational overhead.

2. Label-Guided Source Alignment: Clean foreground-focused adversarial learning In cross-domain adversarial alignment, uniform feature adaptation allows abundant background pixels to dominate gradient updates, washing out the discriminative representations of foreground objects. The SPA module addresses this on the source domain using ground-truth annotations. Source features \(F_s\) are fed into a spatial domain discriminator \(D\) to produce a source probability map \(D^s \in [0, 1]^{H \times W}\). A binary spatial mask \(M \in \{0, 1\}^{H \times W}\) is constructed directly from ground-truth bounding boxes (1 inside bounding boxes, 0 elsewhere), strictly separating foreground and background adversarial objectives: $\(\mathcal{L}_{\text{fg}} = -\frac{1}{\|M\|_1} \sum_{h=1}^H \sum_{w=1}^W M_{h, w} \log D^s_{h, w}\)$ $\(\mathcal{L}_{\text{bg}} = -\frac{1}{\|1_{HW} - M\|_1} \sum_{h=1}^H \sum_{w=1}^W (1 - M_{h, w}) \log (1 - D^s_{h, w})\)$ By assigning a high weight \(\lambda_{\text{fg}} = 0.8\) to foreground alignment and a low weight \(\lambda_{\text{bg}} = 0.05\) to background alignment (\(\mathcal{L}_{\text{src}} = \lambda_{\text{fg}}\mathcal{L}_{\text{fg}} + \lambda_{\text{bg}}\mathcal{L}_{\text{bg}}\)), the discriminator forces the network to concentrate capacity on transferable object regions while attenuating background interference.

3. Soft-Weighted Target Alignment: Bounded continuous weighting against pseudo-label noise In the unlabeled target domain, ground-truth masks are absent. Applying hard thresholding to imperfect teacher pseudo-labels inevitably leads to noisy foreground-background assignments and unstable gradients. To address this limitation, SPA introduces the Soft-Weighted Target Alignment (STA) mechanism. A lightweight convolutional head derives a continuous proposal confidence map \(P \in [0, 1]^{H \times W}\) from target features \(F_t\), optimized via region proposal loss \(\mathcal{L}_{\text{rpn}}\) against teacher pseudo-labels. To prevent severe weight disparity, \(P\) is linearly rescaled into a bounded weight map \(W \in [\tau_{\min}, \tau_{\max}]^{H \times W}\) with \(\tau_{\min} = 0.05\) and \(\tau_{\max} = 0.8\): $\(W = \tau_{\min} 1_{HW} + (\tau_{\max} - \tau_{\min}) P\)$ Target features \(F_t\) are evaluated by the shared domain discriminator to generate domain probability map \(D^t = D(F_t)\), and the soft-weighted target adversarial loss is formulated as: $\(\mathcal{L}_{\text{tgt}} = -\frac{1}{HW} \sum_{h=1}^H \sum_{w=1}^W W_{h, w} \log (1 - D^t_{h, w})\)$ The smooth spatial weight map \(W\) continuously amplifies alignment gradients within salient object proposal regions while gracefully down-weighting ambiguous background areas, eliminating the discrete assignment noise inherent in hard pseudo-labeling.

Loss & Training

DA-F2F adopts a two-stage training strategy over 40k total iterations: a burn-in stage for the first 10k iterations optimizing solely on labeled source samples, followed by a mutual learning stage for 30k iterations training student and teacher jointly with EMA updates (momentum \(\alpha = 0.9996\)). The entire network is optimized end-to-end using the joint objective: $\(\mathcal{L}_{\text{total}} = \lambda_{\text{sup}} \mathcal{L}_{\text{sup}} + \lambda_{\text{unsup}} \mathcal{L}_{\text{unsup}} + \lambda_{\text{rpn}} \mathcal{L}_{\text{rpn}} + \lambda_{\text{spa}} \mathcal{L}_{\text{spa}}\)$ where \(\mathcal{L}_{\text{sup}}\) denotes supervised detection loss on modulated source features, \(\mathcal{L}_{\text{unsup}}\) denotes unsupervised consistency loss against teacher pseudo-labels on target samples, \(\mathcal{L}_{\text{rpn}}\) trains the target proposal map, and \(\mathcal{L}_{\text{spa}} = \min_F \max_D (\mathcal{L}_{\text{src}} + \mathcal{L}_{\text{tgt}})\) denotes the adversarial alignment loss. Loss trade-off hyperparameters are set to \(\lambda_{\text{sup}} = 1.0\), \(\lambda_{\text{unsup}} = 1.0\), \(\lambda_{\text{rpn}} = 1.0\), and \(\lambda_{\text{spa}} = 0.01\). The model is trained using SGD with momentum 0.9 and initial learning rate 0.01 on four NVIDIA RTX A6000 GPUs.

Key Experimental Results

Main Results

The authors evaluate DA-F2F across three standard DAOD scenarios: weather adaptation (Cityscapes \(\to\) Foggy Cityscapes, highest fog density 0.02), small-to-large scale adaptation (Cityscapes \(\to\) BDD100K-daytime), and synthetic-to-real adaptation (Sim10K \(\to\) Cityscapes, car class). All evaluations adopt [email protected].

Table 1: Cross-weather adaptation performance (Cityscapes \(\to\) Foggy Cityscapes, [email protected], %)

Method	Detector	person	rider	car	truck	bus	train	mcycle	bicycle	mAP
DA-Faster	Faster R-CNN	29.2	40.4	43.4	19.7	38.3	28.5	23.7	32.7	32.0
UMT	Faster R-CNN	33.0	46.7	48.6	34.1	56.5	46.8	30.4	37.3	41.7
PT	Faster R-CNN	43.2	52.4	63.4	33.4	56.6	37.8	41.3	48.7	47.1
DDT	Faster R-CNN	60.9	64.9	75.6	36.3	60.6	53.7	42.3	59.3	56.7
DADAOD	Faster R-CNN	59.8	62.8	73.7	40.3	59.4	56.1	47.8	58.3	57.3
DA-Ada	RegionCLIP	57.8	65.1	71.3	43.1	64.0	58.6	48.8	58.7	58.5
DA-F2F (Ours)	Faster R-CNN	60.9	63.0	76.3	46.2	62.8	64.3	52.8	58.2	60.5

Table 2: Small-to-large-scale adaptation performance (Cityscapes \(\to\) BDD100k-daytime, [email protected], %)

Method	Detector	person	rider	car	truck	bus	mcycle	bicycle	mAP
DA-Faster	Faster R-CNN	28.9	27.4	44.2	19.1	18.0	14.2	22.4	24.9
PT	Faster R-CNN	40.5	39.9	52.7	25.8	33.8	23.0	28.8	34.9
CAT	Faster R-CNN	44.6	41.5	61.2	31.4	34.6	24.4	31.7	38.5
HMT	Transformer	52.6	42.5	67.8	33.7	36.3	28.6	33.7	42.2
DATR	Transformer	58.5	42.8	73.4	26.9	39.9	24.2	37.3	43.3
DADAOD	Faster R-CNN	61.4	45.4	75.4	33.0	36.2	29.5	36.7	45.8
DA-F2F (Ours)	Faster R-CNN	64.9	50.0	78.6	33.9	38.4	35.2	39.2	48.6

Table 3: Synthetic-to-real adaptation performance (Sim10k \(\to\) Cityscapes, car class, [email protected], %)

Method	Detector	car mAP	Method	Detector	car mAP
DA-Faster	Faster R-CNN	38.2	DATR	Transformer	64.3
PT	Faster R-CNN	55.1	HMT	Transformer	66.4
REACT	Faster R-CNN	58.6	DA-Ada	RegionCLIP	67.3
HT	FCOS	65.5	DADAOD	Faster R-CNN	69.7
DA-F2F (Ours)	Faster R-CNN	69.9	-	-	-

Ablation Study

Ablation experiments are conducted on the Cityscapes \(\to\) Foggy Cityscapes benchmark to inspect the individual and joint contributions of SFM and SPA.

Table 4: Ablation study of core modules on Cityscapes \(\to\) Foggy Cityscapes

Config	SFM	SPA	STA	mAP (%)	Note
Baseline	-	-	-	57.3	Mean Teacher baseline without modulation or region adversarial alignment
w/ SPA (w/o STA)	-	✓	-	57.4	Discrete hard pseudo-label partitioning in target domain yields marginal +0.1% gain
w/ SPA (full)	-	✓	✓	58.1	Continuous soft-weighted target alignment enables notable gain (+0.8%)
w/ SFM only	✓	-	-	59.4	Style-aware feature modulation brings substantial leap (+2.1%)
Full DA-F2F	✓	✓	✓	60.5	Full integration yields optimal synergy (+3.2% total gain)

Table 5: Sensitivity of feature modulation blending weight \(w\) (Eq. 4)

Weight \(w\)	mAP (%)	Observation
0.2	60.0	Source features dominate; target style transfer is slightly insufficient
0.3	60.5	Optimal balance: injects adequate target style while preserving semantic structure
0.4	60.2	Higher style shift; detection performance remains robust
0.5	60.0	Moderate blending slightly perturbs fine-grained edge textures

Key Findings

1. SFM provides the most significant performance gain: Introducing SFM alone boosts detection accuracy from 57.3% to 59.4% (+2.1% mAP), showing that affine modulation based on first- and second-order latent statistics effectively transfers domain appearance while eliminating the structural artifacts of image-level translation.
2. Soft-weighting is the vital enabler for target region alignment: Replacing continuous soft weighting with discrete pseudo-label bounding boxes (w/o STA) yields a negligible +0.1% improvement (57.4%), whereas continuous soft weighting achieves 58.1%. This proves that smooth bounded weighting is critical to preventing noisy pseudo-labels from destabilizing adversarial training.
3. High robustness across modulation weights: DA-F2F maintains \(\ge 60.0\%\) mAP across modulation weights \(w \in [0.2, 0.5]\), peaking at \(w=0.3\), which indicates solid tolerance to feature blending ratios.
4. Pronounced improvements on challenging and rare categories: In adverse foggy weather, DA-F2F achieves substantial gains on categories frequently degraded by visual occlusion, including truck (46.2% vs 40.3%), train (64.3% vs 56.1%), and motorcycle (52.8% vs 47.8%) compared to the strongest prior baseline.

Highlights & Insights

• Artifact-free feature-level style injection: Deriving affine scaling and shift parameters directly from channel statistics of target features enables lightweight style modulation without the millions of parameters and distortion risks associated with CycleGAN-like translation models.
• Noise-resilient continuous adversarial weighting: Utilizing bounded RPN proposal probabilities \([0.05, 0.8]\) smoothly allocates discriminator attention to salient object candidates, resolving gradient oscillation caused by discrete pseudo-label errors.
• Modular and detector-agnostic design: Both SFM and SPA operate as external feature augmenters and alignment heads without altering the internal architecture of standard detector backbones or heads, allowing direct transfer to other detectors.

Limitations & Future Work

• Expressive limits of global statistics: Channel-wise mean and variance primarily model atmospheric, lighting, and textural shifts; they do not explicitly model structured geometric discrepancies, such as camera viewpoint differences or regional architectural changes.
• Fixed weighting hyperparameters: The modulation factor \(w\) and boundary thresholds \([\tau_{\min}, \tau_{\max}]\) are currently preset constants; learning dynamic scheduling across training iterations could further enhance convergence.
• Extension to modern transformer detectors: While validated on the standard Faster R-CNN benchmark for direct comparison with prior literature, extending DA-F2F to DETR architectures (e.g., DINO or RT-DETR) is a natural next step.

Related Work & Insights

• vs DDT (Dual-Domain Teacher, TMM 2025): DDT relies on bidirectional CycleGAN image synthesis to create cross-domain supervised pairs, incurring substantial training time and memory overhead alongside risk of pixel artifacts; DA-F2F performs affine statistics modulation directly in feature space with near-zero latency and preserved geometric boundaries.
• vs DADAOD (Differential Alignment, AAAI 2025): DADAOD utilizes differential alignment to decouple regional representations but remains vulnerable to pseudo-label thresholding errors in the target domain; DA-F2F replaces discrete thresholds with continuous soft-weighted proposal alignment.
• vs DA-Ada (Domain-Aware Adapter, NeurIPS 2024): DA-Ada inserts domain adapters into a frozen vision-language foundation backbone (RegionCLIP); DA-F2F demonstrates that even with a standard ResNet-50 backbone, pure latent feature modulation and soft adversarial alignment surpass VLM adapter-based detectors (60.5% vs 58.5%).

Rating

• Novelty: ⭐⭐⭐⭐☆ Elegantly replaces cumbersome pixel translation with latent feature modulation and resolves pseudo-label noise via soft proposal alignment.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive validation across adverse weather, cross-scale, and synthetic-to-real benchmarks with insightful ablation tables and t-SNE visualizations.
• Writing Quality: ⭐⭐⭐⭐⭐ Cohesive motivation, clear mathematical formulations, and thorough comparative analysis.
• Value: ⭐⭐⭐⭐⭐ Provides an efficient, robust, and artifact-free paradigm for practical unsupervised domain adaptive object detection.

Related Papers

• [AAAI 2026] Distilling Future Temporal Knowledge with Masked Feature Reconstruction for 3D Object Detection
• [CVPR 2026] H²A²: Homogeneity-Aware and Heterogeneity-Aware Feature Perception for Unified Indoor 3D Object Detection
• [CVPR 2026] PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understanding
• [CVPR 2026] Selfi: Self-improving Reconstruction Engine via 3D Geometric Feature Alignment
• [ECCV 2026] LoMa: Local Feature Matching Revisited