title: >- [Paper Note] Rethinking Pseudo-Labels: Multi-Granularity Supervision for Domain Adaptive Object Detection description: >- [ECCV 2026][Object Detection][Domain Adaptation] Proposes Multi-Granularity Pseudo-Labeling (MGPL) combining dual-competition image-level prediction and coarse-to-fine collaborative filtering for teacher-student DAOD, achieving 51.4% mAP on Clipart1k. tags: - ECCV 2026 - Object Detection - Domain Adaptive Object Detection - Pseudo-Labeling - Multi-Granularity Learning date: 2026-09-19 content_hash: bfdc79c75c40338a
Rethinking Pseudo-Labels: Multi-Granularity Supervision for Domain Adaptive Object Detection¶
Conference: ECCV 2026
Paper: CVF Open Access
Area: Object Detection
Keywords: Domain Adaptive Object Detection, Pseudo-Labeling, Multi-Granularity Supervision, Dual-Competition Aggregation, Coarse-to-Fine Filtering
TL;DR¶
Addressing the vulnerability of conventional teacher-student domain adaptive detectors that rely exclusively on noisy proposal-level pseudo-boxes, this paper presents Multi-Granularity Pseudo-Labeling (MGPL), which extracts noise-resilient image-level predictions via a dual-competition aggregation mechanism and refines proposal selection through coarse-to-fine collaborative filtering, achieving 51.4% mAP on Pascal VOC→Clipart1k and outperforming the fully supervised Oracle by 6.4%.
Background & Motivation¶
Unsupervised Domain Adaptive Object Detection (DAOD) seeks to transfer detection knowledge acquired from a fully labeled source domain to an unlabeled target domain. In realistic deployment scenarios, shifts in environmental lighting, adverse weather conditions, optical sensor characteristics, or substantial stylistic disparities (such as transitioning from natural photographs to artistic clipart illustrations) cause standard object detectors to undergo severe performance degradation. While early research explored adversarial feature alignment via gradient reversal layers and image-to-image translation via generative networks to minimize domain discrepancy, the teacher-student mutual-learning paradigm has emerged as the prevailing standard. In this framework, a teacher detector updated via Exponential Moving Average (EMA) produces pseudo bounding boxes on weakly augmented target instances to guide a student detector trained on strongly augmented inputs.
Nevertheless, existing teacher-student DAOD methodologies remain plagued by fundamental constraints tied to single proposal-level pseudo-supervision. Under acute domain shift, candidate proposals generated by the teacher network suffer from pervasive classification confusion and inaccurate bounding box localization. Propagating these noisy pseudo-boxes directly into the student network initiates an error accumulation feedback loop during iterative self-training. Furthermore, prevailing pipelines enforce fixed, hand-tuned confidence thresholds (typically set to static values such as 0.8) to filter candidate proposals. This inflexible mechanism causes severe dilemmas across categories of varying adaptation difficulty: it fails to suppress false positive detections for categories entirely absent from the image, while simultaneously filtering out lower-confidence yet geometrically valid difficult instances (such as small or partially occluded objects), leading to high false negative rates.
Inspired by multiple instance learning formulations in weakly supervised object detection (WSOD), the authors uncover an intriguing empirical finding: in target domain images, coarse-grained image-level predictions derived by aggregating proposal scores exhibit substantially higher classification accuracy and precision than individual proposal-level pseudo-boxes. This phenomenon arises because spatial pooling over candidate distributions statistically smooths out local localization drift and dilutes isolated misclassifications. However, relying solely on image-level supervision discards critical spatial localization information required for bounding box regression. Core Idea: The paper proposes Multi-Granularity Pseudo-Labeling (MGPL), integrating coarse-grained image-level soft predictions and fine-grained proposal-level pseudo-boxes into teacher-student DAOD for the first time, leveraging a parameter-free dual-competition aggregation mechanism alongside a coarse-to-fine collaborative filtering strategy to establish mutually reinforcing multi-granularity supervision.
Method¶
Overall Architecture¶
MGPL is established upon the Adaptive Teacher (AT) framework with a two-stage Faster R-CNN detector. During training, unlabeled target images are subjected to weak and strong data augmentations before passing through the teacher and student networks, respectively. The teacher network extracts RPN proposal candidates and class logits from weakly augmented inputs. Utilizing a dual-competition aggregation mechanism without introducing additional trainable parameters, the framework computes global category existence probabilities for the entire image. Concurrently, these image-level predictions are coupled with proposal-level distributions through a coarse-to-fine collaborative filtering module, which automatically calculates an image-adaptive threshold via inter-class variance maximization to filter pseudo bounding boxes. Finally, the student detector is optimized via multi-granularity supervision, combining source-domain fully supervised loss, image-level binary cross-entropy soft distillation, and refined proposal-level regression and classification losses.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
In["Unlabeled Target Image<br/>(Weak / Strong Augmentation)"] --> Teacher["Teacher Model (EMA Updated)<br/>Extracts RPN proposals & class logits"]
Teacher --> A["Dual-Competition Aggregation<br/>Top-k class competition + Objectness weighting"]
Teacher --> B["Coarse-to-Fine Collaborative Filtering<br/>Collaborative scoring + Otsu-like thresholding"]
A -->|Global semantic predictions| B
A --> C["Multi-Granularity Supervised Learning<br/>Image-level BCE soft distillation"]
B -->|Filtered pseudo bounding boxes| C
C --> Student["Student Detector<br/>(Backpropagation Optimization)"]
Key Designs¶
1. Dual-Competition Aggregation: Extracting Global Categorical Semantics from Proposals without Extra Parameters
In the absence of ground-truth annotations on the target domain, deriving reliable image-level category probabilities from noisy RPN proposals is the cornerstone of coarse-grained supervision. Directly computing an unweighted average over all proposals introduces substantial background noise from low-quality candidates. To circumvent this, the authors introduce a dual-competition aggregation pipeline that decouples within-proposal semantic categorization from cross-proposal objectness quality assessment. Given \(N\) candidate proposals \(\{r_i\}_{i=1}^N\) with classification logits \(x_i \in \mathbb{R}^C\) and RPN objectness scores \(s_i^{obj}\), the process begins with intra-proposal class competition. To prevent tail-end irrelevant classes from introducing random perturbations, competition is restricted to the top-\(k\) classes \(\mathcal{T}_k(x_i)\) for each candidate proposal: $\(P_{i,c} = \frac{\exp(x_i^c) \cdot \mathbf{1}[c \in \mathcal{T}_k(x_i)]}{\sum_{c' \in \mathcal{T}_k(x_i)} \exp(x_i^{c'})}\)$ This operation sharpens the class probability distribution \(P_{i,:}\). Next, an argmax-based objectness assignment identifies the dominant category \(c_i^* = \arg\max_c x_i^c\) and routes the physical objectness score strictly into this single semantic channel, generating a sparse objectness matrix: $\(\hat{O}_{i,c} = s_i^{obj} \cdot \mathbf{1}[c = c_i^*]\)$ Proposals then participate in objectness-based competition across all candidates within each individual category: $\(w_i^c = \frac{\exp(\hat{O}_{i,c})}{\sum_{j=1}^N \exp(\hat{O}_{j,c})}\)$ The final image-level prediction for category \(c\) is obtained via quality-weighted aggregation \(\hat{y}_c = \sum_{i=1}^N w_i^c \cdot P_{i,c}\). The resulting vector \(\hat{y} \in \mathbb{R}^C\) encapsulates the presence probability of each category in the scene without introducing any learnable parameters.
2. Coarse-to-Fine Collaborative Filtering: Parameter-Free Threshold Derivation Guided by Global Semantics
Conventional pseudo-labeling relies on an empirical static threshold \(\tau\), which inevitably admits false positive bounding boxes for absent categories and discards challenging true positives. MGPL addresses this by employing image-level semantic guidance to regulate proposal filtering. First, a collaborative score is calculated for each candidate proposal by coupling its instance-level Softmax distribution \(P_{i,c}\) with the global image prediction \(\hat{y}_c\): $\(r_i = \max_c \left( P_{i,c} \cdot \hat{y}_c \right), \quad c_i^* = \arg\max_c \left( P_{i,c} \cdot \hat{y}_c \right)\)$ If a category is absent from the target image (\(\hat{y}_c \approx 0\)), the collaborative scores of any corresponding candidate proposals are suppressed close to zero, effectively eliminating hallucinations in background regions.
To remove manual tuning of threshold hyperparameters, the framework derives an image-specific adaptive threshold via inter-class variance maximization (analogous to Otsu's thresholding in image segmentation). Discretizing the collaborative scores \(\{r_i\}_{i=1}^N\) into \(K\) histogram bins yields empirical distribution pairs \(\{p_k, t_k\}_{k=1}^K\). For any candidate splitting threshold \(t\), the cumulative background weight \(\omega(t) = \sum_{r_i \le t} p(r_i)\) and cumulative background mean \(\mu(t) = \sum_{r_i \le t} r_i \cdot p(r_i)\) are computed. The between-class variance between retained and discarded candidates is formulated as: $\(\sigma_B^2(t) = \frac{(\mu_T \cdot \omega(t) - \mu(t))^2}{\omega(t) \cdot (1 - \omega(t))}\)$ where \(\mu_T\) denotes the overall mean across all proposal scores. The optimal threshold \(\tau^* = \arg\max_t \sigma_B^2(t)\) is determined dynamically per image, and proposals satisfying \(r_i \ge \tau^*\) are retained as valid pseudo bounding boxes \(\mathcal{P}_{\text{pseudo}}\). This adaptive partitioning maintains high spatial precision while dramatically improving the recall of challenging, lower-scoring instances.
3. Multi-Granularity Supervised Learning: Joint Dual-Path Optimization
To ensure the student network acquires both robust scene-level semantic representations and accurate spatial localization capabilities, the model is optimized with multi-granularity supervision: $\(\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{src}} + \lambda_{\text{img}} \mathcal{L}_{\text{img}} + \lambda_{\text{prop}} \mathcal{L}_{\text{prop}}\)$ Here, \(\mathcal{L}_{\text{src}}\) represents the fully supervised Faster R-CNN loss on labeled source images, preserving basic object feature representations.
On the unlabeled target domain, \(\mathcal{L}_{\text{img}}\) provides coarse-grained global guidance by matching the student network's image-level predictions \(\hat{y}^s\) to the teacher network's predictions \(\hat{y}^t\) via multi-label binary cross-entropy: $\(\mathcal{L}_{\text{img}} = - \frac{1}{C} \sum_{c=1}^C \left[ \hat{y}_c^t \log(\hat{y}_c^s) + (1 - \hat{y}_c^t) \log(1 - \hat{y}_c^s) \right]\)$ This term regularizes feature representations across domains without being contaminated by proposal localization errors. Meanwhile, fine-grained supervision \(\mathcal{L}_{\text{prop}}\) applies standard detection regression and classification losses on the student network using the filtered pseudo-boxes \(\mathcal{P}_{\text{pseudo}}\). The complementary interaction between image-level category stability and instance-level geometric localization yields superior domain adaptation robustness.
Loss & Training¶
The framework is optimized using Stochastic Gradient Descent (SGD) with momentum 0.9 and an initial learning rate of 0.005. The teacher model is updated via EMA with a momentum coefficient of \(\alpha = 0.9996\). A source-domain warm-up pre-training phase runs for 40k iterations, followed by 100k iterations of mutual learning on combined source and target data. The loss balancing coefficients are set to \(\lambda_{\text{prop}} = 0.5\) and \(\lambda_{\text{img}} = 0.5\), complemented by an adversarial feature alignment loss with weight \(\lambda_{\text{dis}} = 0.1\). The top-\(k\) truncation parameter is set to \(k=3\).
Key Experimental Results¶
Main Results¶
MGPL is thoroughly evaluated across three prominent DAOD benchmarks covering stylistic, cross-weather, and cross-camera domain shifts. Pascal VOC→Clipart1k adopts a ResNet-101 backbone, while Cityscapes→Foggy Cityscapes and KITTI→Cityscapes employ a VGG-16 backbone. Performance is assessed using mean Average Precision at IoU 0.5 (mAP50).
Cross-Style Adaptation Benchmark: Pascal VOC → Clipart1k (ResNet-101)
| Method | Paradigm | aero | bike | bird | boat | bottle | bus | car | cat | chair | cow | table | dog | horse | mtr | prsn | plant | shp | sofa | train | tv | mAP (%) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Source Only | Lower Bound | 23.0 | 39.6 | 20.1 | 23.6 | 25.7 | 42.6 | 25.2 | 0.9 | 41.2 | 25.6 | 23.7 | 11.2 | 28.2 | 49.5 | 45.2 | 46.9 | 9.1 | 22.3 | 38.9 | 31.5 | 28.8 |
| AT (CVPR'22) | Baseline | 33.1 | 66.1 | 35.3 | 44.9 | 57.5 | 44.9 | 51.0 | 5.8 | 59.5 | 54.9 | 34.6 | 23.5 | 64.3 | 84.0 | 75.4 | 51.5 | 17.1 | 30.3 | 43.3 | 37.2 | 45.7 |
| CMT (CVPR'23) | Contrastive | 39.8 | 56.3 | 38.7 | 39.7 | 60.4 | 35.0 | 56.0 | 7.1 | 60.1 | 60.4 | 35.8 | 28.1 | 67.8 | 84.5 | 80.1 | 55.5 | 20.3 | 32.8 | 42.3 | 38.2 | 47.0 |
| CAT (CVPR'24) | Inter-Class | 40.5 | 64.1 | 38.8 | 41.0 | 60.7 | 55.5 | 55.6 | 14.3 | 54.7 | 59.6 | 46.2 | 20.3 | 58.7 | 92.9 | 62.6 | 57.5 | 22.4 | 40.9 | 49.5 | 46.0 | 49.1 |
| MGPL (Ours) | Multi-Gran. | 37.4 | 71.5 | 41.2 | 45.5 | 63.1 | 71.9 | 50.6 | 17.3 | 56.3 | 50.5 | 47.1 | 16.2 | 52.3 | 92.1 | 75.2 | 62.9 | 30.7 | 46.8 | 37.4 | 62.4 | 51.4 |
| Oracle | Supervised | 33.3 | 47.6 | 43.1 | 38.0 | 24.5 | 82.0 | 57.4 | 22.9 | 48.4 | 49.2 | 37.9 | 46.4 | 41.1 | 54.0 | 73.7 | 39.5 | 36.7 | 19.1 | 53.2 | 52.9 | 45.0 |
Cross-Weather & Cross-Camera Adaptation Benchmarks: Cityscapes → Foggy Cityscapes (C→F) & KITTI → Cityscapes (K→C)
| Method | Venue | C→F Person | C→F Rider | C→F Car | C→F Truck | C→F Bus | C→F Train | C→F Motor | C→F Bicycle | C→F mAP (%) | K→C Car AP (%) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Source Only | - | 27.9 | 33.4 | 40.4 | 12.1 | 23.2 | 10.1 | 20.7 | 30.9 | 24.8 | 40.3 |
| DA-Faster | CVPR 2018 | 29.2 | 40.4 | 43.4 | 19.7 | 38.3 | 28.5 | 23.7 | 32.7 | 32.0 | 41.9 |
| PT | ICML 2022 | 43.2 | 52.4 | 63.4 | 33.4 | 56.6 | 37.8 | 41.3 | 48.7 | 47.1 | 47.8 |
| AT | CVPR 2022 | 46.3 | 55.9 | 64.3 | 38.5 | 61.1 | 39.3 | 40.8 | 52.3 | 49.8 | - |
| CMT | CVPR 2023 | 47.0 | 55.7 | 64.5 | 39.4 | 63.2 | 51.9 | 40.3 | 53.1 | 51.9 | 64.3 |
| DSD-DA | ICML 2024 | 49.0 | 59.6 | 65.3 | 35.7 | 61.0 | 46.5 | 43.9 | 57.3 | 52.3 | 49.3 |
| MGPL (Ours) | ECCV 2026 | 51.7 | 56.2 | 64.7 | 40.6 | 62.2 | 52.6 | 44.2 | 57.4 | 52.7 | 64.3 |
| Oracle | - | 41.2 | 49.1 | 61.6 | 32.6 | 56.6 | 49.0 | 37.9 | 42.4 | 46.3 | 64.4 |
Ablation Study¶
Component-Wise Ablation Analysis (Pascal VOC → Clipart1k, ResNet-101)
| Config Index | Image-Level (\(\mathcal{L}_{\text{img}}\)) | Collaborative Filtering | Proposal-Level (\(\mathcal{L}_{\text{prop}}\)) | mAP (%) | Gain vs. AT |
|---|---|---|---|---|---|
| 1 (AT Baseline) | ✗ | ✗ | ✓ | 45.7 | Baseline |
| 2 (Image-Level Only) | ✓ | ✗ | ✗ | 46.1 | +0.4% |
| 3 (MGPL w/o Filter) | ✓ | ✗ | ✓ | 50.6 | +4.9% |
| 4 (MGPL Full Model) | ✓ | ✓ | ✓ | 51.4 | +5.7% |
Sensitivity to Image-Level Loss Weight \(\lambda_{\text{img}}\) (Pascal VOC → Clipart1k)
| \(\lambda_{\text{img}}\) Value | 0.1 | 0.3 | 0.5 (Default) | 1.0 | 2.0 |
|---|---|---|---|---|---|
| mAP (%) | 48.5 | 50.5 | 51.4 | 51.0 | 50.8 |
Key Findings¶
- Multi-Granularity Synergy Significantly Outperforms Single-Granularity Supervision: When relying solely on proposal-level pseudo-boxes, AT achieves 45.7% mAP due to acute localization and classification errors. Utilizing image-level supervision alone yields 46.1% mAP. Crucially, combining both granularities without filtering immediately boosts performance to 50.6% (+4.9%), demonstrating that global scene regularization and spatial localization cues are highly complementary. Integrating collaborative adaptive filtering further improves the result to 51.4% mAP.
- Superior Noise Resilience under Severe Pseudo-Label Corruption: In controlled experiments where candidate pseudo-box labels were corrupted by randomly flipping a fraction \(r \in [0.0, 1.0]\) of classes, AT suffers severe degradation from 45.7% to 38.6% (\(\Delta = -7.1\%\)). In contrast, MGPL exhibits much slower degradation, with its performance margin over AT widening from +5.7% (\(r=0.0\)) to +9.3% (\(r=1.0\)). This demonstrates that aggregated image-level supervision effectively prevents confirmation bias and catastrophic error cascades during self-training.
- Stable and Robust Optimization Dynamics: Exploration of the loss balancing hyperparameter \(\lambda_{\text{img}}\) reveals a smooth unimodal curve peaking at 0.5, while remaining robust across a broad operating window (from 48.5% at 0.1 to 50.8% at 2.0).
Highlights & Insights¶
- Repurposing WSOD Insights into Cross-Domain Self-Training: The paper challenges the dogma that teacher-student DAOD must exclusively rely on pseudo bounding boxes, identifying that aggregated proposal statistics provide inherently cleaner category supervision under domain shift.
- Otsu-Inspired Non-Parametric Adaptive Thresholding: The formulation of inter-class variance maximization on collaborative scores removes arbitrary confidence threshold tuning, simultaneously suppressing hallucinations of absent classes and retrieving low-contrast hard targets.
- Broad Transferability for Structured Prediction: The design paradigm of leveraging noise-reduced coarse-grained aggregates to filter fine-grained structured hypotheses holds significant utility for related fields such as 3D detection, open-vocabulary grounding, and temporal action localization.
Limitations & Future Work¶
- Architectural Dependency on Two-Stage Frameworks: The dual-competition aggregation mechanism relies on RPN proposal candidates and explicit objectness scores, restricting direct applicability to single-stage detectors (e.g., YOLO) or modern query-based Transformer detectors (e.g., DETR variants).
- Absence of Spatial Context and Relation Modeling: Aggregation operates primarily via class-wise Softmax and scalar weighting without explicitly modeling relational topology, geometric overlaps, or scene graphs between neighboring objects.
- Future Directions: Adapting coarse-grained attention pooling mechanisms to DETR object queries, and leveraging multimodal foundation models (e.g., CLIP) for external zero-shot semantic verification.
Related Work & Insights¶
- vs. AT (Adaptive Teacher, CVPR 2022): AT popularized weak-strong augmentation and EMA teacher pseudo-labeling for DAOD but remains vulnerable to fixed-threshold filtering and localized label noise. MGPL introduces dual-competition aggregation and adaptive filtering, delivering a +5.7% mAP improvement on Clipart1k (51.4% vs. 45.7%).
- vs. CMT (Contrastive Mean Teacher, CVPR 2023): CMT incorporated contrastive feature representation learning while remaining confined to proposal-level supervision. MGPL tackles the problem from the supervision granularity axis, surpassing CMT by 4.4% mAP on Pascal VOC→Clipart1k.
- vs. CAT (CVPR 2024): CAT explicitly modeled inter-class dynamics to address class imbalance. In comparison, MGPL suppresses absent-class hallucinations through coarse-to-fine collaborative scores, setting new state-of-the-art benchmarks on both Clipart1k (51.4%) and Foggy Cityscapes (52.7%).
Rating¶
- Novelty: ⭐⭐⭐⭐☆ Insightful realization and empirical verification of image-level noise resilience in DAOD, paired with novel dual-competition aggregation and Otsu-like adaptive thresholding.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive cross-domain benchmark evaluations, granular module ablations, parameter sensitivity investigations, and systematic label noise corruption tests.
- Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical formulations, logically coherent motivation-to-methodology trajectory, and well-structured exposition.
- Value: ⭐⭐⭐⭐☆ Opens up an impactful multi-granularity supervision perspective for cross-domain detection with strong theoretical grounding and practical efficacy.