GoodQ: Zero-Shot Quantization of Object Detectors Using Off-the-Shelf Generative Models¶

Conference: ECCV2026
arXiv: 2606.31456
Code: To be confirmed
Area: Model Compression / Object Detection
Keywords: Zero-Shot Quantization, Object Detection, Diffusion Models, Knowledge Distillation, Model Compression

TL;DR¶

GoodQ leverages off-the-shelf diffusion models to generate diverse training images. It systematically addresses three major challenges in zero-shot quantization of object detectors using three key components: Information-Dense Prompting (IDP), Intrinsic Distribution-Aware Selection (IDAS), and Teacher-Guided Adaptive Noise Reduction (TANR). This achieves significant performance improvements over prior noise-optimization-based methods under extremely low bit-widths such as W4A4 and W3A3.

Background & Motivation¶

Quantization-Aware Training (QAT) is a standard technology for deploying detection models to edge devices, which compresses full-precision models into low-bit-width integers by fine-tuning on a small subset of the original training data. However, in real-world scenarios, the original data is often inaccessible due to privacy regulations or commercial confidentiality—street-view data in autonomous driving and patient information in medical imaging cannot be reused. Zero-Shot Quantization (ZSQ) is precisely designed to resolve this dilemma: it does not rely on real data, instead synthesizing surrogate samples to replace the original training set.

Early ZSQ methods for image classification tasks relied on noise optimization (starting from Gaussian noise and backpropagating to match the Batch Normalization layer statistics of the pre-trained model), resulting in limited diversity of synthesized samples. Recently, works like GenQ and MixQ have demonstrated that leveraging off-the-shelf diffusion models to generate diverse images can significantly improve the quantization quality of classification models. However, directly applying this paradigm to object detection (OD) encounters three unique challenges: First, while image classification typically associates one class per image, OD averages around 8 objects from different classes per image—information density is a native requirement for this task, and single-object images generated by classification prompts are far from sufficient for detector training. Second, the category distribution of OD datasets exhibits an extreme long tail, where the ratio of bounding boxes of high-frequency classes to low-frequency classes in COCO differs by up to 1500 times, and images generated by classification-style random sampling will severely deviate from this actual distribution. Third, the generated images lack ground-truth labels and must rely on pre-trained detectors for pseudo-labeling—but pseudo-label noise can mislead the quantization fine-tuning process.

The core idea of this paper is that the key bottleneck of synthesis methods lies not in the volume of data, but in whether the synthesized data is systematically adapted to the above three unique OD challenges. Core Idea: Without modifying the diffusion model itself, this work introduces three carefully designed components—Information-Dense Prompting (IDP) to ensure multiple object categories per image, Intrinsic Distribution-Aware Selection (IDAS) to align the selected image set with the real data distribution, and Teacher-Guided Adaptive Noise Reduction (TANR) to replace hard pseudo-labels with soft teacher predictions. These components reform off-the-shelf diffusion models into GoodQ, a data generation pipeline tailored for zero-shot quantization of object detectors.

Method¶

Overall Architecture¶

GoodQ is a three-stage pipeline. The first stage uses off-the-shelf Stable Diffusion 1.5 with Information-Dense Prompting (IDP) to generate a library of 160k candidate images. The second stage utilizes Intrinsic Distribution-Aware Selection (IDAS) to select around 2,000 images from this library that best match the real COCO category distribution as the training set. The third stage incorporates Teacher-Guided Adaptive Noise Reduction (TANR) during the quantization-aware training process, substituting hard pseudo-labels with soft predictions from the full-precision teacher as the optimization targets.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Off-the-shelf Diffusion Model<br/>Stable Diffusion 1.5"] -->|IDP Prompts| B["Information-Dense Prompting<br/>~160k Candidate Image Pool"]
    B --> C["Pseudo-Label Annotation<br/>Pre-trained Detector"]
    C --> D["IDAS Distribution-Aware Selection<br/>~2k Training Set"]
    D --> E["Quantization-Aware Training QAT"]
    E --> F["TANR Teacher-Guided<br/>Adaptive Noise Reduction"]
    F --> G["Low Bit-Width Quantized Model<br/>W4A4 / W3A3"]

Key Designs¶

1. Information-Dense Prompting: Enabling Generated Images to Contain Multiple Co-occurring Object Instances

A core characteristic of OD datasets is the high information density per image—COCO averages 7.7 bounding boxes per image. If one directly uses GenQ-style classification prompts (e.g., "A photo of a car"), each generated image contains only one foreground object. Consequently, the detector learns almost zero scene co-occurrence patterns (such as apples and bananas frequently appearing together, or people and cars sharing a street view) during training, which severely limits the detection accuracy after quantization.

The solution of IDP is straightforward: it uses the template "A {D} photo of {Q} {C1} and {Q} {C2}", where {D} is a descriptive adjective (e.g., clean, nice), {Q} is a quantity adjective (e.g., multiple, many, several), and {C1} and {C2} are two randomly selected distinct class names. During each generation, a quantifier is randomly chosen from 6 options, and two categories are randomly selected, resulting in images that naturally contain co-occurring instances of multiple classes. Notably, the design philosophy of IDP is to "guide the diffusion model's output via prompt engineering rather than altering the model itself," which brings two benefits: zero extra training cost and seamless compatibility across different diffusion model backbones.

2. Intrinsic Distribution-Aware Selection: Matching the Real Category Distribution under Data-Free Conditions

The category distribution of OD datasets exhibits a severe long tail—the head category in COCO (e.g., person) accounts for nearly 30% of the bounding boxes, while tail categories (e.g., hair drier) represent less than 0.02%. Without filtering the generated images, a randomly sampled training set will deviate significantly from this distribution, causing the quantized model to overfit on high-frequency classes and degrade severely on low-frequency classes.

IDAS consists of two steps. The first is data-free distribution estimation: since the real data is inaccessible, the category prior is inferred from the bias parameters of the pre-trained detection head. Intuitively, the bias terms of the detection head implicitly encode the occurrence frequency of each class—the bias shifts toward high-prior categories during training. Specifically, the mean of the biases of all output channels in the detection head is calculated for each class and normalized into a target distribution. Experiments verify that the Pearson correlation coefficient between this method and the real COCO category distribution reaches 0.87, proving that the bias terms can effectively convey distribution information. The second step is greedy selection: an iterative algorithm is executed over the generated image library. In each round, the discrepancy between the category distribution of the currently selected set and the target distribution is calculated. The category with the "largest discrepancy and fewest candidate images" is prioritized, and the image with the highest number of bounding boxes is chosen from its candidates to join the set. This design simultaneously ensures distribution alignment (minimizing KL divergence) and information density (selecting images with the most bounding boxes).

3. Teacher-Guided Adaptive Noise Reduction: Circumventing Pseudo-Label Noise with Soft Labels

Generating pseudo-labels for synthesized images via a pre-trained detector inevitably introduces noise—the predicted bounding boxes might be inaccurate on unseen scenes, and the confidence scores might be low. If these hard pseudo-labels are directly used as targets for the QAT detection loss, the model's optimization signal will contain substantial noise, which is easily amplified under low bit-widths.

TANR's core strategy is to abandon reliance on hard pseudo-labels, substituting them directly with the soft predictions of the full-precision teacher model to serve as targets for \(\mathcal{L}_{\text{detect}}\). Specifically, for the three branches (box coordinates, objectness, and class probability), TANR replaces the targets from hard one-hot pseudo-labels with continuous probability distributions from the teacher model. Building on this, a QFocal-style adaptive weighting is introduced: targetness and class probability branch losses are scaled by a coefficient \(|y^{\mathcal{T}} - \hat{y}|^\gamma\), where \(y^{\mathcal{T}}\) represents the teacher's prediction and \(\hat{y}\) is the student's prediction. The intuition behind this coefficient is to automatically increase loss weights for samples where the teacher and student diverge significantly (i.e., where quantization errors are most likely to occur and learning is most needed), while relaxing constraints on samples the teacher is already confident about. Consequently, the QAT process naturally focuses on the regions most affected by quantization noise.

Loss & Training¶

The total training loss is a weighted combination of the detection loss \(\mathcal{L}_{\text{detect}}\) and the distillation loss \(\mathcal{L}_{\text{distill}}\). \(\mathcal{L}_{\text{distill}}\) consists of two terms: KL-divergence for prediction matching and L2 distance of intermediate feature maps, transferring full-precision knowledge from teacher to student. The Adam optimizer is used for training over 100 epochs, with a distillation temperature of 4 and a learning rate of 0.01 (YOLOv5) or 1e-5 (YOLOv11). The coefficient for \(\mathcal{L}_{\text{detect}}\) is set to 0.04 for YOLOv5 and 0.01 for YOLOv11, with the scaling factors \(\gamma\) for QFocal being 1.5 and 2.0, respectively.

Key Experimental Results¶

Main Results¶

Model	Bit-width	GoodQ	TSOD (unlabeled)	LSQ (real data)	Advantage
YOLOv5-s	W4A4	21.0 / 36.1	15.0 / 27.9	19.2 / 34.4	+6.0 mAP vs TSOD†
YOLOv5-m	W4A4	30.7 / 47.6	20.3 / 32.0	27.3 / 44.1	+10.4 mAP vs TSOD†
YOLOv5-l	W4A4	34.8 / 52.0	19.3 / 30.8	31.0 / 48.2	+15.5 mAP vs TSOD†
YOLOv11-s	W3A3	14.2 / 22.1	0.18 / 0.40	11.7 / 19.6	+14.0 mAP vs TSOD†
YOLOv11-m	W3A3	21.6 / 31.8	0.50 / 0.98	19.9 / 31.1	+21.1 mAP vs TSOD†
YOLOv11-l	W3A3	19.5 / 28.7	0.22 / 0.43	19.2 / 29.7	+19.3 mAP vs TSOD†

Note: TSOD† denotes the label-unavailable version of TSOD (completely zero-shot, under identical conditions as GoodQ); LSQ is trained on a subset of real data (with labels, non-zero-shot setting). All methods utilize 2,000 training images.

Ablation Study¶

Configuration	W8A8 (mAP)	W4A4 (mAP)	Description
Diffusion + Random Selection	33.5	16.0	Baseline: using GenQ classification prompts
+IDP (Information-Dense Prompting)	34.3	19.3	IDP improves low bit-width performance
+IDAS (Distribution-Aware Selection)	34.4	19.8	IDAS is also independently effective
+IDP + IDAS	35.0	21.0	Combination of both yields the best results
Baseline Loss	33.9	20.6	TSOD's \(\mathcal{L}_{\text{detect}}\) + \(\mathcal{L}_{\text{distill}}\)
+TANR	35.0	21.0	Teacher soft labels replace hard pseudo-labels

Key Findings¶

Diversity is key to low bit-widths: Diffusion-generated data is more dispersed in the CLIP embedding space compared to noise-optimized data (cosine distance 0.665 vs. 0.539). This diversity is further amplified under W3A3 (0.248 vs. 0.133), demonstrating that data diversity is crucial for maintaining accuracy under extremely low bit-widths.
Distribution alignment improves long-tail recognition: The KL divergence between IDAS and the real distribution (0.26) is far lower than that of random selection (0.48), directly translating to a +2 mAP gain under W4A4. A simplified strategy of "selecting images with the most bounding boxes" actually degrades performance (with a KL divergence up to 1.25), indicating that quantity does not equate to distribution matching.
TANR is more critical for Web/diffusion data: TANR brings a ~1 mAP improvement to diffusion data but has almost no effect on noise-optimized data—this is because the target confidence distribution of diffusion-generated images is more dispersed and pseudo-label noise is more pronounced, whereas noise-optimized generation exhibits concentrated confidence scores where pseudo-labels are already relatively reliable.

Highlights & Insights¶

Estimating category distribution from bias weights is an ingenious data-free technique: without accessing real data, the category prior distribution is inferred solely from the bias parameters of the pre-trained detection head, achieving a Pearson correlation coefficient of 0.87 with the real distribution. This approach does not rely on extra external knowledge and is highly plug-and-play.
Treating generative models as data augmentation tools rather than parameter update targets is a pragmatic design choice: GoodQ does not modify the diffusion model itself, relying only on prompt engineering and data selection to adapt to the OD task. This means it can readily leverage newer generative models without retraining any generative components.
The combination of adaptive weighting and soft labels (TANR) fundamentally sidestepped the pseudo-label noise issue—rather than filtering or correcting noisy pseudo-labels, it outright abandons hard labels and directly adopts the continuous distribution output by the teacher. This imbues the QAT optimization signal with native smoothness and confidence calibration properties.

Limitations & Future Work¶

The current method still relies on Stable Diffusion 1.5 to generate a 160k candidate image library before filtering down to a 2k training set, which incurs high generation costs. The paper reports a notable performance drop on smaller libraries (e.g., 2k candidates) (W4A4 ~19.3 vs. 21.0), implying that the distribution-alignment advantage of IDAS can only be fully exploited with an abundant candidate pool.
The QFocal adaptive weighting introduced by TANR is sensitive to the \(\gamma\) hyperparameter. The paper employs different \(\gamma\) values (1.5 vs. 2.0) across the two YOLO backbones, and the tuning cost remains unanalyzed.
The experiments focus primarily on YOLOv5/v11 single-stage detectors. Although the appendix contains Mask R-CNN results, verification on Transformer-based detectors (such as the DETR family) and more extreme deployment scenarios (such as W2A2 on mobile NPUs) is yet to be established.

vs. GenQ / MixQ: These methods introduce off-the-shelf generative models to classification-based ZSQ, whereas GoodQ extends this paradigm to object detection and systematically addresses the three unique challenges of OD: info-density, category distribution, and pseudo-label noise.
vs. TSOD: TSOD is a pioneering work in ZSQ-OD, synthesizing training sets via noise optimization. It performs acceptably on W8A8 but degrades severely under low bit-widths (W4A4/W3A3). Through diffusion-based data diversity and three targeted components, GoodQ significantly outperforms TSOD under fully zero-shot conditions.
vs. ZeroQ / Qimera: Classic noise-optimization ZSQ methods suffer from the synthetic diversity bottleneck, leading to poor performance under low bit-widths. GoodQ's substitution of noise-optimization with diffusion-based generation represents a fundamental methodological upgrade.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ First to systematically identify and address the three major challenges in OD-based ZSQ; utilizing biases to estimate distributions is one of the most clever engineering designs in this domain.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Covers 6 YOLO backbones, across multiple datasets (COCO/VOC/HomeObjects), generator architectures (SD1.5/SD2.1), and detection frameworks (Mask R-CNN), with comprehensive ablation studies.
Writing Quality: ⭐⭐⭐⭐⭐ The "three challenges, three solutions" narrative is clear and concise; experimental analyses provide causal explanations for each observation rather than merely listing numbers.
Value: ⭐⭐⭐⭐⭐ Directly addresses the most practical bottlenecks in edge deployment—unavailable private data and performance anxiety—providing a pipeline that is immediately applicable to detector quantization in production environments.