GoodQ: Zero-Shot Quantization for Object Detectors Using Off-the-Shelf Generative Models¶

Conference: ECCV2026
arXiv: 2606.31456
Code: TBD
Area: Model Compression / Object Detection
Keywords: Zero-Shot Quantization, Object Detection, Diffusion Models, Knowledge Distillation, Model Compression

TL;DR¶

GoodQ leverages off-the-shelf diffusion models to generate diverse training images, and systematically addresses three major challenges in zero-shot quantization for object detection using three components: Information-Dense Prompting, Inherent Distribution-Aware Selection, and Teacher-guided Adaptive Noise Reduction. Under extremely low bit-widths such as W4A4 and W3A3, it significantly outperforms previous noise-optimization-based methods.

Background & Motivation¶

Quantization-Aware Training (QAT) is a standard technique for deploying detection models to edge devices. It compresses full-precision models into low-bit integers by fine-tuning on a small subset of the original training data. However, in real-world scenarios, the original data is often unavailable due to privacy regulations or trade secrets—street-view data for autonomous driving or patient information in medical imaging cannot be reused. Zero-Shot Quantization (ZSQ) was developed to resolve this dilemma: it does not rely on real data but synthesizes proxy samples to replace the original training set.

Early ZSQ methods for image classification tasks relied on noise optimization (starting from Gaussian noise and back-propagating to match the Batch Normalization (BN) statistics of pre-trained models), yielding synthetic samples with limited diversity. Recent works such as GenQ and MixQ have demonstrated that utilizing off-the-shelf diffusion models to generate diverse images can significantly improve the quantization quality of classification models. However, directly applying this idea to object detection (OD) encounters three unique challenges: First, classification tasks assign one category to one image, whereas each image in OD contains an average of nearly 8 object instances of different categories—making information density a fundamental requirement; single-object images generated by classification prompts are far from sufficient for training detectors. Second, the category distribution of OD datasets exhibits an extreme long-tail distribution; the ratio of bounding boxes for high-frequency classes to low-frequency classes in COCO can differ by up to 1500 times, whereas classification-style random sampling severely deviates from this actual distribution. Third, the generated images lack ground-truth external labels and must rely on pre-trained detectors to generate pseudo-labels—but pseudo-label noise can mislead the quantization fine-tuning process.

The core idea of this paper is that the key bottleneck of synthesis methods lies not in the data volume, but in whether the synthetic data is systematically adapted to the aforementioned three OD-specific challenges. Core Idea: Without modifying the diffusion model itself, GoodQ transforms an off-the-shelf diffusion model into a data generation pipeline tailored for zero-shot quantization of object detectors through three meticulously designed components: Information-Dense Prompting (IDP) to make each image contain multi-class target instances, Inherent Distribution-Aware Selection (IDAS) to align the selected image subset with the real data distribution, and Teacher-guided Adaptive Noise Reduction (TANR) to replace hard pseudo-labels with teacher soft labels.

Method¶

Overall Architecture¶

GoodQ is a three-stage pipeline. In the first stage, an off-the-shelf Stable Diffusion 1.5 model is used under Information-Dense Prompting (IDP) to generate a candidate pool of approximately 160,000 images. In the second stage, Inherent Distribution-Aware Selection (IDAS) is applied to select around 2,000 images from the candidate pool that best match the real COCO category distribution to form the training set. In the third stage, Teacher-guided Adaptive Noise Reduction (TANR) is introduced during the quantization-aware training process, utilizing soft predictions from the full-precision teacher model as optimization targets instead of pseudo-labels.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Off-the-shelf Diffusion Model<br/>Stable Diffusion 1.5"] -->|IDP Prompts| B["Information-Dense Prompting<br/>~160k Candidate Pool"]
    B --> C["Pseudo-label Annotation<br/>Pre-trained Detector"]
    C --> D["IDAS Distribution-Aware Selection<br/>~2k Training Set"]
    D --> E["Quantization-Aware Training (QAT)"]
    E --> F["TANR Teacher-guided<br/>Adaptive Noise Reduction"]
    F --> G["Low-bit Quantized Model<br/>W4A4 / W3A3"]

Key Designs¶

1. Information-Dense Prompting: Generating Images with Multiple Co-occurring Target Instances

A core characteristic of OD datasets is high information density per image—COCO averages 7.7 bounding boxes per image. If one directly uses GenQ-style classification prompts (e.g., "A photo of a car"), each generated image will contain only one foreground object. The detector will learn almost zero scene co-occurrence patterns (e.g., apples and bananas often appearing together, or people and cars sharing street scenes) during training, which severely limits post-quantization detection accuracy.

IDP's solution is straightforward: it uses the template "A {D} photo of {Q} {C1} and {Q} {C2}", where {D} is a descriptive adjective (e.g., clean, nice), {Q} is a quantity indicator (e.g., multiple, many, several), and {C1} and {C2} are two randomly selected class names. For each generation, one of six quantity terms is randomly selected, and two classes are randomly chosen. Consequently, the generated images naturally contain co-occurring instances of multiple categories. Notably, the design philosophy of IDP is to "guide the diffusion model output via prompt engineering without modifying the model itself," which yields two benefits: zero extra training cost and seamless switching between different diffusion model backbones.

2. Inherent Distribution-Aware Selection: Matching Real Category Distribution Globally under Zero-Data Constraints

The long-tail category distribution in OD datasets is highly pronounced—head classes in COCO (such as person) account for nearly 30% of the bounding boxes, while tail classes (such as hair drier) constitute less than 0.02%. Without filtering the generated images, a randomly sampled training set would severely deviate from this distribution, causing the quantized model to overfit on high-frequency classes and collapse on low-frequency classes.

IDAS consists of two steps. The first step is data-free distribution estimation: since the real data is inaccessible, the category prior is derived from the bias parameters of the pre-trained detection head. Intuitively, the bias terms of the detection head implicitly encode the occurrence frequency of each class—the bias shifts toward high-prior classes during model training. Specifically, the average of all output channel biases of the detection head is calculated for each category, and then normalized to form the target distribution. Experiments verify that the Pearson correlation coefficient between this method and the real COCO category distribution is 0.87, proving that bias terms can indeed effectively reflect distribution information. The second step is greedy selection: an iterative algorithm is executed on the generated image pool. In each round, the discrepancy between the category distribution of the currently selected set and the target distribution is computed. Priority is given to categories with "the largest discrepancy and the fewest candidate images," and the image with the highest number of bounding boxes for that category is chosen from the candidate pool and added to the set. This design simultaneously ensures distribution alignment (minimizing KL divergence) and information density (selecting images with the most bounding boxes).

3. Teacher-Guided Adaptive Noise Reduction: Dodging Pseudo-Label Noise via Soft Labels

Labeling generated images with a pre-trained detector inevitably introduces noise—predicted bounding boxes might be inaccurate in unseen scenarios, and category confidence scores can be low. If these hard pseudo-labels are directly used as targets for the QAT detection loss, the model's optimization signals would contain substantial noise, which is easily amplified, especially under low bit-widths.

TANR's core strategy is to transition away from pseudo-labels, instead directly using the soft predictions of the full-precision teacher model as the target for \(\mathcal{L}_{\text{detect}}\). Specifically, for the three branches of bounding box coordinates, objectness, and class probability, TANR replaces target one-hot vectors of pseudo-labels with the continuous probability distribution of the teacher model. On this basis, QFocal-style adaptive weighting is introduced: the losses of the objectness and class probability branches are multiplied by the coefficient \(|y^{\mathcal{T}} - \hat{y}|^\gamma\), where \(y^{\mathcal{T}}\) is the teacher prediction and \(\hat{y}\) is the student prediction. The intuition behind this coefficient is that for samples where the teacher and student diverge significantly (i.e., those most prone to quantization errors and requiring the most learning), the loss weight is automatically increased; for samples the teacher is already confident about, the constraint is relaxed. This allows the QAT process to naturally focus on areas with the most severe quantization noise.

Loss & Training¶

The total training loss is a weighted combination of detection loss \(\mathcal{L}_{\text{detect}}\) and distillation loss \(\mathcal{L}_{\text{distill}}\). \(\mathcal{L}_{\text{distill}}\) consists of the KL divergence of prediction matching and the L2 distance of intermediate feature maps, transferring full-precision knowledge from the teacher to the student. The Adam optimizer is used for training for 100 epochs, with a distillation temperature of 4 and a learning rate of 0.01 (YOLOv5) or 1e-5 (YOLOv11). The coefficient of \(\mathcal{L}_{\text{detect}}\) is set to 0.04 for YOLOv5 and 0.01 for YOLOv11, with QFocal scaling factors \(\gamma\) set to 1.5 and 2.0, respectively.

Key Experimental Results¶

Main Results¶

Model	Bit-width	GoodQ	TSOD (Unlabeled)	LSQ (Real Data)	Gain
YOLOv5-s	W4A4	21.0 / 36.1	15.0 / 27.9	19.2 / 34.4	+6.0 mAP vs TSOD†
YOLOv5-m	W4A4	30.7 / 47.6	20.3 / 32.0	27.3 / 44.1	+10.4 mAP vs TSOD†
YOLOv5-l	W4A4	34.8 / 52.0	19.3 / 30.8	31.0 / 48.2	+15.5 mAP vs TSOD†
YOLOv11-s	W3A3	14.2 / 22.1	0.18 / 0.40	11.7 / 19.6	+14.0 mAP vs TSOD†
YOLOv11-m	W3A3	21.6 / 31.8	0.50 / 0.98	19.9 / 31.1	+21.1 mAP vs TSOD†
YOLOv11-l	W3A3	19.5 / 28.7	0.22 / 0.43	19.2 / 29.7	+19.3 mAP vs TSOD†

Note: TSOD† represents the label-unavailable version of TSOD (completely zero-shot, under the same conditions as GoodQ); LSQ is trained using a subset of real data (with labels, non-zero-shot conditions). All methods use 2000 training images.

Ablation Study¶

Configuration	W8A8 (mAP)	W4A4 (mAP)	Description
Diffusion + Random Selection	33.5	16.0	Baseline: using GenQ classification prompts
+IDP (Information-Dense Prompting)	34.3	19.3	IDP improves low bit-width performance
+IDAS (Inherent Distribution-Aware Selection)	34.4	19.8	IDAS is also independently effective
+IDP + IDAS	35.0	21.0	Best performance when combined
Baseline Loss	33.9	20.6	TSOD's \(\mathcal{L}_{\text{detect}} + \mathcal{L}_{\text{distill}}\)
+TANR	35.0	21.0	Teacher soft labels replacing hard pseudo-labels

Key Findings¶

Diversity is key for low bit-widths: Diffusion-generated data is more scattered in the CLIP embedding space compared to noise-optimized data (cosine distance 0.665 vs 0.539), and this diversity is further amplified under W3A3 (0.248 vs 0.133). This indicates that data diversity is crucial for maintaining accuracy under low bit-widths.
Distribution alignment improves long-tail recognition: The KL divergence between IDAS and the real distribution (0.26) is much lower than that of random selection (0.48), directly translating to a +2 mAP gain under W4A4. A simple strategy of "selecting images with the most bounding boxes" actually degrades performance (KL divergence up to 1.25), proving that quantity does not equal distribution matching.
TANR is more critical for diffusion data: TANR brings ~1 mAP gain to diffusion data but has almost no effect on noise-optimized data. This is because targets in diffusion-generated images have more dispersed confidence distributions and more noticeable pseudo-label noise, whereas objects generated by noise optimization have concentrated confidence, rendering the pseudo-labels already quite reliable.

Highlights & Insights¶

Estimating category distribution from bias weights is a clever data-free technique: without accessing real data, the prior distribution of categories is inferred solely from the bias parameters of the pre-trained detection head, boasting a 0.87 Pearson correlation coefficient with the real distribution. This method does not rely on extra external knowledge and is plug-and-play.
Treating generative models as data augmentation tools rather than parameter update targets is a practical design choice: GoodQ does not modify the diffusion model itself and only adapts to the OD task via prompt engineering and data selection. This means it can seamlessly leverage newer generative models at any time without retraining the generative components.
The combination of adaptive weighting and soft labels (TANR) fundamentally bypasses the pseudo-label noise problem. Instead of filtering or correcting noisy pseudo-labels, it discards hard labels entirely in favor of continuous distributions from the teacher output. This makes the optimization signals for QAT inherently smooth and equipped with confidence-calibration properties.

Limitations & Future Work¶

The current method still relies on Stable Diffusion 1.5 to generate a 160k candidate pool before selecting 2k training images, resulting in high generation costs. The paper reports a significant performance drop (W4A4 ~19.3 vs 21.0) under a smaller candidate pool (2k), indicating that the distribution alignment advantage of IDAS requires an abundant candidate pool to be fully realized.
The QFocal adaptive weighting introduced in TANR is sensitive to the \(\gamma\) hyperparameter. The paper uses different \(\gamma\) values (1.5 vs 2.0) across the two YOLO backbones, and the tuning cost has not been analyzed.
The experimental scope is concentrated on YOLOv5/v11 single-stage detectors. Although the appendix provides Mask R-CNN results, verification on Transformer-based detectors (DETR series) and more extreme deployment scenarios (such as phone-side NPU under W2A2) is yet to be established.

vs GenQ / MixQ: These methods introduced off-the-shelf generative models to classification-based ZSQ, whereas GoodQ extends this to object detection, systematically addressing the three OD-specific challenges of information density, category distribution, and pseudo-label noise.
vs TSOD: TSOD is a pioneering work in ZSQ-OD, synthesizing training sets using noise optimization. While its performance is decent under W8A8, it severely degrades under low bit-widths (W4A4/W3A3). GoodQ significantly outperforms TSOD under completely zero-shot conditions, owing to diffusion-based data diversity and three targeted components.
vs ZeroQ / Qimera: These are classic noise-optimization ZSQ methods. Constrained by the diversity bottleneck of synthetic data, they perform poorly under low bit-widths. Replacing the noise optimization pipeline with a diffusion model pipeline in GoodQ represents a fundamental methodological upgrade.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ This work is the first to systematically identify and address the three challenges of OD-targeted ZSQ, and using bias terms to estimate the distribution is the most engineering-savvy design in this domain.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ It covers six YOLO backbones, cross-datasets (COCO/VOC/HomeObjects), cross-generators (SD1.5/SD2.1), and cross-architectures (Mask R-CNN) with complete ablations.
Writing Quality: ⭐⭐⭐⭐⭐ The narrative of "three challenges, three solutions" is clear without redundancy, and each paragraph in the experimental analysis features causal explanations rather than pure listings.
Value: ⭐⭐⭐⭐⭐ It directly addresses the most realistic dilemmas in edge deployment—unavailable private data combined with performance anxiety—and the pipeline can be directly applied to detector quantization in production environments.