QASA: Quality-Guided K-Adaptive Slot Attention for Unsupervised Object-Centric Learning

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/ouyangtianran/QASA-tianran
Area: Others
Keywords: Slot Attention, Unsupervised Object-Centric Learning, K-Adaptive, Quality-Guided Selection, Gated Decoder

TL;DR

QASA introduces an unsupervised slot-quality metric based on competitive attention purity alongside gated decoders, decoupling slot selection from reconstruction to achieve dynamic object discovery that surpasses fixed-\(K\) baselines without tuning slot counts.

Background & Motivation

Object-centric learning (OCL) aims to decompose visual scenes into structured representations corresponding to physical objects without human supervision. Within this domain, Slot Attention has emerged as the mainstream paradigm by iteratively grouping patch embeddings into a set of latent slot vectors via competitive cross-attention. However, standard Slot Attention models predefine a global fixed number of slots \(K\). In real-world visual scenes, object cardinality varies drastically across images. Setting a rigid \(K\) inevitably triggers severe under-segmentation, over-segmentation, or wasted representational capacity. Furthermore, model segmentation fidelity is notoriously brittle to variations in \(K\), requiring exhaustive hyperparameter sweeps on each new benchmark.

To handle variable object numbers, prior works have explored \(K\)-adaptive Slot Attention variants. For instance, AdaSlot trains a slot selector under an expected slot-count penalty; however, the reconstruction objective inherently pressures the model to activate more slots for fidelity, producing an intractable optimization conflict between slot sparsity and reconstruction precision. Alternative approaches such as CoSA and MetaSlot perform slot filtering via principal component analysis or pre-learned discrete codebooks. Crucially, these existing methods rely primarily on feature prominence, which biases selection toward only the most salient foreground entities while lacking any explicit constraint on the binding quality of individual slots. Consequently, selected slots often absorb diffuse background regions or capture ambiguous mixtures of objects, causing existing \(K\)-adaptive frameworks to lag significantly behind well-tuned fixed-\(K\) baselines in complex real-world scenes.

To resolve the trade-off without introducing conflicting loss penalties, this paper decouples slot selection from the reconstruction objective entirely. Instead of training an auxiliary selector with regularizers, it exploits the intrinsic competition of Slot Attention to assess slot purity under fully unsupervised conditions. Core idea: introduce an unsupervised Slot-Quality metric measuring the purity of attention mass within each slot's winning region, iteratively select a compact subset of high-quality, high-coverage, and novel slots to train a gated decoder, and achieve adaptive object partitioning at inference via token-wise competition.

Method

Overall Architecture

The QASA architecture consists of four primary components: a frozen feature encoder, a Slot Attention module, a quality-guided slot selection module, and a gated decoder. During training, patch tokens \(Y\) extracted from an input image by a frozen DINO encoder are routed to a Slot Attention module with a maximum capacity \(K_{\max}\), yielding slot representations \(U\) and attention maps \(A_{\text{slot}}\). The quality-guided slot selection module evaluates the attention purity of each slot and executes a greedy search under coverage and novelty criteria to yield a binary active mask \(M\). The gated decoder physically suppresses unselected slots in both feature magnitude and attention distribution, reconstructing the DINO patch features against which mean squared error loss is computed. During inference, no selection module is needed: all slots compete at the patch level via \(\arg\max\), naturally inducing image-adaptive object segmentations.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Image X & DINO Encoder<br/>Extract Patch Features Y"] --> B["Slot Attention Module<br/>Produce Kmax Slots U and Attention Aslot"]
    B --> C["Quality Metric Evaluation<br/>Compute Winning-Region Purity Qi"]
    C --> D["Quality-Guided Slot Selection<br/>Greedy Selection under Coverage and Novelty"]
    D --> E["Dual-Gated Decoder<br/>Suppress Inactive Slots and Decode Features"]
    E --> F["Training: Feature Reconstruction MSE Loss<br/>Inference: Patch-Level Hard Winner Partition"]

Key Designs

1. Quality Metric Evaluation: Measuring Attention Purity within the Winning Region

Prior unsupervised object-centric models lack an explicit scalar signal to evaluate the binding quality of individual slots. An ideal object-centric slot should allocate high attention mass over its assigned object (approaching 1) and negligible mass elsewhere (approaching 0). Leveraging the token-wise normalization of Slot Attention, QASA determines the winning slot for each token \(t\) as \(w_t = \arg\max_{i \in \{1,\dots,K_{\max}\}} A_{t,i}\). The quality score \(Q_i\) of slot \(i\) is then formulated as the ratio of its accumulated attention mass within its winning tokens to its total attention mass across all \(N\) tokens:

\[Q_i = \frac{W_i^{\text{win}}}{W_i + \epsilon} = \frac{\sum_{t: w_t = i} A_{t,i}}{\sum_{t=1}^N A_{t,i} + \epsilon}\]

Because \(Q_i \in [0, 1]\), it provides a direct measure of spatial concentration. A diffuse slot that spreads fractional attention across multiple background patches without dominating any token will yield a small numerator and a large denominator, driving \(Q_i \to 0\). In contrast, a slot tightly bound to a coherent physical object commands near-exclusive attention over its territory, driving \(Q_i \to 1\).

2. Quality-Guided Slot Selection: Greedy Selection under Coverage and Novelty

Selecting slots solely by descending quality scores can produce two failure modes: terminating prematurely before explaining all image content, or redundantly picking multiple co-located slots that focus on the same prominent object. QASA addresses this with a greedy selection algorithm governed by token coverage quality \(\tau\) and cumulative image coverage rate \(\rho\). A token \(t\) is deemed covered by active set \(S\) if \(\sum_{i \in S} A_{t,i} \ge \tau\), and the global coverage represents the fraction of covered tokens.

To prevent redundant allocation over already explained image regions, candidate slots ordered by descending quality are filtered through a novelty metric:

\[\text{novelty}(i \mid S) = 1 - \frac{\sum_{t \in C_S} A_{t,i}}{\sum_{t=1}^N A_{t,i} + \epsilon}\]

where \(C_S\) denotes the set of currently covered tokens. A slot is admitted into active set \(S\) only if \(\text{novelty}(i \mid S) \ge \mu\). Iteration terminates immediately once total coverage meets threshold \(\rho\), producing the final selection mask \(M \in \{0, 1\}^{K_{\max}}\). This ensures complete scene explanation while strictly preserving orthogonal, compact object representations.

3. Dual-Gated Decoder: Isolating Inactive Slots Across Information and Logits

Following slot selection, unselected slots must be prevented from leaking information into the reconstructed scene representation. QASA introduces parameter-free gating mechanisms tailored to both Transformer and MLP decoders. In the Transformer decoder, where standard cross-attention can be corrupted by residual slot activations, a dual-gate formulation is derived from mask \(M\). The feature gate \(g_1 = M + (1 - M)\varepsilon_1\) element-wise modulates the Key and Value matrices to extinguish information flow, while the logit gate \(g_2 = M + (1 - M)\varepsilon_2\) injects a severe negative bias into attention logits before the softmax:

\[\text{Output} = \text{softmax}\left(\frac{Q (K \odot g_1)^\top}{\sqrt{d_k}} + \log(g_2)\right) (V \odot g_1)\]

This dual mechanism enforces complete isolation: \(g_1\) physically dampens feature magnitude, while \(g_2\) drives attention probabilities of inactive slots to zero. In the MLP decoder, inactive slots have their mixture logits penalized by \(-C\) (\(C \gg 0\)), restricting mixture weights \(\alpha_{i,t}\) strictly to active slots.

Loss & Training

The network reconstructs self-supervised DINO ViT-S/8 patch representations \(Y \in \mathbb{R}^{N \times d_y}\) using mean squared error:

\[\mathcal{L}_{\text{rec}} = \frac{1}{N \cdot d_y} \| Y - \hat{Y} \|_2^2\]

To ensure stable convergence of slot attention distributions in early training, a warm-up schedule allows all slots to participate in reconstruction for initial epochs (warmup = 10 on VOC, 10-20 on MOVi, and 0 on COCO). Training requires no auxiliary loss functions, expected slot-count penalties, or learned selection heads, eliminating optimization conflicts between reconstruction and capacity control.

Key Experimental Results

Main Results

QASA is evaluated on object discovery across real-world datasets (MS COCO 2017, PASCAL VOC 2012) and synthetic multi-object benchmarks (MOVi-C, MOVi-E), comparing against adaptive and fixed-\(K\) baselines.

Dataset	Metric	QASA (Trans.)	QASA (MLP)	MetaSlot (MLP)	AdaSlot (MLP)	SPOT (Trans. fixed)
COCO	mBOc	45.5	36.5	-	-	44.7
COCO	mBOi	36.7	33.6	29.5	27.4	35.0
COCO	mIoU	35.0	32.5	27.9	-	33.0
VOC	mBOc	57.9	53.8	-	-	55.6
VOC	mBOi	49.7	47.3	43.9	-	48.3
VOC	mIoU	47.9	45.5	42.1	-	-
MOVi-C	mBOi	46.9	42.2	35.0	35.6	47.3
MOVi-C	mIoU	46.1	41.1	-	-	46.7
MOVi-E	mBOi	39.1	34.6	-	29.8	40.1
MOVi-E	mIoU	37.8	33.0	-	-	39.3

In downstream object property prediction on COCO using an MLP probe, QASA achieves 59.6% Top-1 classification accuracy (outperforming MetaSlot at 40.9% and SPOT at 45.9%) and center coordinate regression scores \(R^2\)-X / \(R^2\)-Y of 67.9 / 71.7 (substantially exceeding DINOSAUR's 55.4 / 59.1).

Ablation Study

All ablations are conducted on MS COCO using the Transformer decoder.

Table 1: Influence of key selection components in Slot Selection

Configuration	Coverage Constraint	Quality Constraint	Novelty Constraint	mBOc	mBOi	mIoU	Note
Baseline (No Gating)	\(\times\)	\(\times\)	\(\times\)	25.9	25.5	24.9	All slots participate in decoding
+ Coverage	\(\checkmark\)	\(\times\)	\(\times\)	34.5	25.3	22.7	Coverage alone admits diffuse, low-quality slots
+ Quality	\(\checkmark\)	\(\checkmark\)	\(\times\)	45.0	35.0	32.9	Prioritizing purity yields huge +9.7% mBOi gain
Full Model	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	45.5	36.7	35.0	Novelty filters overlapping redundant slots

Table 2: Ablation of Dual Gating in Transformer Decoder

Gating Setup	Key/Value Gate (\(g_1\))	Logits Gate (\(g_2\))	mBOc	mBOi	mIoU	Note
No Gating	\(\times\)	\(\times\)	25.9	25.5	24.9	Inactive slots degrade reconstruction
Logits Gate Only	\(\times\)	\(\checkmark\)	26.1	25.5	24.9	Unattenuated K/V features leak into outputs
K/V Gate Only	\(\checkmark\)	\(\times\)	43.1	33.2	30.9	Physical suppression provides primary control
Dual Gating (Full)	\(\checkmark\)	\(\checkmark\)	45.5	36.7	35.0	Synergistic suppression achieves optimal results

Key Findings

• The Slot-Quality purity metric is the critical driver of performance: introducing Quality to the coverage search increases mBOi from 25.3% to 35.0%, proving that selecting slots by feature prominence or spatial coverage alone fails to separate clear object boundaries from background sprawl.
• Decoupled selection overcomes the historic performance deficit of adaptive slot models: while AdaSlot and MetaSlot achieve only 27.4% and 29.5% mBOi on COCO, QASA reaches 33.6% with an MLP decoder and 36.7% with a Transformer decoder, establishing a new state of the art that outperforms strong fixed-\(K\) baselines like SPOT (35.0%).
• The framework demonstrates exceptional robustness to the choice of \(K_{\max}\): across MOVi-C, varying \(K_{\max}\) from 15 to 32 keeps mBOi between 44.4% and 46.9%; on MOVi-E, scaling \(K_{\max}\) from 24 to 64 alters mBOi only within 38.3% to 39.5%, confirming that models no longer require exhaustive hyperparameter tuning for slot capacity.

Highlights & Insights

• Attention purity outclasses feature prominence for unsupervised selection: Rather than attempting to guess salient regions via PCA or codebook frequency, evaluating whether a slot cleanly commands its own winning patches provides an uncorrupted indicator of object-level binding.
• Decoupled non-parametric selection eliminates gradient conflict: Eliminating slot-count loss penalties converts cardinality adaptation into a clean inference-like filtering step before decoding, freeing reconstruction gradients from counterproductive regularization pressures.
• Parameter-free dual gating for seamless integration: Modulating both cross-attention feature magnitudes and logit scores ensures inactive slots are eliminated without appending auxiliary prediction heads, maintaining compatibility with diverse decoder families.

Limitations & Future Work

• Slight gap on synthetic datasets with fixed backgrounds: On MOVi-C and MOVi-E where background appearance is strictly uniform and object numbers cluster tightly, QASA (46.9% / 39.1%) trails hand-tuned fixed-\(K\) SPOT (47.3% / 40.1%) slightly, as redundant slots tend to overfit synthetic background modes.
• Slight domain dependence on coverage thresholds: The patch coverage threshold \(\tau\) requires setting to 0.5 on complex real images versus 0.8 on synthetic scenes; formulating a fully parameter-free adaptive threshold remains an open direction.
• Extension to spatiotemporal domains: The current design is developed for static 2D scenes; extending the attention purity criterion to enforce temporal consistency and tracking across video frames is a promising frontier.

Related Work & Insights

• vs AdaSlot: AdaSlot trains an auxiliary slot selector using expected-\(K\) loss penalties, which clashes with reconstruction objectives and limits its COCO mBOi to 27.4%; QASA completely discards slot-count loss penalties in favor of attention purity filtering, outperforming AdaSlot by 9.3% mBOi.
• vs MetaSlot: MetaSlot maintains a global codebook to discretize slots across the dataset, incurring heavy training overhead and struggling on complex backgrounds; QASA performs purely instance-level selection based on intrinsic competition, excelling across both segmentation and property prediction.
• vs SPOT / DINOSAUR: While fixed-\(K\) approaches achieve strong performance when \(K\) matches the dataset average, their accuracy deteriorates sharply under cardinality mismatches; QASA dynamically adjusts slot usage per image and surpasses them on real-world datasets without requiring tuning of \(K\).

Rating

• Novelty: ⭐⭐⭐⭐⭐ Elegant unsupervised Slot-Quality formulation leveraging internal attention competition to resolve a longstanding OCL dilemma.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluations covering four benchmarks, detailed ablations, parameter sensitivity, and downstream property probing.
• Writing Quality: ⭐⭐⭐⭐⭐ Clear logical motivation, mathematically rigorous formulations, and clean structural presentation.
• Value: ⭐⭐⭐⭐⭐ Bridges the performance gap between adaptive and fixed-slot paradigms, significantly advancing practical unsupervised object-centric learning.