CountEx: Fine-Grained Counting via Exemplars and Exclusion

Conference: ECCV2026
Paper: ECCV Paper
Code: https://github.com/bbvisual/CountEx
Area: Multimodal VLM / Object Detection
Keywords: fine-grained counting, negative prompts, visual exemplars, shared prototypes, open-vocabulary detection

TL;DR

CountEx jointly encodes what to count and what to exclude as detection queries, retaining shared features while suppressing negative-exclusive patterns to resolve fine-grained ambiguity, and achieves 26.61 and 12.72 MAE in the novel- and known-category CoCount settings.

Background & Motivation

Visual counting requires more than finding how many objects appear in an image: it must identify which instances the user intends to include. Methods such as CountGD support text and a few boxed exemplars, but positive prompts alone may not delimit the target set when red and green apples or washers of different materials coexist. A model can learn to find apples while still counting unwanted green apples, or exploit training-set bias by counting whichever category dominates the image. Many existing counting images contain only one relevant category, allowing this prompt-insensitive shortcut to achieve respectable results.

Negative prompts appear to offer a direct solution, but “not green apples” does not mean “not apples.” Positive and negative objects share shape, texture, and category semantics; naive subtraction can remove the common evidence needed to recognize the target and leave the model focused on color differences alone. The scene may also contain a third category that the user never mentions, which is neither a positive target nor an explicit negative and must not be counted merely because it differs from the negative class. Predicting two density maps and subtracting them, or detecting the categories separately and suppressing overlapping boxes, therefore does not resolve the semantic boundary at the feature level.

The authors reformulate the problem as identifying visual features shared by both categories, then extracting the residual evidence that distinguishes negative queries from those shared features. They also construct CoCount, where two object types coexist in substantial numbers, instead of evaluating a negation phrase only on single-category counting images. Core Idea: negative prompts should provide exclusive evidence to suppress, rather than act as the opposite vector of a positive prompt; shared-prototype decomposition and selective query suppression retain target-category context while reducing confusion with similar negatives.

Method

Overall Architecture

Inputs comprise an image and a positive text description, optionally supplemented with positive exemplar boxes, negative text, and negative exemplar boxes; outputs are positive-category detections and their count. CountEx uses the open-vocabulary query encoder from LLMDet to condition the same image separately on positive and negative prompts, producing two candidate-query sets. Text becomes tokens through the language encoder; patch features from exemplar crops are average-pooled, linearly projected, and concatenated with the text embeddings. An exemplar is therefore not an extra answer annotation: it specifies the appearance of the object denoted by a word in the current scene. Both query sets contain image context, rather than being global sentence embeddings that are simply subtracted.

Discriminative Query Refinement (DQR) then performs Shared Feature Identification, Exclusive Feature Extraction, and Selective Query Refinement in sequence. Detection heads predict boxes and confidence scores from refined positive queries, and the final count enumerates predictions above a threshold. A density branch supplies auxiliary supervision during training only; inference does not integrate its output and subtract a negative-category count. The diagram shows the main flow when both positive and negative prompts are available; the paper also evaluates configurations without exemplars or negative prompts.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Image and positive/negative prompts<br/>Optional visual exemplars"] --> Encoder["LLMDet query encoding<br/>Positive and negative queries"]
    Encoder --> Shared["Shared Feature Identification"]
    Shared --> Exclusive["Exclusive Feature Extraction"]
    Encoder -->|Negative queries| Exclusive
    Exclusive --> Refine["Selective Query Refinement"]
    Encoder -->|Positive queries| Refine
    Refine --> Output["Detection heads and thresholding<br/>Positive-category count"]
    Encoder -.->|Training only| Density["Density branch and point supervision"]

Key Designs

1. Shared Feature Identification: protect category evidence needed by both object types

This stage asks not which object is negative, but why the positive and negative categories look alike. Learnable prototype vectors cross-attend to the concatenated positive and negative queries, followed by a linear transformation and layer normalization to form image-conditioned shared prototypes. Learnable parameters provide their starting point, but the contextual representation depends on the current query sets; the prototypes are not a fixed global dictionary of colors or shapes. When distinguishing two types of coffee candy, shared prototypes should retain candy shape and packaging texture rather than equate one color with the entire category. They define a shared feature space that subsequently separates a negative query into common content and content more specific to negatives.

Prototype learning combines sharedness and diversity constraints. The sharedness loss encourages each prototype to find at least one high-cosine-similarity match in both the positive and negative query sets, discouraging prototypes that represent only one side. The diversity loss encourages approximately orthogonal prototype embeddings so that they do not all collapse onto one salient appearance and miss other common features. Sharedness is an optimization preference, not a strict guarantee that every prototype contains only pure category semantics. The visualization on page 15 provides partial support: patches associated with one prototype share elongated, rod-like structures across nails, screws, and craft sticks.

2. Exclusive Feature Extraction: retain only the distinctive residual worth excluding

Negative queries still contain substantial shared content and should not directly suppress positive queries in their entirety. The model first computes each negative query's cosine similarity to every shared prototype and uses the maximum as its selection score. A lower score means that even the nearest shared prototype is relatively dissimilar, suggesting more negative-exclusive information. The paper keeps the \(m\) lowest-scoring negative queries, with \(m=64\) by default; these are low-similarity queries, not high-confidence detection boxes.

\[ \sigma_i=\max_j\operatorname{cos}(\mathbf{q}^{\mathrm{neg}}_i,\mathbf{c}_j),\qquad \mathcal{I}_{\mathrm{excl}}=\operatorname{SmallestK}(\{\sigma_i\},m). \]

This notation expresses the selection rule described on page 9, where \(\mathbf{c}_j\) is a shared prototype and \(\mathcal{I}_{\mathrm{excl}}\) contains selected query indices. Each selected negative query is then projected onto the space spanned by the shared prototypes, and its projected shared component is removed. The remaining residuals form the negative-exclusive reference set used for selective suppression. Selection and projection are not redundant: selection reduces the candidate pool and removes obviously shared queries, while projection removes common content remaining within each selected query. Equation (2) in the cached extraction has missing operators and damaged dimensional notation, so this note preserves the reliable textual mechanism rather than reconstructing the authors' exact projection expression.

3. Selective Query Refinement: apply exclusion according to each positive query's relevance

With negative-exclusive references available, DQR uses positive queries as cross-attention queries and negative residuals as keys and values. The attention response measures a positive candidate's association with negative patterns, rather than merely checking whether two predicted boxes overlap. A residual suppression connection with a learnable gate controls subtraction strength, with layer normalization and dropout participating in the update. Positive queries weakly aligned with negative-exclusive patterns should remain largely unchanged, whereas strongly aligned queries receive stronger suppression. The gate regulates suppression magnitude instead of rewriting every candidate with equal strength.

This places negative information before the final decision, rather than performing arithmetic on two independently estimated counts. Retained positive queries still carry shared category evidence, allowing the detection head to assess whether a candidate belongs to the target category rather than merely being different from the negative class. Box regression and classification heads produce candidate locations and scores, and the output enumerates detections above a confidence threshold. The consulted main text does not specify the threshold value, so no conventional default is assumed here. The reduction in false counts still depends on the decomposition learned by the preceding stages; preserving shared features is not a mathematical guarantee against missed instances.

A Worked Example

Figure 5a uses the same image containing black and brown coffee candies to test whether changing positive and negative intent changes the counted objects. First, black coffee candy is specified as positive and brown coffee candy as negative, with visual exemplars supplied according to the available configuration. Dual encoding produces two candidate sets; Shared Feature Identification retains common candy appearance, while Exclusive Feature Extraction produces distinctive references associated with brown candies. Selective Query Refinement suppresses brown-candy patterns in positive candidates, and the detection head outputs 103 against a ground-truth black-candy count of 96.

Swapping positive and negative prompts also swaps the exclusion target: the model predicts 79 brown candies against a ground-truth count of 94. These results show that it does not always return the same dominant-category count or simply divide the total number of objects evenly. Both predictions remain imperfect, so the example demonstrates conditional control rather than flawless instance recognition. The semantic explanation of shared and exclusive features illustrates the pipeline; Figure 5a does not annotate the internal query residuals for this image individually.

Loss & Training

Training uses point annotations and jointly optimizes matched-query classification, predicted-box center localization, density prediction, and prototype learning. The overall objective on pages 9–10 has the following compact form:

\[ \mathcal{L}=\lambda_{\mathrm{cls}}\mathcal{L}_{\mathrm{cls}}+\mathcal{L}_{\mathrm{loc}}+\lambda_{\mathrm{den}}\mathcal{L}_{\mathrm{den}}+\mathcal{L}_{\mathrm{proto}}. \]

Classification uses focal loss on matched predictions to separate positive instances from non-target candidates through their confidence scores. Localization compares predicted box centers with ground-truth point coordinates using \(L_1\) distance; this does not imply full box-size supervision for every training instance. The density branch starts from text-fused visual tokens before query decoding and predicts a spatial distribution through an FPN-based density head. Point annotations are converted into density supervision with two-dimensional Gaussian kernels, and mean squared error supplies dense spatial learning signals to the encoder. The prototype term combines weighted sharedness and diversity losses, encouraging common responses across positive and negative sets and variation among prototypes, respectively. The model requires end-to-end training; using negative prompts at inference time does not make the overall method training-free. Unlike methods that synthesize data and adapt separately for each target category, CountEx treats negative prompts as conditions for a forward pass without optimizing again for each test category. The consulted main text does not specify all loss weights, the optimizer, or the training duration, and this note does not infer those reproduction settings.

Key Experimental Results

Main Results

CoCount contains 1,780 videos and 10,086 annotated frames, covering 55 categories and 97 category pairs, including 50 inter-category pairs and 47 intra-category variant pairs. The training, validation, and test splits contain 7,417, 1,335, and 1,334 annotated frames, respectively; an additional Train2 split lacks dense annotations and is not used in the experiments. Training point annotations and 3 exemplar boxes per category are propagated from manually annotated frames with CoTracker3, followed by sampling and quality filtering. The NC protocol holds out each of the five supercategories in turn; KC training covers all supercategories, but training and test videos remain strictly disjoint. MAE averages absolute differences between predicted and ground-truth counts, while RMSE is the square root of the mean squared count error; lower is better for both.

The following results come from Table 1 on page 11. Methods use the same training splits, but CountEx receives inclusion and exclusion cues whereas conventional baselines receive inclusion cues only, so prompt information is not held equal.

Method	NC MAE	NC RMSE	KC MAE	KC RMSE
CAD-GD	34.08	50.39	16.00	27.52
GroundingREC	29.29	42.43	17.54	27.41
CountGD	33.78	48.29	15.55	28.32
LLMDet	33.22	47.66	16.82	29.23
LLMDet + Detection-Suppression	28.93	42.17	16.94	29.71
LLMDet + Density-Subtraction	47.71	74.05	24.19	38.93
LLMDet + Count-All-then-Halve	31.51	46.80	19.77	31.66
CountEx	26.61	38.86	12.72	23.99

Relative to LLMDet, NC MAE falls from 33.22 to 26.61, a reported relative reduction of 19.9%; KC MAE falls from 16.82 to 12.72, a reported reduction of 24.4%. Density subtraction instead substantially worsens performance, showing that accepting negative prompts and using them correctly are separate problems: negative information does not automatically help.

Ablation Study

The following selection from Tables 3a–3c on page 14 reports KC results for prompt combinations, DQR components, and training supervision; the groups are not one cumulative module-addition experiment.

Group	Config	MAE	RMSE
Prompts, Table 3b	Positive text only	15.96	28.76
Prompts, Table 3b	Positive text and positive exemplars	15.75	27.59
Prompts, Table 3b	Positive text and negative text	13.22	26.23
Prompts, Table 3b	Positive/negative text and exemplars	12.72	23.99
Architecture, Table 3c	Without Shared Feature Identification	14.18	26.48
Architecture, Table 3c	Without Exclusive Feature Extraction	13.40	25.30
Architecture, Table 3c	Without Selective Query Refinement	16.10	30.49
Losses, Table 3a	Without density and prototype losses	15.20	28.33
Losses, Table 3a	Without prototype loss, retaining density loss	13.00	25.22
Losses, Table 3a	Without sharedness loss	13.64	25.07
Losses, Table 3a	Without diversity loss	14.05	26.28
Full model, Tables 3a–3c	All components and supervision	12.72	23.99

Key Findings

• Adding negative text alone reduces KC MAE from 15.96 to 13.22; all prompts improve it further to 12.72, so additional exemplar boxes do not explain the entire gain.
• Removing Selective Query Refinement causes the largest DQR degradation, reaching 16.10 MAE: identifying distinctive features is insufficient without using them to update positive queries.
• In Table 3c, increasing negative queries from 16 to the default 64 reduces MAE from 13.87 to 12.72; 128 queries yield 12.99 MAE but slightly lower RMSE, 23.70 versus 23.99, so 64 is not optimal on every metric.
• Table 2a on page 12 reports 18.53 MAE and 30.46 RMSE for CountEx trained on CoCount and transferred zero-shot to LOOKALIKES; the per-category synthesis-and-adaptation method achieves 10.00 MAE, so an unconditional best-result claim would ignore adaptation conditions.
• Table 2b on page 12 reports PairTally inter-scene and intra-scene MAE of 15.61 and 12.57; CountGD trained on the same CoCount data obtains 19.67 and 15.67, respectively.
• The text on page 12 reports 8.63 MAE after fine-tuning on FSC-147 with positive prompts only, still behind CountGD at 5.74; this is not cross-dataset evaluation without training.
• The efficiency test on pages 12–13 averages 1000 images on one NVIDIA A5000 with batch size 1: latency with and without DQR is 194.0 and 190.2 ms, an increase of 3.8 ms or approximately 2%.

Highlights & Insights

• Negation requires contextual interpretation. Protecting evidence for being a candy before suppressing evidence for the unwanted candy type better matches fine-grained semantics than subtracting the entire negative representation.
• The dataset makes prompt use consequential. Both categories occur in substantial numbers and positive/negative intent can be swapped, exposing the shortcut of counting only the dominant category.
• Computation operates at the query level. Selected negative references participate in cross-attention with little measured additional latency; this result applies only to the reported hardware and inference configuration.

Limitations & Future Work

• Exclusion relevance still matters. Page 14 reports 28.43 MAE and 42.29 RMSE with irrelevant negative prompts in NC, worse than 26.61 MAE with relevant negatives; incorrect prompts are not cost-free assistance.
• Extreme similarity and density remain difficult. Figure 4 reports KC MAE of 43.1 for counts above 200, with still larger errors for certain challenging category pairs; overall averages do not mean all fine-grained objects are handled reliably.
• Dataset coverage has boundaries. CoCount primarily features manually arranged scenes with two co-occurring object types; excluding more categories simultaneously and handling more complex backgrounds or capture conditions require dedicated evaluation, which is this note's assessment of experimental coverage.
• Text extraction limits reproduction details. Some formulas on pages 8–9 and dense labels in Figure 2 are damaged, although method prose and Tables 1 and 3 remain readable; this note does not reconstruct projection operators, the exact gating expression, or unspecified hyperparameters.
• Useful additional experiments. Future evaluations could hold positive/negative prompt information constant across exclusion mechanisms and report false-positive and missed-instance rates under incorrect negatives, rather than only aggregate counting errors.

Related Work & Insights

• Relation to LLMDet: CountEx reuses its open-vocabulary query encoding; the main addition is joint positive/negative query modeling and DQR for counting, not training a new general-purpose large language model.
• Relation to CountGD: Both use text and visual exemplars, but CountEx explicitly models objects to exclude; the single-category FSC-147 results also establish a clear task boundary for this advantage.
• Relation to FiGO: The cited work by D'Alessandro and colleagues improves fine-grained counting through synthetic confounding categories and test-time adaptation, whereas CountEx uses lightweight inference-time exclusion cues; their test budgets differ.
• Relation to T-Rex-Omni: Negative visual prompts have also been used in generic object detection; CountEx specifically separates shared and negative-exclusive features before refining few-shot counting queries.
• Transferable direction: Retrieval or localization tasks that require finding A while excluding a similar B could investigate representation decomposition that preserves shared semantics before suppressing distinctive negatives, but this is a research direction rather than a cross-task result established here.

Rating

These are the note author's subjective assessments on a 5-point scale. - Novelty: 4/5. Negative prompts have precedents, but shared prototypes, exclusive residuals, and counting-query suppression form a clearly task-specific design. - Experimental Thoroughness: 4/5. Novel/known categories, external datasets, and multiple ablations are covered; equal-information prompt comparisons and more complex exclusion conditions could strengthen the evidence. - Writing Quality: 4/5. Motivation and experiments align clearly; damaged cached formulas are an extraction limitation, not evidence about the original paper's typesetting quality. - Value: 4/5. The model and dataset support controllable fine-grained counting, particularly when similar categories coexist and exclusion intent is explicit.