Task-driven Processing with Coarse-to-Fine Glimpse-based Active Perception

Conference: ECCV2026
Paper: ECCV 2026
Area: Robotics & Embodied AI
Keywords: active perception / instance detection / glimpse / log-polar / high-resolution vision

TL;DR

CF-GAP is a task-driven perception front-end: it convolves multi-view examples of the search target over a downscaled scene to obtain a coarse search heatmap, picks 30 coarse glimpse locations with winner-takes-all plus inhibition-of-return, then drives a log-polar sensor in a closed loop on each coarse location so that the 2D centroid of a fine search map walks the fixation onto the target; only a handful of high-resolution RoIs and the glimpse locations are handed to an off-the-shelf instance detector, raising the AP of several state-of-the-art detectors by up to about 20 points on HR-InsDet and Robotools and letting lightweight detectors surpass their heavier counterparts.

Background & Motivation

Instance detection today — not finding every object of a category, but localizing one specific instance given a few photographs of that object — almost always follows the same two-stage pipeline: a foundation model pretrained on large-scale data (SAM, GroundingDINO) exhaustively proposes every object-like region in the whole image, and DINOv2 features then match each proposal against the target examples (OTS-FM, NIDS-Net). Both stages process the image uniformly. The first is completely blind to what is being searched for — to avoid missing anything it must surface every object-like region; the second does use the task information, but by then the entire image has already been encoded. The tension becomes acute at high resolution: HR-InsDet scenes are 6144×8192, yet ViT-style backbones must split the image into a fixed number of patches to keep the quadratic cost of self-attention in check, so input is capped below 2048×2048 and a small target may be left with only a few dozen pixels — critical detail is erased by the resize itself.

Human vision does not work that way. The eye has a fovea whose sampling density is highest at the center and decays logarithmically toward the periphery, so it keeps both fine detail and a wide field of view; coupled with eye movements, humans search coarse-to-fine: low-resolution peripheral cues drive large saccades (macrosaccades) toward promising locations, and small microsaccades then inspect those locations in detail. Existing saccadic vision models — GAP, which this paper builds on directly, is one — have shown that processing only selected image parts yields strong out-of-distribution generalization and sample efficiency, but they choose fixation points by saliency, or by a policy learned indirectly through the task loss, and at inference time they cannot be steered by "which object am I looking for." That is the gap this paper fills: making the coarse-to-fine fixation process directly driven by the search target (the specific instance to be localized), and packaging all of it as a front-end that plugs into any existing detector rather than a new detector trained from scratch.

Core idea: an inner/outer pair of nested glimpsing processes — coarse search-map location selection plus a closed-loop log-polar sensor that refines each location — turns high-resolution scene processing into processing a few high-signal-to-noise local regions, and lifts task information (the target examples) to the very front of the perception pipeline, so the downstream detector needs neither modification nor a heavier architecture.

Method

Overall Architecture

CF-GAP frames detection as steering a virtual log-polar sensor around the scene. The input is a high-resolution scene plus several multi-view examples of the target; the output is (with the downstream detector's help) the best-matching instance in the scene. In between sit two nested levels of glimpsing. The outer level is coarse glimpsing: on a 2× downscaled version of the whole scene, it repeatedly takes the maximum of a "coarse search map" — a 2D heatmap whose value rises with the likelihood that the target is present at that location — computed by convolving target features with scene features, and masks out already-visited locations with inhibition-of-return (IoR), collecting \(N_c=30\) coarse glimpse locations. Each coarse location then triggers one inner round of fine glimpsing: a log-polar sensor extracts a local view of the full-resolution scene that is fine at the center and logarithmically compressed at the periphery, a fine search map is computed on that view, and its 2D centroid becomes the next fixation; the loop iterates \(N_f=3\) times. At the end of every fine loop, CF-GAP hands the downstream instance detector three things: a fixed-size RoI cropped around the last fine glimpse location, the fine glimpse locations themselves, and the target's multi-view examples. The detector proposes candidates on those inputs and matches them, and after all \(N_c\) coarse rounds the globally best match is returned.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}}%%
flowchart TD
    A["High-resolution scene<br/>+ target examples"] --> B["Task-driven coarse glimpse selection<br/>coarse search map → WTA + IoR"]
    B --> C
    subgraph C["Log-polar closed-loop fine glimpsing"]
        direction TB
        D["Log-polar sensor<br/>sharp center + compressed periphery"] --> E["Perceiver-style encoder<br/>compress → self-attention → decode"]
        E --> F["Fine search map and centroid"]
        F -->|another round| D
    end
    C --> G["Glimpse interface<br/>RoI + glimpse locations + target examples"]
    G --> H["Downstream detector proposes and matches"]
    H -->|fewer than Nc rounds| B
    H -->|Nc rounds done| I["Return best match"]

Key Designs

1. Task-driven coarse glimpse selection: decide where to look first by convolving target examples over the downscaled scene

Today's two-stage instance detection leaves the task in the second stage: target examples only enter the picture after the entire image has been encoded by SAM / GroundingDINO and every proposal has been extracted. CF-GAP moves that information to the very front. What it uses is still the classic "match target features against scene features" scheme (the search-map formulation comes from Transporter Networks and "Finding any Waldo"), but it runs it once on a 2× downscaled scene, which costs almost nothing. Both the scene encoder and the target encoder are lightweight MobileNet-V3: the features of the target examples are first averaged over the spatial dimensions into a single feature vector, which is then convolved over the downscaled scene features to produce the coarse search map.

Location selection is deliberately plain. Each round takes the highest response on the search map (winner-takes-all), masks out a neighborhood around that location on the map (inhibition-of-return) so the same region is not chosen repeatedly, and takes the next maximum. Thirty rounds yield thirty coarse glimpse locations. The benefit is that these locations spread out naturally over the few regions that resemble the target instead of piling onto one peak — the paper's visualizations show the glimpses covering only a small fraction of the scene rather than spreading uniformly (Fig. 7).

The reason this works is that it makes task conditioning as cheap as possible: target information is used only to produce one low-resolution heatmap, so there is no need to push the target into the downstream detector's feature space, nor to retrain or fine-tune it — the same front-end can serve any detector. The price is that coarse locations come from a downscaled scene, so each is only a rough estimate that can sit noticeably off the true target. That spatial error is exactly what the next level of fine glimpsing exists to fix.

2. Log-polar closed-loop fine glimpsing: walking an offset coarse location onto the target

A coarse location is a low-resolution estimate and may miss the target, while downstream detectors — especially SAM-like ones — crop the object at the location they are given, so being off target means losing the point outright. The intuitive fix is to cut a small Cartesian crop and zoom in, but the crop size is extremely hard to tune: too small and there is not enough context (the target may not even be inside), too large and a crowd of distractors comes along. The right size also depends sensitively on how far the coarse location is from the object and on the object's own size, so no single setting generalizes.

CF-GAP instead uses a fovea-inspired log-polar transform: sampling is centered on the fixation point, resolution is finest at the center, and sampling density decays logarithmically outward, packing "sharp locally, wide globally" into one fixed-size image. Its key property is that it is visually insensitive to size — sweeping the sampling diameter from 256 all the way to 2048 leaves the log-polar image looking almost unchanged (the differences are squeezed into the outermost handful of pixels), whereas a Cartesian crop turns into a completely different image at each size (Fig. 5). The paper's explanation is that because peripheral resolution decays logarithmically, the content near the center of the receptive field carries the same relative weight regardless of diameter, so fine search-map extraction becomes robust to peripheral distractors and the fine fixation stays stable; Fig. 6 confirms this numerically — log-polar performance is flat over a wide range of sizes while the crop-based variant is highly size-sensitive.

Non-uniform resolution, however, is precisely a CNN's blind spot: convolution implicitly assumes every input location has the same resolution and treats all regions alike. The fine-stage scene and target encoders are therefore replaced by Perceiver-style modules (Fig. 2B-C). A glimpse is split into non-overlapping patches projected into a \(D\)-dimensional space, and the scene encoder uses a small set of learnable embeddings as queries to compress the \(H\times W\times D\) spatial input into \(N\) latent embeddings via cross-attention (\(N \ll H\times W\)), which also removes the quadratic cost of the subsequent self-attention; self-attention then integrates the high-frequency detail near the fixation with the broader peripheral context; finally a second cross-attention decodes back to the original spatial dimensions, producing the glimpse feature map \(F_s^{*}\). The decoding cross-attention uses a distinct set of learnable embeddings as values, decoupling the feature space used for compression from the one used for the output, which the authors describe as letting the output feature map be optimized specifically for comparison with target features. Those equations are rendered as garbage in the cached main text, so the following is a reconstruction from the prose (⚠️ refer to the original paper):

\[F_s^{*} = \mathrm{CrossAttn}\Big(\mathbf{e}_o,\ \mathrm{SelfAttn}\big(\mathrm{CrossAttn}(\mathbf{e}_q,\ F_s)\big)\Big),\qquad \mathbf{e}_q,\mathbf{e}_o \in \mathbb{R}^{N\times D}\]

where \(\mathbf{e}_q\) is the query embedding used for compression and \(\mathbf{e}_o\) the value embedding used for decoding. The target encoder reuses the same cross- and self-attention blocks but iteratively attends to \(M\) target glimpses: at round \(m\) the target glimpse \(F_m\) supplies keys and values and the previous output supplies the query for cross-attention, followed by self-attention, with the round-0 query initialized to \(\mathbf{e}_q\); the final output \(\mathbf{h}_T\) is a compact set of target features. Those features are correlated with the glimpse feature map by convolution, yielding \(N\) correlation maps that are averaged into a single fine search map.

Fine glimpsing is closed-loop: the 2D centroid of the fine search map directly gives the next fixation, the sensor moves there, extracts a new glimpse, and computes the next search map. The precise meaning of "task-driven" here is worth stating — it is not a fixation policy learned by reinforcement learning. The entire guiding signal is the centroid of the "target features × current scene" correlation map: task information (the target examples) enters the correlation through the encoders, and the centroid collapses that map into one coordinate, so the fixation moves toward whatever looks more like the target. As for why the centroid rather than the maximum response is used, the paper states only that the 2D centroid determines the next location and does not argue for the choice (⚠️ refer to the original paper); a natural reading is that the correlation map near the target is a broad response rather than a single spike, so the centroid smooths it and keeps the movement from jumping to an isolated noisy peak.

3. Glimpse interface: making CF-GAP plug into any instance detector

CF-GAP is deliberately a front-end rather than a new detector: it never emits boxes, it only decides where to look, and the final judgement belongs to the downstream model. To that end it hands the downstream three things at the end of every fine loop: ① a fixed-size high-resolution RoI cropped around the last fine glimpse location; ② the fine glimpse locations themselves; ③ the target's multi-view examples (used to match the detected candidates). How ② is consumed depends on the downstream. Models that support point prompting, such as SAM / MobileSAM, can swap the dense grid of 2D point prompts that would otherwise blanket the whole image for a few glimpse locations, shrinking both the area examined and the number of calls. GroundingDINO does not support point prompting, so glimpse locations act as spatial filters that discard any detected box not containing them — fewer candidates means fewer errors in the subsequent Stable Matching. The paper ablates this item on its own: with the RoI but without glimpse locations, CF-GAP + NIDS-Net reaches 41.8 AP on small objects and 49.5 AP on hard scenes; adding the locations lifts those to 52.1 / 63.2.

A non-trivial consequence of this interface is that the downstream model can be made smaller. Because CF-GAP has already filtered out the vast majority of irrelevant content, what the downstream receives is a high-signal-to-noise local region, so it no longer needs the bulky model that was sized "to cope with complex scenery containing numerous objects and varied visual intricacies." The paper calls this a division of labour between looking and seeing — CF-GAP decides where to look, the downstream sees clearly there. Pairing the lightweight OTS-FM_MobileSAM with CF-GAP lets it overtake the original OTS-FM_SAM and OTS-FM_GroundingDINO on the hardest subsets. Note, however, that CF-GAP is not free for the downstream: the downstream is invoked once per coarse glimpse (Tab. 5), so the accuracy/compute trade-off is governed by \(N_c\), and even under a matched compute budget (\(N_c=1\)) the CF-GAP extension still beats the plain baseline.

A Worked Example

Take an HR-InsDet scene of 6144×8192 (resized to 4096×5460 during experiments for faster iteration) in which the goal is to find one particular small tool.

1. Coarse level: the scene is downscaled 2×, encoded by MobileNet-V3, and convolved with the averaged target-example feature to produce the coarse search map; WTA takes the peak → IoR masks its neighborhood → the next peak is taken, and so on for 30 rounds, giving 30 coarse glimpse locations. They do not tile the image uniformly but concentrate on a few regions that resemble the target.
2. Fine level: for the first coarse location, a log-polar sensor of diameter 4096 pixels extracts a glimpse from the full-resolution scene and rescales it to 245×245; the Perceiver-style encoder computes the fine search map, whose centroid yields a new coordinate; the sensor moves there, extracts another glimpse, and repeats — three times. After those iterations the fixation has typically moved from the offset coarse position onto the target (the zoom-in panels of Fig. 7 visualize exactly this).
3. Hand-off: a fixed-size RoI is cropped around the final fine glimpse location and, together with the three fine glimpse locations, passed to the downstream model (NIDS-Net / SAM / GroundingDINO). The downstream proposes candidates inside that small region, matches them against the target examples, and records its best match.
4. Loop: return to the second coarse location and repeat; after 30 rounds the globally best match is returned. The entire pipeline sends 30 local RoIs rather than the whole image to the downstream — for comparison, the naive "patched" high-resolution recipe sends 165 patches of the same scene.

Loss & Training

The conference main paper says almost nothing about CF-GAP's internal training. What is certain is the module composition: the coarse search-map scene and target encoders are lightweight MobileNet-V3, while the fine-stage Perceiver-style encoders are this paper's own contribution and are the part that must be trained. Training data comes from the 200 random-background images HR-InsDet provides, using the cut-paste-learn (CPL) strategy that resizes and pastes targets onto arbitrary backgrounds. The paper relegates the full search-map derivation and the training/reproduction details to Appendices A and B, which the cached main text does not include, so the objective and hyper-parameters cannot be confirmed from this note — ⚠️ refer to the original appendices.

What can be confirmed is the evaluation configuration: coarse glimpsing runs on the 2× downscaled scene and fine glimpsing at full resolution; the log-polar sensor diameter is 4096 pixels and the resulting glimpses are resized to 245×245; defaults are \(N_c=30\) and \(N_f=3\). In addition, Robotools forbids using its 20 targets for training, so CF-GAP is trained only on HR-InsDet objects and the Robotools numbers measure generalization to unseen objects.

Key Experimental Results

Main Results

Evaluation uses two benchmarks. HR-InsDet contains 100 object instances with 24 examples each and 160 high-resolution scenes spanning 14 indoor scenarios, split by clutter/occlusion (easy / hard) and by object size (small / medium / large); Robotools contains 20 instances and 1581 test images from 24 indoor scenarios, and its targets may not be used for training. The metric is average precision (AP) at IoU thresholds from 0.5 to 0.95 in steps of 0.05, plus AP50 at IoU 0.5. The table aggregates HR-InsDet results for four downstream detectors with and without CF-GAP ("baseline" means the same detector without CF-GAP, not the strongest method in the table):

Dataset	Subset	Metric	Plain baseline (same downstream)	CF-GAP extension	Gain
HR-InsDet	small	AP	12.4 (OTS-FM_MobileSAM)	29.3	+16.9
HR-InsDet	hard	AP	22.0 (OTS-FM_MobileSAM)	41.7	+19.7
HR-InsDet	small	AP	14.6 (OTS-FM_SAM)	32.4	+17.8
HR-InsDet	hard	AP	28.0 (OTS-FM_SAM)	47.1	+19.1
HR-InsDet	small	AP	28.8 (OTS-FM_GroundingDINO)	39.6	+10.8
HR-InsDet	hard	AP	37.2 (OTS-FM_GroundingDINO)	50.2	+13.0
HR-InsDet	small	AP	32.4 (NIDS-Net)	52.1	+19.7
HR-InsDet	hard	AP	39.9 (NIDS-Net)	63.2	+23.3

The abstract phrases the improvement as up to 20% AP; by the numbers above, 22.0 → 41.7 (+19.7) and 39.9 → 63.2 (+23.3) on the hard subset reach or exceed that, and even the smallest gain, OTS-FM_GroundingDINO on small objects, is +10.8. On Robotools, CF-GAP likewise raises the AP of every baseline: the lightweight OTS-FM_MobileSAM paired with CF-GAP reaches an AP50 comparable to the heavier baselines, although its AP stays below theirs — the authors attribute this to less precise boxes from its weaker detector backbone.

For the direct rival question of "what should one do about high resolution," the paper builds a naive high-resolution baseline: split each 6144×8192 scene into 1024×1024 patches with 50% overlap and let the baseline treat each patch as a separate RoI, yielding 165 RoIs per scene — more than 5× the 30 RoIs CF-GAP produces.

Model	AP (small)	AP (hard)
OTS-FM_MobileSAM	12.4	22.0
Patched OTS-FM_MobileSAM	18.4	23.2
CF-GAP + OTS-FM_MobileSAM	29.3	41.7
OTS-FM_SAM	14.6	28.0
Patched OTS-FM_SAM	18.9	24.2
CF-GAP + OTS-FM_SAM	32.4	47.1
OTS-FM_GroundingDINO	28.8	37.2
Patched OTS-FM_GroundingDINO	22.4	28.1
CF-GAP + OTS-FM_GroundingDINO	39.6	50.2
NIDS-Net	32.4	39.9
Patched NIDS-Net	50.1	49.1
CF-GAP + NIDS-Net	52.1	63.2

A second main experiment separates "found" from "recognized." Instance detection decomposes into finding a candidate region likely to contain the target and matching it against the target examples for recognition; CF-GAP mainly improves the first, so the paper measures how often the search target is found regardless of whether it is later recognized correctly (the authors note they cannot report standard average recall because CF-GAP does not directly output boxes):

Model	Target found % (small)	Target found % (hard)
OTS-FM_SAM	42.1	56.3
NIDS-Net	73.3	75.2
CF-GAP	81.9	89.6
CF-GAP w/o fine glimpsing	69.5	74.6

Ablation Study

The table merges the paper's two ablations, both run with CF-GAP + NIDS-Net on HR-InsDet; values are AP:

Config	small	medium	large	easy	hard
Full model	52.1	79.7	86.1	78.2	63.2
w/o glimpse locations (RoI only)	41.8	73.1	86.0	74.5	49.5
w/o fine glimpsing (coarse only)	44.6	77.3	85.5	75.6	56.2

A further size-sensitivity study on glimpse type (Fig. 6) sweeps the axis from 64 to 2048: log-polar performance stays flat over a wide range while the Cartesian crop fluctuates sharply with size, on both the small and the hard subset.

Key Findings

• Gains concentrate on small objects and hard scenes; easy scenes barely move. Removing fine glimpsing costs 7.5 points on small and 7.0 on hard, yet leaves large (85.5 → 86.1) and easy (75.6 → 78.2) essentially unchanged. CF-GAP is compensating for losses caused by resizing and clutter, not improving recognition in general — it addresses a resolution problem, not a recognition problem.
• Fine glimpsing buys localization precision, not visibility. CF-GAP finds the target in 81.9% / 89.6% of scenes, dropping to 69.5% / 74.6% without fine glimpsing. The paper's explanation is that the coarse search map is low-resolution and its locations sit off the true object, while the downstream is prompted to detect objects at specific locations and fails outright when the location is wrong; the closed fine loop exists to correct that spatial error.
• Glimpse locations are a cheap but important signal. Giving the downstream only the RoI yields 41.8 on small and 49.5 on hard; adding the fine locations lifts these to 52.1 / 63.2 (+10.3 / +13.7). For SAM-like models they replace the dense point prompts (saving compute), for GroundingDINO they act as a spatial filter (cutting mismatches) — both uses pay off.
• Naive patching is not a free substitute. Although 165 patches can in principle recover detail lost to resizing, the OTS-FM family actually gets worse (GroundingDINO drops from 28.8 to 22.4 on small and 37.2 to 28.1 on hard) because more RoIs means a flood of candidates and more matching errors; only NIDS-Net, with fine-tuned matching features, benefits (32.4 → 50.1 on small), and it still trails its own CF-GAP version (52.1) by a wide margin on hard scenes (49.1 vs 63.2) — with fewer than a fifth of the RoIs.
• Compute is almost entirely in the downstream. Tab. 5 breaks the cost down: 60 GFLOPs per scene for the coarse search map (MobileNet), 1 GFLOP per glimpse for the fine search map (Perceiver encoders), so CF-GAP's internal cost is \(60 + N_c \times N_f \times 1\), about 150 GFLOPs at the default setting. A single downstream call costs 80 (STT) / 130 (MobileSAM) / 800 (NIDS-Net) / 5800 (SAM) GFLOPs, so 30 calls range from roughly 2.4k to 174k GFLOPs — two to three orders of magnitude above CF-GAP itself. Total cost is CF-GAP internal cost + Nc × [downstream cost], so what really needs shrinking is the bulky downstream, which is why a lightweight downstream turns out to be the better deal.
• There is plenty of room for early stopping. The grey cumulative curve in Fig. 9 shows that in about half the scenes the search target is hit within the first 8–10 coarse glimpses, so \(N_c\) could be cut substantially if a more reliable matching stage could halt the glimpsing process once the target is recognized — a direction the paper itself lists as future work.

Highlights & Insights

• Pushing task conditioning to the very front is the paper's cheapest move. Target examples only contribute to a low-resolution heatmap; nothing needs to be fine-tuned and the target never has to be encoded into the downstream's feature space, so one front-end can serve any detector — the authors can even discuss compatibility with a baseline like IDOW whose weights are unavailable. This "don't touch the downstream, only change where it looks" interface is what makes active perception deployable on top of existing models.
• The log-polar choice is not cosmetic; it removes a tuning dimension. The right Cartesian crop size depends on two variables at once — how far the coarse location is from the object and how large the object is — and must be tuned per dataset in practice. Log-polar compresses the periphery into a fixed tiny region, so the relative content at the center stays nearly the same across diameters (the visual comparison in Fig. 5 is striking: log-polar images at diameters 256 through 2048 look nearly identical while crops at sizes 256 through 2048 look nothing alike). Trading a geometric prior for a hyper-parameter is a transferable trick.
• Abandoning CNNs for non-uniform resolution is a targeted architectural decision. Convolution's translation equivariance encodes the assumption that every location has equal resolution, which a log-polar image violates. The Perceiver-style "compress — self-attention — decode" replacement does global interaction in a low-dimensional latent space, and a second set of value embeddings decouples the output feature space so it can serve the specific downstream purpose of comparison against target features; Appendix D supports the choice with a CNN comparison.
• The looking/seeing split yields a counter-intuitive but reusable conclusion: the better the front-end, the smaller the back-end can be. The usual instinct is to reach for a bigger model when accuracy is short; here, because the front-end has already raised the signal-to-noise ratio of the input, lightweight downstreams overtake heavy ones on the hardest subsets. Any system with a front-end that filters or recommends regions can redo its compute allocation along the same lines.
• Measuring "found" separately from "recognized" is a worthwhile evaluation design. Because a standard recall metric is not available, the paper directly measures whether the target appears inside the region handed to the downstream. This both gives the AP upper bound under a perfect matching stage and isolates CF-GAP's contribution from the downstream's recognition ability — otherwise one cannot tell whether an AP gain comes from looking in the right place or from seeing it clearly.

Limitations & Future Work

• The guidance signal is texture-level correlation only. Both search maps are convolutions of appearance features against scene features, whereas human search also exploits high-level object semantics and the spatial layout of the scene. For targets with little textural distinctiveness, or scenes full of look-alike distractors, this guidance is likely to break down.
• Inhibition-of-return suppresses too little. IoR masks only visited locations and their immediate neighborhoods rather than entire task-irrelevant regions, so the process can repeatedly revisit the same distractor and waste part of the \(N_c\) budget. A mechanism that suppressed by semantic region instead of pixel neighborhood would be more economical.
• The RoI is a fixed size and must be conservatively large. To accommodate objects of varying size the crop has to be sized for the worst case, which both wastes downstream compute and dilutes precision; an adaptive mechanism that adjusts the crop to the content of the task-relevant region would improve both.
• Cost grows linearly with the number of coarse glimpses. Every coarse location triggers a full downstream call, so \(N_c\) is effectively the compute budget. The authors' proposal is a more robust matching stage that halts the glimpsing process once the target is confidently recognized — given that about half of all scenes are hit within the first 8–10 glimpses, the savings could be substantial.
• Evaluation breadth is still narrow (added by the note author). Both benchmarks are indoor tabletop/tool scenes with rigid-object targets and very similar scene types; whether the method holds outdoors, in street scenes, or for the text-query task definition the authors propose as future work is untested. The paper also relegates the network and training details to appendices, leaving no objective or hyper-parameters in the main text, which raises the reproduction barrier.
• An untested concern: the fine loop depends entirely on the centroid of the fine search map, and that centroid uses only appearance correlation with the target. If the target is occluded until only a small part is visible, the correlation map may show several peaks of similar strength, and the centroid would then land between them — treating a location that is on none of them as the fixation. Whether the failure cases in Appendix F cover this situation cannot be judged from the main text.

Related Work & Insights

• vs OTS-FM / NIDS-Net (two-stage instance detection): they exhaustively propose candidates over the whole image and then match with DINOv2 features; the first stage is task-agnostic and the second already pays the full cost. CF-GAP reverses the order — the task decides which few regions to look at, and the detector works inside them. The advantage is both compute and accuracy at 6144×8192; the disadvantage is an extra fixation loop whose total cost grows linearly with the number of rounds.
• vs GAP (Kolner et al., ICLR 2025): CF-GAP inherits GAP's glimpse-based perception skeleton, but GAP selects by saliency for synthetic visual reasoning tasks and its fixation policy cannot be conditioned on the task at inference. CF-GAP replaces saliency with a "target examples × scene" search map and adds the coarse-to-fine nesting, pushing the framework to real high-resolution instance detection.
• vs STT (Segment This Thing): both use fovea-inspired imagery and both require a location input. STT uses foveated tokenization to encode a fixed view, whereas CF-GAP uses the log-polar representation to navigate — the same sensor is moved repeatedly, closing in on the target. Precisely because STT needs a location, it can only be evaluated together with CF-GAP and is excluded from the pairwise comparisons between standalone baselines and their CF-GAP extensions.
• vs patched baselines: patching throws semantics away and adds compute to recover resolution, while CF-GAP adds semantics and removes compute — 165 RoIs versus 30, and the patched candidate explosion feeds back into matching errors. The comparison makes the point that the high-resolution problem is not about how many pixels to look at but about which few to look at.

Rating

• Novelty: ⭐⭐⭐⭐ Combining saccadic active perception with task conditioning for instance detection, and using log-polar imagery to remove the crop-size tuning problem, is a clean composition; the individual ingredients (search maps, IoR, log-polar, Perceiver) all pre-exist.
• Experimental Thoroughness: ⭐⭐⭐⭐ Four downstream detectors × two benchmarks × five ablations, with a purpose-built patched competitor and a "found but not necessarily recognized" metric; deductions for presenting the HR-InsDet results only as figures (Fig. 3/4) with no full tables, and for pushing all training details into the appendices.
• Writing Quality: ⭐⭐⭐⭐ The mapping from the motivation (human coarse-to-fine search) to the method (two nested glimpsing levels) is very clear, and the looking/seeing framing is memorable; some of the method equations are hard to read in the typeset version.
• Value: ⭐⭐⭐⭐ As a plug-and-play front-end it turns a "replace the model" problem into an "add a preprocessing layer" problem, and it delivers the transferable allocation rule that a better front-end permits a smaller back-end; directly relevant to robotics settings that need high-resolution target finding on edge devices.

Related Papers

• [ECCV 2026] NavWM: A Unified Navigation World Model for Foresight-Driven Planning
• [ECCV 2026] LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Models
• [ECCV 2026] AutoSpeed: Annotation-Free Stage-Adaptive Motion Speed Learning for Robot Manipulation
• [ECCV 2026] Pose Anything Anywhere: Model-free Object Poses from Arbitrary References
• [ECCV 2026] Towards Generalizable Robotic Manipulation in Dynamic Environments