Skip to content

โœ‚๏ธ Segmentation

๐Ÿง  NeurIPS2026 ยท 25 paper notes

๐Ÿ“Œ Same area in other venues: ๐ŸŽž๏ธ ECCV2026 (96) ยท ๐Ÿ“ท CVPR2026 (125) ยท ๐Ÿ”ฌ ICLR2026 (31) ยท ๐Ÿงช ICML2026 (14) ยท ๐Ÿค– AAAI2026 (29) ยท ๐Ÿง  NeurIPS2025 (45)

๐Ÿ”ฅ Top topics: Segmentation ร—21 ยท Adversarial Robustness ร—3 ยท Multimodal/VLM ร—2 ยท Few-/Zero-Shot Learning ร—2 ยท Object Tracking ร—2

COCOTree: A Dataset and Benchmark for Open Tree-Structured Visual Decomposition

This paper introduces the task of open tree decomposition, constructs the large-scale benchmark COCOTree (over 21K images and 1.8M instance nodes) via an automated recursive LVLM-SAM 3 pipeline, and establishes a standardized evaluation metric, Open Tree Quality (OTQ).

Complementing DINO Features with Image Structure for Part Discovery

Addressing the blurry part boundary bottleneck caused by pixel mixing across rigid ViT patch grids, this paper proposes Part-DINO, a training-free framework that complements low-level image contrast-derived geometric regions with DINO semantic features for high-precision part discovery.

Cracks in the Foundation: A Civil Infrastructure Dataset to Challenge Vision Foundation Models

IBM Research and ETH Zurich introduce Cracks in the Foundation (CiF), the largest and most fine-grained civil infrastructure instance segmentation benchmark (โ‰ˆ150k images, 250k masks), revealing that cutting-edge vision foundation models and VLMs suffer from a severe breakdown in zero-shot industrial inspection while specialized supervised models plateau near 25% mAP.

Efficient SAM3 Adaptation for Multi-Class Semantic Segmentation via Dense Competitive Representations

Addressing the prohibitive \(O(N)\) inference latency and semantic misclassifications caused by the absence of inter-class competition in SAM3, this paper proposes DCR-SAM, which decouples visual conditioning to yield shared global visual representations, leverages vision-conditioned text refinement with dynamic pooling, and applies a dense competitive head to achieve a \(33\times\) speedup alongside \(2.3\%\sim 10.7\%\) mIoU improvements.

From Pixels to Concepts: Do Segmentation Models Understand What They Segment?

Introduces CAFE, an attribute-level counterfactual evaluation benchmark that preserves target regions and ground-truth masks while manipulating surface appearance, context, or material composition, revealing that modern open-vocabulary promptable segmentation models consistently rely on visual attribute shortcuts rather than faithfully understanding underlying semantic concepts.

FuseAdapt: Parameter-Efficient Multimodal Fusion for Semantic Segmentation with Missing Modalities

FuseAdapt adapts a frozen vision foundation model for multimodal semantic segmentation by isolating private self-attention low-rank adaptation from receiver-conditioned FFN low-rank residual exchange, complemented by an EMA-supervised Predictive Modality Modeling (PMM) training objective that dramatically improves robustness under missing sensors with zero inference overhead.

Geometry-Guided Semantic Reconstruction for 3D Instance Segmentation

Addressing the issues of cross-modal distribution gaps and passive feature ingestion when directly concatenating lifted 2D foundation model priors, this paper proposes a geometry-guided semantic reconstruction framework that modulates the 2D feature stream using raw 3D geometry and enforces active semantic distillation via masked background superpoint reconstruction.

GMOS: Grounding Moving Object Segmentation in 3D Space and Time

Addressing the heavy reliance of conventional moving object segmentation on pre-computed 2D optical flow/trajectories and coarse sequence-level motion assumptions, GMOS grounds segmentation directly in 3D space and time from raw RGB video, achieving new state-of-the-art results across MOS and fine-grained MOS-I benchmarks while running significantly faster.

Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design

Affinity Explainer (AffEx) exploits the dense cross-attention and correlation matching mechanisms of matching-based few-shot segmentation models alongside multi-level feature ablation to produce pixel-level support attribution maps without backpropagation, establishing new benchmarks on dedicated few-shot causal perturbation metrics.

Pixels to Tokens: Token Space Efficient Active Learning for Low-Budget Semantic Segmentation

TEAL introduces the first ViT-based pixel-level active learning framework for extremely low-budget semantic segmentation by performing two-level MaxHerding diversity selection directly in the native token space to eliminate interpolation artifacts, followed by margin uncertainty refinement on a lightweight decoder grid, surpassing prior baselines by up to +24.36 mIoU with only 1 labeled pixel per image.

Preference-Guided Adaptation for Open-Vocabulary Semantic Segmentation via Prompt Disagreement

The paper converts localized segmentation disagreement between prompt templates into binary supervision and adapts open-vocabulary segmentation with region-localized preference optimization and outside-region consistency; with a ground-truth-based preference oracle, CAT-Seg-L improves its mean MESS mIoU from 35.26 to 45.88.

Prompting Diffusion Models for Zero-Shot Instance Segmentation

Addressing the lack of structural priors in discriminative models and the suboptimal post-hoc click handling in prior generative segmenters, Prompt2Seg injects dual-scale Gaussian spatial prompts into a frozen single-step diffusion backbone via ControlNet, delivering state-of-the-art zero-shot interactive instance segmentation across unseen artistic, X-ray, and cluttered domains using only synthetic training data.

Redefining Instance Matching: A Unified Framework for Part-Aware Matching in Panoptic Segmentation Evaluation

Recasts panoptic segmentation segment matching as a constrained bipartite graph assignment problem with vertex-based TP/FN/FP accounting, unifying One-to-One, Many-to-One, One-to-Many strategies and threshold-agnostic AUTC with decoupled part-aware evaluation.

Rethinking Vector Field Learning for Generative Segmentation

Addressing gradient vanishing near semantic centroids and trajectory traversing caused by the lack of inter-class repulsion in vanilla flow matching, this paper introduces a vector field reshaping strategy with distance-aware repulsive/attractive corrections and quasi-random Kronecker encoding, substantially closing the gap between generative segmentation and discriminative specialists.

S2D: Sparse-To-Dense Keymask Distillation for Unsupervised Video Instance Segmentation

Addressing the limitation of synthetic translational video datasets that fail to model complex real-world dynamics, S2D discovers temporally-coherent sparse keymasks using deep point-tracking motion priors and trains a student-teacher distillation pipeline equipped with Temporal DropLoss to achieve implicit mask propagation across real unannotated videos.

S^2-RL: Sample-Set Dual Reinforcement Learning for Generative Semantic Segmentation Dataset Distillation

Addressing the twin bottlenecks of prohibitive memory costs from bi-level gradient unrolling and structural rigidity in static-mask proxy generation, S^2-RL reformulates semantic segmentation dataset distillation as a text-conditioned diffusion policy optimization problem via GRPO, leveraging a dual sample-set reward mechanism to establish new state-of-the-art benchmarks on ADE20K and COCO.

SetCon: Towards Open-Ended Referring Segmentation via Set-Level Concept Prediction

SetCon overturns the conventional paradigm of relying on implicit special tokens (such as [SEG]) for mask decoding by reformulating open-ended referring segmentation as explicit set-level natural language concept prediction with hierarchical semantic decomposition, achieving state-of-the-art results across both multi-target image and video benchmarks.

Sparse All-Layer Connector for Domain Generalised Semantic Segmentation

Addressing the limitation of conventional depth-aligned fusion in multi-foundation-model domain generalised semantic segmentation, this paper introduces the Sparse All-Layer Connector (SALC) to dynamically select and aggregate candidate layers across depths via Sparsegen and frequency-based candidate dropout regularisation.

Towards Fairness under Label Bias in Image Segmentation: Impact, Measurement and Mitigation

Addressing the critical challenge of systematic demographic annotation disparities in image segmentation, this paper introduces a Confident Learning-based auditing framework to measure label bias without clean ground-truth masks and develops a Subgroup-Conditioned Decoder with asymmetric boundary masking to effectively eliminate performance gaps.

Towards Visual Query Segmentation in the Wild

Addressing the core limitations of conventional visual query localization (VQL) that only locates the last target appearance using bounding boxes, this paper introduces the visual query segmentation (VQS) paradigm along with the first large-scale benchmark VQS-4K, and develops VQ-SAM, a multi-stage progressive baseline with adaptive memory evolution that segments all pixel-level occurrences of an external visual query in untrimmed videos.

UGO: Unified Architecture for General Multi-Object Tracking by Segmentation

UGO unifies an exemplar-conditioned detector (GECO2) and a video object segmentation head (SAM2) under a shared foundation backbone, utilizing a training-free energy-minimization consolidation module and hierarchical adaptive memory to resolve proposal overlaps into mutually exclusive instance masks for state-of-the-art general multi-object tracking.

Universal Cross-Prompt Adversarial Attacks on Promptable Concept Segmentation

AdvPCS examines vulnerabilities in promptable concept segmentation through prompt variation, concept perception, and temporal memory, reducing three-model average video mIoU on SA-CO with text prompts from 70.24% to 4.61% in controlled digital-input experiments, without establishing failure guarantees for arbitrary prompts, models, or real-world inputs.

When Noise Meets Long-Tail: Feature-Threshold Dual Calibration for Robust Pseudo-Labeling

FTC-Seg combines prototype-based residual feature reconstruction with pseudo-label thresholds driven by labeled accuracy and class-distribution bias in teacherโ€“student segmentation, improving DINOv2-S over UniMatch V2 by 2.09โ€“8.54 mIoU percentage points across four benchmarks with 2% labels, without winning every configuration.

When Predicting Nothing Beats SAM 3: Revisiting Evaluation in Video Object Segmentation

Reveals that the standard Video Object Segmentation metric \(\mathcal{J}\&\mathcal{F}\) collapses into absence-classification reward under low temporal visibility, and introduces the FaVOS benchmark alongside a volumetric evaluation metric \(\mathcal{J}\&\mathcal{F}_v\).

When Trackers Fail: VLM-Guided Verification and Recovery for Robust Video Object Segmentation

Dual-Agent reframes recursive mask tracking into a closed-loop verify-and-recover framework, using an untuned Qwen3-VL-8B reasoning model to verify tracking validity and a failure-fine-tuned Qwen3-VL-4B model to re-localize targets, recovering severe drift across VOS and VOT benchmarks with sparse interventions.