SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation

Conference: CVPR 2025
arXiv: 2603.06572
Code: GitHub
Area: 3D Vision / Point Cloud Segmentation
Keywords: Incremental Few-Shot Learning, 3D Semantic Segmentation, Prototype Network, Background Mining, Point Cloud

TL;DR

SCOPE proposes a plug-and-play background-guided prototype enrichment framework. After training on base classes, a class-agnostic segmentation model is utilized to mine pseudo-instances from background regions to establish an Instance Prototype Bank (IPB). When novel classes emerge in a few-shot manner, background prototypes are fused with few-shot prototypes using Contextual Prototype Retrieval (CPR) and Attention-Based Prototype Enrichment (APE), achieving up to 6.98% improvement in novel class IoU on ScanNet/S3DIS.

Background & Motivation

Background: 3D point cloud semantic segmentation performs outstandingly under full supervision, but heavily relies on intensive point-wise annotations. Incremental few-shot learning balances multi-step novel class learning, few-shot adaptation, and retaining old knowledge.

Limitations of Prior Work: Few-shot learning does not preserve old knowledge; class-incremental learning requires extensive supervision; generalized few-shot learning only performs a single update. IFS-PCS is virtually unexplored.

Key Challenge: During base class training, novel class objects exist as "background" in the scenes, containing rich structural information, but are compressed into indistinguishable single representations.

Goal: Mine the unlabeled object structures within base class scene backgrounds to assist incremental few-shot learning of novel classes.

Key Insight: Mine pseudo-instances from the background using a class-agnostic segmentation model to construct a reusable prototype bank.

Core Idea: Unlabeled objects in the background are a "gold mine" for future novel classes. Few-shot representations can be enriched through prototype retrieval and attention-based fusion.

Method

Overall Architecture

Three stages: Base Training \(\rightarrow\) Scene Contextualisation (IPB Construction) \(\rightarrow\) Incremental Class Registration (CPR + APE).

Key Designs

1. Instance Prototype Bank (IPB):

   • Use a class-agnostic model \(\Theta\) (Mask3D) to detect pseudo-instances in background regions with confidence \(>\tau\).
   • Perform masked average pooling on each pseudo-instance to obtain prototype \(\mu_{i,j} \in \mathbb{R}^D\).
   • Aggregate all scenes to form the IPB \(\mathcal{P}\), which is frozen after a single construction.

2. Contextual Prototype Retrieval (CPR):

   • For a few-shot prototype \(\mathbf{p}^c\) of novel class \(c\), calculate the cosine similarity with each prototype in the IPB.
   • Retrieve the Top-\(R\) most relevant prototypes to form \(\mathcal{B}^c\).

3. Attention-Based Prototype Enrichment (APE):

   • Parameter-free cross-attention: the few-shot prototype serves as the query, and retrieved prototypes serve as key/value.
   • Fusion: \(\tilde{\mathbf{p}}^c = \lambda \mathbf{p}^c + (1-\lambda) \sum_r \text{Attn}_r \bar{\mu}_r^c\).
   • Final classification is completed via cosine similarity in the enriched prototype space.

Key Experimental Results

Main Results (ScanNet, K=5, IFS-PCS)

Method	mIoU	mIoU-N (Novel)	HM
JT (Oracle)	45.34	36.97	42.03
GW (GFS SOTA)	35.35	23.20	—
Ours (SCOPE)	37.23	25.57	—

Ablation Study

Configuration	mIoU-N	Gain
GW Baseline	23.20	—
+ IPB Random	24.10	+0.90
+ CPR Cosine Retrieval	25.00	+1.80
+ APE Attention	25.57	+2.37

Key Findings

• Novel class IoU increases by up to 6.98% (ScanNet) / 3.61% (S3DIS), while base class performance is barely degraded.
• Top-\(R=10\) prototypes yield the best performance; a larger \(R\) introduces noise.
• The gains are more significant under the 1-shot setting—background prototype compensation is most crucial when support samples are extremely scarce.
• Plug-and-play—effective for any prototype-based 3D segmentation method.

Highlights & Insights

• Background is a "gold mine": Base class scene backgrounds contain rich structural information of future novel class objects.
• Fully parameter-free enrichment: CPR+APE introduces no learnable parameters, adhering to the principle of minimal adaptation for few-shot learning.
• Clever utilization of class-agnostic models: Mask3D is used for objectness detection without requiring any novel class priors.
• Transferable: This paradigm can be generalized to 2D few-shot segmentation or other incremental learning tasks.

Limitations & Future Work

• The quality of the IPB depends on the performance of the class-agnostic segmentation model on the target domain.
• Cosine retrieval is static—it does not update with the incremental stages.
• Validation is limited to indoor datasets (ScanNet/S3DIS).
• Integration with active learning or continual learning remains unexplored.

Related Work & Insights

• vs GW: SCOPE improves novel class performance by +2.37 mIoU-N over GW through background enrichment.
• vs HIPO: Hyperbolic prototypes underperform compared to the GFS baseline; SCOPE is more effective in Euclidean space.
• vs CLIMB-3D: Incremental methods collapse under the \(K=5\) setting; SCOPE is specifically designed for few-shot scenarios.

Rating

• Novelty: ⭐⭐⭐⭐ The concept of background mining and prototype enrichment is novel.
• Experimental Thoroughness: ⭐⭐⭐⭐ Validated on two benchmarks across multiple settings with extensive ablations.
• Writing Quality: ⭐⭐⭐⭐ Mathematically rigorous formulation.
• Value: ⭐⭐⭐⭐ Blazes a new trail for IFS-PCS.

Area: 3D Vision
Keywords: Incremental Few-Shot 3D Segmentation, Prototype Network, Background Mining, Catastrophic Forgetting, Point Cloud Segmentation

TL;DR

SCOPE proposes a background-guided prototype enrichment framework for incremental few-shot 3D segmentation. By mining pseudo-instances from background regions and storing them in an Instance Prototype Bank (IPB), it utilizes Contextual Prototype Retrieval (CPR) and Attention-Based Prototype Enrichment (APE) to fuse context with few-shot prototypes. Without retraining the backbone, it achieves a +6.98% novel class IoU and a +2.25% mean IoU improvement on ScanNet.

Background & Motivation

1. Background: Few-shot 3D segmentation requires learning new classes from extremely sparse annotations.
2. Limitations of Prior Work: Incremental learning of new classes easily suffers from catastrophic forgetting of old classes.
3. Key Challenge: Few-shot prototype information is insufficient to accurately represent novel classes, but background regions contain transferable contextual clues.
4. Goal: How to improve the quality of few-shot prototypes using scene context without retraining the backbone?
5. Key Insight: Class-agnostic segmentation results in background regions can be extracted as pseudo-instance prototypes to enrich the contextual information of few-shot prototypes.
6. Core Idea: Mine pseudo-instances from background \(\rightarrow\) Instance Prototype Bank \(\rightarrow\) Contextual retrieval + Attention fusion \(\rightarrow\) Enriched few-shot prototypes.

Method

Key Designs

1. Instance Prototype Bank (IPB): Extract pseudo-instances from background regions using class-agnostic segmentation, and store their features as contextual prototypes.
2. Contextual Prototype Retrieval (CPR): Retrieve semantically aligned prototypes from the IPB based on few-shot prototypes.
3. Attention-Based Prototype Enrichment (APE): Fuse retrieved contextual prototypes with few-shot prototypes without additional parameters.

Key Experimental Results

Dataset	Novel IoU Gain	Mean IoU Gain	Description
ScanNet	+6.98%	+2.25%	Low forgetting
S3DIS	+3.61%	+1.70%	Cross-scene generalization

Key Findings

• Pseudo-instances in background regions serve as effective sources of contextual signals.
• Prototype manipulation alone can yield improvements without retraining the backbone.

Highlights & Insights

• Background is a resource, not waste: Moving away from the "background = useless" assumption, this work systematically mines transferable knowledge from the background.
• Practicality of training-free adaptation: The backbone is kept frozen, and APE relies purely on attention without adding parameters.

Limitations & Future Work

• Assumes the background contains meaningful structures for future classes—which may not hold in simple scenes.
• The effectiveness of the IPB is affected by the quality of the class-agnostic segmentation.

Related Work & Insights

• vs Standard Prototype Networks: They generate prototypes only from the support set, leading to insufficient context. SCOPE enriches prototypes through background mining.

Rating

• Novelty: ⭐⭐⭐⭐ The idea of background mining and prototype enrichment is novel.
• Experimental Thoroughness: ⭐⭐⭐⭐ Thorough ablation on two datasets.
• Writing Quality: ⭐⭐⭐⭐ Clear description of the methodology.
• Value: ⭐⭐⭐⭐ Practical value for 3D incremental few-shot learning.

Related Papers

• [CVPR 2026] Few-Shot Incremental 3D Object Detection in Dynamic Indoor Environments
• [CVPR 2025] Synthetic Prior for Few-Shot Drivable Head Avatar Inversion
• [CVPR 2025] FFaceNeRF: Few-Shot Face Editing in Neural Radiance Fields
• [CVPR 2025] Zero-Shot Monocular Scene Flow Estimation in the Wild
• [CVPR 2025] Rethinking End-to-End 2D to 3D Scene Segmentation in Gaussian Splatting