P3-SAM: Native 3D Part Segmentation¶

Conference: ECCV2026
Paper: ECCV Paper
Area: 3D Vision
Keywords: native 3D supervision, part segmentation, point prompts, multi-scale masks, automatic segmentation

TL;DR¶

P3-SAM learns a single-point-prompted, two-stage multi-mask network from approximately 3.7 million native 3D assets and combines quality prediction with deduplication for automatic segmentation, reaching an average IoU of 59.88 on PartObj-Tiny without connectivity, compared with PartField's 53.93.

Background & Motivation¶

Decomposing a complete 3D object into parts supports local editing, asset reuse, animation, and part generation. Traditional part segmentation typically specifies object categories and part labels before learning pointwise classification, making unfamiliar assets difficult to handle directly. An alternative inspired by SAM renders multiple views, lifts image masks or features into 3D, and uses them for clustering or supervision. SAMPart3D, PartField, and Point-SAM use such 2D data engines, but different views may disagree about the boundary of the same part. For heavily occluded or intricate assets, these errors affect not only the final output but also the supervision used to train the 3D model.

The paper addresses two connected questions: how to obtain part labels without 2D projection, and how to segment an object without requiring its part count beforehand. Artists often construct individual components before assembling an object, so connected components in the original mesh retain information about its creation process. However, connected components are not automatically meaningful parts: fragments and decorative details need merging, while scanned or generated watertight surfaces lack the same topological separation. Connectivity can therefore help construct supervision without becoming a prerequisite at test time.

P3-SAM reformulates the task as predicting the part containing a given 3D point, then repeats this operation with automatically selected points. A point can belong to a local component or a larger assembly, so the network retains alternatives at multiple scales instead of forcing all training sources to share one granularity. Core Idea: produce 3D supervision from native mesh structure, represent part-granularity ambiguity with multiple masks, and use learned quality estimation to assemble single-point segmentation into complete segmentation without a prescribed part count.

Method¶

Overall Architecture¶

The input is an object mesh, while the network operates on sampled surface coordinates and normals; the output assigns a part-instance identifier to each face rather than predicting semantic category names. Native 3D Supervision constructs labels before training, Two-Stage Multi-Mask Prediction learns candidates for a single point, and Quality-Guided Automatic Segmentation selects and combines candidates at inference time. The default automatic mode requires no user clicks; interactive mode supplies one positive point to the same network.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Assets["Artist meshes"] --> Data["Native 3D<br/>Supervision"]
    Data -.->|Training labels only| Masks["Two-Stage<br/>Multi-Mask Prediction"]
    Input["Input mesh<br/>or point cloud"] --> Masks
    Masks --> Auto["Quality-Guided<br/>Automatic Segmentation"]
    Auto --> Output["Part masks<br/>and mesh-face labels"]

The training branch does not generate labels for test assets, and the inference branch does not invoke 2D SAM. Point-cloud inputs can directly receive point-level segmentation; mapping sampled points back to a mesh is needed only when face labels are required. The three designs address label sources, single-point ambiguity, and whole-object coverage, respectively; predicted masks and the post-processed complete segmentation are distinct outputs.

Key Designs¶

1. Native 3D Supervision: recover labels from construction structure and cover watertight surfaces

Training assets come from Objaverse, Objaverse-XL, ShapeNet, PartNet, and other internet repositories. The algorithm first splits meshes into connected components and measures their surface areas, avoiding treating every fragment as a separate part. It builds adjacency in a voxel space with resolution 128: two submeshes are adjacent if they occupy a shared voxel. Components covering less than 1% of the object's total surface area are iteratively merged into larger adjacent components. Objects with fewer than 2 or more than 50 parts are removed, followed by additional filtering for severe area imbalance. The resulting dataset contains approximately 3.7 million objects, with supervision derived from 3D assets themselves rather than fused multi-view SAM masks.

This construction has a specific bias: assembled source assets often contain internal structures, and their part boundaries may coincide with disconnected meshes. Training only on these examples could encourage the network to exploit topological cues instead of recognizing geometric transitions on scanned or generated surfaces. The authors therefore convert filtered assets into watertight models, successfully obtaining approximately 2.3 million versions containing only exterior surfaces. Each face in a watertight version inherits the label of the nearest face in its original model, without repeating 2D annotation. When both versions exist, training selects the watertight version with 80% probability so that the model encounters both geometric distributions. Page 5 also gives 0.85% as the large-part filtering threshold, contradicting the requirement in the same passage that every part exceed 1%; this uncertainty is preserved rather than silently changed to 85%.

2. Two-Stage Multi-Mask Prediction: use initial candidates to guide global context aggregation

The network uses a Sonata-pretrained Point Transformer V3 encoder and reduces its input voxel size to retain more intricate geometry. Features from different levels are aggregated through a shared MLP into 512-dimensional pointwise features, extracted only once for each asset. Each query concatenates these features, original point coordinates, and the prompt coordinates broadcast to all points, without a separate general-purpose prompt encoder. The basic prompt is therefore a single positive point; the model does not take on SAM's broader mixture of negative points, boxes, and other prompt types. Three first-stage MLPs each predict a candidate mask, allowing a point to admit different interpretations of part granularity. These heads are not three fixed semantic classes, nor does the paper assign them universal fine, medium, and coarse labels.

Although the first stage uses encoder features, its segmentation heads do not explicitly summarize the overall candidate structure associated with the current prompt. The second stage feeds the mixed input and all three initial masks into an MLP, then max-pools over points to produce a prompt-dependent global feature. Three additional MLPs read the original input, global feature, and initial masks to predict three more accurate candidates. The important distinction is not merely greater depth: initial predictions indicate which region global aggregation should focus on. This lets geometrically similar surfaces with different part memberships use overall component shape to refine their boundaries. Interactive inference caches the asset's pointwise features; global features that depend on the prompt and initial masks still require per-query computation and are not prompt-independent caches.

3. Quality-Guided Automatic Segmentation: rank candidates and consolidate repeated queries into parts

Multiple heads preserve ambiguity but create a selection problem. The IoU predictor reads the mixed input, global feature, and three second-stage masks, then uses an MLP, max pooling, and another MLP to output three quality scores. During training it predicts candidate-to-ground-truth IoU; at test time it selects a candidate using only those predictions, without ground-truth access. Automatic mode uses farthest point sampling (FPS) to choose 400 surface prompts, deliberately exceeding the expected part count to improve coverage opportunities. Each prompt contributes one mask and its predicted quality; prompts within the same part can generate many duplicate candidates.

Non-maximum suppression (NMS) sorts candidates by predicted quality, retains the highest-ranked mask, and removes other candidates whose mask IoU with it exceeds 0.9. After repeated selection and suppression, the number of remaining masks becomes the predicted part count, eliminating the need to supply a clustering count beforehand. The quality score and NMS overlap are different quantities: the former estimates agreement with unknown ground truth, whereas the latter directly measures overlap between two predicted masks. Although the paper sometimes calls this merging, the specified operation suppresses duplicates rather than taking the set union of overlapping masks. Point labels are projected to their corresponding mesh faces and resolved by voting at the patch level; flood filling then completes unlabeled faces. Filling uses majority labels among neighboring faces, or several nearest faces when connectivity is unavailable, and repeats until every face has a label.

A Worked Example¶

Consider a chair mesh with a seat, backrest, and supports as an explanatory example, not an additional quantitative experiment from the paper. First sample 100,000 surface points and normals, run the encoder once, and use FPS to choose 400 queries automatically. A seat prompt may yield both a local-seat candidate and a larger structural candidate; two-stage prediction refines the boundaries, and the IoU predictor selects one. Another seat prompt may produce a nearly identical mask that NMS suppresses, while a low-overlap backrest mask remains. Point labels are finally projected to the mesh and propagated to faces missed by sampling; the candidates and suppression determine the part count, with no invented fixed count here. In the multi-prompt application where users specify individual target parts, the paper instead chooses masks to maximize whole-object coverage and reduce overlap, rather than selecting heads solely by predicted IoU. Hierarchical segmentation clusters average point features of already segmented parts; it is an application after basic segmentation, not a required training stage of automatic mode.

Loss & Training¶

Each training asset supplies \(K=8\) randomly selected ground-truth parts, with one positive prompt sampled inside each part. Both stages use Dice and Focal losses, but each prompt back-propagates only through the lowest-loss candidate head, avoiding pressure for all heads to reproduce identical boundaries. The following compact restatement follows the loss definition on page 8, where \(t\) denotes the stage, \(i\) the candidate head, and \(j\) the prompt:

\[ \mathcal{L}_{\mathrm{mask}}^{(t)} =\frac{1}{K}\sum_{j=1}^{K}\min_{i\in\{1,2,3\}} \left[0.5\mathcal{L}_{\mathrm{Dice}}(m_{ij}^{(t)},m_j^{\mathrm{gt}}) +\mathcal{L}_{\mathrm{Focal}}(m_{ij}^{(t)},m_j^{\mathrm{gt}})\right]. \]

The quality branch applies mean squared error to all three predicted IoUs, using targets computed against ground truth after thresholding second-stage masks at 0.5. The overall loss sums the mask losses from both stages and the IoU regression loss, rather than training only the final masks. Random noise perturbs coordinates, normals, and prompt points, while normals are removed with probability 0.3 to reduce dependence on ideal inputs. Training, evaluation, and inference all use 100,000 points; optimization uses Adam with learning rate \(10^{-5}\). The authors train on 64 H20 GPUs with batch size 2 per GPU for 9 epochs, taking approximately 4 days. Table 1 on page 3 reports approximately 112M parameters, 8 s for automatic segmentation, and 3 ms for interaction; the interactive figure concerns lightweight queries after feature extraction, not first-time processing of an entire asset.

Key Experimental Results¶

Main Results¶

Table 2 on page 10 evaluates PartObj-Tiny, containing 200 assets across 8 categories; the table below selects averages within matching tasks. The metric is the reported average IoU, retaining the source's percentage-scale values; interactive evaluation samples 10 prompts per ground-truth part and averages prediction-to-ground-truth IoU.

Dataset / Setting	Comparator	Comparator IoU	P3-SAM IoU	Gain (percentage points)
PartObj-Tiny, without connectivity	SAMPart3D	53.47	59.88	6.41
PartObj-Tiny, without connectivity	PartField	53.93	59.88	5.95
PartObj-Tiny, with connectivity	PartField	79.18	81.14	1.96
PartObj-Tiny, single-point interaction	Point-SAM	27.91	51.23	23.32

The connectivity setting supplies additional structure; the text also explains that PartField selects the best hierarchical level, while P3-SAM incorporates connected components and part prompts. Thus, 81.14 is not the default automatic score without structural assistance, and subtracting 59.88 from it would not isolate an independent module's effect.

Table 3 on page 10 extends the comparison to watertight meshes and point clouds, neither using a part-level connected-component prior. PartObj-Tiny-WT contains 189 successfully converted assets, while PartNetE contains 1,906 shapes from 45 object categories.

Dataset / Setting	Comparator	Comparator IoU	P3-SAM IoU	Gain (percentage points)
PartObj-Tiny-WT, full segmentation	PartField	51.54	58.10	6.56
PartObj-Tiny-WT, single-point interaction	Point-SAM	24.16	49.11	24.95
PartNetE, full segmentation	PartField	59.1	65.39	6.29
PartNetE, single-point interaction	Point-SAM	45.85	63.48	17.63

Not using connectivity on watertight data means not exploiting connected components that directly separate parts; it does not mean watertight meshes mathematically lack adjacency. The original Table 3 caption mentions only watertight PartObj-Tiny, although its rows also include PartNetE; the table above distinguishes them by their actual row names.

Ablation Study¶

Table 4 on page 14 evaluates segmentation-head variants on the authors' own test set, reporting mIoU on its original 0-to-1 scale rather than the percentage scale above.

Config	Augmentation	mIoU	Note
Single-Head	No	0.2801	One first-stage head only
Stage 1 Only	No	0.4265	First-stage multiple heads
Stage 2 Only	No	0.6647	Stage with global features
Stage 1 + Stage 2	No	0.7464	Complete two-stage predictor
Full	Yes	0.7906	Two stages and augmentation

Key Findings¶

Moving from Stage 1 Only to Stage 2 Only adds 0.2382 mIoU, supporting the importance of global context; combining both stages further reaches 0.7464.
Adding augmentation to the two-stage architecture increases mIoU from 0.7464 to 0.7906, an absolute gain of 0.0442.
Figure 7 illustrates removal of NMS and flood filling only qualitatively, so it supplies no independent numerical drops; the paper also gives no numerical ablation removing the IoU predictor.

Highlights & Insights¶

Using connected components during supervision construction without requiring topologically separated parts at test time is more important than simply cutting a mesh into connected pieces. Watertight augmentation makes that distinction operational.
Multiple heads accommodate different definitions of part granularity rather than adding fixed categories. Lowest-loss-head supervision gives the candidates an opportunity to specialize.
Automation depends on ranking candidates reliably, not only on producing accurate individual masks. Other promptable segmentors could reuse dense querying, learned scoring, and duplicate suppression, but would still need score calibration.

Limitations & Future Work¶

The authors acknowledge a reliance on surface geometry and limited understanding of part volume; this is not a complete volumetric segmentation or occluded-part reconstruction model.
Connected components, area thresholds, and nearest-face label transfer provide heuristic supervision rather than guaranteed semantic parts. The original 0.85% filtering condition still requires clarification from official materials.
A 3.7 million-scale dataset and 64 H20 GPUs mean the gains cannot be attributed entirely to the segmentation heads; existing ablations do not isolate dataset scale, native label sources, and encoder pretraining.
The fixed 400 prompts and 0.9 NMS threshold determine coverage and deduplication, but no systematic sensitivity curves are provided; tiny parts and conflicts between overlapping granularities deserve separate evaluation.
The available description does not establish a complete deduplication protocol between training assets and external evaluation sets, nor report repeated-run variance. Handling arbitrary objects is therefore an author objective, not a universal guarantee established by finite benchmarks.

vs Point-SAM: Both use point prompts for 3D segmentation; P3-SAM restricts the prompt to one positive point and uses native 3D supervision, lightweight queries, and automatic candidate selection for complete segmentation. This does not imply support for all SAM interaction capabilities.
vs PartField / SAMPart3D: These methods emphasize feature learning and clustering, whereas P3-SAM directly predicts prompt-conditioned masks and then determines the part count; comparisons must still account for training scale and connectivity protocols.
Sonata and HoloPart: Sonata provides pretrained point features, while HoloPart is a downstream part-generation method consuming segmentation results. Figure 6 demonstrates application feasibility rather than independent quantitative evidence of generation quality.
Research direction: Combining surface segmentation with volumetric constraints could test whether predicted parts form plausible spatial entities. This extends the authors' stated limitation and is not an implemented component of the paper.

Rating¶

Novelty: 4/5. Native data construction, simplified prompting, and complete automation form a distinctive combination, although individual operators are conventional.
Experimental Thoroughness: 4/5. The study covers meshes, point clouds, interaction, and architectural ablations, but lacks stronger data-source ablations, deduplication details, and sensitivity analysis.
Writing Quality: 3/5. The core pipeline is understandable, while the filtering threshold and some evaluation naming need clarification.
Value: 4/5. The method offers direct lessons for 3D asset processing and part editing, subject to its training cost and surface-geometric scope.