Skip to content

GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure

Conference: ECCV2026
Paper: ECCV 2026
Area: Self-Supervised Learning
Keywords: LiDAR point cloud, self-supervised pretraining, 3D object detection, occlusion modeling, self-distillation

TL;DR

GhostPoint observes that the learning signal in LiDAR self-supervised pretraining is almost exclusively defined on returns from visible surfaces and leaves occluded and no-return regions unconstrained; it therefore uses instance voxel dilation to generate neighborhood voxel sets around pseudo-instances that contain visible, masked, and no-return voxels, trains a lightweight predictor to hallucinate features at those positions from observed context, and supervises them with asymmetric teacher targets (occupied voxels match the teacher encoder, no-return voxels match the teacher predictor), unifying pseudo-occlusion and true occlusion under one self-distillation objective — reaching state-of-the-art downstream 3D detection on nuScenes and Waymo (67.5 mAP / 71.2 NDS on nuScenes after full fine-tuning).

Background & Motivation

LiDAR 3D object detection is a core component of autonomous driving perception: it must output an object's position, orientation, and full extent, feeding directly into collision avoidance, motion planning, and multi-object tracking. Training such detectors, however, requires large-scale and precisely annotated 3D boxes, which are expensive to obtain, whereas the unlabeled LiDAR logs produced daily by autonomous fleets are virtually unlimited. Self-supervised learning is therefore a natural path to scalable pretraining: learn an encoder on massive unlabeled point clouds, then transfer it downstream. Most such methods follow a teacher–student self-distillation route (Sonata, DOS) or a masked reconstruction / occupancy-prediction route (Occ-MAE, NOMAE, GD-MAE). Their transfer to per-point tasks such as semantic and panoptic segmentation is already close to supervised training, and everything looks fine.

The problem shows up in detection. The authors first ran a direct controlled comparison: freeze the pretrained encoder and train only a lightweight decoder, with PTv3 as the shared backbone for comparability. The SSL methods reach 79–80 mIoU on nuScenes semantic segmentation, essentially matching supervised PTv3 (80.3). On 3D detection, the same encoders show a clear gap to the supervised baseline (63.8 mAP / 68.4 NDS), with the strongest, PointINS, reaching only 56.7 mAP / 62.5 NDS. More telling is a negative result: adding box fitting and a box regression head to PointINS degrades performance (56.7 → 56.2 mAP). The authors' conclusion is pointed — the problem is not missing geometric supervision but the incompleteness of the observation itself: the regression target is still derived from the same occluded point cluster, so geometric objectives defined on visible points inherit exactly the bias they aim to correct. Instance-aware pretraining is necessary but not sufficient.

Explaining this requires a representation-level account. Let \(V = V_o \cup V_u\) be the full scene voxel set, where \(V_o\) holds measured returns and \(V_u\) has no observations. Transformer encoders such as PTv3 operate only on \(V_o\) and produce no representation for \(v \in V_u\); SSL objectives are therefore naturally defined on \(V_o\) and encourage representations aligned with observed measurements. For per-point tasks this is harmless, since semantic labels are themselves defined on \(V_o\). Detection is annotated differently: a box \(B_k\) specifies an object's full extent in \(V\), including occluded parts. Fine-tuning for detection must extrapolate beyond sparse returns, whereas SSL pretraining constrains representation quality only on occupied voxels. The two are systematically mismatched, and this is especially damaging in outdoor LiDAR scenes with heavy occlusion.

Core idea: rather than adding stronger geometric supervision on visible surfaces, turn the unobserved regions themselves into the pretraining target — use instance voxel dilation to generate neighborhood voxel sets around pseudo-instances that contain visible, masked, and no-return voxels, let a lightweight predictor hallucinate features at those positions from observed context, and supervise them with asymmetric teacher targets (occupied voxels from the teacher encoder, no-return voxels from the teacher predictor) so that pseudo-occlusion and true occlusion share a single self-distillation objective.

Method

Overall Architecture

GhostPoint is a LiDAR pretraining framework built on top of self-distillation: the input is a single LiDAR scan and the output is an encoder that is more robust to occlusion. The pipeline is as follows. A scan is augmented into two independent views, and for each view the teacher encoder sees the unmasked points while the student encoder sees a randomly masked subset. Beyond encoder-level distillation, teacher features are clustered into pseudo-instances; the occupancy grid of each instance is dilated into a neighborhood voxel set \(Q\); and the student, aided by a lightweight predictor, produces semantic and geometric predictions at the non-visible voxels of \(Q\), whose targets are constructed asymmetrically — teacher encoder for occupied voxels, teacher predictor for no-return voxels. After pretraining, the neighborhood sampling module and the predictor are discarded entirely, leaving only the encoder to be attached to a CenterPoint detector.

Concretely, the teacher receives \(P\) and the student receives a randomly masked subset \(P_{vis}\). After voxelization, let \(V_o\) be the occupied voxels of the teacher view, \(V_{vis} \subseteq V_o\) those of the student view, and \(V_{mask} = V_o \setminus V_{vis}\) the masked occupied voxels. Student and teacher share the same architecture and the teacher parameters are an EMA of the student; both produce pseudo-instances and the base learning signal through a Softmap head (prototype soft assignments) and an Offset head (center-offset prediction). GhostPoint adds a prediction branch on top that extends supervision into the occluded space of \(Q\). Relative to the masking, \(Q\) splits into \(Q_{vis} = Q \cap V_{vis}\), \(Q_{mask} = Q \cap V_{mask}\), and \(Q_{unobs} = Q \setminus V_o\); all subsequent initialization and losses are defined on this partition.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Randomly masked two views<br/>teacher sees all / student sees masked"] --> B["Teacher encoder + student encoder<br/>Softmap head / Offset head"]
    B --> C["Neighborhood sampling by instance voxel dilation"]
    C --> D["Lightweight predictor and KNN spatial initialization"]
    D --> E["Asymmetric teacher targets and two-level distillation"]
    E --> F["Pretrained encoder<br/>→ CenterPoint 3D detector"]

Key Designs

1. Neighborhood sampling by instance voxel dilation: place queries next to objects instead of spreading them uniformly through free space

To model unseen regions without labels, one must first decide where to place new queries. Uniform sampling in free space is computationally prohibitive at outdoor scene scale, and the vast majority of the resulting locations are background with little structure to predict (the limitation NOMAE runs into). The authors' observation is that LiDAR returns are spatially clustered, and that missing object surfaces are most likely to occur adjacent to observed instance occupancy — both occlusion and no-return gaps happen near object boundaries — whereas voxels far from any instance are predominantly background. Sampling therefore proceeds in two steps. First, visibility filtering drops pseudo-instances that are fully masked in the student view (an instance is kept as long as at least one of its voxels remains visible), yielding the union of occupied voxels \(V_o^{inst}\) over the surviving instances. Second, the binary occupancy grid of \(V_o^{inst}\) is dilated with a 3D kernel of size \(k_s > 1\) to obtain the candidate neighborhood \(Q_{dil} \supseteq V_o^{inst}\). To control cost, a fixed-size subset \(Q\) is sampled from \(Q_{dil}\) (by default only half of it, see the ablation), and all subsequent queries and losses are computed on this fixed set.

The dilation is the hinge of the whole method: it pushes the observed object shell outward, so the newly activated voxels land exactly where an object is known to be but the LiDAR did not return — avoiding both the waste of uniform free-space sampling and the risk that supervision lands off the object structure. The kernel size \(k_s\) governs the dilation extent; the ablation peaks at 5 and degrades beyond it, because larger dilations introduce too many background voxels and dilute the instance signal. The clustering radius \(\tau_d\), the minimum cluster size \(\tau_{min}\), and the subsample ratio are the three accompanying hyper-parameters.

2. Lightweight predictor and KNN spatial initialization: propagate observed context into empty voxels

\(Q\) contains both occupied voxels and newly activated no-return voxels, but the student encoder has tokens only on the visible occupied subset \(Q_{vis}\); positions in \(Q \setminus Q_{vis}\) have no input representation at all, so the predictor has nothing to start from. These tokens must therefore be created explicitly. The default scheme is distance-weighted KNN interpolation: for each voxel \(q\) to initialize, take its \(k = 3\) nearest visible occupied voxels and form a weighted sum of their features with weights given by the inverse of the 3D center distance.

\[\tilde{\mathbf{f}}_q = \sum_{v \in \mathcal{N}_k(q)} w_v \mathbf{f}_v, \qquad w_v = \frac{1/d_v}{\sum_{v' \in \mathcal{N}_k(q)} 1/d_{v'}}\]

Here \(d_v = \|c(v) - c(q)\|_2\) is the center distance and \(\mathbf{f}_v\) is the student encoder feature at visible occupied voxel \(v\). The authors also evaluate a cheaper alternative in which all voxels in \(Q \setminus Q_{vis}\) share a single learnable mask token.

The initialized tokens, together with the encoder tokens on \(Q_{vis}\), are fed into the lightweight predictor \(G_\theta\) (two PTv3-style transformer blocks), which aggregates information from visible voxels into the new tokens through self-attention over all tokens. So that the predictor refines only the newly queried positions, visible tokens use an identity overwrite: the visible features from before the predictor are copied verbatim to the corresponding output positions, leaving representations on \(Q_{vis}\) unchanged and keeping predictor outputs only on \(Q \setminus Q_{vis}\). This matters because it guarantees the prediction branch cannot pollute the encoder-level representations; the two supervision paths each act on their own voxel set. The Softmap and Offset heads then map these features into semantic and geometric predictions over \(Q\).

3. Asymmetric teacher targets and two-level distillation: hallucinate only in no-return space, so pseudo-occlusion and true occlusion share one objective

With predictions in hand, the remaining question is what to supervise them with. \(Q\) contains both voxels the LiDAR genuinely hit (including masked ones) and voxels with no return at all — the latter have no labels and no teacher encoder token to draw on. The authors therefore construct asymmetric teacher targets that are anchored on measured occupied voxels and hallucinate only in no-return space. For occupied voxels \(Q \cap V_o\) (including \(Q_{mask}\)), the target Softmaps and offsets come directly from the teacher encoder, since LiDAR returns support them. For unobserved voxels \(Q_{unobs}\), where no encoder token exists, targets come instead from the teacher predictor, which produces context-based semantic and geometric predictions conditioned on the full (unmasked) voxel set. The teacher predictor is itself an EMA of the student predictor, so these hallucinated targets become progressively more reliable as training proceeds.

What closes the design is the choice of supervision scope: the predictor-level loss is applied on \(Q_{\neg vis} = Q \setminus Q_{vis} = Q_{mask} \cup Q_{unobs}\), meaning that "voxels under pseudo-occlusion" and "voxels that are genuinely unobserved" are placed under one and the same Softmap + offset objective. To fill in pseudo-occlusion (the artificial masking), the student must learn to infer missing structure from context — and that ability is then required to transfer unchanged to true occlusion, all without any annotation. This is also the most essential difference from masked-reconstruction methods: MAE-style approaches hide the points that were seen and reconstruct them, so the target surface is still one that was observed; GhostPoint places its target directly in space that was never observed, using pseudo-occlusion as a supervisable practice ground.

A Worked Example

Take a nuScenes scan containing several cars. Voxelizing the teacher view yields tens of thousands of occupied voxels \(V_o\); the student view is randomly masked at a ratio of 0.6 (mask block size 1 m), leaving \(V_{vis}\). Teacher encoder features are passed through the Offset head to predict center offsets, and BFS clustering over the PIT-normalized predicted centroids (20 neighbors, 5 m search radius) groups non-ground points into pseudo-instances; connected components smaller than 10 points are discarded. Suppose one of the cars forms a single instance whose occupied voxels partly fall inside the masked region — as long as at least one voxel remains visible, the instance is kept.

Its occupancy grid is then dilated with a kernel of \(k_s = 5\), activating a ring of voxels around the visible half of the car and pulling in the no-return voxels on its far side and rear; a fixed-size \(Q\) is sampled at a ratio of 0.5. The voxels in \(Q\) now fall into three classes: visible occupied (using student encoder features directly), masked occupied (KNN-interpolated initialization), and no-return (KNN-interpolated initialization). The interpolated tokens and the visible tokens pass together through two predictor blocks with self-attention, and the visible positions are overwritten by identity. Finally the Softmap and Offset heads predict only on \(Q \setminus Q_{vis}\): masked voxels match the targets from the teacher encoder, while no-return voxels match the hallucinated targets from the teacher predictor. Both share the same loss.

Loss & Training

GhostPoint distills at two levels. Encoder-level distillation follows the standard recipe: on visible occupied voxels \(V_{vis}\), teacher and student encoder outputs pass through the prototype head to produce Softmap distributions and through the Offset head to produce center offsets, giving a semantic loss \(\mathcal{L}_{\text{sem,vis}}\) (KL divergence between the Softmap distributions) and a geometric loss \(\mathcal{L}_{\text{geo,vis}}\) (constraining both the magnitude and the direction of the offsets, i.e. a magnitude difference plus a \(1 - \cos\) term). Predictor-level distillation reuses the same pair of semantic and geometric objectives on the non-visible neighborhood voxels \(Q_{\neg vis}\), with targets selected asymmetrically as described above.

\[\mathcal{L} = \underbrace{\mathcal{L}_{\text{sem,vis}} + \lambda\,\mathcal{L}_{\text{geo,vis}}}_{\text{encoder level}} + \underbrace{\mathcal{L}_{\text{sem},\neg\text{vis}} + \lambda\,\mathcal{L}_{\text{geo},\neg\text{vis}}}_{\text{predictor level}}\]

⚠️ This equation is corrupted in the PDF text extraction and has been reconstructed here from context and the ablation settings; refer to the original paper for the exact notation. \(\lambda = 0.1\) balances the semantic and geometric terms.

Training uses a two-stage warmup before joint optimization: for the first 10% of epochs only the encoder's Softmap branch is trained to stabilize representations; for the next 10% the encoder offset branch and the predictor branches are activated with their gradients detached from the student encoder; only then is everything optimized end-to-end. The appendix shows performance is insensitive to the warmup ratios ((0.1, 0.1) is best at 59.5 mAP; other combinations land between 59.2 and 59.4) as long as the two-stage order is preserved. The warmup matters a lot in the ablation: without it the predictor yields only 57.2 mAP, versus 59.5 with the two-stage schedule.

Remaining settings: AdamW with cosine annealing, learning rate 2e-4, weight decay 4e-2, batch size 16, mask ratio 0.6 with 1 m mask blocks, 50 pretraining epochs, \(k_s = 5\) (occupancy dilation), \(k = 3\) (token initialization), \(\lambda = 0.1\), Zipf prototype shape parameter \(\alpha_{zipf} = 1.3\), and teacher/student temperatures of 0.035 / 0.05. Pretraining uses 2 H200 GPUs in DDP mode and downstream fine-tuning a single H200. In terms of cost, GhostPoint pretrains in 26 hours versus 20 for PointINS and 15 for DOS; the roughly 30% overhead comes entirely from neighborhood sampling and the predictor, both of which are discarded at fine-tuning time, so downstream cost is identical to prior methods.

Key Experimental Results

Main Results

Table 2 compares GhostPoint against recent self-supervised methods on nuScenes and Waymo under two protocols: decoder probing (freeze the PTv3 encoder, train the remaining CenterPoint components from scratch) and full fine-tuning. nuScenes uses mAP and NDS; Waymo uses AP at L2 difficulty.

Method nuScenes mAP (probe) nuScenes NDS (probe) Waymo mAP (probe) nuScenes mAP (ft) nuScenes NDS (ft) Waymo mAP (ft)
PTv3 (sup.) 63.8 68.4 67.9 — — —
PSA 41.3 52.8 42.4 63.9 68.9 68.1
SONATA 44.6 55.0 47.4 64.2 69.0 68.6
NOMAE 53.5 60.1 52.4 65.8 69.8 69.5
DOS 55.4 61.6 57.1 65.5 69.7 69.2
PointINS 56.7 62.5 57.5 66.3 70.1 69.5
GhostPoint 59.5 64.2 60.0 67.5 71.2 70.1

Under probing, GhostPoint beats the second-best SSL method by 2.8 mAP / 1.7 NDS on nuScenes and by 2.5 mAP on Waymo. Under full fine-tuning the gains grow further: 67.5 mAP / 71.2 NDS on nuScenes, 3.7 mAP above the supervised PTv3 baseline (63.8 mAP). Qualitatively (Figure 4), GhostPoint is the only method that correctly localizes partially observed cars at mid-to-long range and reduces duplicate boxes while improving orientation, though the authors acknowledge it can still hallucinate occasional boxes in empty space.

Ablation Study

Table 3 adds components one at a time starting from the PointINS baseline (nuScenes, decoder probing):

Config mAP NDS Note
Baseline (PointINS) 56.7 62.5 —
+ Neighborhood sampling (no predictor, direct KNN interpolation) 57.3 62.7 extending supervision into occluded regions helps on its own
+ Predictor (no warmup) 57.2 63.0 NDS rises but mAP slightly drops
+ Predictor (one-stage warmup) 58.5 63.6 mAP starts improving clearly
+ Predictor (two-stage warmup) 59.5 64.2 full model
+ Box regression 59.1 63.9 adding box regression hurts

To confirm the gains really come from hallucination rather than from an extra regularizer or extra supervised positions, Table 4 runs three controlled ablations:

Config mAP NDS Note
PointINS 56.7 62.5 baseline
+ No hallucination (predictor applied only to masked observed points) 57.3 63.1 mild regularization benefit only
+ Zero targets (keep ghost neighborhoods, set targets to zero offsets + uniform Softmaps) 57.6 63.3 only marginally more
+ Swap (replace masked points with an equal number of hallucinated points) 59.4 64.1 matches the full model
GhostPoint 59.5 64.2 full model

Label Efficiency

Table 5 reports detection performance on nuScenes with only 0.1% / 1% / 10% of labels:

Method 0.1% mAP 0.1% NDS 1% mAP 1% NDS 10% mAP 10% NDS
PTv3 (sup.) 18.1 35.3 37.3 48.0 57.9 65.0
NOMAE 30.5 46.8 45.7 56.0 59.7 66.6
DOS 34.0 49.1 49.0 58.0 61.4 67.4
PointINS 34.8 50.8 50.8 60.2 62.1 67.8
GhostPoint 36.9 52.1 53.1 62.5 63.5 68.8

Key Findings

  • The hallucination target itself is the source of the gain — not regularization, and not more supervised positions. The three controls in Table 4 eliminate the alternatives one by one: supervising only masked observed points (no hallucination) gains just 0.6 mAP; keeping the ghost neighborhoods but zeroing their targets gains almost nothing more; and swapping an equal number of masked points for hallucinated points matches the full model. Together these attribute the benefit to the informative hallucination targets in object-adjacent unobserved regions.
  • The predictor needs warmup, and it does not need to be deep. Without warmup the predictor (57.2 mAP) is actually worse than neighborhood sampling alone (57.3 mAP); only after the two-stage warmup does it pull ahead to 59.5. Figure 5 shows that without KNN initialization, performance saturates once the predictor reaches 3 blocks; with it, performance is consistently higher and stable from 2 blocks onward, indicating that a spatially coherent initial guess substantially reduces the predictor's burden.
  • Adding geometric supervision does not fix the mismatch. Adding box regression to PointINS (Table 1, 56.7 → 56.2) and to GhostPoint (Table 3, 59.5 → 59.1) both cost performance, confirming the paper's central claim from two directions: the limitation is not missing geometric learning but incomplete observation.
  • The largest gains come under sparse sensing and small label budgets. At 0.1% labels GhostPoint is 2.1 mAP / 1.3 NDS above the second-best method, widening to 2.3 mAP / 2.3 NDS at 1%; with 10% of labels (63.5 mAP / 68.8 NDS) it essentially matches fully supervised PTv3 (63.8 / 68.4). Waymo shows the same pattern, with the largest gains on pedestrians and cyclists, the categories most affected by occlusion.
  • Per-point transferability is not sacrificed for detection accuracy. Under linear probing on nuScenes, GhostPoint reaches 74.2 mIoU on semantic segmentation, essentially matching the strongest SSL baseline PointINS (74.4); on panoptic segmentation it is the best SSL method at 62.8 PQ, +0.6 over PointINS and +5.4 over DOS, with the top SQ/RQ among SSL methods (84.6 / 73.2).
  • Transfer holds across datasets and architectures. Pretraining on Waymo and transferring to nuScenes under probing reaches 43.4 mAP / 54.1 NDS, +1.8 / +1.7 over PointINS. When PTv3 is replaced by the sparse CNN encoder from CenterPoint, pretraining the CNN directly with GhostPoint transfers poorly (48.0 L2 mAP under probing), but distilling into that CNN from a frozen GhostPoint PTv3 teacher reaches 57.3, and 67.2 after full fine-tuning — better than pretraining the same CNN directly (65.4) and than the supervised CNN baseline (64.5).
  • Better robustness to corruption. On the Robo3D nuScenes-C benchmark, GhostPoint attains the highest mRR (97.5) and the lowest mCE (68.8), clearly better on mCE than PointINS (70.4) and DOS (71.2).
  • Robust hyper-parameters. The dilation kernel \(k_s\) peaks at 5 and the KNN neighbor count \(k\) plateaus after 3; the subsample ratio of unobserved voxels is best at 0.5 (58.9 / 59.5 / 59.2 for 0.2 / 0.5 / 0.8), since low ratios concentrate new voxels at instance boundaries with insufficient coverage while high ratios make targets in unobserved regions noisier; the two-stage warmup ratios vary by less than 0.3 mAP over 0.1–0.2.

Highlights & Insights

  • Turn "unobservable" into a supervisable target instead of approximating it with stronger geometric losses. The key step is not a more elaborate regression head but the realization that pseudo-occlusion (artificial masking) and true occlusion are the same kind of missingness at the representation level, so the former can serve as a practice ground and the learned completion ability transfers directly to the latter. This "simulate unsupervised missingness with supervised missingness" idea transfers to any modality with a systematic observation blind spot — sparse radar returns, motion-blur blind spots in event cameras, or depth-sensor failures on transparent surfaces.
  • Dilation-based sampling injects an inductive bias at almost zero cost. Rather than scattering queries uniformly through free space (wasteful and overwhelmingly background), simply dilate the occupancy grid of discovered instances with a 3D kernel so new queries naturally hug object boundaries. A one-line operation solves both "where to place them" and "is there any structure to learn," and is worth reusing in any self-supervised framework that must actively choose query locations.
  • Identity overwrite keeps the two supervision paths from interfering. The predictor attends over all tokens for simplicity, but its outputs are used only at non-visible positions; visible tokens are copied over verbatim. The prediction branch therefore cannot push gradients back into the encoder's visible representations, and the two distillation levels each act cleanly on their own voxel set — a pitfall many multi-branch self-supervised designs fall into.
  • The asymmetric target construction exploits the difference in teacher competence. Where returns exist, the teacher encoder supplies targets (reliable but covering only visible surfaces); where none exist, the teacher predictor supplies them (a guess, but covering the full space). Because the teacher predictor is the student's EMA, hallucinated targets improve on their own during training. This "pick the target by evidence availability" pattern is directly reusable.

Limitations & Future Work

  • The authors acknowledge two limitations: under probing, performance still trails fully supervised training, and GhostPoint has not been evaluated on other sparse sensing modalities affected by occlusion (e.g. radar).
  • The hallucination targets ultimately come from the teacher predictor's own guesses with no ground-truth geometry to anchor them. Qualitative results show that when objects are extremely sparse due to severe occlusion, the hallucination fails to recover the complete shape, and that it is sensitive to background outliers, producing blurred or incorrectly extended instance boundaries (Figure 4 also acknowledges occasional hallucinated boxes in empty space). Errors can therefore be self-reinforcing.
  • Pretraining costs 30% more than PointINS (26h vs 20h), and the method introduces a set of coupled hyper-parameters (dilation kernel \(k_s\), clustering radius \(\tau_d\), minimum cluster size \(\tau_{min}\), subsample ratio of unobserved voxels). Sensitivity analyses are provided, but retuning is still needed on a new dataset.
  • Evaluation is limited to two autonomous-driving datasets, nuScenes and Waymo; indoor or non-driving scenes are not covered.
  • Possible improvements: bring in occupancy prediction or multi-frame accumulated point clouds as a more reliable hallucination signal (anchoring "self-guessing" to cross-frame consistency); make \(k_s\) adaptive to instance scale rather than globally fixed; and explicitly down-weight background outlier voxels during training to mitigate boundary contamination.
  • vs DOS / PointINS (self-distillation and instance awareness): they introduce instance-aware cues into pretraining, but the objectives remain defined on visible returns and the instance centroids are computed only from observed points, so they are biased under occlusion. GhostPoint reuses their student–teacher setup and Softmap/offset mechanism as the base signal, but extends supervision to \(Q \setminus Q_{vis}\) and uses the teacher predictor's hallucinations to cover no-return space. The difference is fundamentally the domain of the representation: theirs is meaningful only on \(V_o\), whereas GhostPoint requires it to be constrained on \(V_u\) as well.
  • vs NOMAE / Occ-MAE (masked occupancy modeling): they treat voxels with no point returns as empty and reconstruct occupancy within local neighborhoods and at multiple scales. GhostPoint shares the local-neighborhood motivation but explicitly treats no-return voxels as potentially occluded space to be predicted rather than as empty, and needs no occupancy labels. The ablation also shows that merely extending supervision into occluded regions (adding neighborhood sampling) gains only 0.6 mAP — what matters is the content of the hallucination target.
  • vs PSA (pose- and size-aware SSL): PSA takes a different route, explicitly encoding geometric priors such as object size and orientation into the pretext task. It is the weakest under probing (41.3 mAP), suggesting hand-designed geometric proxy tasks transfer poorly in outdoor LiDAR scenes; GhostPoint shows that introducing no geometric prior at all and merely widening the supervision domain to unobserved space improves detection more effectively.
  • vs point cloud completion (PCN / FoldingNet / implicit occupancy networks): they also aim to infer unobserved geometry, but rely on complete shapes or box annotations for supervision and are used as standalone completion tools at inference. GhostPoint shares the intuition that modeling unobserved structure is valuable, but turns it into a purely self-supervised representation-learning objective serving downstream perception rather than shape output.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ The problem formulation itself — moving the pretraining target from visible surfaces to unobserved voxels — is sharp, and the combination of instance voxel dilation with asymmetric teacher hallucination targets is a clean original design, backed by a reproducible negative result on why extra geometric supervision does not help.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Two datasets, probing and fine-tuning protocols, label efficiency, segmentation transfer, cross-dataset transfer, architectural transfer, OOD robustness, runtime, and five hyper-parameter sensitivity analyses, plus three controlled ablations that cleanly separate "hallucination" from "regularization."
  • Writing Quality: ⭐⭐⭐⭐ Clear problem formalization (representation mismatch) and a well-mapped correspondence between figures and ablations; the method section leans on notation fairly heavily and some equations are not easy to parse as typeset.
  • Value: ⭐⭐⭐⭐⭐ Pushes downstream detection substantially higher in a crowded LiDAR SSL field (3.7 mAP above the supervised baseline after fine-tuning), and both the predictor and the sampling module are discarded downstream, so the deployment cost is zero.