C2E: Boosting Ego-Only 3D Object Detection via Multi-Teacher Contrastive Knowledge Distillation¶

Conference: ECCV2026
Paper: ECCV Official Page
PDF: Full Paper
Area: Model Compression
Keywords: knowledge distillation, collaborative perception, 3D object detection, instance point cloud reconstruction, contrastive learning

TL;DR¶

C2E uses the M2S multi-teacher distillation framework to transfer dense knowledge from multi-agent training observations into a student receiving only ego LiDAR, raising AttFuse's V2XSet test [email protected] from 58.06 to 66.70 without inter-agent communication at deployment.

Background & Motivation¶

Ego-only 3D detection is limited not just by network capacity but by what the sensor can observe: occluded vehicles, sparsely sampled distant objects, and blind spots all remove geometric evidence from the ego point cloud. Collaborative perception supplements these observations with scans from other vehicles or infrastructure, but requires transmitting point clouds or features and aligning agent poses. Methods such as Where2comm and CoSDH reduce communication, yet their deployment still depends on cooperation; latency and localization errors do not disappear.

This paper changes when additional observations are available: multi-agent information supplies teacher knowledge only during training, while inference returns entirely to ego-only input. The student does not somehow acquire other vehicles' sensor data at test time. Instead, it learns better geometric priors and feature organization from training examples. The difficulty is that teachers see denser point clouds with broader spatial coverage, so directly matching their features exposes the student to substantial input-distribution differences. Teachers can also disagree about the current scene, making uniform averaging a potential source of misleading supervision.

C2E names the collaborative-to-ego perception paradigm; M2S is the training framework that implements it. Core Idea: enhance sparse student features, supervise their geometry through instance-level point cloud reconstruction, and adaptively combine local contrastive and global feature distillation according to each teacher's localization quality on the current sample, retaining collaborative knowledge in the ego-only model's parameters.

Method¶

Overall Architecture¶

During training, three pretrained teachers—AttFuse, V2X-ViT, and CoSDH—receive multi-agent point clouds from the ego vehicle, other vehicles, and infrastructure. The student receives only ego points and predicts 3D bounding boxes. Multi-Level Feature Enhancement improves the student's pillar features before bird's-eye-view (BEV) projection and the detection backbone; Auxiliary Point Cloud Reconstruction and Multi-Teacher Contrastive Distillation constrain geometry and feature representations, respectively. These are training supervision branches, not point cloud generators executed sequentially at inference.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Ego point cloud"] --> B["Multi-Level Feature Enhancement"]
    B --> C["Student BEV features<br/>and detection backbone"]
    C --> D["Auxiliary Point Cloud Reconstruction"]
    C --> E["Multi-Teacher Contrastive Distillation"]
    T["Multi-agent point clouds<br/>three pretrained teachers"] --> E
    G["Multi-agent instance points<br/>ground-truth boxes"] --> D
    G --> E
    C --> F["Ego-only 3D detections"]

Both supervision branches operate only during training. Reconstruction requires multi-agent instance points cropped using ground-truth boxes; distillation requires teacher features and ground-truth boxes for teacher weighting. Deployment retains only the student detection path, without accessing teachers or other agents' inputs. Some students still use the full collaborative architecture, including fusion modules, but receive only ego data. “Ego-only input” therefore does not mean that the paper redesigns an entirely different detection backbone.

Key Designs¶

1. Multi-Level Feature Enhancement: prepare sparse student representations for distillation

Teacher features are dense because they incorporate additional observations, whereas the student must infer object structure from fewer points. Multi-Level Feature Enhancement (MLFE) does not directly synthesize observations. It applies channel-wise, pillar-wise, and global-wise attention sequentially to pillar features. The paper defines the tensor dimensions as channels, points per pillar, and the number of global pillars. Its pillar-wise stage should not be confused with running an additional object detector; it reallocates attention along the corresponding feature dimension.

Each stage summarizes responses through average and max pooling, then uses convolutions, ReLU, and sigmoid to produce weights. Element-wise reweighting and residual addition update the features. Serial processing matters because each stage operates on representations refined by the previous one: useful semantic channels are emphasized first, followed by local geometry and global context. This supplies a shared student representation for reconstruction and distillation but does not replace collaborative supervision. The ablation likewise evaluates MLFE separately from multi-teacher distillation.

2. Auxiliary Point Cloud Reconstruction: narrow the observation-density gap with instance geometry

Matching teacher BEV features alone may indicate which activations the student should produce without providing sufficiently explicit geometric constraints. Auxiliary Point Cloud Reconstruction (APCR) crops object instances from multi-agent point clouds using ground-truth boxes and re-voxelizes them into supervision targets. It focuses on instances rather than reconstructing the whole scene, concentrating supervision on object structures relevant to detection and avoiding a reconstruction budget dominated by road and background points. The student learns how sparse ego observations relate to more complete instance geometry, rather than requesting external point clouds at test time.

Directly outputting all dense points would be expensive, so APCR separates voxel occupancy prediction from point offset estimation. It expands student 2D BEV features into a 3D representation and processes them with multiple \(1\times1\) 3D convolutions and GELU to predict occupied voxels and point offsets from voxel centers. Occupancy uses binary cross-entropy adjusted for foreground and background counts; offsets use L1 regression on foreground voxels. Together they supervise reconstruction. Reconstruction remains an auxiliary learning task, not a source of points fed into another inference-time detector.

3. Multi-Teacher Contrastive Distillation: select reliable knowledge per sample and align local and global features

Multi-Teacher Contrastive Distillation (MTCD) first determines which teacher to trust. For each training sample, it measures the Smooth L1 localization loss between each teacher's predicted boxes and the ground truth. Teachers with better localization receive larger weights, rather than being selected by model name or a fixed ranking. The paper uses a softmax over negative localization losses:

\[ W_i=\frac{\exp(-\alpha\mathcal{L}_{\mathrm{loc}}^{t_i})}{\sum_{j=1}^{T}\exp(-\alpha\mathcal{L}_{\mathrm{loc}}^{t_j})},\qquad \alpha=0.8. \]

Here, \(T\) is the number of teachers and \(\mathcal{L}_{\mathrm{loc}}^{t_i}\) is the localization loss of teacher \(i\) on the current sample. This label-dependent weight measures current-sample detection quality, not unsupervised uncertainty. Multi-teacher learning also does not simply alternate between imitating three sets of output boxes. The same weights govern global feature fusion and the combination of teacher-specific local contrastive losses, preventing a teacher that performs poorly on the current scene from retaining equal influence in another supervision branch.

Local contrast partitions BEV features into non-overlapping regions and constructs pairs from teacher–student spatial correspondences: matching spatial indices form positive pairs, while different indices form negatives. Squared Euclidean distance measures feature distance, and spatial contrasting is combined with channel contrasting. The target is not merely an activation value at one location, but which locations or channel groups should be similar and which should remain distinct; the temperature is \(\tau=0.07\). Unlike a label-based objective that pulls all same-class objects together, this design specifically depends on teacher–student spatial alignment.

The global branch uses the same weights to fuse dense BEV features from the teachers at each layer and applies KL-divergence distillation to the corresponding student layers, with layer weights \(\{0.2,0.3,0.5\}\). Local constraints preserve fine-grained correspondence structure, while global constraints transfer the aggregate feature-map distribution. Some equations are corrupted in the cached text, and full spatial/channel contrast details are deferred to an appendix. This note therefore retains the verifiable correspondence rules and teacher weighting without inventing a negative-sample denominator or channel-grouping implementation.

Loss & Training¶

The student objective combines weighted classification, box localization, auxiliary reconstruction, global distillation, and local contrastive losses. Classification uses focal loss and localization uses Smooth L1; occupancy and offset reconstruction targets are described above. Each of the five losses has its own weight, but their concrete overall weighting values are not provided in the cached main text, so this description is not a complete reproduction recipe.

The experiments use three pretrained teachers and evaluate the ego agent. Students include the conventional ego-only PillarNeSt model and ego-input versions of several collaborative models. Teachers provide privileged training information, requiring synchronized multi-agent training data and ground-truth annotations; this is not equivalent to training with ego-only data. The supported deployment benefit is the removal of inter-agent communication and teacher dependence, not proof from the available tables that every student adds zero computation relative to its original ego-only baseline.

Key Experimental Results¶

Main Results¶

V2XSet is simulated, whereas V2V4Real and DAIR-V2X are real-world datasets. All rows below use ego-only input and report AP at an IoU threshold of 0.7, with higher values being better; the V2XSet table calls this 3D mAP. Gains are score differences in AP percentage points, not relative percentages or improvements over full collaborative inference.

Dataset and split	Student model	Ego baseline [email protected] ↑	With M2S [email protected] ↑	Gain (percentage points)
V2XSet Test	AttFuse	58.06	66.70	+8.64
V2XSet Test	Where2comm	62.18	70.28	+8.10
V2XSet Test	CoAlign	65.07	71.66	+6.59
V2XSet Test	CoSDH	62.05	66.50	+4.45
V2XSet Test	PillarNeSt	59.83	64.16	+4.33
V2V4Real Test	PillarNeSt	16.18	25.24	+9.06
DAIR-V2X Val	AttFuse	33.25	39.86	+6.61

The headline gain of 8.64 describes the V2XSet comparison, not the maximum across all datasets and thresholds: PillarNeSt gains 9.06 percentage points on V2V4Real test at the same threshold. Dataset difficulty and splits differ, so raw AP values across datasets should not be used directly to assess transfer effectiveness.

Ablation Study¶

The following results come from Table 3 on the V2XSet test split with AttFuse fixed as the student. Components are added sequentially. The single-teacher row uses AttFuse alone; the multi-teacher configurations use AttFuse, V2X-ViT, and CoSDH.

Config	[email protected] ↑	Gain over ego baseline (percentage points)
Ego-only baseline	58.06	0.00
Single-teacher distillation only	58.29	+0.23
Adaptive multi-teacher global distillation	62.76	+4.70
Global distillation + MLFE	64.58	+6.52
Global distillation + MLFE + APCR	66.50	+8.44
Full M2S, adding local contrast	66.70	+8.64

Starting from multi-teacher global distillation, MLFE and APCR add 1.82 and 1.92 percentage points, respectively; the final local contrastive component adds 0.20. This is an ordered incremental ablation rather than a factorial experiment over all component combinations. It supports complementary benefits but does not isolate fully independent causal contributions.

Key Findings¶

Not every result improves. On V2V4Real validation, Where2comm's [email protected] falls from 27.03 to 25.55, and CoSDH falls from 25.07 to 24.42. Overall test improvements do not imply gains for every model, split, and threshold.
More teachers require additional training resources. Table 5 reports 7.54 hours and 7640 MB for the single-AttFuse-teacher configuration, versus 12.69 hours and 10992 MB for three teachers, measured on one RTX 3090. These configurations should not be conflated with the “single-teacher distillation only” row in Table 3.
The inference-efficiency comparison is between full collaborative perception and ego-only M2S. For AttFuse, FLOPs are 143.8 G versus 92.8 G, speed is 10.3 versus 20.7 FPS, and communication latency is 266.6 versus 0 ms. This does not establish equal computational cost relative to the original ego-only baseline.

Highlights & Insights¶

Additional sensor information need not become a deployment dependency. Using collaborative observations as training supervision derives practicality from changing the input requirements, not merely from more aggressive communication compression.
Geometry and representation gaps receive distinct constraints. Instance reconstruction asks the student to explain object shape, while feature distillation teaches information organization, rather than asking a single feature-distance loss to resolve both discrepancies.
Teacher reliability comes from measurable current-sample error. Reusing the same weights for local and global supervision prevents a downweighted teacher from continuing to propagate unreliable information through fixed averaging in another branch.

Limitations & Future Work¶

The authors explicitly limit the current method to LiDAR detection; multimodal inputs such as LiDAR–camera fusion remain untested. Teacher weighting also requires ground-truth boxes, and ground-truth-free teacher voting is presented as future work rather than an implemented capability.
Training still requires collaborative data collection, alignment, and multiple teacher evaluations. Insensitivity to inter-agent localization noise at test time follows from excluding other agents' inputs; it does not establish robustness to ego LiDAR noise or all distribution shifts.
The final local contrastive increment is small, and the main text does not report variance across repeated runs, leaving its statistical stability uncertain. The local cache lacks the appendix and contains damaged mathematical formatting, further limiting implementation-level verification.
Knowledge distillation can improve the use of available observations but cannot guarantee recovery of newly appearing objects that are entirely invisible to the ego sensor. Comparisons against ego-only baselines support effectiveness, not equality with full collaborative perception in every scene.

vs Where2comm / CoSDH: These methods optimize communication and fusion during collaborative inference. C2E moves collaborative information into training, so its ego-only deployment benefits are not the same metric as communication compression ratios.
vs MKD-Cooper / DiscoNet: Knowledge distillation has already been applied to collaborative perception, and multiple teachers are not a new concept. This paper's main positioning is to transfer collaborative teacher knowledge into an ego-input-only student while handling the resulting density and feature gaps.
vs Sparse2Dense: APCR builds on the idea of avoiding direct generation of all dense points, but uses instance supervision cropped with ground-truth boxes rather than reconstructing the entire scene. This supervision scope is more informative than describing it generically as point cloud completion.

Rating¶

Novelty: 4/5. The collaborative-training/ego-only-deployment objective is clear, while its components build on established attention, reconstruction, and multi-teacher distillation ideas.
Experimental Thoroughness: 4/5. Simulated and real-world datasets, multiple students, and incremental ablations are covered, but statistical variation and fuller ego-only cost comparisons are missing.
Writing Quality: 3/5. Motivation and module relationships are clear, but some conclusions are too broad and cached equations and appendix availability limit readability.
Value: 4/5. The method is a useful reference for ego-only detection deployments with collaborative training data but unreliable communication at inference.