UECP: Uncertainty-Enhanced Collaborative Perception¶

Conference: ECCV 2026
Paper: CVF Open Access
Area: Autonomous Driving
Keywords: collaborative perception, uncertainty quantification, LiDAR point density, BEV feature fusion, robustness

TL;DR¶

Addressing the issue where detector-coupled confidence maps trigger false-positive amplification and feature corruption in collaborative perception, UECP establishes physically grounded uncertainty maps from raw LiDAR point densities and aggregates multi-agent BEV features via uncertainty-weighted downsampling and residual fusion.

Background & Motivation¶

Collaborative perception is indispensable for autonomous driving to overcome single-vehicle occlusions, field-of-view blind spots, and long-range truncation hazards. Among early, intermediate, and late fusion paradigms, intermediate Bird's-Eye-View (BEV) feature fusion has emerged as the mainstream direction, striking an optimal balance between communication bandwidth and 3D detection precision. Nevertheless, in complex open-world vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) deployments, viewpoint discrepancies, spatial pose offsets, and network latencies inevitably introduce severe collaborative noise. Seeking reliable evidence to quantitatively assess and dynamically weigh each participating agent's sensory contribution remains a fundamental challenge for robust multi-agent synergy.

Mainstream collaborative perception frameworks (such as Where2comm and HEAL) typically rely on confidence maps produced by the 3D detector's classification head to modulate feature interaction weights and communication sparsity. However, classification confidence essentially measures the network's semantic posterior belief, which is tightly intertwined with detection errors and classification bias. This coupling triggers a catastrophic "confidence self-reinforcement" cycle: when an agent produces a spurious high-activation false positive, its self-assigned high confidence misleads the fusion network into trusting and broadcasting the erroneous representation, corrupting the global feature representation. Conversely, in distant or sparse regions, weak initial confidence often causes informative observations to be prematurely discarded by hard thresholding, aggravating false negatives.

The core tension stems from the lack of an objective, physically grounded observation quality metric decoupled from the detector's classification head. In LiDAR-based detection, diverse sensing impairments—such as physical occlusion, geometric truncation, long-range beam divergence, and specular surface absorption—uniquely manifest as a drastic decrease in returned point cloud density. Grounded in this physical insight, this paper introduces an independent uncertainty map directly supervised by real-time LiDAR point density. In collaborative perception, high uncertainty does not indicate that an area should be discarded, but rather signifies that this location urgently requires neighboring supportive evidence. Core idea: supervise an independent uncertainty map via physical LiDAR point density to decouple detection noise from feature trust, and embed this reliability prior into multi-scale downsampling and ego-residual aggregation via an Uncertainty-Aware Pyramid Fusion (UAPF) architecture to achieve highly robust, noise-resilient collaborative perception.

Method¶

Overall Architecture¶

The UECP pipeline consists of single-agent BEV feature extraction, physical uncertainty map prediction, cross-agent spatial coordinate alignment and lightweight transmission, and the Uncertainty-Aware Pyramid Fusion (UAPF) module. Each agent independently encodes local raw point clouds into BEV features using a PointPillars backbone, while a dedicated uncertainty head simultaneously predicts a dense reliability map. During collaboration, transmitting agents project their spatial features and single-channel uncertainty maps into the ego agent's coordinate system. The UAPF module then performs coarse-to-fine multi-scale fusion, applying uncertainty-weighted downsampling for feature fidelity and uncertainty-guided residual fusion for ego-centric feature enhancement before feeding the fused representation to the 3D detection head.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multi-Agent Raw Sensor Inputs<br/>Ego and Collaborator Point Clouds"] --> B["BEV Feature Encoding<br/>PointPillars Backbone Extraction"]
    B --> C["Physical Uncertainty Modeling<br/>Density Supervision & Gradient Consistency"]
    C --> D["Cross-Agent Pose Alignment & 1-Channel Transfer<br/>Ego Frame Projection & Low-Bandwidth Exchange"]
    D --> E["Uncertainty-Weighted Downsampling<br/>Reliability-Weighted AvgPool & Pyramid Construction"]
    E --> F["Uncertainty-Guided Residual Fusion<br/>Spatially Smoothed Weighting & Ego Residual Update"]
    F --> G["Multi-Scale Coarse-to-Fine Aggregation & 3D Detection"]

Key Designs¶

1. Physical Uncertainty Modeling: Decoupling Sensing Reliability from Classification Noise

Traditional confidence maps fail to supply unbiased physical evidence and tend to amplify false positives across collaborators. To overcome this limitation, UECP incorporates a dedicated uncertainty head to produce a dense uncertainty map $\mathcal{U}_i \in [0, 1]^{H \times W}$. Ground-truth uncertainty is constructed directly from raw sensor observations: LiDAR point returns are projected onto a BEV plane with grid resolution $\Delta x = \Delta y = 0.4\,\text{m}$ to compute point count density $\mathcal{D}_i$, normalized as $\mathcal{U}_i^{\text{gt}} = \text{norm}(\mathcal{D}_i)$, where sparser point returns directly correspond to higher sensing uncertainty. Because this representation reflects raw physical evidence rather than semantic objectness, high-density background areas provide reliable negative evidence ("empty space here") that systematically suppresses background false alarms.

To enable the network to regress continuous physical density from abstract BEV features while preserving geometric object silhouettes, the uncertainty head is supervised by a joint objective comprising a continuous focal regression term and a Sobel gradient consistency term: $$L_{\text{un}} = \alpha \frac{1}{HW} \sum_{x,y} |e_i[x,y]|^{\rho} (1 - \alpha) + w_g \left(\|\nabla_x \mathcal{U}_i - \nabla_x \mathcal{U}_i^{\text{gt}}\|_1 + \|\nabla_y \mathcal{U}_i - \nabla_y \mathcal{U}_i^{\text{gt}}\|_1\right)$$ where $e_i = \mathcal{U}_i - \mathcal{U}_i^{\text{gt}}$ represents the point-wise prediction error, $\rho$ denotes the focal exponent weighting difficult regions, and $\nabla_x, \nabla_y$ represent fixed Sobel filtering operators. This objective forces the branch to accurately model low-density sparse frontiers while maintaining sharp structural boundaries.

2. Uncertainty-Weighted Downsampling: High-Fidelity Feature Pyramid Construction

Constructing multi-scale BEV representations requires downsampling high-resolution features without diluting critical object evidence. Standard max-pooling is vulnerable to local noise spikes, whereas conventional average-pooling uniformly blurs high-response foreground cues with sparse background clutter. To resolve this trade-off, the Uncertainty-Weighted Downsampling (UWD) module adaptively weights feature locations by their reliability $(1 - \mathcal{U})$ during spatial compression.

At pyramid scale $s$, the downsampled feature map $\mathcal{F}_s$ is computed via the element-wise quotient of two standard average-pooling operations: $$\mathcal{F}_s = \frac{\mathrm{AvgPool}_s\left(\mathcal{F} \odot (1 - \mathcal{U})\right)}{\mathrm{AvgPool}_s(1 - \mathcal{U}) + \epsilon}$$ where $\mathrm{AvgPool}_s$ denotes 2D average pooling with the designated kernel size and stride, and $\epsilon = 10^{-8}$ prevents numerical division by zero. By weighting feature points with their reliability prior before pooling, high-confidence regions dominate the pooled representation, while noisy or occluded grids are attenuated. Operating via standard pooling primitives, UWD achieves zero-parameter, noise-resilient cross-scale feature propagation.

3. Uncertainty-Guided Residual Fusion: Ego-Anchored Robust Multi-Agent Interaction

A central vulnerability in collaborative fusion is feature contamination caused by unreliable collaborators corrupted by pose errors, occlusions, or latency. The Uncertainty-Guided Residual Fusion (UGRF) module establishes a resilient defense mechanism combining adaptive confidence-decay weighting with an ego-anchored residual update. For the ego agent and aligned neighbors $n \in \mathcal{N}(i)$, spatial fusion weights $\lambda_n$ are computed from their respective uncertainty maps: $$\lambda_n = \frac{\exp(-\gamma \mathcal{U}_n) v_n}{\sum_{m=1}^{N} \exp(-\gamma \mathcal{U}_m) v_m + \epsilon}$$ where $\gamma = \mathrm{softplus}(\tilde{\gamma}) > 0$ is a learnable scaling factor, and $v_n \in \{0, 1\}^{H \times W}$ is the spatial visibility mask under the ego frame. A $3 \times 3$ Gaussian blur filter is subsequently applied to $\lambda$ to guarantee spatial weight continuity.

Rather than directly replacing the ego feature with the multi-agent sum, UGRF formulates the collaboration as a controlled residual update over the ego base representation: $$\hat{\mathcal{F}}_i = \mathcal{F}_i + \beta \cdot \phi_{\text{post}}\left(\sum_{n=1}^{N} \lambda_n \mathcal{F}_n - \mathcal{F}_i\right)$$ where $\phi_{\text{post}}$ is a shallow convolutional refinement network and $\beta \in (0, 1)$ is a learnable gating parameter. This formulation ensures that the ego representation acts as the primary anchor, while collaborative messages provide supplementary refinement signals. When collaborators suffer from severe environmental noise or occlusions, weights shift to the ego vehicle, safeguarding the system from catastrophic collaborative degradation.

Loss & Training¶

The network is optimized end-to-end using a joint multi-task loss formulation: $$L_{\text{total}} = \lambda_{\text{reg}} L_{\text{reg}} + \lambda_{\text{cls}} L_{\text{cls}} + \lambda_{\text{dir}} L_{\text{dir}} + \lambda_{\text{un}} L_{\text{un}}$$ where $L_{\text{reg}}$ is a Smooth L1 3D bounding box regression loss, $L_{\text{cls}}$ is a focal classification loss, $L_{\text{dir}}$ is a direction classification cross-entropy loss, and $L_{\text{un}}$ is the uncertainty objective defined in Eq. (4). All models are trained for 60 epochs using the AdamW optimizer and a One-Cycle learning rate scheduler within a unified codebase.

Key Experimental Results¶

Main Results¶

Evaluated on the DAIR-V2X vehicle-to-infrastructure benchmark and the V2V4REAL vehicle-to-vehicle benchmark using a PointPillars LiDAR backbone, UECP sets new state-of-the-art detection marks against both non-collaborative baselines and leading intermediate-fusion methods:

Dataset	Method	[email protected] (%)	[email protected] (%)	[email protected] (%)	Highlights & Characteristics
V2V4REAL (V2V)	No Collaboration	46.12	44.03	27.71	Single-vehicle baseline
	F-Cooper	47.97	40.83	30.64	Max-pooling feature fusion
	AttnFuse	48.18	40.06	28.75	Multi-agent attention fusion
	V2VNet	45.12	43.24	31.75	GNN spatial-temporal message passing
	V2X-ViT	40.11	37.18	24.53	Heterogeneous multi-scale attention
	CoBEVT	63.13	60.10	33.94	Sparse axial-attention Transformer
	Where2comm	49.61	45.10	34.45	Confidence-directed sparse communication
	Who2comm	52.99	46.80	18.14	Handshake communication protocol
	HEAL	51.57	48.70	35.24	Heterogeneous collaborative framework
	CodeFilling	60.15	59.78	35.07	Discrete codebook compression (runner-up)
	UECP (Ours)	65.69	63.38	38.90	Outperforms runner-up by +2.56 / +3.28 / +3.66
DAIR-V2X (V2I)	No Collaboration	59.97	54.23	41.27	Single-vehicle baseline
	F-Cooper	76.94	70.57	54.65	Max-pooling feature fusion
	AttnFuse	69.83	64.06	48.98	Multi-agent attention fusion
	V2VNet	69.25	65.52	47.84	Graph convolutional fusion
	V2X-ViT	77.28	72.47	57.97	Multi-agent Transformer
	CoBEVT	75.67	69.34	49.77	Axial-attention BEV Transformer
	Where2comm	74.92	67.90	48.86	Confidence map thresholding
	Who2comm	77.08	71.22	52.17	Communication handshake mechanism
	HEAL	79.11	73.95	55.56	Unified heterogeneous alignment
	CodeFilling	79.85	75.18	57.26	Codebook feature filling (runner-up)
	UECP (Ours)	81.78	77.59	61.12	Outperforms runner-up by +1.93 / +2.41 / +3.15

Ablation Study¶

1. Cumulative Component Contribution (DAIR-V2X Validation Set)

Config Index	Uncertainty Map (UM)	Multi-Scale (MS)	Weighted Downsampling (UWD)	Residual Fusion (UGRF)	[email protected] (%)	[email protected] (%)	[email protected] (%)	Analysis & Incremental Gain
1	-	-	-	-	76.94	70.57	54.65	F-Cooper max-pooling baseline
2	✓	-	-	-	78.52	72.39	56.92	Adding physical uncertainty guidance (+2.27@70)
3	✓	✓	-	-	79.98	75.73	59.74	Adding multi-scale pyramid backbone (+2.82@70)
4	✓	✓	✓	-	80.08	76.05	60.63	Adding UWD fidelity downsampling (+0.89@70)
5 (Full)	✓	✓	✓	✓	81.78	77.59	61.12	Full model with UGRF residual update (+0.49@70)

2. Fine-Grained Architectural Design Choices (DAIR-V2X Validation Set)

Ablation Dimension	Specific Configuration	[email protected] (%)	[email protected] (%)	[email protected] (%)	Empirical Analysis
Pyramid Scales	(1) Single Scale	80.03	75.43	59.03	Misses multi-receptive field context
	(2, 1) Two Scales	81.00	76.12	59.79	Progressive performance gains
	(4, 2, 1) Three Scales	81.78	77.59	61.12	Coarse-to-fine hierarchy achieves optimal precision
Guidance Map Type	No Map	77.89	72.63	55.91	Lacks spatial quality metric
	Classification Confidence	78.68	73.60	56.47	Limited by false-positive self-reinforcement
	Raw Point Density (Direct)	81.73	77.28	60.77	Slightly below learned uncertainty
	Learned Uncertainty	81.78	77.59	61.12	Distills density into feature-level ambiguity prior
Fusion Operator	Max Pooling	80.33	75.13	57.66	Sensitive to isolated extreme noise
	Mean Average	80.93	76.01	59.86	Dilutes contributions from high-quality views
	Weighted Sum	81.78	77.59	61.12	Retains comprehensive collaborative evidence

Key Findings¶

Substantial Gains at High-Precision Thresholds: UECP demonstrates marked superiority under the strict [email protected] metric (reaching 61.12% on DAIR-V2X and 38.90% on V2V4REAL, outperforming the closest competitors by +3.15% and +3.66%). This confirms that physical uncertainty modeling eliminates spurious high-confidence predictions that degrade spatial regression accuracy.
Universal Plug-and-Play Applicability: Directly replacing the classification confidence map in Where2comm and HEAL with UECP's learned uncertainty map yields average AP improvements of +3.71% and +2.03% respectively, with [email protected] leaping by +5.86% and +3.37%. This highlights that physical uncertainty successfully circumvents the confidence self-reinforcement failure mode across diverse model backbones.
Superior Noise Resilience and Minimal Overhead: Under simulated spatial pose noise ($\sigma \in [0, 1]\,\text{m}$) and temporal communication delays ($0 \sim 500\,\text{ms}$), UECP exhibits much slower performance degradation than competing methods. Importantly, this resilience requires transmitting only a single extra scalar uncertainty channel, incurring an overhead of merely $1/256 \approx 0.39\%$ bandwidth for 256-channel features, while operating at 8.34M parameters and 9 FPS.

Highlights & Insights¶

Decoupled Physical Evidence via LiDAR Density: Using raw LiDAR point density as ground truth decouples spatial reliability from the 3D detector's classification score. High-density background regions provide trustworthy negative evidence that directly suppresses false alarms, systematically addressing false-positive runaway in multi-agent networks.
Closed-Loop Reliability from Downsampling to Residual Fusion: UWD implements quality-preserving multi-scale compression via the quotient of two average-pooling operations with zero parameter bloat, while UGRF frames multi-agent aggregation as an ego-anchored residual update, guaranteeing graceful degradation when collaborators provide degraded cues.
Minimal Communication Footprint with High Generality: By adding only a single 2D uncertainty channel, the design seamlessly integrates with existing communication pruning algorithms (e.g., Where2comm) while offering substantial robustness against transmission latency and pose misalignment.

Limitations & Future Work¶

Author-Acknowledged Limitations: The uncertainty map's ground-truth generation relies on LiDAR point returns; for pure camera-based collaborative perception without LiDAR, an alternative physical grounding must be established.
Experimental Scope & Assumptions: While evaluated under synthetic pose errors and communication delays, real-world deployment challenges involving severe packet loss, network outages, and weather-induced sensor degradation (such as heavy rain or dense fog causing artificial point attenuation) warrant further validation.
Future Improvements: Extending implicit physical uncertainty modeling to multi-modal camera-LiDAR systems and coupling it with temporal trajectory prediction networks to jointly compensate for multi-agent asynchronous misalignment.

vs Where2comm: Where2comm guides communication and feature aggregation using detector-derived confidence maps, which are prone to false-positive self-reinforcement. UECP replaces this with a physical density-supervised uncertainty map, decoupling observation quality from detection scores and outperforming Where2comm by +12.26% [email protected] on DAIR-V2X.
vs HEAL: HEAL focuses on aligning heterogeneous feature spaces but lacks an objective physical reliability metric. Plug-in experiments show that equipping HEAL with UECP's uncertainty map boosts its [email protected] by +3.37%, while UECP's ego-centric residual fusion provides significantly tighter robustness bounds under pose noise.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ Formulating LiDAR point density as a decoupled physical uncertainty prior represents an insightful solution to the confidence self-reinforcement problem in collaborative perception.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous evaluations across DAIR-V2X and V2V4REAL spanning main results, two-stage ablations, plug-and-play validation, pose/latency stress tests, and qualitative visualizations.
Writing Quality: ⭐⭐⭐⭐⭐ Clear logical organization, restrained mathematical formulation, and deep analysis of collaborative noise mechanics.
Value: ⭐⭐⭐⭐⭐ Achieving substantial gains under strict IoU thresholds with only 0.39% extra communication overhead provides high practical value for connected autonomous driving deployments.