CooperScene: Multi-Modal Cooperative Autonomy Benchmark with C-V2X Communication Characterization¶

Conference: ECCV 2026
arXiv: 2606.31219
Code: https://cisl.ucr.edu/CooperScene (Project Website)
Area: autonomous_driving
Keywords: cooperative perception, C-V2X communication, autonomous driving dataset, multi-agent collaboration, benchmark

TL;DR¶

CooperScene is the first collaborative autonomous driving dataset that synchronously captures C-V2X communication characteristics (throughput, latency, packet loss rate, jitter) and multi-modal sensor data (LiDAR, camera, GNSS/IMU) in real-world road scenarios. It covers interactions among three intelligent vehicles and one road-side unit (RSU) across diverse scenes such as intersections, highway ramps, and parking lots. Benchmark evaluations on this dataset reveal that the mAP of existing cooperative perception methods drops by up to 49% under realistic C-V2X bandwidth constraints compared to ideal network assumptions, highlighting the massive gap between idealized evaluations and deployable performance.

Background & Motivation¶

Single-vehicle autonomous driving, represented by Waymo, has accumulated significant operating mileage on public roads. However, issues such as inherent occlusion blind spots, limited sensor detection ranges, and insufficient capability to handle long-tail events make it difficult to achieve qualitative breakthroughs in safety solely by upgrading single-vehicle sensor schemes. Cellular Vehicle-to-Everything (C-V2X) technology offers a distinct alternative: enabling real-time sharing of perceptual information among vehicles (V2V) and between vehicles and infrastructure (V2I), thereby breaking the physical boundaries of single-vehicle line-of-sight. Recently, cooperative perception methods represented by V2VNet, CoBEVT, and V2X-ViT have made significant progress on public datasets like OPV2V and V2V4Real, achieving detection accuracy far exceeding single-vehicle perception by sharing intermediate feature maps.

However, existing works suffer from three critical blind spots. First, real-world V2X communication bandwidth is extremely limited (the measured throughput of 3GPP Release 14 commercial automotive-grade C-V2X modules is typically below 1 Mbps), while state-of-the-art cooperative perception models require exchanging from dozens of KB to dozens of MB of feature data per frame. This implies that many methods designed under ideal network assumptions cannot complete data exchange in time over realistic C-V2X links, failing to complete transmission before the decision window expires. Second, most existing datasets only involve two-party collaboration (one vehicle + one RSU, or two vehicles), lacking benchmark evaluations for multi-vehicle (three or more) collaboration, where channel contention and feature accumulation effects brought by more vehicles might alter the judgment of whether cooperation is indeed beneficial. Third, data quality (sensor synchronization accuracy, spatial alignment accuracy, and calibration precision) remains inconsistent across existing datasets. Cross-device time synchronization errors of 30-50 ms lead to temporal misalignment and degraded perception performance, yet this has rarely been systematically validated. Fortunately, commercial automotive-grade C-V2X communication modules (e.g., Cohda MK6) and low-cost high-precision LiDARs have matured to a deployable level for real-world collection, and modular sensor acquisition platforms (CMS) have significantly lowered the barriers to multi-vehicle physical replication—making a high-quality, multi-vehicle, multi-scenario dataset with real-world communication data technically feasible.

Core Idea: To build a real-world cooperative autonomous driving dataset consisting of three vehicles and one road-side unit, all equipped with multi-modal sensors and commercial-grade C-V2X modules. Precision synchronization, centimeter-level spatial alignment, and MoCap-grade sensor calibration ensure data quality, while C-V2X communication characteristics are provided as an evaluation dimension, allowing researchers to evaluate both perception accuracy and communication efficiency to uncover the gap between ideal and deployable performance.

Method¶

Overall Architecture¶

Fundamentally, CooperScene is an end-to-end pipeline spanning from data collection to benchmark evaluation. The collection layer comprises three vehicle-mounted CMS (Cooperative Multi-Modal Sensing) platforms and one roadside infrastructure, each integrated with LiDAR, camera, GNSS-IMU, and commercial-grade C-V2X communication modules. The synchronization layer achieves microsecond-level alignment of on-board sensors via PTP + hardware triggering, followed by cross-vehicle synchronization achieved via GNSS global time. The spatial alignment layer refines the local point clouds of each vehicle into the global ENU coordinate system using the proposed Spatio-Temporal ICP (ST-ICP). Under the calibration layer, millimeter-level extrinsic parameters among sensors are obtained with a MoCap motion-capture system and eye-hand calibration. The collected raw data undergoes a semi-automatic annotation pipeline (incorporating detection pre-labeling, tracking association, cross-vehicle post-fusion, and manual refinement) to generate globally consistent 3D annotations. Finally, the evaluation layer integrates all baseline methods into a unified OpenCOOD-based pipeline, overlaying a configurable C-V2X network simulator to report performance under both ideal networks and realistic C-V2X conditions.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["CMS Acquisition Platform<br/>3 Veh + 1 RSU"] --> B["Sensor Sync<br/>PTP + Hardware Trigger"]
    B --> C["Spatio-Temporal Alignment<br/>ST-ICP to Global Frame"]
    C --> D["Sensor Calibration<br/>MoCap + Eye-Hand"]
    D --> E["Scene Data Collection<br/>Intersections/Ramps/Parking"]
    E --> F["Semi-Auto Annotation<br/>Detection→Tracking→Fusion→Manual"]
    F --> G["Unified Eval Pipeline<br/>OpenCOOD + C-V2X Sim"]
    G --> H["Dual-Condition Benchmark<br/>Ideal Network vs Real C-V2X"]

Key Designs¶

1. CMS Cooperative Acquisition Platform: Unifying Multi-Vehicle Hardware Configuration and Calibration Workflows

The data quality of CooperScene relies on the self-developed CMS platform—an open-source portable universal sensing hub that integrates LiDAR (Ouster OS1-128), camera (Lucid 5.4 MP), GNSS/IMU (Xsens MTi-680 RTK), and C-V2X modules (Cohda MK6) via a PoE switch to a vehicle-mounted ROS node. It addresses a practical bottleneck: multi-vehicle collection requires consistent, replicable sensor configurations, synchronization schemes, and calibration processes across all vehicles, whereas individual vehicle sensor mounting variations mean extrinsic calibrations cannot be directly reused. CMS ensures cross-vehicle consistency through a unified hardware integration scheme and a shared calibration program. Three CMS vehicles and one road-side unit (equipped with LiDAR and MK6, mounted on an intersection traffic light pole) captured 24 interactive segments across three scenarios: urban intersections (16 segments), highway ramps (4 segments), and parking lots (4 segments). Vehicles were scheduled to merge from different start points into the interaction area simultaneously, simulating occlusion, long-range targets, and multi-directional merges. The campaign yielded 59,000 synchronized LiDAR frames and 53,000 images, with 344,000 labeled 3D bounding boxes.

2. High-Precision Synchronization and Spatio-Temporal Alignment: Crucial System Design for Eliminating Multi-Modal Multi-Vehicle Misalignment

High-quality cooperative perception data demands strict requirements for temporal synchronization and spatial alignment, which are precisely the weakest links in existing datasets. Temporally, CooperScene addresses alignment at both inter-sensor and cross-vehicle levels: on each vehicle, a Cohda MK6 serves as the PTP master clock to synchronize all sensors, and uses 10 Hz GPIO pulses generated by the LiDAR to hardware-trigger the camera exposure—aligning the camera shutter with the LiDAR scan cycle to eliminate network jitter and scheduling latencies in software timestamps. Across vehicles, the GNSS global time is utilized to lock the MK6 master clocks of each platform, securing a shared temporal reference. Spatially, a Spatio-Temporal ICP (ST-ICP) method is proposed: GNSS/IMU first projects each vehicle's point cloud into the global ENU coordinate system, after which coarse ICP calibration is conducted on the single frame with the largest point cloud overlap between each vehicle and the RSU, followed by a joint ICP on the frame that minimizes the sum of all vehicle distances. With alignment achieved using only a single frame per vehicle throughout the process, the final alignment RMSE stands at 0.2 m, outperforming V2V4Real and DAIR-V2X-C.

3. C-V2X Communication Characterization: Embedding Real-world Bandwidth Constraints into Benchmarks

This is the core feature that distinguishes CooperScene from all prior works. During data collection, the Cohda MK6 on each CMS simultaneously records real-time throughput, latency, packet loss rate, and jitter between all V2V and V2I links, sampled at 100 ms intervals. During evaluation, these realistic communication traces are fed into the unified evaluation pipeline, forcing the baseline methods to run under the constraint that "data transmission must adhere to real-world C-V2X bandwidth limits." The feature datasets are packetized into 2100-byte packets sent at 1 ms intervals, strictly conforming to the physical resource allocation of the 3GPP Release 14 C-V2X standard. This yields a previously overlooked insight: even when exchanging only 59.5 KB of features (the scale of CoSDH), transmission under C-V2X still takes approximately 298 ms. A vehicle traveling at 40 km/h covers 3.3 meters in 300 ms. Data exceeding the decision window is not only unhelpful, but such stale information can also mislead the perception system.

A Concrete Example: The C-V2X Dilemma of CoBEVT under V+2V+I Cooperation¶

Consider CoBEVT under a full four-agent (V+2V+I) configuration. The feature volume to be exchanged per frame is approximately 1.5 MB. Under an ideal infinite-bandwidth assumption, all collaborators receive each other's features almost instantaneously, and the fused three-vehicle plus one RSU data yields an [email protected] of 0.75. However, on a realistic C-V2X channel, transmitting 1.5 MB of data takes about 12.4 seconds—far exceeding the decision window of autonomous driving. During this interval, vehicles must rely on their local perception. Crucially and counter-intuitively, adding the third vehicle not only fails to improve performance but actually degrades it due to intensified channel contention (where more vehicles share the same spectrum and interfere with each other), causing the actual [email protected] to drop from 0.55 (under V+V) to 0.50 (under V+2V+I). This result highlights the fundamental contradiction of current cooperative methods: designed on the assumption of infinite bandwidth, but strictly bound by physical bandwidth bottlenecks in deployment.

Loss & Training¶

Because this work presents a dataset and benchmark, it does not introduce new model training strategies. All baseline methods use the default configurations in OpenCOOD: PointPillars backbone, LiDAR voxel resolution of 0.4 m, maximum of 32,000 non-empty voxels per frame, trained on a single GPU for approximately 40 epochs, with data augmentation including random flipping, \(\pm45^\circ\) rotation, and global scaling.

Key Experimental Results¶

Main Results¶

The table below compares the perception accuracy of different cooperative models under the V+2V+I configuration (all four agents participating) under C-V2X bandwidth constraints versus infinite network conditions.

Model	Network Condition	[email protected]	[email protected]	[email protected]	Shared Data Size per Frame	C-V2X Transmission Latency
V2VNet	Infinite	0.81	0.75	0.50	30.0 MB	107.4 s
V2VNet	C-V2X	0.32	0.23	0.13	—	—
V2X-ViT	Infinite	0.80	0.74	0.50	1.01 MB	10.0 s
V2X-ViT	C-V2X	0.60	0.52	0.33	—	—
V2VAM	Infinite	0.84	0.78	0.53	1.27 MB	11.3 s
V2VAM	C-V2X	0.62	0.54	0.34	—	—
CoBEVT	Infinite	0.82	0.75	0.54	1.50 MB	12.4 s
CoBEVT	C-V2X	0.59	0.50	0.32	—	—
ERMVP	Infinite	0.87	0.81	0.57	1.04 MB	5.2 s
ERMVP	C-V2X	0.64	0.56	0.36	—	—
CoSDH	Infinite	0.81	0.73	0.52	0.18 MB	0.9 s
CoSDH	C-V2X	0.75	0.61	0.41	—	—

Estimated transmission latency is based on the data volume and the measured C-V2X throughput; mAP under C-V2X conditions is simulated using real-world communication traces. Even though CoSDH requires only 0.9 seconds of transmission, the high packet loss rate (approx. 38%) and jitter of C-V2X still degrade its performance.

Ablation Study: Number of Vehicles and Communication Conditions¶

Configuration	C-V2X [email protected]	Infinite Network [email protected]	Description
V (Single Veh)	0.50	0.54	V2X-ViT baseline, no cooperation
V+I	0.50	0.54	Single vehicle plus roadside
V+V+I	0.56	0.70	Single vehicle plus roadside plus one vehicle
V+2V+I	0.60	0.80	All participants

C-V2X [email protected] from V+I to V+2V+I only improves from 0.50 to 0.60, whereas under infinite network it leaps from 0.54 to 0.80, demonstrating that the theoretical advantages of multi-vehicle fusion fail to materialize under bandwidth constraints.

Key Findings¶

Bandwidth bottlenecks are the primary limiting factor: Communication-inefficient methods (V2VNet, CoBEVT) show almost no gains under C-V2X, performing worse than single-vehicle perception; although communication-efficient methods (CoSDH, ERMVP) demonstrate relative advantages, their transmission times still significantly exceed safety-critical decision requirements.
Packet loss and latency must be resolved independently of bandwidth: Even in oracle experiments where bandwidth constraints are eliminated, C-V2X's inherent 38% packet loss rate and 31 ms latency still degrade detection accuracy relative to infinite networks, indicating that robust cooperative perception must address both bandwidth compression and communication resilience.
C-V2X throughput exhibits massive dynamic fluctuations: Vehicle mobility and channel contention cause effective throughput to fluctuate dramatically frame-by-frame. No single agent can continuously dominate the channel, which demands that future cooperative protocols possess self-adaptive capabilities.

Highlights & Insights¶

Embedding communication characteristics into the dataset itself is a methodological contribution: Prior cooperative perception datasets treated communication as a transparent pipe, assuming "transmitted data always arrives, and arrives on time." CooperScene shatters this assumption with real-world V2X module measurements, quantifying the massive chasm between laboratory performance and real-world deployment, and providing a benchmark for network-perception co-design.
Counter-intuitive findings on multi-vehicle cooperation: Under limited bandwidth, adding more participants does not necessarily translate to better performance; indeed, it may even degrade due to channel contention. This requires future cooperative architectures to explicitly model the trade-off between "fusion gain" and "transmission cost."
The efficiency of ST-ICP: Achieving a multi-vehicle alignment of 0.2 m RMSE relying on just a single frame per vehicle is significantly more efficient than TUM-Traf's practice of performing ICP every 10 frames.
Transferable research paradigm: The paradigm of embedding a "real-world network condition layer" into benchmarks is directly applicable to other bandwidth-constrained multi-agent systems such as V2I communications, UAV swarms, and industrial robot collaboration.

Limitations & Future Work¶

Currently, annotations only cover the "Car" class; pedestrians and cyclists will be added in future versions, which is crucial for safety-critical applications.
C-V2X communication features only record network-layer metrics (throughput, latency, packet loss). Physical-layer measurements, such as wireless resource scheduling and received signal strength indicators (RSSI), have not yet been integrated. Including these will drive cross-layer cooperative perception designs.
The roadside infrastructure is only equipped with LiDAR (no camera), and each vehicle only has a single forward-facing camera, lacking wider sensor perspectives, which limits the diversity of multi-modal fusion evaluations.
The collection scenarios are concentrated near a single intersection type and do not yet cover more challenging communication environment classes, such as long highway straights or urban canyons. Additionally, the geographic characteristics of Riverside, California, may introduce geographical bias.
The evaluation format is "sender compresses \(\rightarrow\) C-V2X transmits \(\rightarrow\) receiver fuses," which neglects the additional latency brought by feature compression/encoding itself—a factor that cannot be ignored in safety-critical systems.

vs OPV2V / V2V4Real: Pioneered the cooperative perception evaluation paradigm but only involved two vehicles and lacked real-world communication characteristics. CooperScene scales up to 3 vehicles + 1 RSU and introduces real-world C-V2X physical data for the first time.
vs DAIR-V2X-C / V2X-Seq: Contains vehicle-road cooperation but only has one vehicle per scene. Its paper notes that cross-modal projection exhibits non-trivial spatial misalignment and lacks systematic synchronization validation.
vs TUM-Traf: High positioning accuracy but only supports a single host vehicle and RSU. TUM-Traf requires running ICP every 10 frames, whereas CooperScene's ST-ICP achieves similar accuracy with just a single frame.
vs UrbanIng-V2X: Broad multi-vehicle coverage but limited multi-modal sensing capabilities, whereas each vehicle in CooperScene is fully equipped with multi-modal sensors and C-V2X modules.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ The first dataset to synchronously record real-world C-V2X communication characteristics in multi-vehicle scenarios, closing the gap between ideal evaluations and deployable performance with highly original insights.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Evaluates 7 baseline methods across 5 cooperative configurations and 2 network conditions on both detection and prediction tasks, backed by meticulous validation of synchronization, alignment, and calibration.
Writing Quality: ⭐⭐⭐⭐⭐ The problem motivation is distinct and strongly backed by quantitative evidence (the contrast between bandwidth demands and C-V2X capacity on OPV2V is exceptionally compelling), with systematic methodological descriptions and clear conclusions.
Value: ⭐⭐⭐⭐⭐ Introducing realistic C-V2X communication constraints to the CV community is poised to drive a paradigm shift in cooperative perception from "accuracy-oriented" to "communication efficiency + robustness-oriented" designs, offering direct practical guidance for industry deployment.