RESOLVE: A Multi-Resolution and Multi-Modal Dataset for Roadside Cooperative Perception¶

Conference: ECCV 2026
arXiv: 2606.31895
Code: https://github.com/ASU-Suo-Lab/RESOLVE
Area: Autonomous Driving
Keywords: Roadside cooperative perception, Multi-resolution LiDAR, Multi-modal fusion, 3D object detection, Roadside dataset, Cooperative perception

TL;DR¶

RESOLVE is the first real-world multi-resolution, multi-modal roadside cooperative perception dataset. By simultaneously deploying three LiDARs with 16/64/128 channels at real urban intersections while strictly maintaining sensor configuration and environmental consistency, it systematically evaluates the impact of LiDAR resolution variation on 3D object detection, multi-object tracking, and cooperative perception. It reveals how multi-modal fusion compensates for perception degradation under sparse point clouds and exposes the domain drift issue caused by the mismatch between training and inference resolutions.

Background & Motivation¶

By fusing data from distributed cameras and LiDARs on infrastructure, roadside cooperative perception expands sensing range and mitigates occlusion, playing an increasingly important role in intelligent transportation systems and autonomous driving. However, the deployment of roadside LiDARs in the real world faces an often-overlooked challenge: LiDAR configurations can vary significantly due to budget constraints, hardware upgrade cycles, and procurement batch differences—meaning the same intersection may utilize LiDARs with different beam counts across different operational phases. Furthermore, even with the same LiDAR, objects further away from the sensor naturally become sparser due to the angular resolution sampling characteristics. Such variations in point cloud distribution, caused by both sensor configurations and scene-dependent factors, pose a severe challenge to the robustness of perception models. However, existing roadside datasets (such as UrbanV2X and RCooper) were collected under fixed LiDAR configurations, failing to systematically investigate the impact of resolution changes on detection and fusion performance, let alone answer how much multi-modal fusion can compensate for perception degradation under sparse point clouds.

This data-level gap leaves several critical questions unanswered. Currently, top-performing LiDAR detection models (such as LION and DSVT) and camera-LiDAR fusion models (such as BEVFusion and UniTR) are evaluated under matched sensor sparsity. How will their performance degrade when deployed in real-world scenarios facing resolution changes from different LiDAR beam counts? What is the magnitude of the domain drift when the training LiDAR resolution does not match the inference resolution? More importantly, can multi-modal fusion help models break free from point cloud density dependence, and if so, how? Answering these questions requires a controlled experimental platform that can isolate the resolution factor and conduct cross-architecture comparisons while keeping all other sensing and environmental variables consistent—a capability that no open-source dataset has provided until now.

Core Idea: This paper proposes RESOLVE, the first real-world multi-resolution, multi-modal roadside cooperative perception dataset. By simultaneously deploying three sets of LiDARs (16-channel, 64-channel, and 128-channel) at a real urban intersection and strictly maintaining mounting positions, scene content, and other sensing variables, it achieves systematic isolation of the LiDAR resolution factor. This provides, for the first time, a controlled evaluation platform to analyze the impact of resolution variation on single-modal and multi-modal roadside perception.

Method¶

The design logic of RESOLVE is to construct a real roadside perception experimental platform that can simultaneously provide data in three different LiDAR resolutions while ensuring all other variables remain identical. Based on this, systematic evaluation is performed across 3D detection, multi-object tracking, and cooperative perception, extracting patterns regarding how resolution changes affect perception performance.

Overall Architecture¶

The dataset construction and evaluation pipeline consists of five stages: First, a multi-resolution sensor array (4 directional cameras + 6 cross-resolution LiDARs) is deployed at a real urban intersection. Next, high-precision spatial-temporal calibration and synchronization are performed. This is followed by large-scale data collection and 3D manual annotation. The data is then partitioned into High, Mid, and Low subsets corresponding to different LiDAR resolutions while maintaining consistent annotations. Finally, systematic benchmarking is conducted on three major tasks based on these subsets.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Urban Intersection Scenes<br/>Multi-weather & Multi-lighting Conditions"] --> B["Multi-Resolution Sensor Deployment<br/>4 Directional Cameras + 6 LiDARs"]
    B --> C["High-Precision Spatial-Temporal Calibration<br/>PTP/NTP Synchronization + ICP Registration"]
    C --> D["Large-Scale Data Collection & Annotation<br/>26k Point Cloud Frames · 220k 3D Boxes · 10 Classes"]
    D --> E["Multi-Resolution Subset Partitioning<br/>High/Mid/Low Three Subsets"]
    E --> F["3D Object Detection Evaluation<br/>9 Models × 3 Resolutions"]
    E --> G["Multi-Object Tracking Evaluation<br/>4 TBD Trackers × 3 Resolutions"]
    E --> H["Cooperative Perception Evaluation<br/>4 Fusion Paradigms × 3 Resolutions"]
    F --> I["Resolution Impact Analysis &<br/>Multi-Modal Compensation Insights"]
    G --> I
    H --> I

Key Designs¶

1. Multi-Resolution Sensor Co-deployment: Controlled Experimental Design Isolating Resolution Variables

The core design of RESOLVE is to make the comparison of LiDAR resolution truly "clean." The paper deploys a set of LiDARs on pedestrian signal poles at two diagonal corners of the intersection (northwest and southeast). Each set consists of three LiDARs: a 16-channel (RoboSense Helios-16), a 64-channel (Ouster OS1-64), and a 128-channel (Ouster OS1-128) sensor, totaling six LiDARs. To minimize confounding factors other than resolution, the horizontal positions and zero-phase orientations of the two LiDAR sets are strictly aligned. The only minor difference is their mounting height due to physical constraints (the height difference between high and mid-resolution LiDAR is only 0.17 m, resulting in a 99% ground coverage overlap; the low-resolution LiDAR is mounted slightly lower due to its narrower vertical field of view, but maintains a 95% effective coverage overlap through a unified point cloud range constraint). Consequently, at any given moment under the same scene, RESOLVE offers three sets of observation data that are almost identical except for point cloud density—which is its key differentiating capability from previous datasets. In addition, four AXIS P1455-LE network cameras are installed on traffic poles facing four directions, providing 360° coverage and synchronized RGB images.

2. High-Precision Spatial-Temporal Calibration and Synchronization: The Foundation for Fair Comparison

The prerequisite for cross-resolution comparison is achieving sufficient accuracy in temporal and spatial alignment. RESOLVE adopts a hierarchical synchronization strategy: PTP (Precision Time Protocol) is utilized across LiDARs via industrial switches to achieve microsecond-level synchronization, and phase-locking is used to align rotation cycles. Since the AXIS network cameras do not support hardware synchronization via external triggers, a post-processing matching strategy is applied to match each LiDAR frame with the closest camera frame in time, ensuring the temporal offset does not exceed half a frame interval (outlying matched pairs are discarded). In terms of spatial calibration, camera intrinsic parameters are first calibrated using checkboards, camera-to-LiDAR extrinsic matrices are computed by minimizing 2D-3D re-projection errors, and the relative poses between LiDARs are finely aligned using the ICP algorithm. Together, these steps establish a dataset foundation with bounded spatial and temporal errors, ensuring that the performance discrepancies observed in subsequent cross-resolution comparisons can be attributed solely to the resolution variation itself rather than calibration errors.

3. Multi-Resolution Consistent Annotation Pipeline: Fair Annotating Methodology

This is a highly clever design trade-off. If the point clouds of the three different resolutions were labeled separately, annotator inconsistency would introduce confounding variables; if only the high-resolution point clouds were annotated, the data sparsity in low- and mid-resolution point clouds would cause some boxes to contain almost zero points. The strategy in RESOLVE is: first fuse the point clouds from the two synchronized 128-channel LiDARs into a unified coordinate system and annotate 3D bounding boxes solely on this dense point cloud. After annotation, these 3D boxes are back-projected onto the 64-channel and 16-channel LiDAR coordinate systems using the calibration results. For each projected box, the number of internal points is counted. A box containing \(\ge 5\) points is considered a valid box in the statistics. However, "invalid" boxes containing fewer than 5 points are still kept in the ground-truth set for training and evaluation—they are simply marked with their validity flags during statistical analysis. This ensures that all three resolutions utilize identical 3D ground-truth boxes but differ only in their local point cloud density, fundamentally establishing annotation consistency for cross-resolution comparisons. The overall dataset covers 10 traffic participant classes (Car, Truck, Bus, Van, Construction Vehicle, Trailer, Motorcyclist, Cyclist, Pedestrian, and Golf Cart), containing over 100k images and 26k point cloud frames with 220k 3D bounding boxes, spanning diverse conditions such as sunny, cloudy, rainy, night, and strong backlight.

Key Experimental Results¶

Main Results¶

RESOLVE systematically evaluates 9 detection models, 4 trackers, and 4 cooperative perception paradigms across three LiDAR resolutions for the three major tasks. The following table showcases the main results of 3D object detection:

Model	Modality	Backbone	Low mAP	Mid mAP	High mAP
PointPillars	L	SparseConv	75.1	79.7	80.6
CenterPoint	L	SparseConv	79.9	86.4	87.4
TransFusion-L	L	SparseConv	82.5	86.6	89.1
VoxSeT	L	Transformer	87.4	89.1	90.1
DSVT	L	Transformer	85.9	94.1	94.6
Voxel Mamba	L	Mamba	85.7	94.9	95.4
LION	L	Mamba	88.1	95.1	95.9
BEVFusion	L+C	SparseConv	86.3	92.6	93.1
UniTR	L+C	Transformer	89.7	94.4	94.9

All models were trained using a uniform protocol (20 epochs, Adam, OneCycle LR=0.001) to ensure fair comparison.

Ablation Study¶

One of the core analytical experiments compares single-modal and multi-modal performance under the same LiDAR backbone to reveal the extent to which visual information compensates for sparse point clouds:

Configuration	Low mAP	Mid mAP	High mAP	Remarks
TransFusion-L (LiDAR-only)	82.5	86.6	89.1	LiDAR-only, SparseConv backbone
BEVFusion (Camera+LiDAR Fusion)	86.3	92.6	93.1	Multi-modal fusion, same backbone
Multi-modal Gain	+3.8	+6.0	+4.0	Fusion gain peaks at median resolution

In addition, the paper quantifies the impact of resolution drift by comparing the performance degradation when training and inference resolutions are mismatched (the table below presents the experiments with Voxel Mamba):

Train Res	Inference Low	Inference Mid	Inference High
Low	85.7	~40% drop	~40% drop
Mid	~40% drop	94.9	~4.6% drop
High	~40% drop	~4.6% drop	95.4

Key Findings¶

Diminishing marginal returns of resolution: The average mAP of all models improves by 7.4% from Low \(\rightarrow\) Mid, but only by 1.0% from Mid \(\rightarrow\) High. The medium resolution (64-channel) already captures most discriminative geometric information; further increasing point cloud density offers limited new information, and information bottlenecks at encoding stages (such as voxelization) prevent additional points from translating proportionally into performance gains.
Mamba architectures are most sensitive to resolution: Due to their sequential modeling behavior, Mamba-based models demand higher token quality, exhibiting a substantially higher performance improvement from mid to high resolution (+9.3%) compared to Transformer (+5.7%) and SparseConv (+8.0%). Transformers are relatively the most robust to sparsity due to their global attention mechanism.
Multi-modal fusion matches LiDAR-only high-resolution performance at lower resolutions: The mAP of BEVFusion at low resolution (86.3) exceeds that of TransFusion-L at medium resolution (86.6), implying that a lower-cost LiDAR (16-channel) coupled with cameras can perform comparably to a 64-channel LiDAR-only setup.
Resolution mismatch causes severe domain drift: Shifting a low-resolution model to high-resolution inference leads to an approximate 40% drop in mAP, and shifting a high-resolution model to low-resolution inference also suffers a ~40% loss. This mismatch is even more devastating for multi-modal models (e.g., UniTR), where it causes catastrophic failure with \(\text{mAP} \approx 0\), as resolution changes disrupt both single-modal feature representation and cross-modal alignment.
Simple downsampling cannot replace real low-beam LiDAR: The performance discrepancy of 16-channel data downsampled from 128-channel point clouds compared to real 16-channel LiDAR is highly irregular—underestimating pedestrian detection significantly (AP falls by 29-44%) while being overly optimistic in certain range intervals. This is because downsampling only changes sparsity and cannot simulate real low-beam LiDAR's vertical angle distribution variations, receiver-transmitter optical characteristics, and missing measurements.

Highlights & Insights¶

Pioneering controlled multi-resolution experimental design: Synchronously deploying three LiDAR resolutions at a real-world roadside intersection while strictly keeping other parameters identical is conceptually simple but extremely challenging to execute—it demands precise handling of placement coordinates, field-of-view overlap, and calibration errors. This provides the community with a "gold standard" platform to systematically study the impact of LiDAR resolution.
Quantitative reveal of multi-modal compensation mechanisms: Through controlled gradient-clipping experiments (retaining/severing gradients back-propagating from multi-modal loss to the LiDAR backbone), the paper finds that multi-modal training "reshapes" feature learning in the LiDAR backbone. Although categorical separability in early layers decreases due to gradient interference, it increases significantly in the fusion layer, demonstrating that camera features assist the LiDAR backbone in reducing dependency on point cloud density cues and shifting focus towards better alignment with visual signals.
Practical guidance on roadside cost-accuracy trade-offs: Experimental results on RESOLVE deliver a clear message—in roadside deployments, a 64-channel LiDAR paired with cameras represents the optimal cost-performance trade-off, as moving further to 128 channels yields negligible marginal gains. This insight has direct reference value for infrastructure investment planning.

Limitations & Future Work¶

Currently, RESOLVE covers only a single intersection scenario, lacking diverse continuous road sections and multi-intersection setups. The authors plan to extend it to multi-intersection and corridor-level environments.
The data distribution exhibits class imbalance—mainstream car categories achieve high and stable performance, whereas small targets (motorcyclists, cyclists, pedestrians) and shape-variable categories (trailers, vans) perform poorly at low resolutions, indicating that the conclusions may be asymmetric for large and small objects.
The current evaluation covers 9 detection models and 4 trackers, while more recent perception architectures (e.g., Occupancy networks, end-to-end models) have yet to be systematically evaluated on RESOLVE.
The analysis of resolution mismatch is mainly focused on quantifying "how much it drops." The paper does not propose effective domain adaptation methods to address this—representing one of the most valuable directions for future work.

vs RCooper: RCooper is also a cooperative perception dataset across infrastructure, but only contains single-resolution 64-channel LiDAR, making it incapable of exploring resolution variation. RESOLVE introduces multi-resolution comparative capabilities for the first time.
vs UrbanV2X: UrbanV2X covers broader road environments (including intersections and road segments) and deploys more sensors (3 LiDARs), but remains single-resolution. The unique value of RESOLVE lies in its resolution-isolated experimental design.
vs nuScenes/KITTI: These classic autonomous driving datasets provide single-resolution LiDAR data (32- or 64-channel) primarily from a vehicle-centric perspective. RESOLVE targets the roadside infrastructure perspective, bringing distinct data characteristics and challenges (higher mounting height, longer sensing range, broader field-of-view).
vs DAIR-V2X / V2X-Real: These V2X datasets cover vehicle-to-infrastructure scenes, but LiDAR is mainly utilized on the vehicle end or only features single-resolution on the roadside. RESOLVE focuses on infrastructure-to-infrastructure (I2I) cooperative perception rather than vehicle-to-infrastructure communication.
Key Insight: The multi-modal compensation mechanisms revealed by RESOLVE point toward a design direction: under limited sensor budgets, smart camera-LiDAR fusion strategy can compensate for insufficient LiDAR resolution, eliminating the sheer need to increase LiDAR beam count. Moreover, the severity of the train-test resolution mismatch calls for community attention to the cross-resolution generalization capabilities of perception models.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ The first real-world roadside cooperative perception dataset offering three LiDAR resolutions with controlled comparisons, successfully filling the data gap.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ A comprehensive evaluation covering 3 major tasks \(\times\) 3 resolutions \(\times\) multiple model architectures, featuring multi-angle mechanism analyses (such as t-SNE visualizations and gradient-clipping analysis).
Writing Quality: ⭐⭐⭐⭐ Well-structured with detailed data; the appendix provides extensive implementation details and supplementary experiments, though some large tables require readers to extract key insights themselves.
Value: ⭐⭐⭐⭐⭐ Beyond providing a high-value dataset, it delivers practical insights into resolution impacts and multi-modal compensation mechanisms through systematic baseline experiments, offering direct engineering references for roadside perception system design and cost optimization.