Explainability-aware Frustum Attack: Exposing Structural Vulnerabilities in LiDAR-Based 3D Object Detectors¶
Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/SecMindLab/Saliency_LiDAR
Area: Autonomous Driving
Keywords: 3D Object Detection, Point Cloud Saliency, Integrated Gradients, Frustum Adversarial Attack, Adversarial Robustness
TL;DR¶
This paper introduces SALL, an explainability framework that aggregates instance-level Integrated Gradients across driving scenes to derive universal class-level saliency maps for LiDAR 3D detectors, and proposes EFA, a geometry-consistent frustum attack that selectively perturbs only critical frustums to drop detection recall to near zero while reducing required perturbation budgets by 25%–50%.
Background & Motivation¶
LiDAR-based 3D object detectors serve as an indispensable cornerstone of autonomous vehicle perception. Despite rapid advancements in benchmark accuracy, the fundamental structural dependencies governing model decisions and their implications for adversarial robustness remain poorly understood. While explainability-driven investigations have explored point cloud attribution, prior studies predominantly targeted isolated 3D object classification on normalized CAD datasets like ModelNet40. Such formulations completely sidestep the harsh realities of real-world driving environments—where background clutter, partial occlusions, sensor sparsity scaling with distance, region proposal networks, and non-maximum suppression (NMS) fundamentally alter the decision topology.
Concurrently, physical LiDAR spoofing attacks have evolved from sparse point displacements to wide-angle frustum-level disruptions (e.g., PRA, HFR, A-HFR). By manipulating or erasing return signals across entire angular frustums, these approaches reliably induce critical object-hiding failures. However, they suffer from severe operational inefficiencies: lacking insight into detector behavior, attackers uniformly blast entire spatial sectors (typically spanning a 30° field of view). This brute-force footprint imposes heavy transmission budgets and risks laser emitter overheating or physical tracking dropouts under high-speed driving conditions.
The core tension lies in the trade-off between attack capability and efficiency: whole-frustum suppression is powerful but physically exorbitant, whereas sparse point perturbations are hardware-friendly yet struggle to consistently blind complex 3D detection backbones. Resolving this tension demands understanding where 3D detectors actually concentrate their spatial reliance in multi-object scenes, and whether such attribution patterns are universal across objects, datasets, and architectures. Core idea: aggregate instance-level Integrated Gradients into universal class-level 3D saliency maps (SALL) and use them to guide an Explainability-aware Frustum Attack (EFA) that perturbs only a tiny subset of critical angular frustums along physically consistent LiDAR rays.
Method¶
Overall Architecture¶
The proposed pipeline unifies explainability-guided attribution mining with targeted physical attack execution. In the offline attribution phase, SALL processes raw 3D point cloud scenes, isolates target objects from background clutter, and evaluates point attributions using physically anchored Integrated Gradients. It then projects these attributions via an adaptive voxelization mechanism into standardized 2D contribution matrices, which are accumulated across thousands of diverse scenes and detector architectures to form a universal class-level saliency map. In the online attack phase, EFA discretizes the target object into fine-grained \(1^\circ\) angular frustums based on the sensor's optical origin, ranks frustum criticality according to the SALL saliency prior, and executes ray-consistent point removal or displacement on only the top-ranked frustums before merging the adversarial points back into the scene.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Raw Input LiDAR Scene S"] --> B["Target Object Extraction & Background Separation"]
B --> C["Adaptive Projection and Cross-Scene Aggregation<br/>Compute point attributions and aggregate universal saliency map"]
C --> D["Ray-Consistent Frustum Geometric Discretization<br/>Discretize object space into 1° angular frustums"]
D --> E["Saliency-Guided Minimal Budget Frustum Selection<br/>Rank and perturb only the most critical frustums"]
E --> F["Perturbed Target and Background Merge"]
F --> G["Adversarial Point Cloud S' to Victim Detector"]Key Designs¶
1. Adaptive Projection and Cross-Scene Aggregation: Bridging Instance Heterogeneity to Universal Priors
In complex driving streams, instances of a given class vary dramatically in distance, heading, aspect ratio, and occlusion, rendering raw point-wise gradient maps highly erratic and non-transferable. To extract robust, category-wide attribution patterns, SALL adapts Integrated Gradients (IG) under strict sensing constraints. For each target cloud \(T^i\), it sets an anchor base point \(b_0 = (x_0, y_0)\) at the sensor origin or the nearest corner of the region of interest (ROI, expanded by a factor of 1.5 around the ground-truth box). Along the displacement vector \(z_j = (x_j - x_0, y_j - y_0)\), points evolve linearly across \(M=25\) steps: $\(p_j' = p_j + \frac{N}{M} z_j, \quad N \le M\)$ The perturbed target clouds are merged with background points and passed through detector \(D\). By matching predicted bounding boxes to the ROI via 3D IoU, gradients from non-maximal predictions are filtered out, generating a clean point-level attribution map \(C_p\). To unify diverse object dimensions into a canonical reference frame, SALL applies Adaptive Indexing: target point clouds are transformed into local bounding-box coordinates and adaptively mapped onto a standardized \(64 \times 32\) 2D pixel contribution matrix \(C_v\). Summing matrices across \(k\) scenes yields the universal saliency map \(C_{uv} = \sum_{k} C_v^{(k)}\). The resulting heatmaps reveal that detectors consistently rely on geometric boundaries: PointPillars anchors heavily on the front-left corner, while SECOND concentrates on characteristic "L-shaped" contour walls.
2. Ray-Consistent Frustum Geometric Discretization: Structuring 3D Space into Physical LiDAR Ray Bundles
Physical spoofing devices cannot arbitrarily translate floating 3D points in Euclidean space; they manipulate return pulses along straight optical lines-of-sight. To ensure digital and physical consistency, EFA projects the continuous target region into discrete angular frustums matching physical sensor characteristics. Given the LiDAR sensor position \(P_0 = (x_0, y_0, z_0)\), each target point \(P_j = (x_j, y_j, z_j)\) is converted into an azimuth angle \(D_j\): $\(D_j = \arctan\left(\frac{y_j - y_0}{x_j - x_0}\right) \times \frac{180^\circ}{\pi}\)$ Evaluating all object points and ROI corners establishes the minimum bounding azimuth \(D_{\min}\). With an angular resolution step \(D_{\text{step}} = 1^\circ\) (reflecting standard automotive LiDARs), each point is mapped to a relative frustum index \(f_{id}\): $\(f_{id} = \left\lfloor \frac{D_j - D_{\min}}{D_{\text{step}}} \right\rfloor\)$ This transformation maps unstructured 3D point clusters into an ordered array of physical beam corridors, allowing the 2D pillar-based saliency scores from SALL to be directly projected onto discrete angular attack channels.
3. Saliency-Guided Minimal Budget Frustum Selection: Breaking Reliance on Full-Span Brute Force
Existing frustum spoofing methods uniformly target a wide 30° span covering the entire vehicle, resulting in excessive transmission overhead. EFA replaces heuristic or central targeting with a "Critical Frustum First" budget allocation strategy. Under a constrained budget \(B\) (e.g., \(B \le 15\) for cars, \(B \le 3\) for pedestrians), EFA aggregates the saliency scores of all points within each frustum according to \(C_{uv}\) and selects only the top \(B\) highest-scoring channels. Within the selected frustums, EFA supports both "Remove Perturbation" (suppressing or discarding pulses, as in PRA) and "Shift Perturbation" (altering arrival times along rays, as in HFR). A crucial empirical insight emerges: the frustum selection strategy is the decisive factor governing attack success; once critical geometric boundary frustums are targeted, the specific perturbation mode (shift vs. drop) has virtually no impact on degrading detection logits.
A Worked Example¶
Consider an autonomous vehicle tracking a passenger car located 5–8 meters ahead, spanning approximately 30 continuous \(1^\circ\) frustums in the ego-vehicle's forward field of view: 1. SALL loads the pre-aggregated universal vehicle saliency map (\(64 \times 32\) matrix), which exhibits concentrated attribution peaks along the exterior boundaries and the foreground corner facing the sensor. 2. EFA maps the vehicle's spatial points to frustum indices \(f_{id} \in [0, 29]\) and computes the cumulative saliency weight for each frustum by querying the universal matrix. 3. The sorting step reveals that just 10 frustums along the outer contour and corner edge encapsulate over 70% of the detector's discriminative evidence. 4. EFA selectively applies pulse-delay shifts to only these 10 targeted frustums, leaving the remaining 20 frustums in the vehicle center unperturbed. 5. The target detector's (PointPillars) prediction confidence plunges from 0.92 to 0.18, falling below the detection threshold. The vehicle is completely hidden from the tracking stack while consuming only a fraction of the hardware power required by full-span baselines.
Key Experimental Results¶
Main Results¶
Experiments were conducted on the KITTI and nuScenes benchmarks against PointPillars and SECOND detectors. Target objects include near-field cars (5–8 m) and pedestrians (0–20 m). EFA is benchmarked against state-of-the-art frustum spoofing baselines PRA and HFR.
| Dataset / Target Object | Attack Method | Perturbation Budget | Attack Success Rate (ASR %) | Budget Reduction vs. Baseline |
|---|---|---|---|---|
| nuScenes (Car) | PRA (Baseline) | 20 Frustums | 73.47% | 0% (Reference) |
| nuScenes (Car) | HFR (Baseline) | 20 Frustums | 78.29% | 0% (Reference) |
| nuScenes (Car) | EFA (SALL-Shift, Ours) | 20 Frustums | 96.55% | 0% (+18.26% ASR) |
| nuScenes (Car) | EFA (SALL-Shift, Ours) | 10 Frustums | 78.50% | -50.0% Frustums Required |
| KITTI (Car) | PRA (Baseline) | 20 Frustums | ~62.00% | 0% (Reference) |
| KITTI (Car) | HFR (Baseline) | 20 Frustums | ~67.00% | 0% (Reference) |
| KITTI (Car) | EFA (SALL-Shift, Ours) | 20 Frustums | 97.00% | 0% (+30.00% ASR) |
| KITTI (Car) | EFA (SALL-Shift, Ours) | 15 Frustums | ~75.00% | -25.0% Frustums Required |
| KITTI (Pedestrian) | PRA (Baseline) | 3 Frustums | 73.61% | 0% (Reference) |
| KITTI (Pedestrian) | HFR (Baseline) | 3 Frustums | 74.25% | 0% (Reference) |
| KITTI (Pedestrian) | EFA (SALL-Shift, Ours) | 3 Frustums | 97.00% | 0% (+22.75% ASR) |
Ablation Study¶
Transferability of universal saliency maps across different detector architectures and dataset distributions on Car objects (evaluated with a 20 or 30 frustum budget):
| Saliency Map Source | Target Model / Dataset | Attack Success Rate (ASR %) | Key Mechanism & Note |
|---|---|---|---|
| SECOND (KITTI) | SECOND | 88.12% | Native single-model baseline |
| PointPillars (KITTI) | SECOND | 94.84% | Cross-architecture transfer gains +6.72% ASR |
| Joint (PointPillars+SECOND) | SECOND | 95.29% | Architecture aggregation reinforces shared structural cues |
| SECOND (KITTI) | PointPillars | 97.63% | High cross-model transferability in reverse direction |
| PointPillars (KITTI) | PointPillars | 98.88% | Native single-model baseline |
| Joint (PointPillars+SECOND) | PointPillars | 99.44% | Joint multi-model map achieves highest disruption rate |
| KITTI (PointPillars) | nuScenes | 96.64% | Flawless cross-dataset transfer without retuning |
| nuScenes (PointPillars) | KITTI | 83.82% | Sparse-to-dense sensor transfer retains strong efficacy |
| Joint (KITTI+nuScenes) | nuScenes | 96.90% | Multi-dataset aggregated prior yields optimal generalization |
Key Findings¶
- Frustum selection dominates over perturbation mechanics: SALL's Critical Frustum First policy drives detector recall to 0.0 with 30 frustums, whereas random selection fails to achieve complete suppression even with 50 frustums (the entire ROI). Once high-saliency boundaries are identified, pulse shifting and pulse deletion exhibit negligible performance divergence.
- Distant targets are drastically more vulnerable: Vehicles at 25 meters are hidden with a 94.5% success rate using merely 5 perturbed frustums, since long-range LiDAR point returns are already inherently sparse.
- Saliency priors capture intrinsic object geometry rather than model artifacts: Saliency maps computed on PointPillars fool SECOND with higher success (94.84%) than SECOND's own native map (88.12%). This confirms that SALL isolates fundamental 3D structural dependencies (such as L-shape vehicle corners) common to spatial point representations.
Highlights & Insights¶
- From Instance-Level Optimization to Category-Level Universal Priors: Unlike traditional point cloud adversarial attacks that require iterative gradient descent for every target sample, SALL standardizes and aggregates multi-scene attributions into static universal priors that enable real-time, zero-overhead online deployment.
- Exposing the "Geometric Redundancy" Fallacy: Point cloud perception is widely assumed to be naturally robust due to returning thousands of reflection points per frame. This work proves that modern deep networks disproportionately rely on a razor-thin band of structural edge points, creating severe physical attack surfaces.
- Transferable Physical-Constraint Modeling: The formalization of ray-consistent angular slicing and projection provides a reusable analytical blueprint for assessing physical robustness in other beam-based sensors, such as automotive millimeter-wave and 4D imaging radars.
Limitations & Future Work¶
- Single-Frame and Single-Sensor Scope: The current evaluation is restricted to single-frame, LiDAR-only 3D detection backbones, without examining temporal tracking filters (e.g., Kalman filters) or multi-modal LiDAR-camera fusion networks.
- Operational Alignment Assumptions: The attack assumes knowledge of the victim's approximate expanded bounding box to index angular frustums. In dynamic high-speed driving, severe vehicle pitch and roll could induce alignment jitter relative to ideal \(1^\circ\) frustum bins.
- Defense Vulnerabilities: Existing shadow-based removal detectors yield a prohibitive 60.96% false alarm rate on KITTI benign scenes due to mistaking natural sensor sparsity for malicious pulse deletion. Effective defense requires fundamental architecture-level training regularization that compels detectors to distribute feature attribution redundantly across full object surfaces.
Related Work & Insights¶
- vs Tan et al. (WACV 2023): Tan et al. pioneered Integrated Gradients for isolated CAD classification on ModelNet40; this work scales gradient attribution to complex outdoor driving scenes with clutter, occlusion, distance sparsity, and full detection heads, while achieving cross-scene universal aggregation.
- vs HFR / A-HFR (NDSS 2024 / NDSS 2025): Prior frustum attacks relied on wide-angle brute-force spoofing over 30° sectors, causing severe transmitter overheating; EFA achieves superior attack success rates while cutting required frustums by 25%–50%, maintaining over 92% efficacy even at 50 km/h under constrained physical budgets.
- vs ORA (AutoSec 2021): ORA executes point-level coordinate shifts; when SALL is extended to point-level attacks, it matches ORA's attack potency using 40%–50% fewer spoofed points, validating the efficiency gains of explainability guidance across both granularities.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Successfully pioneers category-level universal 3D saliency aggregation and bridges explainability with physically realizable LiDAR attacks.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorously evaluated across KITTI and nuScenes, multiple detector architectures, physical angular jitter, and velocity-dependent constraints.
- Writing Quality: ⭐⭐⭐⭐⭐ Clear exposition, elegant mathematical formulation of ray-consistent frustums, and tightly argued narrative.
- Value: ⭐⭐⭐⭐⭐ Uncovers structural perception vulnerabilities in autonomous driving, establishing critical insights for robust 3D network design and physical defense benchmarking.