PointLAM: Local Attentive Mamba for Efficient Point-based 3D Object Detection

Conference: ECCV 2026
Paper: ECCV Paper
Code: https://pointlam.github.io/
Area: Autonomous Driving
Keywords: 3D object detection, point cloud, Mamba, state space model, local attentive gating

TL;DR

Addressing the longstanding dilemma between high computational latency in point-based architectures and quantization loss in voxel-based detectors, PointLAM introduces the Laplacian Point Sampler (LPS) with doubly sorted sampling and the Local Hadamard Aggregator (LHA) with transient grid routing coupled with bidirectional Mamba, achieving top-tier accuracy and voxel-rivaling inference speed while strictly preserving fine-grained point geometry.

Background & Motivation

LiDAR-based 3D object detection is a vital perception foundation for autonomous vehicles, yet it has long been confronted with a fundamental trade-off between geometric fidelity and computational efficiency. Mainstream voxel-based detectors, relying on 3D sparse convolutions (SpCNN) or sparse Transformers, regularize unstructured points into discrete grids to gain high inference throughput. However, spatial quantization inevitably destroys subtle geometric edges and surface structures, leading to severe performance degradation on small targets and distant sparse objects. Conversely, point-based methods (such as PointNet++ and 3DSSD) operate directly on continuous point coordinates, avoiding quantization errors, but remain computationally impractical for real-world deployment due to severe systemic latency bottlenecks.

These systemic bottlenecks stem from two core operations in the point-based paradigm: downsampling and local neighborhood modeling. First, standard Farthest Point Sampling (FPS) exhibits an \(O(N^2)\) time complexity whose iterative Euclidean distance searches become an extreme computational bottleneck on large-scale point clouds, all while remaining completely agnostic to semantic foreground geometry; meanwhile, semantic-guided downsamplers incur heavy network overhead and lack robust geometric priors. Second, local context modeling has traditionally relied on dynamic k-NN or ball queries, which are computationally expensive and density-sensitive—frequently pulling distant, irrelevant background points into sparse target features—while continuous relative positional embeddings add further latency.

Although state space models like Mamba provide linear-complexity sequence modeling for long sequences, flattening 3D spatial points into 1D sequences intrinsically scrambles spatial locality and continuous neighborhood topology, leading to severe spatial contextual disruption when applied naively to point clouds. PointLAM circumvents heavy continuous spatial queries and costly sorting heuristics through discrete geometric analysis and local-global structural decoupling. Core Idea: employ an implicit discrete Laplacian high-pass filter with doubly sorted sampling (LPS) to rapidly preserve foreground skeletons, and leverage transient grid routing with Hadamard gating (LHA) to continuously anchor local geometric topology across bidirectional Mamba sequence scans, achieving linear-complexity global perception while preserving fine-grained point fidelity.

Method

Overall Architecture

PointLAM adopts an end-to-end point-based detection framework composed of two primary stages: the Laplacian Point Sampler (LPS) for rapid structure-aware downsampling, and a hierarchical 3D backbone consisting of \(N=4\) stacked Local Attentive Mamba (LAM) blocks.

The overall processing flow is organized as follows: first, raw LiDAR point clouds enter LPS, where the Deviation Network extracts high-frequency geometric deviations within partitioned regions, followed by Doubly Sorted Sampling (DSS) to select geometrically salient foreground points while guaranteeing uniform spatial distribution. Next, the sampled points are forwarded to the cascaded LAM blocks. Within each block, X-axis and Y-axis Bi-Directional Mamba (BDM) layers establish global contextual dependencies, while interleaved Local Hadamard Aggregator (LHA) modules continuously anchor local geometric topology via transient grid routing and element-wise gating. Finally, after multi-stage feature extraction, point features are projected onto a Bird's-Eye-View (BEV) feature map for bounding box regression and object classification by the detection head.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Raw LiDAR point cloud<br/>N unordered continuous points"] --> DevNet["Deviation Network (DevNet)<br/>computes local mean deviation for Laplacian filtering"]
    DevNet --> DSS["Doubly Sorted Sampling (DSS)<br/>dual sorting on saliency and spatial region"]
    DSS --> Backbone["3D Backbone: N cascaded LAM blocks"]
    subgraph LAM["Local Attentive Mamba Block (LAM)"]
        direction TB
        BDM["Bi-Directional Mamba (BDM)<br/>axis-aligned 1D scanning for global dependencies"] --> LHA["Local Hadamard Aggregator (LHA)<br/>transient grid routing + Hadamard gating"]
    end
    Backbone --> LAM
    LAM --> BEV["BEV Projection<br/>transforms 3D point features into 2D BEV representation"]
    BEV --> Head["3D Detection Head<br/>predicts category logits and 3D bounding boxes"]

Key Designs

1. Deviation Network (DevNet): discrete Laplacian high-pass filtering for geometric singularity preservation

Standard Point Feature Networks (PFNs) rely heavily on max-pooling for intra-partition aggregation. Max-pooling acts as an aggressive low-pass filter with sparse, winner-take-all gradient assignment (zero gradient for non-maximal points), causing numerically weaker but structurally vital foreground edge signals to be suppressed by dominant flat backgrounds. DevNet replaces max-pooling with a local deviation operator, computing the feature deviation of each point \(h_i \in \mathbb{R}^D\) from the local spatial mean \(\bar{h}\):

\[\delta_i = h_i - \bar{h} = h_i - \frac{1}{N}\sum_{j=1}^N h_j\]

Mathematically, this deviation formulation mirrors the discrete Laplace operator. As a second-order spatial derivative measuring local signal roughness, it yields near-zero responses across smooth low-frequency regions (where \(h_i \approx \bar{h}\), such as roads and walls) while producing high-magnitude responses at high-frequency geometric singularities like object corners and structural boundaries. Crucially, this closed-form formulation produces dense, collaborative gradients across all points during backward propagation:

\[\frac{\partial \mathcal{L}}{\partial h_i} = \frac{\partial \mathcal{L}}{\partial \delta_i}\left(1 - \frac{1}{N}\right) - \frac{1}{N}\sum_{j \neq i} \frac{\partial \mathcal{L}}{\partial \delta_j}\]

This gradient mechanism forces the network to optimize statistical distinctiveness relative to local neighborhoods. The final enriched representation is constructed by concatenating intrinsic point features with their Laplacian responses: \(F'_{p_i} = \text{Concat}(h_i, \delta_i)\).

2. Doubly Sorted Sampling (DSS): latency-free downsampling via saliency and regional sorting

To eliminate the computational latency of iterative distance queries in Farthest Point Sampling (FPS), DSS computes a scalar saliency score \(S(p_i) = \|F'_{p_i}\|_2\) from DevNet outputs and transforms downsampling into a deterministic two-step sorting procedure. Points are first globally sorted by saliency \(S(p_i)\) in descending order, and then stably sorted by their regional partition indices \(r'_j\). This ensures that the most geometrically distinctive points consistently lead each local spatial bin.

Once doubly sorted, points \(p'_j\) are selected using a deterministic sliding-window rule: points with global rank \(j < k\) are retained immediately; for points where \(j \ge k\), a point is accepted only if its region index differs from that of the \(k\)-th preceding selected point (\(r'_j \neq r'_{j-k}\)). This selection rule reduces downsampling complexity to that of sorting, strictly guaranteeing uniform spatial coverage while concentrating the point budget along object geometric skeletons at negligible runtime cost.

3. Local Hadamard Aggregator (LHA): transient grid routing and topology-aware Hadamard modulation

Dynamic k-NN queries in continuous space incur heavy search overhead and suffer from density sensitivity, often retrieving irrelevant background points in sparse point regions. LHA decouples spatial neighbor indexing from feature representation by using a discrete "transient grid" purely as a deterministic spatial router. Each continuous point \(p_i\) is mapped to a grid coordinate \(v_i = \mathcal{V}(p_i)\), defining the neighborhood \(\mathcal{N}(i)\) as points located within a bounded kernel window centered on \(v_i\) (e.g., \(3\times 3\times 3\)). Point features are never quantized or pooled into voxel embeddings; rather, sparse convolution weights \(W\) aggregate local context directly across continuous points:

\[f^{local}_i = \sum_{j \in \mathcal{N}(i)} W(\mathcal{V}(p_j) - \mathcal{V}(p_i)) f_j\]

After mapping \(f^{local}_i\) to context vector \(c_i = \text{Linear}(f^{local}_i)\), LHA replaces continuous relative positional embeddings with Hadamard Gating. A parallel linear layer projects the point feature \(f_i\) into gating vector \(g_i\), and an element-wise Hadamard product modulates the aggregated context, stabilized by a residual skip connection:

\[f^{out}_i = (g_i \odot c_i) + f_i\]

This mechanism enforces a strictly bounded spatial receptive field that excludes distant noise, dynamically modulating channel-wise responses according to local topology with near-zero computational overhead.

4. Bi-Directional Mamba (BDM) and LAM architectural synergy: low-latency sequence scanning with local geometric anchoring

While standard Mamba layers provide linear-time global context modeling, flattening 3D point clouds into 1D sequences disrupts spatial adjacency. Complex space-filling curves like Hilbert or Z-order curves incur substantial sorting latencies (5-11 ms) without fully mitigating higher-order spatial disruptions.

PointLAM resolves this dilemma through the architectural synergy of the LAM block: complex space-filling curves are replaced by simple coordinate axis sorting (Axis Sort), which requires only 0.1 ms of sorting latency. Within each LAM block, point streams are serialized along the X-axis and scanned by a bidirectional Mamba layer to establish long-range context. An interleaved LHA module immediately re-anchors local geometric topology within the continuous coordinate space, restoring spatial proximity weakened during 1D serialization. A second Mamba layer then scans along the Y-axis, followed by further LHA refinement. This continuous local anchoring prevents serialization errors from compounding across deep layers, rendering global perception highly robust to the exact scanning trajectory.

Loss & Training

PointLAM is trained in an end-to-end, single-stage manner on NVIDIA A800 GPUs using the OpenPCDet codebase. On the nuScenes dataset, the model is trained without Class-Balanced Grouping and Sampling (CBGS) for 36 epochs (totaling 63,324 optimization steps, compared to the 154,480 steps required by standard 20-epoch CBGS schedules used by DSVT and HEDNet), significantly reducing training compute. On Waymo, PointLAM is trained for 24 epochs following standard protocol. LPS spatial partition bins are set to \(0.3\text{m} \times 0.3\text{m} \times 0.25\text{m}\) for nuScenes and \(0.32\text{m} \times 0.32\text{m} \times 0.1875\text{m}\) for Waymo, setting \(k=1\) salient point per partition. In each LHA module, the effective \(5\times 5\times 5\) receptive field is factorized into two cascaded \(3\times 3\times 3\) sparse convolutions to minimize neighbor indexing overhead.

Key Experimental Results

Main Results

PointLAM achieves competitive detection accuracy on both the nuScenes and Waymo Open Dataset benchmarks, matching top-performing voxel-based detectors while significantly raising the performance bar for point-based architectures.

Dataset / Benchmark	Metric	PointLAM (Ours)	Representative Baselines / Prev. SOTA	Comparison & Gain
nuScenes Val	NDS / mAP	72.2 / 67.8	DSVT (71.1 / 66.4) LION (72.1 / 68.0) UniMamba (72.6 / 68.5)	Outperforms voxel Transformer DSVT (+1.1 NDS); exceeds voxel Mamba baseline LION (+0.1 NDS)
nuScenes Test	NDS / mAP	73.0 / 68.8	3DSSD (56.4 / 42.6) DSVT (72.7 / 68.4) Voxel Mamba (73.0 / 69.0)	Improves over prior point detector 3DSSD by +16.6 NDS / +26.2 mAP; strictly matches Voxel Mamba
Waymo Val	L1 / L2 mAPH	79.7 / 73.6	DSVT (78.2 / 72.1) HEDNet (79.4 / 73.4) Voxel Mamba (79.6 / 73.6)	Outperforms DSVT (+1.5 L2) and HEDNet (+0.2 L2); matches Voxel Mamba
Waymo Test	L1 / L2 mAPH	79.8 / 74.4	SAFDNet (79.8 / 74.6) Voxel Mamba (79.6 / 74.3)	Matches fully sparse voxel SOTA; strong pedestrian detection at 75.2 L2 APH

In computational footprint and inference efficiency (benchmarked on a single NVIDIA A800 GPU), PointLAM delivers substantial advantages:

Detector	Architecture Type	Representation	Params (M)	FLOPs (G)	Latency (ms)	nuScenes NDS	Waymo L2 mAPH
VoxelNeXt	spCNN	Voxel	15.5	97.5	192.9	66.7	70.1
HEDNet	spCNN	Voxel	4.6	106.2	96.7	71.4	73.4
DSVT-voxel	Transformer	Voxel	2.7	108.6	115.6	-	72.1
LION	Mamba	Voxel	10.1	165.8	195.3	72.1	74.0
Voxel Mamba	Mamba	Voxel	15.1	246.2	109.8	71.9	73.6
PointLAM (Ours)	LAM (Mamba+LHA)	Point	8.6	90.7	93.1	72.2	73.6

Compared to voxel Mamba detector LION, PointLAM reduces computational FLOPs by 45% (90.7G vs. 165.8G) and accelerates inference by over \(2\times\) (93.1 ms vs. 195.3 ms). Compared to Voxel Mamba, it achieves equivalent accuracy using only 37% of its FLOPs.

Ablation Study

The ablation experiments systematically isolate the contribution of each module on the nuScenes validation set under an 18-epoch schedule:

Ablation Dimension	Configuration	mAP (%)	NDS (%)	Key Mechanism & Impact
LPS Module Decoupling	PFN + Pooling (Standard baseline)	67.41	71.42	Conventional point feature network with local average/max pooling
	DevNet + Pooling	67.82	71.59	Adding Laplacian deviation features captures high frequencies (+0.17 NDS)
	PFN + DSS	67.77	71.56	Replacing pooling with doubly sorted sampling preserves skeletons (+0.14 NDS)
	DevNet + DSS (Full LPS)	68.14	71.82	Combines geometric prior and fast selection, gaining +0.40 NDS over baseline
LHA Aggregation Mode	Addition	67.13	70.98	Feature addition lacks expressive channel-wise interaction
	Concatenation	67.63	71.51	Channel concatenation introduces extra projection without dynamic filtering
	Subtraction	67.51	71.53	Direct difference aggregation
	Hadamard Gating	68.14	71.82	Element-wise gating provides adaptive modulation, outperforming others by +0.29 NDS
LAM Architectural Synergy	LHA only (w/o BDM)	64.37	69.40	Lacks long-range context, leading to poor global perception
	BDM only (w/o LHA)	64.87	69.58	1D sequence scanning disrupts spatial structure, overfitting to scan artifacts
	BDM (X-axis) + LHA	67.26	71.23	Single-axis scan combined with local topology anchoring
	Full LAM (BDM X+Y + LHA)	68.14	71.82	Bidirectional scanning + topological anchoring boosts performance by +2.24 NDS

Spatial indexing and scan sequence ablations further highlight the efficiency profile: 1. Neighbor Query Routing (Waymo 1/5 subset): Continuous k-NN queries consume 44.5 ms (74.61 L1 mAPH); PTv3 serialization requires 11.2 ms (72.52 L1 mAPH); while LHA's transient grid routing slashes latency to 2.7 ms while achieving the highest accuracy of 75.95 L1 mAPH. 2. Scan Trajectory Comparison: Complex curves such as Hilbert (71.26 NDS, 5.5 ms) and Z-order (71.23 NDS, 5.6 ms) incur heavy sorting overhead; the simple Axis Sort requires only 0.1 ms and achieves an identical 71.23 NDS. Even a completely Random scan achieves 71.08 NDS, underscoring that LHA's local anchoring makes global sequence modeling resilient to scan order.

Key Findings

• Synergy between LHA and BDM: Standalone Mamba layers perform poorly on unstructured 3D points (69.58 NDS), but coupling them with LHA provides a +2.24 NDS surge, demonstrating that 1D sequence models require continuous geometric anchors to prevent topological drift.
• Robustness on Sparse and Small Objects: When point cloud density is uniformly reduced to 1/16, PointLAM retains 21.86 L2 mAPH (vs. 20.60 for voxel-based LION). Under extreme instance sparsity (5-10 points per object), PointLAM maintains 19.76 L2 mAPH (vs. 18.39 for LION). On small targets with volume \(\le 1.0\text{m}^3\), PointLAM outperforms the k-NN baseline by +1.24 L1 mAPH, proving that bounding receptive fields via transient grids successfully eliminates distant noise.

Highlights & Insights

• Laplacian high-pass filtering intuition: Traditional point pooling acts as a spatial low-pass filter, retaining only dominant background energy and smoothing away subtle foreground contours. Formulating feature deviations as a discrete Laplacian operator introduces an explicit high-pass physical prior that yields dense, contrast-maximizing gradients during training.
• Transient grid routing decoupled from quantization: Instead of quantizing point clouds into voxel embeddings, PointLAM uses the grid strictly as an \(O(1)\) spatial routing index while updating point representations continuously, avoiding both k-NN computational overhead and voxel quantization errors.
• Axis sorting suffices for 3D state space models: PointLAM proves that complex space-filling curves are unnecessary for 3D Mamba architectures. Interleaving local geometric anchors with 0.1 ms coordinate axis sorting achieves equal or superior global contextual perception.

Limitations & Future Work

• Information collapse during BEV projection: While point features are preserved throughout the 3D backbone, the architecture flattens multi-stage features onto a 2D BEV grid to interface with conventional detection heads, compressing fine-grained elevation geometry; exploring grid-free detection heads represents a natural next step.
• Fixed grid resolution across varying ranges: The transient grid employs a uniform kernel size across both dense foreground points and ultra-sparse distant regions; introducing distance-adaptive kernel strides could further improve long-range (>150m) LiDAR perception.

Related Work & Insights

• vs 3DSSD / PointNet++: Early point-based detectors suffered from \(O(N^2)\) FPS overhead and memory explosions during continuous k-NN queries. PointLAM's LPS and transient grid routing replace these with sorting and table lookups, accelerating inference to voxel-level speeds while outperforming 3DSSD by +16.6 NDS on nuScenes.
• vs DSVT / Voxel Mamba: Voxel-based Transformers and Mamba detectors lose fine geometric structures during voxelization and typically require 150G-250G FLOPs. PointLAM matches their detection accuracy with only 90.7G FLOPs and 8.6M parameters, providing a compelling option for edge deployment.
• vs PTv3: Point Transformer V3 relies on patch serialization and Hilbert curves, requiring 11.2 ms for spatial ordering. PointLAM demonstrates that with local Hadamard gating, simple 0.1 ms axis sorting provides equivalent context, making it well-suited for autonomous driving systems requiring tight latency budgets.

Rating

• Novelty: ⭐⭐⭐⭐⭐ Elegant integration of discrete Laplacian high-pass filtering and transient grid routing into point-based 3D state space modeling.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluations across nuScenes and Waymo, accompanied by extensive sparsity, object scale, and component ablations.
• Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical derivations, coherent architectural narratives, and strong alignment between diagrams and text.
• Value: ⭐⭐⭐⭐⭐ Effectively breaks the computational efficiency barrier of point-based 3D detectors, establishing a practical blueprint for high-fidelity LiDAR perception.

Related Papers

• [ECCV 2026] PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detection
• [ECCV 2026] Event-LiDAR: 3D Eventification for Efficient Point Cloud Processing
• [ECCV 2026] Real-Time Source-Free Object Detection
• [ECCV 2026] BEVOpen3D: Towards Open-World 3D Object Detection in Bird's-Eye-View
• [CVPR 2025] Trajectory Mamba: Efficient Attention-Mamba Forecasting Model Based on Selective SSM