Skip to content

Hierarchical Direction Perception via Atomic Dot-Product Operators for Rotation-Invariant Point Clouds Learning

Conference: AAAI2026
arXiv: 2511.08240
Code: DiPVNet
Area: 3D Vision
Keywords: Point Cloud, Rotation Invariance, dot-product operator, spherical Fourier transform, Equivariance

TL;DR

This paper proposes DiPVNet, which builds a local L2DP operator and a global DASFT module based on the dual properties of the atomic dot-product operator (directional selectivity + rotation invariance) to achieve hierarchical direction-aware rotation-invariant point cloud learning.

Background & Motivation

  • 3D point cloud processing is widely applied in scenarios such as autonomous driving and embodied AI, but arbitrary rotations scramble the spatial distribution, leading to feature inconsistency.
  • The key challenge is that rotational perturbations destroy the inherent multi-scale directional features of point clouds (local: edge orientations, normals; global: principal axis directions, structural symmetry).
  • Explicit methods (such as ODF, spatial direction partitioning) rely on fixed partitioning and cannot adapt to non-uniform distributions.
  • Although implicit methods (such as VNN) maintain equivariance/invariance, they do not utilize directional information sufficiently; VNN only uses a single global directional vector for gating, failing to capture complex hierarchical directional structures.

Core Problem

How to adaptively perceive the multi-scale directional features of point clouds while maintaining rotational symmetry, thereby enhancing rotationally robust discriminative representations?

Method

Overall Architecture

DiPVNet consists of three core components: (1) the L2DP operator extracts local directional features -> (2) the DASFT module constructs a global directional response spectrum -> (3) Cross-Attention fuses local and global features. Meanwhile, the equivariant branch of the VNN Block is retained.

Key Designs

1. Atomic Dot-Product Operator

  • Reveals the dual properties of the dot-product: directional selectivity (acting as a directional filter) + rotation invariance.
  • Encapsulated as a differentiable atomic operator: \(\Phi(\mathbf{a}, \{\mathbf{b}_i\}; \Theta) = \text{FFN}(\{\langle \mathbf{a} \cdot \mathbf{b}_i \rangle\}_{i=1}^K; \Theta)\)

2. Learnable Local Dot-Product (L2DP) Operator

  • Performs a dot-product with relative position encoding between the central point \(\mathbf{v}_j\) and its K-nearest neighbors \(\mathbf{g}_j^{(k)}\): \(I_j^{(\mathcal{G}, \text{rel})} = \{\langle \mathbf{v}_j, \mathbf{g}_{jk} - \mathbf{v}_j \rangle \mid k=1,\dots,K\}\)
  • Where \(\langle \mathbf{v}_j, \mathbf{g}_{jk} \rangle\) encodes directional information, and \(\langle \mathbf{v}_j, \mathbf{v}_j \rangle\) injects position encoding.
  • Two aggregation strategies: DLP (Direct Linear Projection, preserving all neighbor interactions) and SAP (Statistics-Aware Projection, computing max/var/avg before projection, suitable for large-scale neighborhoods).

3. Direction-Aware Spherical Fourier Transform (DASFT)

  • Treats the point cloud as a discrete signal in 3D space, performing a dot-product with a spherical sampling vector \(\Omega = r \cdot \omega\).
  • Proves that this operation is equivalent to a direction-aware spherical Fourier transform: \(\mathcal{F}(\mathcal{P}, \{\Omega\}) = \sum_{j=1}^n \exp(-ir\omega^\top v_j)\)
  • Constructs the energy spectrum \(E(\mathcal{P}, \{\Omega\}) = |\mathcal{F}|^2\), obtaining rotation-invariant descriptors through spherical averaging.
  • Uniformly samples \(N_{\text{dir}} = 36\) directions; the results remain stable when \(\geq 36\).

4. Feature Fusion

  • Cross-Attention: L2DP features serve as Query, and DASFT features serve as Key/Value.
  • The equivariant features of the VNN Block are projected onto a learned canonical basis to generate scalar tokens, which are then concatenated with the fused invariant features.

Key Experimental Results

Method ModelNet40 (z/SO(3)) ScanObjectNN (z/SO(3)) ShapeNetPart (z/SO(3))
VN-DGCNN 89.5 83.5 81.4
LGR-Net 90.9 81.2 80.0
TetraSphere 90.5 87.3 82.3
PaRot 91.0 - -
DiPVNet 91.4 87.5 82.5
  • Ablation: DASFT only (Model A) = 89.5 (no improvement); L2DP-DLP only (Model C) = 90.6; full model = 91.4.
  • Gated fusion (90.9) < Cross-Attention fusion (91.4), showing that dynamic feature calibration outperforms static weight assignment.

Highlights & Insights

  • Starting from the fundamental mathematical properties of the dot-product, it uniquely models direction awareness and rotation invariance with a novel perspective.
  • L2DP + DASFT capture local and global directional features respectively, showing strong complementarity (DASFT alone brings no improvement, indicating that local features are crucial).
  • The theoretical connection between DASFT and generalized harmonic analysis is rigorous, going beyond mere empirical design.
  • Consistent performance is maintained under noise and large-angle rotations (\(z/z = z/SO(3) = SO(3)/SO(3)\)).

Limitations & Future Work

  • Based on the VN-DGCNN baseline, the model capacity is limited and has not been verified on large-scale pre-trained point cloud models.
  • The dataset sizes for classification and segmentation tasks are relatively small (ModelNet40 has only 12k samples).
  • Experiments on large outdoor scenes (such as full-scene segmentation in S3DIS or ScanNet) are lacking.
  • DASFT involves spherical sampling and Fourier transform; the computational overhead has not been analyzed in detail.
  • Only coordinate information is processed; multi-modal point cloud features such as normals and colors are not involved.
  • vs VNN: VNN utilizes a single global direction vector for gating, whereas DiPVNet achieves adaptive multi-scale direction awareness via the dot-product operator.
  • vs SGMNet: SGMNet also uses dot-product, but its sorting mechanism disrupts spatial relationships and it lacks a directional aggregation strategy.
  • vs TFN: TFN uses spherical harmonics to construct group equivariant convolution kernels, resulting in large computational overhead; DiPVNet is more lightweight.
  • vs PaRot/TetraSphere: DiPVNet outperforms them on both ModelNet40 and ScanObjectNN.

Inspirations & Connections

  • The "atomic operator" concept can be extended to other geometric operations (e.g., cross-product for normal perception).
  • The spherical frequency domain analysis of DASFT can be applied to point cloud generation or shape retrieval.
  • The adaptive neighborhood directional learning scheme of L2DP can be integrated into local feature extraction for 3D object detection.

Rating

  • Novelty: ⭐⭐⭐⭐ (Unique theoretical perspective starting from the fundamental properties of the dot-product)
  • Experimental Thoroughness: ⭐⭐⭐⭐ (Comprehensive ablation studies, but lacks large-scene experiments)
  • Writing Quality: ⭐⭐⭐⭐ (Rigorous mathematical derivation, clear structure)
  • Value: ⭐⭐⭐⭐ (New SOTA for rotation-invariant point cloud learning)