SpikCommander: A High-Performance Spiking Transformer with Multi-View Learning for Efficient Speech Command Recognition

Conference: AAAI 2026
arXiv: 2511.07883
Code: https://github.com/JackieWang9811/SCommander
Area: Spiking Neural Networks / Speech Recognition
Keywords: Spiking Neural Networks, Speech Command Recognition, Spiking Transformer, Multi-View Learning, Energy Efficiency

TL;DR

The authors propose SpikCommander, a fully spike-driven Transformer architecture. By jointly enhancing temporal and channel feature modeling via Multi-View Spike Temporal-Aware Self-Attention (MSTASA) and Spiking Context Refinement MLP (SCR-MLP), SpikCommander outperforms state-of-the-art SNN methods on SHD, SSC, and GSC benchmarks with fewer parameters.

Background & Motivation

1. Background: Spiking Neural Networks (SNNs) exhibit significant energy-efficiency advantages due to their event-driven nature, making them highly suitable for Speech Command Recognition (SCR) tasks. Spiking Transformers such as Spikformer and SDT have recently made progress in computer vision tasks.
2. Limitations of Prior Work: (a) Existing SNN speech models struggle to capture rich temporal dependencies and contextual information, limited by the sparsity of binary spike representations; (b) most existing spiking self-attention mechanisms consume high computational costs due to their global attention complexity (\(O(N^2)\)); (c) traditional channel MLPs lack context refinement capabilities.
3. Key Challenge: The binary sparsity of spikes limits effective feature extraction, and traditional continuous-valued attention operations perform poorly in the spike domain.
4. Goal: Design an efficient and highly expressive fully spike-driven Transformer architecture tailored for speech command recognition.
5. Key Insight: Leverage a multi-view learning framework to simultaneously capture complementary temporal information from three paths: local (sliding window), global (long-range), and convolutional (shift-invariant).
6. Core Idea: Incorporate three-branch complementary temporal-aware attention and selective context refinement MLP to achieve rich temporal modeling under fully spiking constraints.

Method

Overall Architecture

The input speech signal is converted into spike representations via a Spike Embedding Extractor (SEE). The backbone consists of alternating stacked Transformer blocks of MSTASA and SCR-MLP. The classification head outputs predictions via soft-max after summing over all time steps. Training is conducted end-to-end using BPTT and surrogate gradients.

Key Designs

1. Spike Temporal-Aware Self-Attention (STASA) and its Multi-View Extension (MSTASA)

   • Function: Capture complementary temporal dependencies in the spike domain with linear complexity \(O(ND)\).
   • Mechanism: STASA applies temporal masks to spiking queries \(Q\) and keys \(K\), then aggregates them along the temporal dimension: \(\hat{Q}_S = \sum_{t=1}^T Q'_S[:,t,:]\). The attention weights are computed as \(S_{attn} = \beta(\hat{Q}_S + \hat{K}_S)\), which are passed through spiking neurons and then element-wise multiplied with values \(V_S\) via broadcasting. MSTASA incorporates three branches: (a) Sliding Window STASA (SWA-STASA)—restricting attention to a \(2w+1\) window to model local dependencies; (b) Long-Range STASA (LRA-STASA)—using full-sequence attention to model global dependencies; (c) V-branch—injecting shift-invariant positional patterns into the value representation via depthwise convolution (kernel=9×1) followed by pointwise convolution. The two STASA branches are fused through dual-attention projection before merging with the V-branch.
   • Design Motivation: The \(QK^T\) matrix multiplication of typical Spiking Self-Attention (SSA) incurs an \(O(N^2)\) complexity in the spike domain. STASA reduces this to \(O(ND)\) via temporal aggregation. Multi-view learning captures complementary information, spanning local details, global context, and shift-invariant patterns.

2. Spiking Context Refinement MLP (SCR-MLP)

   • Function: Enhance channel mixing and temporal context modeling.
   • Mechanism: A three-stage process: (i) Forward projection: expanding to dimension \(\alpha D\) (\(\alpha=4\)) via PCBlock + LinBlock; (ii) Selective context refinement: splitting the dimensions in half along the channel dimension, where one half captures local temporal context through a depthwise convolution with kernel=31, while the other half passes through directly, followed by concatenation; (iii) Backward projection: compressing back to dimension \(D\). All operations maintain fully spike-driven propagation via {Conv-BN-SN} blocks.
   • Design Motivation: Traditional channel MLPs only perform channel mixing and lack temporal context. The selective splitting design—applying depthwise convolution to only half of the channels—injects context while saving computation.

3. Spike Embedding Extractor (SEE)

   • Function: Map acoustic inputs to structured spike representations.
   • Mechanism: Apply depthwise separable convolutions (pointwise 1D + depthwise 1D with kernel=7) to extract local time-frequency features, followed by a linear transformation with residual connections to elevate channel projections, with all outputs converted to spikes through SN.

Loss & Training

Standard cross-entropy loss is employed, and the network is trained end-to-end with BPTT and surrogate gradients (ArcTan). The number of timesteps is set to T=100.

Key Experimental Results

Main Results

Dataset	Method	Parameters (M)	Timesteps	Accuracy (%)
SHD	SpikeSCR (1L)	0.26	100	95.60
SHD	Pfa-SNN	0.20	100	96.26
SHD	SpikCommander (1L)	0.19	100	96.41
SSC	SpikeSCR (2L)	3.30	100	82.79
SSC	SpikCommander (2L)	2.13	100	83.49
GSC	Spiking LMUFormer	1.69	-	96.12
GSC	d-cAdLIF (2L)	0.61	100	95.69
GSC	SpikCommander (2L)	2.13	100	96.92

Ablation Study

Configuration	SHD Acc	Description
LRA-STASA Only	Lower	Lacks local information
SWA-STASA Only	Lower	Lacks global information
MSTASA (Three Branches)	Optimal	Complementary gains
Standard MLP	Lower	No context refinement
SCR-MLP	Higher	Selective refinement is effective

Key Findings

• SpikCommander outperforms all SOTA SNN methods on three datasets with fewer parameters (e.g., SHD: 0.19M vs SpikeSCR 0.26M).
• On GSC, it even surpasses the ANN-based model LMUFormer (96.92% vs 96.53%), which is highly rare in the SNN domain.
• Each of the three branches in the multi-view design contributes to performance; removing any single branch leads to performance degradation.
• The selective refinement strategy of SCR-MLP is more efficient and effective than full-channel depthwise convolutions.

Highlights & Insights

• Spiking Attention with Linear Complexity: By aggregating and summing over the temporal dimension, complexity is reduced from \(O(N^2)\) to \(O(ND)\), making Spiking Transformers with long sequences feasible.
• SNN Model Outperforming ANN: The results on GSC demonstrate that a meticulously designed SNN architecture can close or even reverse the performance gap with ANNs, which holds significant value for Neuromorphic Computing.
• Refinement Strategy via Selective Channel Splitting: Applying depthwise convolution to only half of the channels cleverly balances computational efficiency with context modeling.

Limitations & Future Work

• The model was only validated on speech command recognition tasks and was not extended to more complex speech tasks (e.g., ASR).
• Energy efficiency was not verified on real neuromorphic hardware (e.g., Loihi, Tianjic).
• The number of timesteps is fixed to 100; adaptive timesteps could be more energy-efficient.
• Implementation details regarding dynamically relating the sliding window size to the input length were not fully discussed.

Related Work & Insights

• vs Spikformer/SDT: While these models use \(O(N^2)\) SSA/SDSA, the linear STASA in SpikCommander is more efficient.
• vs SpikeSCR: Although SpikeSCR combines attention with convolution, it lacks a multi-view design and has more parameters.
• vs DCLS-Delays: Unlike delay-learning methods, SpikCommander replaces explicit delay modeling with an attention mechanism.

Rating

• Novelty: ⭐⭐⭐⭐ The design of Multi-View Spiking Attention + SCR-MLP is novel.
• Experimental Thoroughness: ⭐⭐⭐⭐ Complete evaluations on three benchmark datasets along with extensive ablation studies.
• Writing Quality: ⭐⭐⭐⭐ The architectural diagrams are clear, and comparisons are comprehensive.
• Value: ⭐⭐⭐⭐ Significantly advances the domain of SNN-based speech processing.

Related Papers

• [ACL 2025] Sparsify: Learning Sparsity for Effective and Efficient Music Performance Question Answering
• [CVPR 2026] OmniRet: Efficient and High-Fidelity Omni Modality Retrieval
• [ICLR 2026] Latent Speech-Text Transformer
• [AAAI 2026] Do LLMs Feel? Teaching Emotion Recognition with Prompts, Retrieval, and Curriculum Learning
• [AAAI 2026] HQ-SVC: Towards High-Quality Zero-Shot Singing Voice Conversion in Low-Resource Scenarios