High-Throughput Event-Based Feature Detection and Tracking on an Embedded CPU¶

Conference: ECCV 2026
Paper: ECCV Official
Code: https://0thane.github.io/SPEEDTrack/
Area: Others
Keywords: Event Camera, Feature Detection, Feature Tracking, Embedded Systems, Task-Graph Scheduling

TL;DR¶

Addressing the severe tension between the massive event rates of neuromorphic cameras and the tight compute limits of edge devices, SPEEDTrack presents a learning-free, GPU-free pipeline that leverages motion-defined concurrent slices and task-graph scheduling to surpass benchmark event throughput on an embedded CPU.

Background & Motivation¶

Event sensors offer microsecond-level temporal resolution, high dynamic range, and minimal power consumption, establishing them as an attractive vision modality for agile edge platforms such as autonomous micro-aerial vehicles and augmented reality headgear. In practical deployments, event-based perception algorithms must maintain sufficiently high throughput to keep pace with continuous, asynchronous event streams that often output millions of events per second. Most state-of-the-art feature detection and tracking methods rely on deep neural networks with GPU acceleration (such as attention-driven or recurrent models), making them too power-hungry for constrained edge devices. Conversely, traditional non-learning methods typically depend on fixed time windows or fixed event counts for frame accumulation, which fail to adapt to variable camera dynamics—causing severe motion blur during high-speed maneuvers and data starvation during slow periods.

A critical systems bottleneck stems from the fundamental misalignment between algorithmic structures and multi-core CPU architectures. Conventional event pipelines bind discrete processing stages to dedicated threads in a serial pipeline, which prevents concurrent processing of multiple input batches and incurs heavy inter-thread synchronization penalties that leave CPU cores underutilized. Furthermore, classical optimization-based trackers (such as HASTE or RATE) carry substantial per-feature computational footprints, causing throughput to fall well below the average sensor event rate. Once the event rate exceeds system throughput, the pipeline is forced to drop or sub-sample events on the fly, which breaks spatio-temporal continuity and discards crucial high-speed dynamics.

Rather than stacking heavier networks or relying on global serialized updates, this work explores the deep co-optimization of low-level hardware characteristics and event spatio-temporal sparsity. Core idea: build a Motion-Defined Concurrent Slices (MDCS) architecture that adaptively partitions the event stream into self-contained temporal slices based on active pixel event density, executes lock-free slice-local corner detection and linear tracklet fitting across multi-core CPUs via task-graph dynamic scheduling, and stitches global trajectories through a lightweight Kalman filter.

Method¶

Overall Architecture¶

SPEEDTrack maps a continuous asynchronous event stream $\mathcal{E}_T = \{e_i\}_{i=1}^{N_T}$ into long-lived feature tracks $\mathcal{T}_j$ through four tightly coupled, lock-free stages: an Activity Slicer that segments incoming events into motion-defined slices based on active pixel density; an Event-Only Corner Detector that computes slice-local ordinal surfaces and filters redundant calculations via an ignore mask; a Feature Tracklet Detector that fits local linear spatio-temporal segments within each slice; and an Asynchronous Feature Tracker orchestrated by a dynamic task-graph scheduler that associates tracklets using lightweight constant-velocity Kalman filters.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Asynchronous Event Stream<br/>(t, x, y, p)"] --> B["Activity Slicing<br/>dynamic partitioning by active pixel density"]
    B --> C["Slice-Local Corner Detection<br/>ordinal surfaces + ignore mask"]
    C --> D["Feature Tracklet Detection<br/>closed-form spatio-temporal linear fitting"]
    D --> E["Task-Graph Scheduling & Tracking<br/>dynamic dispatch + KD-tree Kalman filter association"]
    E --> F["Global Stable Feature Tracks<br/>real-time low-latency position & velocity"]

Key Designs¶

1. Activity Slicing: Dynamic Motion-Adaptive Slice Partitioning Fixed-time slicing causes extreme event accumulation and motion blur during rapid motion while creating over-sparse frames in static intervals. Fixed-size batching similarly fails when spatial texture density varies. To overcome this, the Activity Slicer monitors the activity score $\alpha$, defined as the mean number of events per active pixel within the current slice $\mathcal{S}_j$: $$\alpha = \frac{|\mathcal{S}_j|}{A_{\text{active}}}, \quad A_{\text{active}} = |\{(x,y) \mid \mathbf{A}(x,y)=1\}|$$ where $\mathbf{A}(x,y)$ flags whether pixel $(x, y)$ has fired within $\mathcal{S}_j$. The system evaluates $\alpha$ periodically at intervals $\delta\alpha$ aligned with sensor physics (e.g., 100 times the sensor refractory period). Once $\alpha$ exceeds a predefined threshold $\alpha_{\text{th}}$ (set to 3), the slice is finalized and dispatched. This guarantees consistent structural appearance across both fast and slow dynamics, throttling computational load in quiet periods while producing high-cadence slices during rapid motion.

2. Slice-Local Corner Detection: Decoupling Global State with an Ignore Mask Standard event corner detectors like eHarris maintain a persistent global surface across all events, which incurs significant synchronization locks and cache thrashing across multiple CPU cores. SPEEDTrack confines corner extraction entirely within each independent slice by constructing a slice-local ordinal surface $\mathbf{\Sigma}_o(x, y)$, storing only the relative index of the latest event at that pixel. A binary patch is extracted using the $\Theta$-th largest index within an $L \times L$ neighborhood and convolved with Sobel kernels to evaluate the Harris matrix $\mathbf{M}$ and score $H(e'_c)$. To further eliminate redundant floating-point operations, an ignore mask $\mathbf{V}(x, y)$ records each pixel's status: negative Harris scores as flat background ($0$), intermediate values as edges ($1$), and strong responses as corners ($2$). Subsequent events only evaluate the full Harris response if $\mathbf{V}(x, y) = 0$, skipping costly convolutions on established edges and corners while suppressing spatial feature clustering.

3. Feature Tracklet Detection: Local Linear Fitting for Slice-Level Concurrency Traditional tracking relies on continuous temporal propagation across frames, creating a strict serial dependency that prevents multi-core parallelization. Because the activity slicer restricts the temporal extent of each slice proportionally to motion magnitude, corner displacement within $\mathcal{S}_j$ can be accurately approximated by a local constant-velocity linear trajectory: $x(t) = v_x t + b_x$ and $y(t) = v_y t + b_y$. After applying non-maxima suppression (NMS) within radius $r_{\text{suppress}}$ to select the top $K$ corners with highest local density, the system solves for $(v_x, v_y, b_x, b_y)$ via closed-form linear least squares over neighboring events. Evaluating endpoints at $t_{\min}$ and $t_{\max}$ yields self-contained tracklets $f_k = \langle \boldsymbol{\tau}_{\text{start}}^{(k)}, \boldsymbol{\tau}_{\text{end}}^{(k)} \rangle$, completely isolating intra-slice computation from cross-slice dependencies.

4. Task-Graph Scheduling & Tracking: Concurrency Runtime with Lightweight Kalman Filters To eliminate idle wait states where CPU cores stall on fixed thread boundaries, the entire processing flow is structured as an asynchronous Directed Acyclic Graph (DAG) executed by a work-stealing task-graph runtime. Slice-level detection and tracklet extraction are registered as unconstrained concurrent tasks that execute on any ready core. At the track assembly stage, an asynchronous tracker maintains independent constant-velocity Kalman filters tracking state $[x, y, v_x, v_y]^\top$. When new tracklets arrive, feature positions are propagated to the earliest tracklet timestamp, and candidates are collected via KD-tree queries within radius $r_{\text{match}}$ for greedy association and filter update. An active spatial grid of size $g \times g$ prevents duplicate track spawning, and unassociated stale tracks are discarded, achieving multi-second global trajectory continuity with minimal CPU overhead.

Key Experimental Results¶

Main Results¶

On the high-dynamic Event-aided Direct Sparse Odometry (EDS) benchmark and the classic Event Camera (EC) dataset, trackers were benchmarked using the Stable Feature Age (SFA) and Expected Feature Age (EFA) metrics under standardized initial corner seeding without in-flight re-initialization.

Method	Modality	EDS: SFA ↑	EDS: EFA ↑	EC: SFA ↑	EC: EFA ↑
EKLT	Events+Frames	0.230	0.150	0.795	0.760
DeepEvT	Events+Frames	0.480	0.400	0.815	0.805
EM-ICP	Events Only	0.115	0.090	0.320	0.310
HASTE	Events Only	0.070	0.050	0.425	0.410
AEB-Tracker	Events Only	0.300	0.290	0.540	0.460
DeepEvT*	Events Only	0.515	0.430	0.785	0.770
ETAP	Events Only (GPU)	0.742	0.613	0.870	0.851
SPEEDTrack (Ours)	Events Only (CPU)	0.558	0.421	0.822	0.713

In relative pose estimation (evaluating the Area Under the Curve [AUC] of relative rotation errors across two-second baselines), SPEEDTrack achieved 14.4% (5°), 20.8% (10°), and 27.2% (20°) on EC, and 10.9% (5°), 14.0% (10°), and 17.5% (20°) on EDS. It achieved the second-highest overall performance among all event-only trackers, surpassed only by SuperEvent which uses a large learning-based backbone, while significantly outperforming classical methods such as RATE, LLAK, and EventPoint.

Ablation Study¶

Ablation experiments were performed directly on an embedded NVIDIA Jetson Orin NX development kit (using only its 8-core Arm Cortex A78AE CPU and 16GB RAM, consuming an average of 12W total power) to evaluate real-world embedded latency, event throughput, and tracking longevity.

Config	Latency (µs)	Throughput (Mev/s)	EDS SFA	EDS EFA	EC SFA	EC EFA	Note
Fixed time slicing (15ms)	71.2	4.01	0.401	0.298	0.658	0.541	Degraded consistency under high speed
Nearest tracklet (w/o KF)	45.1	5.02	0.438	0.330	0.829	0.672	Lowers latency but loses dynamic smoothing
Recompute H (w/o ignore mask)	83.2	3.52	0.491	0.375	0.774	0.662	Redundant convolutions throttle pipeline
Thread-bound concurrency	96.4	2.97	0.558	0.421	0.822	0.713	Identical accuracy but severe core stalling
Full model (SPEEDTrack)	49.7	4.92	0.558	0.421	0.822	0.713	Optimal throughput, latency, and accuracy

Key Findings¶

Task-graph concurrency (MDCS) is the primary driver of high throughput: replacing dynamic task-graph dispatch with traditional thread-bound pipelines dropped throughput from 4.92 Mev/s to 2.97 Mev/s (a ~40% penalty) and nearly doubled latency from 49.7 µs to 96.4 µs due to pipeline imbalances and thread lock contention.
The ignore mask is an essential algorithmic acceleration trick: bypassing redundant Harris score calculations on existing edge and corner pixels increased throughput by ~40% (3.52 to 4.92 Mev/s) and reduced detection latency from 83.2 µs to 49.7 µs.
Activity slicing provides vital structural stability: switching to a standard 15 ms fixed-time window reduced SFA on EDS from 0.558 to 0.401, demonstrating that normalizing event density per active pixel is crucial for stable downstream geometric feature tracking.

Highlights & Insights¶

Hardware-Algorithm Co-Design for Embedded Vision: Demonstrates that high-accuracy, long-lived feature tracking does not inherently require heavy deep neural networks; aligning algorithmic batch independence with multi-core CPU scheduling achieves competitive accuracy at a fraction of the power budget.
Motion-Defined Active Pixel Partitioning: Normalizing event accumulation by active pixel count rather than wall-clock time or raw event volume creates a remarkably robust, parameter-lean mechanism for generating consistent spatio-temporal representations across variable motion speeds.
Self-Contained Slice Tracklets: Decoupling the tracking pipeline into slice-local linear least squares fitting followed by global Kalman association removes inter-slice lock dependencies, allowing CPU cores to operate at peak occupancy without synchronization bottlenecks.

Limitations & Future Work¶

Vulnerability to Bandwidth Saturation and Lost Packets: The method relies on spatio-temporal smoothness across the event stream. In ultra-aggressive high-speed scenarios, sensor USB bandwidth limits may trigger hardware-level packet drops, corrupting the local linear motion assumption.
Artifacts from AC Lighting Modulation: Because event pixels trigger on any temporal brightness variation, high-frequency flickering from AC artificial illumination creates phantom events and spurious feature tracks indistinguishable from true camera egomotion.
Absence of Long-Term Track Re-identification: The current association logic uses greedy matching; once a feature track is broken by severe occlusion or rapid out-of-frame motion, it cannot be recovered without integrating a keyframe map or SLAM loop-closure mechanism.

vs ETAP / DeepEvT / SuperEvent: Modern learning-based methods achieve top tracking accuracy using Transformers or recurrent memory backbones, but demand dedicated high-wattage GPUs; SPEEDTrack runs entirely on a 12W embedded CPU without GPU acceleration, offering a deployable solution for power-critical robotics.
vs eHarris / eFAST / FA-Harris: Early event corner detectors either suffered from heavy computational burdens (eHarris) or poor geometric repeatability over time (eFAST); SPEEDTrack adapts eHarris within local ordinal surfaces and adds an ignore mask, preserving geometric repeatability while improving throughput by an order of magnitude.
vs HASTE / RATE: Classical asynchronous trackers perform iterative non-linear optimization for each tracked feature, causing computational overhead to scale steeply with feature count; SPEEDTrack decouples feature extraction into closed-form linear fits per slice and lightweight Kalman filtering, sustaining multi-million event throughput under dense feature loads.

Rating¶

Novelty: ⭐⭐⭐⭐☆ [Combines motion-defined slicing with lock-free task-graph scheduling, introducing a compelling systems-oriented paradigm for event vision]
Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive evaluations spanning detection stability, track longevity, relative pose estimation, embedded ablations, and real-time camera demonstrations]
Writing Quality: ⭐⭐⭐⭐⭐ [Clear structural narrative, rigorous hardware bottleneck analysis, and well-motivated design choices]
Value: ⭐⭐⭐⭐⭐ [Sets a strong benchmark for real-time edge deployment of event cameras on resource-constrained platforms like UAVs and AR headsets]