Following the Flow: Advection-Consistent Modeling for Event-based Small Object Detection¶

Conference: ECCV2026
arXiv: 2606.22378
Code: https://github.com/fulongcai/PACT
Area: Object Detection / Event Camera
Keywords: Event Camera, Small Object Detection, Advection Consistency, Motion Continuity, Physics-Guided

TL;DR¶

PACT explicitly models the evolution of event stream features as a physical transport process constrained by the advection equation. Through trajectory-level consistency, weak target responses accumulate along the velocity field while noise naturally attenuates, improving the IoU from 55.18% to 75.90% on the EV-UAV dataset.

Background & Motivation¶

Event cameras asynchronously capture luminance changes with microsecond-level resolution, offering unparalleled advantages over traditional frame-based cameras in high-speed dynamic scenes. However, event streams are inherently sparse, irregular, and highly sensitive to noise. This challenge becomes particularly severe when detecting small objects, where the spatial support is extremely small and the triggered event responses are scattered and discontinuous, making them easily overwhelmed by complex backgrounds and intense noise. Existing methods generally fall into two categories: frame-based pipelines aggregate events into static representations before detection, but this aggregation process discards continuous temporal evolution, removing crucial cues for distinguishing weak signals from noise; spiking neural networks (SNNs) implicitly integrate temporal information via neuronal dynamics, but weak and intermittent responses often fail to trigger stable spiked activities, making them equally difficult to separate from noise.

A key but often overlooked fact is that although event responses of small objects are sparse and discontinuous, their underlying physical motion is continuous. Within a sufficiently short time window, the event responses triggered by the same moving object can be closely approximated by a local velocity field describing its spatial translation, whereas background noise lacks such directional consistency. Therefore, motion continuity serves as the final (and most fundamental) cue to differentiate meaningful signals from random triggers under intense noise. However, existing methods model temporal continuity in an implicit manner, lacking explicit propagation rules to link fragmented weak responses.

Our Key Insight is to explicitly characterize the spatio-temporal evolution of event features using the physical advection equation (advection equation). Core Idea: By treating the feature representation of the event stream as a continuous field transported by a local velocity field, the advection consistency constraint allows weak responses conforming to motion continuity to accumulate along the velocity field into coherent trajectories, while noise violating the advection constraint naturally misaligns and attenuates during transport, thereby restoring the spat-temporal continuity of small objects amidst heavy background activity.

Method¶

Overall Architecture¶

PACT adopts an encoder-decoder architecture operating on sparse voxel feature tensors. The input consists of sparse voxels (including coordinate indices, normalized attributes, and polarities). The encoder stacks four Trajectory-Guided Feature Extraction (T-FE) modules to extract motion-aware features hierarchically, forming a pyramid {F_enc^i}. At each stage, the Advection-based Trajectory Consistency (ATC) module estimates the local velocity field and outputs advection-consistency gating weights, trying to ensure that only features conforming to motion continuity are reinforced and propagated. The decoder employs four Advection-consistent Feature Reconstruction (A-FR) modules to perform multi-hypothesis propagation along the velocity field and fuse them with encoder skip connections. This connects fragmented responses into a continuous trajectory feature pyramid {F_traj^i}, which finally outputs segmentation masks and localization results.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Sparse Voxel Input<br/>Coordinates + Normalized Attributes + Polarity"] --> B["T-FE ×4<br/>Trajectory-Guided Feature Extraction"]
    B --> C["ATC<br/>Velocity Field Estimation + Advection Consistency Score"]
    C --> D["A-FR ×4<br/>Multi-Hypothesis Advection Propagation"]
    E["Encoder Skip Connections"] --> D
    D --> F["Trajectory Feature Pyramid<br/>F_traj[1..4]"]
    F --> G["Detection Output<br/>Segmentation Mask + Localization"]

Key Designs¶

1. Advection Consistency Representation: Quantifying Motion Continuity with Transport Residuals

Regarding the sparse voxel features as a spat-temporal continuous field $u(\mathbf{x},t)$, its short-term evolution is approximately described by the advection equation: $\partial u / \partial t + \mathbf{v} \cdot \nabla u \approx 0$, where $\mathbf{v}$ is the local velocity field. Along the feature trajectory $\mathbf{x}(t)$ satisfying $d\mathbf{x}/dt = \mathbf{v}$, the feature is approximately conserved: $d u(\mathbf{x}(t),t)/dt \approx 0$. Consequently, the transport operator is defined as $\mathcal{T}_{\mathbf{v}}(u)(\mathbf{x},t) = u(\mathbf{x} - \tau\mathbf{v}, t - \tau)$ and the advection residual as $\mathcal{R}_{adv} = \|u - \mathcal{T}_{\mathbf{v}}(u)\|$. Features with small residuals are considered to originate from the same moving target and receive high confidence for subsequent propagation, while features with large residuals are classified as noise and dynamically suppressed. This formulation transforms the vague concept of "conformance to motion continuity" into a computable numerical criterion, serving as the fundamental physical basis of the entire method.

2. T-FE + ATC: Trajectory-Constrained Feature Encoding and Velocity Gating

Each level of the encoder first aggregates spatio-temporal context via multi-branch sparse convolutions (dilation rates of 1, 2, 3, 4) to obtain $\mathrm{F}_{multi}$. For each active voxel, ATC predicts a 2D displacement $\Delta\mathbf{p}_i = v_{max}\tanh(\mathrm{MLP}(\mathrm{F}_i))$ and obtains the transported feature $\mathrm{F}_{adv}$ through Gaussian kernel interpolation. It then computes the trajectory consistency gating weight: $$g = \exp\left(-\frac{\|\mathrm{F}_{adv}-\mathrm{F}\|_1}{\|\mathrm{F}\|_1+\varepsilon}\right) \odot \frac{1}{1+\alpha(|v_x|+|v_y|)}$$ $g$ assigns higher weights to components that are "consistent before and after transport" and have "moderate velocity." The final fused feature is obtained as $\mathrm{T}_{adv} = g\cdot\mathrm{F}_{adv} + (1-g)\cdot\mathrm{F}$. This gating mechanism ensures that only features conforming to motion consistency are preserved and enhanced during the encoding phase, whereas isolated noise (which differs significantly from itself after transport) is automatically down-weighted, preventing noise amplification in subsequent layers from the source.

3. A-FR: Multi-Hypothesis Advection-Guided Trajectory Propagation

Instead of direct upsampling, the decoder performs advection propagation along the velocity field to link fragmented responses. To handle the uncertainty of velocity estimation in sparse regions, $K$ perturbed hypotheses $\mathbf{v}_i^k = \mathbf{v}_i + \delta_k$ are generated for each voxel from $\mathbf{v}_i$. Transport candidates $\mathrm{A}_k$ and confidence scores $\mathrm{G}_k$ are computed for each hypothesis. A magnitude penalty $\mathrm{S}_k$ (channel-average energy) is introduced to prevent abnormally high activation values from dominating the propagation, followed by Softmax aggregation: $\bar{\mathrm{A}} = \sum_k \pi_k \mathrm{A}_k$, where $\pi_k = \mathrm{Softmax}(\eta\mathrm{G}_k - \mathrm{S}_k)$. The aggregated feature is concatenated with the original source feature and aligned through channel attention, then fused with encoder skip connections. Finally, it is refined by multi-dilation sparse convolutions to yield the trajectory feature $\mathrm{F}_{traj}$. This multi-hypothesis design renders the propagation robust to velocity estimation errors—hypotheses with slightly inaccurate velocities contribute less but do not completely lose the trajectory, while completely incorrect hypotheses are automatically excluded due to their low confidence.

Loss & Training¶

The joint training objective is formulated as $\mathcal{L} = (1-\lambda)\mathcal{L}_{seg} + \lambda\mathcal{L}_{vel}$ with $\lambda=0.3$. $\mathcal{L}_{seg}$ is the voxel-level binary cross-entropy (segmentation mask). $\mathcal{L}_{vel}$ is the Smooth-L1 regularization on the predicted velocity. Since dense optical flow annotations are unavailable, the pseudo-labels $\mathbf{v}^*_i = (\mathbf{p}_j - \mathbf{p}_i)/(t' - t)$ are constructed via nearest-neighbor matching of foreground voxels between adjacent frames. The velocity regularization stabilizes the learning of the transport field, while the primary supervision remains driven by the detection loss.

Key Experimental Results¶

Main Results¶

Evaluation on the EV-UAV dataset (with an average target size of only 6.8×5.4 pixels, representing extremely small targets under complex lighting conditions). PACT substantially outperforms previous state-of-the-art methods with 75.90% IoU and 91.84% P_d, reducing the false alarm rate by an order of magnitude.

Method	Type	IoU (%)	ACC (%)	P_d (%)	F_a (×10⁻⁴)	Params
EV-SpSegNet (ICCV'25)	Point Cloud Seg.	55.18	65.02	77.53	1.63	4.0M
COSeg (CVPR'24)	Point Cloud Seg.	51.89	60.93	71.32	9.21	23.4M
KPConv (ICCV'19)	Point Cloud Seg.	48.19	57.28	68.59	16.32	50.1M
Spike-YOLO (ECCV'24)	SNN Detection	43.94	48.26	59.62	55.38	69.0M
RVT (CVPR'23)	Voxel Detection	43.21	51.38	60.35	55.68	9.9M
PACT (Ours)	Advection Prop.	75.90	80.05	91.84	0.76	2.9M

Ablation Study¶

Configuration	IoU (%)	ACC (%)	P_d (%)	Description
Baseline (No transport constraint)	59.90	63.18	78.80	Weak responses lack a propagation mechanism and easily disconnect
+ ATC only	55.11	69.95	83.35	Imposing advection constraints on unstructured features induces false correspondences
+ T-FE only	66.54	69.67	84.48	Suppresses noise but does not link trajectories
+ A-FR only	63.26	65.34	79.97	Conservative propagation with limited gains
+ T-FE + ATC	71.95	73.48	85.61	Closed-loop of extraction + constraint
+ T-FE + A-FR	70.55	76.04	88.76	Unconstrained propagation is not precise enough
Full (All components activated)	75.90	80.05	91.84	Complete closed-loop

Key Findings¶

All Components are Indispensable: T-FE generates motion-consistent features, ATC imposes physical transport constraints, and A-FR propagates along the flow to form stable trajectories. Together, their coupling establishes a complete physical closed-loop of "extraction $\rightarrow$ constraint $\rightarrow$ propagation".
Adding Only ATC Leads to a Surge in False Alarms (rising from 0.76 to 4.76×10⁻⁴) because forcing advection consistency on unstructured features induces false temporal correspondences, which in turn reinforces pseudo-trajectories of noise.
Scenarios with Multiple Targets Having Large Velocity Differences are a Weakness: A single first-order velocity field cannot simultaneously align multiple targets with vastly different displacements within the same window, which narrows the performance margin.
Advection Residual Evolution Analysis is Highly Convincing: During training, the advection residuals of the foreground gradually concentrate near zero, while background residuals maintain a wide distribution, visually validating that the model successfully learns to leverage transport validity to maintain weak responses.

Highlights & Insights¶

The most ingenious design is introducing the physical advection equation into event feature propagation, converting the vague principle of "motion continuity" into computable transport residuals $\mathcal{R}_{adv}$ and gating weights $g$. The physical assumption only requires transport consistency under local velocity fields, which is weaker and more general than global contrast maximization.
The multi-hypothesis propagation mechanism ($K=5$ velocity perturbations + Softmax aggregation) elegantly handles the uncertainty of velocity estimation in sparse regions, avoiding single-path propagation errors.
The velocity pseudo-labels are automatically constructed via nearest-neighbor matching between adjacent frames without requiring manual annotations of dense optical flows, offering strong practicality.
With only 2.9M parameters (smaller than EV-SpSegNet's 4.0M) and an inference time of 58ms/window, the method achieves an optimal balance between accuracy and efficiency.

Limitations & Future Work¶

The current model adopts a local constant velocity approximation, causing accuracy degradation in scenarios with acceleration or large-scale directional changes. Extending it to second-order or piecewise linear motion models is a natural next step.
The velocity pseudo-labels rely on nearest-neighbor matching, which may become inaccurate when targets are highly dense or heavily occluded.
The authors struggle with scenarios where "multiple targets within the same window exhibit excessively large velocity differences" where the performance gain of the proposed method narrows. Future time-varying motion models are expected to mitigate this issue.

vs. Frame-based Methods (SSD/Faster R-CNN/DETR/YOLOv10): A 50ms event aggregation window discards temporal structures, performing far worse than temporal modeling methods when facing weak responses of small objects.
vs. SNN Methods (Spike-YOLO/EMS-YOLO): Implicit temporal integration makes it difficult for weak responses to trigger stable spikes, and the spiking mechanism itself introduces additional computational constraints.
vs. Point Cloud Segmentation Methods (KPConv/RandLA-Net/COSeg/EV-SpSegNet): While preserving sparse geometry, these methods lack explicit propagation rules, leaving many disconnected trajectory points. PACT increases detection continuity from approximately 0.3 (normalized average tracking length) to around 0.7.

Rating¶

Novelty: ⭐⭐⭐⭐☆ The physics-vision cross-disciplinary perspective of introducing the advection equation into event feature propagation is highly novel, and the gated aggregation design of multi-hypothesis propagation is elegant.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive and deep execution covering main experiments, module-by-module ablation, advection residual evolution analysis, continuity quantitative analysis, and failure case analysis.
Writing Quality: ⭐⭐⭐⭐☆ The motivation is clear (motion continuity vs. implicit modeling) and the physical formulation is natural, though the methodology section introduces many equations, which slightly increases the reading burden.
Value: ⭐⭐⭐⭐⭐ Small object detection with event cameras holds broad application prospects (e.g., UAVs, autonomous driving, industrial inspection). PACT achieves massive improvements on extremely small targets.