WiFlow: Estimating Optical Flow using WiFi Channel State Information

Conference: ECCV 2026
Paper: CVF Open Access
Code: https://visinf.github.io/wiflow
Area: Video Understanding
Keywords: WiFi sensing / optical flow estimation / channel state information (CSI) / RAFT / cross-modal motion estimation

TL;DR

WiFlow introduces the first framework to estimate dense 2D optical flow fields solely from commodity WiFi Channel State Information (CSI), achieving robust cross-subject motion reconstruction in privacy-sensitive and unlit indoor environments through antenna quotient normalization and a decoupled flow-mask architecture.

Background & Motivation

Motion serves as a foundational cue for interpreting dynamic physical scenes. In computer vision, optical flow remains the canonical representation for capturing pixel-wise apparent motion across consecutive temporal frames, underpinning critical downstream applications such as human action recognition, video object tracking, frame interpolation, unsupervised scene segmentation, and robotic path planning. However, conventional optical flow estimators rely almost exclusively on high-resolution RGB image sequences captured by optical cameras. In privacy-critical indoor environments—such as patient monitoring in residential homes, bathrooms, and private workspaces—deploying optical cameras introduces unacceptable privacy intrusion risks. Furthermore, optical sensors are inherently fragile to severe illumination fluctuations, failing completely in darkness, adverse glare, or smoke-occluded environments.

Device-free WiFi sensing presents a promising, non-intrusive alternative for indoor motion understanding. As radio frequency (RF) signals propagate between wireless transceivers, their multipath reflections and Doppler shifts are dynamically modulated by static objects (walls, furniture) and moving human bodies. Commercial off-the-shelf WiFi Network Interface Cards (NICs) continuously compute Channel State Information (CSI) across subcarriers to equalize channel distortions during packet reception, naturally capturing fine-grained physical disturbances. Nevertheless, existing CSI-based sensing paradigms predominantly predict task-specific discrete outputs (such as discrete gesture classification, fall alarms, 3D metric coordinate trajectories, or skeleton keypoints) or attempt full-frame RGB/mesh synthesis, which suffers from severe cross-modal ill-posedness.

The core tension lies in bridging the dimensional and semantic gap between raw, hardware-corrupted RF subcarrier measurements and dense, continuous 2D motion fields without sacrificing spatial fidelity or suffering from catastrophic background hallucination. Core idea: map raw WiFi CSI sequences directly into continuous 2D image-plane optical flow fields by eliminating transceiver hardware phase offsets via antenna quotient normalization and decoupling motion localization from flow refinement with a parallel dual-branch network (WiFlowCombo), enabling camera-free, illumination-invariant dense motion sensing.

Method

Overall Architecture

WiFlow estimates the 2D dense optical flow field \(F \in \mathbb{R}^{2 \times H \times W}\) between two consecutive timestamps using solely complex-valued CSI measurements captured by distributed WiFi transceivers. The hardware setup comprises a single-antenna transmitter (Tx) broadcasting 80 MHz bandwidth WiFi packets at 1 kHz on an interference-free channel and four distributed 4-antenna commercial receivers (Rx1 to Rx4), yielding 16 synchronized spatial channels. The end-to-end framework operates in three sequential stages: multi-antenna quotient normalization for phase artifact removal, spatiotemporal feature extraction across time snapshots, and iterative decoupled optical flow and motion mask decoding.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Raw CSI Measurement Sequences<br/>16 Spatial Channels × K Subcarriers × N Snapshots"] --> B["Multi-Antenna Quotient Normalization<br/>Canceling shared transceiver phase offsets and CFO"]
    B --> C["Spatiotemporal Cross-Modal Feature Encoding<br/>Context and feature extraction across time intervals"]
    C --> D["Decoupled Flow and Mask Decoding<br/>Parallel iterative flow refinement and motion mask prediction"]
    D --> E["Point-wise Mask Gating & Flow Output<br/>Filtering background noise to yield clean 2D optical flow"]

Key Designs

1. Multi-Antenna Quotient Normalization: Canceling Shared Transceiver Hardware Phase Offsets Raw CSI data extracted from commercial WiFi chipsets is heavily contaminated by unsynchronized oscillator drift, carrier frequency offsets (CFO), and sampling time offsets (STO), causing severe, non-physical phase rotations over time. Feeding raw amplitudes and phases directly into neural networks leads to severe overfitting on hardware artifacts rather than physical human motion. To overcome this, WiFlow employs multi-antenna quotient normalization. For each receiver device, subcarrier \(k\), and timestamp \(t\), the complex CSI measurement of antenna \(r\), \(H_{r,k}(t)\), is divided by the measurement of an assigned on-device reference antenna \(H_{\text{ref},k}(t)\):

\[ \tilde{H}_{r,k}(t) = \frac{H_{r,k}(t)}{H_{\text{ref},k}(t)} \]

Because all receiver RF chains on the same physical board share identical local oscillators and clock timing, complex division cancels out shared common-mode phase offsets while preserving the relative phase shifts induced by multipath reflections of moving subjects. Ablation experiments demonstrate that quotient normalization slashes endpoint error on moving pixels by more than 22% compared to raw CSI, Fourier-domain transforms, or temporal smoothing filters.

2. RAFT-Inspired Cross-Modal Iterative Flow Regression: Bridging RF Perturbations and Pixel Displacements Mapping unstructured frequency-domain antenna perturbations onto continuous 2D spatial pixel motion fields is fundamentally ill-posed. WiFlow adapts the core architecture of RAFT into a modular WiFlow Block. Each block integrates a modified ResNet Feature Network with widened input channels matching preprocessed CSI dimensions to construct all-pairs feature correlation representations between timestamps \(t_0\) and \(t_1\), an auxiliary Context Network extracting spatial structure priors, and a convolutional gated recurrent unit (ConvGRU) Refinement Network. The refinement network iteratively updates the flow predictions across \(M\) recurrence cycles:

\[ F^{(m)} = F^{(m-1)} + \Delta F^{(m)}, \quad m=1,\dots,M \]

By projecting high-dimensional RF disturbance patterns onto an implicit 2D geometry and refining velocity vectors over multiple recurrence steps, the network recovers smooth, continuous flow fields without accessing visual pixel textures.

3. Decoupling Motion Localization and Flow Estimation (WiFlowRoI & WiFlowCombo): Suppressing Static Background Hallucination In static indoor environments with fixed transceivers, dynamic motion is spatially sparse, meaning the vast majority of image pixels exhibit exactly zero flow. A monolithic regression baseline (WiFlowSimple) struggles with this sparsity, frequently generating pervasive, low-magnitude background noise while underestimating localized peak motion velocities. WiFlow addresses this structural challenge through two decoupled architectures: - WiFlowRoI: Adopts a two-stage detection-and-estimation paradigm where a pretrained Mask Block first detects moving candidate regions and extracts bounding box contours, after which RoIAlign pools context features within each box to drive a convolutional decoder for local flow estimation; - WiFlowCombo: Deploys a parallel dual-branch pipeline where an upper Flow Block predicts the full unconstrained velocity field while a lower Mask Block predicts a binary motion activation probability map. The final flow output is obtained via element-wise multiplication:

\[ F_{\text{final}} = F_{\text{flow}} \odot M_{\text{mask}} \]

This multiplicative gating mechanism eliminates residual background drift at negligible computational overhead, driving static-pixel endpoint error (\(EPES\)) down from 0.40 to 0.02 while maintaining sharp motion boundaries.

Loss & Training

To prevent optimization from collapsing to a trivial all-zero prediction across dominant static backgrounds, WiFlow optimizes an exponentially weighted sequence loss penalizing non-zero flow errors:

\[ \mathcal{L} = \sum_{m=1}^{M} \gamma^{M-m} \| F_m - F_{\text{gt}} \|_{1, \text{weighted}} \]

where \(M=4\) refinement iterations are computed with decay factor \(\gamma = 0.8\). Moving pixels with \(\|F_{\text{gt}}\|_2 > 0.5\) receive fourfold amplification. Models are trained using the AdamW optimizer with cosine annealing schedule (initial learning rate \(1\times 10^{-3}\)) for 60,000 steps on a single NVIDIA RTX 6000 Ada GPU.

Key Experimental Results

Main Results

WiFlow is benchmarked on the newly collected multi-view, multi-subject benchmark across both Subject Split (cross-person generalization) and Time Split protocols. We report overall Endpoint Error (\(EPE\)), moving-pixel error (\(EPEM\)), and static-pixel error (\(EPES\)):

Perspective	Architecture	Subject Split EPE (↓)	Subject Split EPEM (↓)	Subject Split EPES (↓)	Time Split EPE (↓)	Time Split EPEM (↓)	Time Split EPES (↓)
sideview	WiFlowSimple	0.51	2.22	0.40	0.53	2.07	0.43
sideview	WiFlowRoI	0.18	2.36	0.04	0.16	2.21	0.04
sideview	WiFlowCombo	0.16	2.36	0.02	0.15	2.18	0.03
birdview	WiFlowSimple	0.55	3.53	0.40	0.56	3.24	0.43
birdview	WiFlowRoI	0.21	3.73	0.05	0.20	3.40	0.04
birdview	WiFlowCombo	0.19	3.71	0.03	0.18	3.38	0.03
birdview+	WiFlowSimple	0.45	3.24	0.32	0.41	2.89	0.29
birdview+	WiFlowRoI	0.21	3.41	0.05	0.19	3.13	0.05
birdview+	WiFlowCombo	0.18	3.50	0.02	0.17	3.13	0.02

Ablation Study

1. CSI Preprocessing Strategy Ablation (WiFlowSimple on birdview+, Time Split):

Preprocessing	EPE (↓)	EPEM (↓, Moving)	EPES (↓, Static)	EPEA (↓, Amplified)	Note
Zero Prediction	0.17	3.70	0.00	0.84	Trivial zero-flow baseline with catastrophic moving errors
SavGol	0.59	3.64	0.44	1.14	Temporal smoothing over-smooths dynamic transients
Fourier	0.41	3.71	0.25	0.95	Frequency transform fails to isolate transceiver hardware drift
Raw	0.40	3.71	0.24	0.94	Raw phase drift heavily degrades moving-pixel tracking
Quotient	0.41	2.89	0.29	0.78	Cancels common hardware offsets, cutting EPEM by 22%
PCA	0.47	2.95	0.35	0.82	Denoising 150 components preserves structure but lags Quotient

2. Computational Complexity and Inference Overhead (Averaged over 1000 runs on RTX 6000 Ada GPU):

Architecture	Time (ms)	FLOPs (\(\times 10^9\))	VRAM (MB)	Evaluation
WiFlowSimple	23	22	380	Lowest latency, single stream, but suffers background noise
WiFlowRoI	26	22	380	Two-stage box cropping, clean background, minimal overhead
WiFlowCombo	47	43	763	Dual parallel streams, under 1GB VRAM, best overall accuracy

Key Findings

• Quotient Preprocessing is Crucial for Cross-Modal Learning: While Raw CSI achieves an deceptively low raw EPE due to the zero-dominant background, its error on actual moving pixels (\(EPEM\)) is 3.71. Quotient normalization effectively eliminates receiver clock drift, dropping \(EPEM\) to 2.89 and yielding the lowest amplified error (\(EPEA=0.78\)).
• Decoupled Mask Filtering Eliminates Hallucinated Noise: WiFlowCombo reduces background static pixel error (\(EPES\)) by an order of magnitude (from 0.40 to 0.02). Although WiFlowSimple exhibits slightly lower \(EPEM\) due to unconstrained diffusive flow leakage into static areas, its visual outputs are corrupted with severe false-positive background flow artifacts.
• Spatial Diversity Yields Saturation around 3 Receivers: Systematically increasing the receiver count from 1 device (2 antennas) to 4 devices (16 antennas) demonstrates rapid error reductions up to 3 distributed devices, after which performance gains plateau, confirming that 3-4 spatial viewpoints adequately resolve indoor multipath ambiguity.
• Robust Cross-Subject Generalization: The marginal discrepancy between Subject Split and Time Split (< 0.02 pixels EPE) confirms that WiFlow learns invariant physical relationships between RF Doppler perturbations and 2D velocity fields rather than memorizing individual body shapes.

Highlights & Insights

• Pioneering Low-Level Vision via Ambient RF: Establishes the first proof-of-concept that raw ambient WiFi CSI contains sufficient geometric information to reconstruct continuous, dense 2D pixel motion fields without cameras.
• Elegant Multiplicative Gating Architecture: The parallel combination of dedicated flow and mask blocks in WiFlowCombo elegantly resolves the severe spatial sparsity problem inherent to indoor motion sensing.
• Comprehensive Dual-Perspective Benchmark: Provides 164 minutes of multi-antenna 1 kHz CSI synchronized with dual-view video and multi-method ensemble pseudo-ground-truth optical flow, creating a standardized foundation for future RF-vision research.

Limitations & Future Work

• Environment and Layout Specificity: Similar to classical WiFi sensing systems, the learned multipath mapping is closely tied to the specific room geometry and furniture layout, necessitating retraining or domain adaptation for unseen physical rooms.
• Physical Inconsistency in Optical PseudoGT: Pseudo-ground-truth flow generated from optical cameras inherently tracks moving cast shadows, whereas RF signals penetrate or disregard optical shadows, introducing conflicting supervisory signals.
• Multi-Person Overlap and Output Resolution: When multiple individuals cross paths, superimposed multipath signals create complex interferences; furthermore, the current output resolution of \(128 \times 168\) remains coarse compared to camera-based optical flow estimators.

Related Work & Insights

• vs MM-Fi / Person-in-WiFi (Multimodal Wireless Sensing): Prior benchmarks focus on discrete human actions, bounding boxes, or 3D skeletal keypoints. WiFlow pushes sensing output down to dense pixel displacement vectors, enabling seamless integration into downstream computer vision pipelines.
• vs RAFT (Optical Flow Estimation): While RAFT computes correlation volumes from dense RGB feature pyramids, WiFlow re-engineers this recurrence mechanism to iteratively map 1D frequency-domain RF perturbations into 2D continuous flow vectors.

Rating

• Novelty: ⭐⭐⭐⭐⭐ [Pioneers dense 2D optical flow estimation purely from commodity WiFi CSI]
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ [New synchronized multi-view benchmark with rigorous preprocessing, architecture, resolution, and generalization studies]
• Writing Quality: ⭐⭐⭐⭐⭐ [Exceptionally clear physical motivation, methodical error analysis, and well-grounded architectural designs]
• Value: ⭐⭐⭐⭐⭐ [Unlocks camera-free, privacy-preserving, illumination-agnostic motion sensing for smart homes, healthcare, and security]

Related Papers

• [ECCV 2026] On the real-world generalisability of Optical Flow models
• [ECCV 2026] FlowPainter: Inpainting Optical Flow via Confidence-Guided Completion
• [ECCV 2026] MegaFlow: Zero-Shot Large Displacement Optical Flow
• [ICLR 2026] Map the Flow: Revealing Hidden Pathways of Information in VideoLLMs
• [ECCV 2026] Triangular Consistency as a Universal Constraint for Learning Optical Flow