Skip to content

Estimating Velocity and Spin of Spherical Objects from Rolling-Shutter Image(s)

Conference: ECCV 2026
Paper: ECCV Official
Area: 3D Vision
Keywords: Rolling Shutter, Spherical Motion Reconstruction, Velocity and Spin Estimation, Back-projection Geometric Consistency

TL;DR

Rather than treating rolling-shutter distortions as artifacts to be eliminated, this paper introduces a correspondence-free back-projection framework that decouples translation from rotation, enabling high-precision estimation of 3D velocity and spin for high-speed spherical objects from a single rolling-shutter image.

Background & Motivation

In sports analytics (such as trajectory and spin tracking for golf, tennis, or baseball) and high-speed industrial inspection, capturing the 3D translational velocity and spin of fast-moving spherical objects is essential. Contemporary solutions rely heavily on specialized, capital-intensive hardware setups, including synchronized high-speed global-shutter camera arrays, radar Doppler tracking units, or asynchronous event sensors. These systems require complex trigger synchronization and remain prohibitively expensive for commodity consumer devices. Conversely, ubiquitous CMOS rolling-shutter sensors capture pixel rows sequentially, which inevitably induces severe geometric wobble and skew artifacts when an object traverses the camera view at high speeds (e.g., exceeding 50 m/s). Traditional vision pipelines almost uniformly dismiss these distortions as undesirable image degradation to be undone.

Directly extracting 6-DoF rigid-body motion from rolling-shutter captures of spherical bodies presents an acute dilemma. The core challenge stems from two fundamental barriers: first, conventional rolling-shutter kinematic estimators (such as the seminal formulation by Ait-Aider et al.) demand explicit 3D-to-2D feature correspondences or structured straight-line primitives. Yet rapidly spinning spheres inherently lack persistent linear geometry and often exhibit repetitive, symmetric patterns that become severely degraded by motion blur and row-wise skew, making reliable matching intractable. Second, standard forward-projection formulations suffer from an implicit temporal dependency: the physical acquisition timestamp of a point depends on its projected vertical scanline coordinate, which itself depends on the unknown 3D position and motion parameters. This circular dependency renders the joint optimization over position, orientation, linear velocity, and angular velocity highly non-convex, unstable, and prone to divergence under standard initializations.

This paper fundamentally reconsiders rolling-shutter distortion, viewing scanline delay not as a flaw, but as a dense, high-frequency temporal encoder. Because each image row samples the scene at a distinct microsecond timestamp, the observed deformation traces the continuous space-time trajectory of the object. Core idea: by back-projecting image observations into 3D sight rays in the object frame and exploiting spherical rotational symmetry to decouple translation from rotation, a two-stage correspondence-free optimization achieves stable, robust recovery of 3D velocity and angular velocity from rolling-shutter imagery.

Method

Overall Architecture

By the principle of relative kinematics, viewing a dynamic sphere with a stationary camera is mathematically equivalent to observing a stationary sphere centered at the origin of its object frame \(\mathcal{F}_o\) from a moving virtual camera. The proposed framework exploits this perspective and operates in four sequential stages: 2D feature and silhouette extraction, silhouette-driven translational velocity estimation, icosahedral pattern-driven angular velocity estimation, and inter-frame temporal refinement for multi-view or multi-frame setups.

The input comprises one or more frames exhibiting rolling-shutter distortion. The system detects the 2D contour points of the sphere along with internal feature markers. For translational motion, because a sphere's silhouette is fundamentally invariant to axial rotation, the algorithm optimizes position and linear velocity solely by minimizing the distance between back-projected contour rays and the spherical surface. Once translation is fixed, detected surface features are back-projected onto the known 3D sphere, and spin is recovered by penalizing deviations from an isometric pairwise distance invariant. When additional frames or stereo views are available, an inter-frame truncated Chamfer loss further refines the rotational velocity estimate.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Rolling-shutter image(s) (Single/Stereo)"] --> B["Stage 1: Ray Alignment via Back-Projection<br/>Map pixel rays and row-dependent timestamps to object frame"]
    B --> C["Stage 2: Contour-Constrained Translation Estimation<br/>Optimize 3D position and linear velocity via rotation-invariant silhouette"]
    C --> D["Stage 3: Pairwise Distance Variance Spin Estimation<br/>Minimize metric distance variance of icosahedral feature pairs"]
    D -->|Single-frame output| E["Output: 3D Translational Velocity v & Angular Velocity ฯ‰"]
    D -->|Multi-frame/Stereo| F["Stage 4: Truncated Chamfer Temporal Refinement<br/>Align back-projected 3D point sets across frames"]
    F --> E

Key Designs

1. Ray Alignment via Back-Projection: Eliminating Implicit Forward-Projection Coupling

In forward projection, the scanline row \(u_y\) determines the capture time \(\Delta t(\mathbf{u}) = \Delta t_g + \tau u_y\), which simultaneously dictates the instantaneous camera pose, locking the optimization in an implicit nonlinear loop. To break this dependency, the authors employ a back-projection ray formulation. For each measured pixel \(\mathbf{u} = [u_x, u_y]^\top\) captured at timestamp \(k = k_s + \tau u_y\), a sight ray is cast from the camera center into 3D space. Using Lie algebra kinematics, the ray is transformed to a common temporal reference timestamp \(k_0\) within the object coordinate frame \(\mathcal{F}_o\): $\(\mathbf{x}_{o,k_0}(\mathbf{u}) = \exp\left(-\Delta t(\mathbf{u})(\mathbf{R}_{co}^\top \boldsymbol{\omega}_{co})^\wedge\right) \left(\mathbf{x}_o - \Delta t(\mathbf{u})\mathbf{R}_{co}^\top \mathbf{v}_{co}\right)\)$ $\(\mathbf{d}_{o,k_0}(\mathbf{u}) = \exp\left(-\Delta t(\mathbf{u})(\mathbf{R}_{co}^\top \boldsymbol{\omega}_{co})^\wedge\right) \mathbf{d}_o(\mathbf{u})\)$ Because \(u_y\) is an explicit observation, each ray receives an exact, deterministic capture time, allowing closed-form geometric ray transformations that decouple the imaging physics from unknown motion parameters.

2. Contour-Constrained Translation Estimation: Parameter Decoupling via Spherical Symmetry

Because a sphere's projected silhouette is perfectly rotation-invariant, its occluding boundary rays are governed exclusively by the initial translation \(\mathbf{t}_{co}\) and translational velocity \(\mathbf{v}_{co}\), remaining entirely unaffected by spin \(\boldsymbol{\omega}_{co}\). Setting \(\boldsymbol{\omega}_{co} = \mathbf{0}\), the algorithm extracts \(N\) contour points \(\mathbf{u}_i\). For each back-projected ray, the 3D point nearest the sphere center is determined in closed form by \(\hat{a}_i = \max(0, -\mathbf{d}_i^\top \mathbf{x}_i)\), producing point \(\mathbf{P}_i = \mathbf{x}_i + \hat{a}_i \mathbf{d}_i\). For true motion hypotheses, all \(\mathbf{P}_i\) must reside on the sphere surface of known radius \(r\). To prevent trivial convergence where all points collapse into a single degenerate cluster, a spatial dispersion regularization term \(\mathcal{R}_{\mathbf{v}}\) is incorporated into the objective: $\(\mathcal{J}_{\mathbf{v}}(\mathbf{t}_{co},\mathbf{v}_{co}) = \frac{1}{N}\sum_{i=0}^{N-1} \big|\|\mathbf{P}_i\| - r\big|^2 + \max\left(0, \sigma r^2 - \frac{1}{N}\sum_{i=0}^{N-1} \|\mathbf{P}_i - \bar{\mathbf{P}}\|^2\right)\)$ This reduces the optimization dimensionality from 12 joint parameters to just 6, achieving robust, fast convergence via L-BFGS-B even for entirely textureless spheres.

3. Pairwise Distance Variance Spin Estimation: Correspondence-Free Icosahedral Optimization

To recover the angular velocity \(\boldsymbol{\omega}_{co}\), the sphere is imprinted with an icosahedron-inspired pattern comprising 20 equilateral triangular clusters with shaded interior centroids. Rather than establishing explicit 2D-to-3D landmark correspondences, the formulation relies on a distance invariance principle: the metric Euclidean distance between any pair of vertices within an equilateral cluster is constant on the physical sphere. Back-projected rays from detected surface points are intersected with the sphere: $\(\hat{a}_i = -\mathbf{d}_i^\top \mathbf{x}_i - \sqrt{(\mathbf{d}_i^\top \mathbf{x}_i)^2 - (\|\mathbf{x}_i\|^2 - r^2)}\)$ Yielding reconstructed 3D surface points \(\mathbf{P}_i\). For the set of equidistant feature pairs \(\mathcal{E}\), the angular velocity loss is formulated directly as the empirical variance of pairwise distances: $\(\mathcal{J}_{\boldsymbol{\omega}}(\boldsymbol{\omega}_{co}) = \frac{1}{|\mathcal{E}|}\sum_{(i,j)\in\mathcal{E}} \|\mathbf{P}_i - \mathbf{P}_j\|^2 - \left(\frac{1}{|\mathcal{E}|}\sum_{(i,j)\in\mathcal{E}} \|\mathbf{P}_i - \mathbf{P}_j\|\right)^2\)$ When the candidate angular velocity accurately inverts the scanline-induced rotational skew, reconstructed surface triangles recover their true equilateral geometry, driving the variance to zero without requiring point identity tracking.

4. Truncated Chamfer Temporal Refinement: Multi-View and Inter-Frame Extension

Single-frame estimation may exhibit ambiguity along the optical axis (\(z\)-axis) or parallel to scanlines. When multi-frame or stereo rolling-shutter observations are available (e.g., separated by a 2 ms baseline), the preliminary spin estimate initializes a refined temporal alignment. The truncated squared Chamfer distance measures structural discrepancy between back-projected 3D point sets \(\mathcal{P}_i\) and \(\mathcal{P}_j\): $\(d_{\mathrm{ch}}(\mathcal{P}_i, \mathcal{P}_j) = \frac{1}{|\mathcal{P}_i|} \sum_{\mathbf{a} \in \mathcal{P}_i} \min\left(D, \min_{\mathbf{b} \in \mathcal{P}_j} \|\mathbf{a} - \mathbf{b}\|^2\right)\)$ Here, threshold \(D\) rejects features that rotate out of the field of view. Minimizing the multi-view loss \(\mathcal{J}_{\boldsymbol{\omega}, 2}\) enforces tight temporal consistency and achieves sub-pixel unwarped texture alignment.

Loss & Training

The pipeline is a pure physics-based numerical optimization framework requiring no offline training or data collection. Translation parameters are optimized with L-BFGS-B across a coarse grid search over initial translational hypotheses \(\{-25, 0, 25\}\,\mathrm{m/s}\) along each axis. The angular velocity optimization is initialized at \(\boldsymbol{\omega}_{co} = [0, 0, 0]^\top\). In multi-frame settings, single-frame estimates initialize the secondary Chamfer refinement.

Key Experimental Results

Main Results

Evaluated on a Blender-based rolling-shutter simulator (\(2048 \times 2048\) resolution, \(10\,\mu\mathrm{s/row}\) readout, sphere velocities up to \(50\,\mathrm{m/s}\), spin up to \(5000\,\mathrm{rpm}\), across 100 random trials), reporting Mean Absolute Error (MAE) for translational and angular velocities across four configurations.

Configuration Method Velocity MAE (m/s) [x / y / z] Angular Velocity MAE (rad/s) [x / y / z] Description
1. Single Frame Global Shutter (PnP) - / - / - - / - / - Cannot infer kinematics from single image
1. Single Frame Ait-Aider et al. [1] (w/ GT matches) 3.24 / 10.14 / 14.33 164.16 / 172.60 / 138.25 Severely diverges under fast 12-DoF search
1. Single Frame Ours (Correspondence-free) 0.23 / 4.61 / 2.95 49.82 / 17.18 / 27.89 Robust single-frame recovery without 3D-2D links
2. Stereo Baseline Ait-Aider et al. [1] (w/ GT matches) 3.25 / 7.80 / 10.87 163.96 / 188.64 / 137.85 Large errors persist despite stereo cues
2. Stereo Baseline Ours (Correspondence-free) 0.12 / 0.23 / 0.19 33.11 / 20.60 / 10.13 Stereo parallax resolves depth ambiguity
3. Multi-Frame (2ms) Global Shutter (PnP) 11.57 / 8.03 / 30.58 131.00 / 126.50 / 120.88 Ignoring scanline delay causes massive drift
3. Multi-Frame (2ms) Ait-Aider et al. [1] (w/ GT matches) 1.44 / 1.12 / 12.40 142.31 / 133.22 / 81.99 Significant depth and spin residual errors
3. Multi-Frame (2ms) Ours (Correspondence-free) 0.16 / 0.12 / 0.36 8.48 / 7.22 / 7.51 Temporal baseline dramatically enhances spin accuracy
3. Multi-Frame (2ms) Ours + Chamfer Refinement 0.16 / 0.12 / 0.36 4.58 / 2.41 / 1.10 Chamfer alignment suppresses residual errors
4. Stereo Multi-Frame Global Shutter (PnP) 38.42 / 8.55 / 39.21 132.52 / 123.50 / 119.31 Complete failure due to unmodeled row timings
4. Stereo Multi-Frame Ait-Aider et al. [1] (w/ GT matches) 2.76 / 1.16 / 9.84 153.16 / 151.77 / 114.63 Trapped in local minima
4. Stereo Multi-Frame Ours + Chamfer Refinement 0.19 / 0.10 / 0.26 10.39 / 3.93 / 2.68 Best overall spatiotemporal consistency

Ablation Study

Sensitivity analysis evaluating sphere radius variation in the single-image setting (evaluated at 80%, 100%, and 120% of nominal radius):

Sphere Radius Setting \(e_{vx}\) (m/s) \(e_{vy}\) (m/s) \(e_{vz}\) (m/s) \(e_{\omega x}\) (rad/s) \(e_{\omega y}\) (rad/s) \(e_{\omega z}\) (rad/s) Key Observations
80% Radius 0.32 3.46 4.90 45.32 31.54 44.02 Smaller pixel footprint increases depth and spin variance
100% Radius 0.23 4.61 2.95 49.82 17.18 27.89 Baseline physical dimensions exhibit balanced error metrics
120% Radius 0.19 3.50 3.63 38.14 14.86 22.28 Higher apparent resolution yields denser contour constraints

Real-world golf launch monitor benchmarking against commercial SkyTrak system across 10 shots:

Evaluation Metric Ball Speed (mph) Launch Angle (deg) Side Angle (deg) Backspin (rpm) Sidespin (rpm)
All 10 Shots MAE 2.4 1.0 3.2 841 402
MAE excluding Outlier #10 2.4 1.0 3.5 107 247
SkyTrak Stated Tolerance \(\pm 1.0\) \(\pm 1.0\) \(\pm 2.0\) \(\pm 250\) \(\pm 250\)

Key Findings

  • Traditional coupled 12-parameter formulations fail severely in high-speed regimes: even with ground-truth 3D-to-2D correspondences provided, Ait-Aider et al. yield translation errors above \(10\,\mathrm{m/s}\) and spin errors above \(130\,\mathrm{rad/s}\). In contrast, decoupling translation from rotation reduces our translation MAE to below \(0.2\,\mathrm{m/s}\) in multi-frame scenarios.
  • In single-view setups, errors concentrate along the optical depth axis (\(z\)) and vertical scan axis (\(y\)) due to projection ambiguity (\(e_{vy}=4.61\,\mathrm{m/s}, e_{vz}=2.95\,\mathrm{m/s}\)). Introducing stereo baseline or a tiny 2 ms inter-frame delay resolves this ambiguity, dropping error by an order of magnitude (\(e_{vy}=0.23\,\mathrm{m/s}, e_{vz}=0.19\,\mathrm{m/s}\)).
  • In real-world testing, Shot 10 exposed an anomaly where SkyTrak reported a sudden backspin spike to 11,254 rpm (versus our 3,805 rpm). Overlaying unwarped consecutive images using our estimated spin confirmed near-perfect visual texture alignment, revealing that SkyTrak's global-shutter cross-frame correlation suffered stroboscopic pattern aliasing, whereas rolling-shutter continuous scanline timestamps provided unambiguous motion disambiguation.

Highlights & Insights

  • Turning distortion into a temporal sensor: Rather than spending compute removing rolling-shutter wobble, treating each scanline as an exact time-stamp converts a static 2D image into a continuous temporal velocity sensor.
  • Decoupling via spherical symmetry: Exploiting the rotation invariance of spherical contours reduces an otherwise intractable 12D non-convex problem into sequential 6D translation and 3D rotation sub-problems, guaranteeing rapid convergence and enabling textureless velocity tracking.
  • Correspondence-free pairwise distance metric: Enforcing variance minimization over icosahedral equilateral feature pairs circumvents the notoriously fragile requirement of 2D-to-3D landmark correspondence matching under motion blur.
  • Hardware scanline orientation co-design: Mounting rolling-shutter sensors with a 90-degree in-plane rotation so that the scanline progression opposes the ball flight direction substantially widens the acoustic trigger timing window, preventing fast balls from escaping the FOV mid-frame.

Limitations & Future Work

  • Strict dependence on spherical geometry: The current derivation relies strictly on perfect spherical contours and known radius, limiting immediate applicability to irregular or ellipsoidal geometries.
  • Constant velocity assumption: Kinematics are modeled assuming constant velocity over short exposures, omitting rapid aerodynamic deceleration or transient impact deformations.
  • Spin estimation requires custom markings: While translational tracking works on blank spheres, spin determination necessitates printing the designed icosahedral pattern onto the surface.
  • Future directions: The authors plan to generalize the formulation to arbitrary rotationally symmetric quadrics and ellipsoids, and develop self-supervised spin recovery for natural sports textures.
  • vs Ait-Aider et al. (ECCV 2006 / CVPR 2007): Ait-Aider established foundational kinematic estimation from single rolling-shutter images but required explicit 2D-3D point or line correspondences in a 12-parameter joint optimization. The proposed method eliminates correspondences and decouples translation from spin, drastically boosting stability.
  • vs Global-Shutter Multi-Camera Arrays (Tamaki et al. / Commercial Trackers): Standard launch monitors depend on multi-camera synchronized global-shutter arrays and microsecond strobes. This work proves that low-cost commodity rolling-shutter CMOS sensors can achieve comparable or superior precision.
  • vs Event Camera Tracking (Nakabayashi et al. CVPRW 2024): Event sensors provide high temporal resolution for spin recovery but suffer from low spatial resolution and drift in absolute metric depth. This method preserves high spatial resolution and metric scale at commodity sensor costs.

Rating

  • Novelty: โญโญโญโญโญ Elegant inversion of rolling-shutter artifacts into temporal velocity cues, combined with correspondence-free spherical decoupling.
  • Experimental Thoroughness: โญโญโญโญโญ Thorough validation spanning Blender ray-tracing simulations, stereo/multi-frame setups, and real-world prototype comparison against commercial monitors.
  • Writing Quality: โญโญโญโญโญ Mathematically rigorous, clear geometric derivations, and candid analysis of failure modes and hardware nuances.
  • Value: โญโญโญโญโญ Substantial industrial and engineering relevance for sports analytics, automated tracking, and high-speed robotic manipulation.