PS-EIP: Robust Photometric Stereo Based on Event Interval Profile¶

Conference: CVPR 2025
arXiv: 2503.18341
Code: None
Area: Human/Scene Understanding
Keywords: Photometric Stereo, Event Camera, Surface Normal Estimation, Outlier Detection, Event Interval Profile

TL;DR¶

This paper proposes a robust photometric stereo method based on the Event Interval Profile (EIP). By utilizing the temporal continuity and profile shape of event interval time series, it detects outliers caused by shadows and specular reflections, significantly outperforming EventPS-FCN without requiring deep learning.

Background & Motivation¶

Background: Photometric Stereo is a classic method for 3D reconstruction that recovers pixel-level surface normals by observing objects under varying illumination directions. Recently, event cameras have been introduced to the photometric stereo domain due to their low power consumption, high dynamic range, and microsecond-level temporal resolution. A representative method, EventPS, reconstructs surface normals using events triggered by a moving directional light source.

Limitations of Prior Work: EventPS processes each event interval independently, making it highly sensitive to noise, shadows, and non-Lambertian reflections. When shadow areas or specular highlights are present in the scene, individual event intervals can deviate severely, leading to inaccurate normal estimation. Even its deep learning variant, EventPS-FCN, cannot fully resolve this robustness issue.

Key Challenge: Event intervals contain rich time-series info, but existing methods only utilize single interval values for computation, discarding the temporal continuity and structural information.

Goal: To design a robust, non-deep-learning photometric stereo method that fully utilizes the time-series profile of event intervals to detect and reject outliers.

Key Insight: Event intervals present a specific continuous profile shape as the light source moves. The profile in Lambertian reflection regions is smooth and predictable, whereas shadows and specular reflections introduce obvious shape anomalies into the profile.

Core Idea: Replace individual event intervals with the complete time-series profile of event intervals (EIP) for normal estimation, and perform outlier detection based on the profile shape.

Method¶

Overall Architecture¶

The input is an event stream collected by an event camera under a moving directional light source, and the output is a pixel-level surface normal map. First, the event interval sequence of each pixel is organized into an Event Interval Profile (EIP). Then, outliers are detected using the continuity and shape features of the EIP. Finally, surface normals are solved using the clean data.

Key Designs¶

Event Interval Profile (EIP):
- Function: Maps pixel-level event intervals from isolated values to complete time-series representations.
- Mechanism: For each pixel, the trigger intervals of all events during a full rotation cycle of the light source are recorded, forming a continuous curve that varies with the light source direction. Under the Lambertian reflection model, the event interval is related to the logarithmic derivative of the change in illumination direction, so the EIP should exhibit smooth and periodic variation. Anomalous regions (shadows, specular reflections) create abrupt changes or abnormal peaks in the EIP.
- Design Motivation: EventPS processes each interval independently, making it impossible to distinguish between inliers and outliers. EIP leverages temporal continuity to transform outlier detection into a curve shape analysis problem.
Outlier Detection Based on Profile Shape:
- Function: Automatically identifies and rejects intervals in the EIP affected by shadows and specular reflections.
- Mechanism: The EIP generated by Lambertian reflection has specific smooth shape characteristics. Shadow regions correspond to sudden increases in event intervals (sharp brightness drops leading to sparse events), and specular reflection regions correspond to sudden decreases in event intervals (sharp brightness increases leading to dense events). The proposed method analyzes the local shape changes of the EIP, detects intervals that do not conform to the expectations of the Lambertian model, marks them as outliers, and excludes them from the normal estimation solver.
- Design Motivation: Traditional outlier handling in photometric stereo mostly relies on RANSAC or deep learning, whereas the profile shape of EIP provides a physically interpretable criterion for outliers.
Robust Normal Estimation Based on Clean EIPs:
- Function: Solves surface normals using EIP data after outlier rejection.
- Mechanism: After outlier detection, only event intervals marked as normal participate in normal estimation. Due to the deterministic relationship between event intervals and normal directions under Lambertian reflection (modeled via the logarithmic brightness change rate with respect to the illumination direction), clean data can accurately solve the normals via least squares or similar methods. The continuity of EIP also allows for smoothing of residual noise.
- Design Motivation: The data quality is vastly improved after outlier rejection, enabling highly accurate normals to be obtained using simple solving methods.

Loss & Training¶

PS-EIP is a training-free (non-learning) method, involving no loss functions or training strategies. The normals are obtained through optimization solving based on physical models.

Key Experimental Results¶

Main Results: Normal Estimation Accuracy on 3D Printed Objects¶

Method	Mean Angular Error (°) ↓	Training Required	Robustness to Shadows	Robustness to Specular Reflections
EventPS	Higher	No	Poor	Poor
EventPS-FCN	Moderate	Yes (Deep Learning)	Moderate	Moderate
PS-EIP (Ours)	Lowest	No	Strong	Strong

Ablation Study: Contribution of Outlier Detection Modules¶

Configuration	Mean Angular Error (°)	Description
No outlier detection	Higher	Equivalent to directly using all event intervals
Shadow detection only	Moderate	Rejects outliers in shadow regions
Specular detection only	Moderate	Rejects specular reflection outliers
Full PS-EIP	Lowest	Simultaneously detects both types of outliers

Key Findings¶

PS-EIP significantly outperforms EventPS-FCN on real-world event data, demonstrating that outlier detection based on profile shape is more effective than deep learning approaches.
The method requires no training data or GPU computation, offering better generalization and practicality.
The outlier detection modules for shadows and specular reflections are complementary, and their combination yields the best results.
The effectiveness of the method is validated on 3D-printed objects with complex reflection properties.

Highlights & Insights¶

Proposal of the EIP Concept: Organizing event intervals from isolated values into time-series profiles is an ingenious representation shift. This transformation changes outlier detection from a difficult single-point decision to simple curve shape analysis, greatly reducing the difficulty of the task.
Training-Free Nature: In an era dominated by deep learning, PS-EIP demonstrates that physics-based methods can outperform learning methods given the correct representation. The physical interpretability of EIP also makes the method more trustworthy.
General Outlier Detection Paradigm: The idea of profile-shape-based outlier detection can be transferred to other event camera applications, such as optical flow or depth estimation, where processing anomalous events is necessary.

Limitations & Future Work¶

The method is currently only validated on 3D-printed objects; hence the scene complexity is limited and it has not been tested on large-scale natural scenes.
It assumes that the motion trajectory of the light source is known and is a simple rotation; real-world light source configurations can be more complex.
The method relies on the prior assumption of a Lambertian reflection model, which may have limited effectiveness on strongly non-Lambertian materials (such as metals or glass).
No comparison has been made with the latest neural-network-based photometric stereo methods (such as NeRF-based methods).

vs EventPS: EventPS processes each event interval independently, whereas PS-EIP dramatically improves robustness by utilizing the complete time-series profile.
vs EventPS-FCN: EventPS-FCN handles noise using deep learning, but its generalization is inferior to the physics-based approach of PS-EIP.
vs Traditional Photometric Stereo: Traditional methods require multiple images and precise illumination control, while event-camera-based methods are more efficient and can handle dynamic scenes.

Rating¶

Novelty: ⭐⭐⭐⭐ The EIP concept is novel, introducing time-series analysis into event-driven photometric stereo.
Experimental Thoroughness: ⭐⭐⭐ Only validated on 3D printed objects; the scenes and baselines are relatively limited.
Writing Quality: ⭐⭐⭐⭐ The problem definition is clear, and the method description is well-structured.
Value: ⭐⭐⭐⭐ Provides a new paradigm for the application of event cameras in photometric stereo.