Event Ellipsometer: Event-based Mueller-Matrix Video Imaging¶

Conference: CVPR 2025
arXiv: 2411.17313
Code: None
Area: LLM Evaluation
Keywords: Event Camera, Mueller Matrix, Polarization Imaging, High-Speed Imaging, HDR

TL;DR¶

The first system to achieve 30fps video-rate Mueller matrix imaging. By capturing intensity modulations caused by a rapidly rotating QWP via an event camera, the system maps event time differences to Mueller matrix ratios and reconstructs physically valid Mueller matrix videos using SVD estimation combined with spatiotemporal propagation.

Background & Motivation¶

Background¶

Background: The Mueller matrix fully describes the polarization transformation properties of materials and is a core tool in photoelastic analysis, transparent material inspection, and biological tissue imaging. Traditional Mueller matrix measurement requires acquiring multiple images under different polarization states, which is time-consuming and unable to handle dynamic scenes.

Limitations of Prior Work: (1) Traditional frame-based camera schemes require multi-frame synthesis and cannot operate in real-time. (2) High-speed camera schemes are limited by low dynamic range and synchronization timing. (3) Existing polarization cameras can only measure Stokes parameters (4 parameters) and cannot measure the full Mueller matrix (16 parameters).

Key Challenge: The full Mueller matrix requires multiple measurements under different polarization configurations, but in dynamic scenes, the scene has already changed between consecutive measurements.

Goal: To achieve dynamic, single-scan Mueller matrix video imaging using the asynchronous high-speed properties of event cameras.

Key Insight: Event cameras record timestamps of luminance changes (rather than absolute intensities). A rapidly rotating QWP generates periodic luminance changes where the time differences of events can be directly mapped to ratios of Mueller matrix elements—without requiring absolute intensities, inherently providing HDR capabilities.

Core Idea: Map the time difference signals of the event camera (rather than intensity) to Mueller matrix ratios, and achieve 30fps dynamic Mueller matrix imaging through the modulation of a rotating QWP.

Method¶

Overall Architecture¶

Hardware: Event camera (EVK4) + LED light source, each equipped with a rapidly rotating QWP + a fixed linear polarizer. The camera QWP rotates at 5 times the speed of the light source QWP \(\rightarrow\) events record luminance change timestamps at the pixel level \(\rightarrow\) the ellipsometric image formation model relates \(\partial\log(I)/\partial t\) to the Mueller matrix \(\rightarrow\) SVD pixel-wise estimation + Cloude filtering physical constraints + spatiotemporal propagation refinement.

Key Designs¶

Ellipsometric-Event Image Formation Model:
- Function: Maps event time differences to the Mueller matrix.
- Mechanism: The light intensity \(I(t)\) generated by the rotating QWP is a function of the Mueller matrix elements and rotation angles. Events record \(\partial\log(I)/\partial t\), which can be expressed as a linear combination of the system matrix \(B_{tk}\) and the normalized Mueller matrix. Multiple events provide an overdetermined system of equations.
- Design Motivation: Using time differences instead of absolute intensities makes the system insensitive to ambient light variations and sensor gain, inherently providing HDR.
SVD Estimation + Cloude Filtering:
- Function: Restores physically valid Mueller matrices from noisy event data.
- Mechanism: Weighted least squares solves the initial estimate of the Mueller matrix via SVD. Cloude filtering projects the estimate into the physically realizable Mueller matrix space (satisfying Stokes transformation constraints). Iterative reweighting (5 rounds) suppresses outliers.
- Design Motivation: The raw SVD estimation MSE is 0.11, which drops to 0.04 after adding Cloude filtering and reweighting.
Spatiotemporal Propagation Refinement:
- Function: Fills event-sparse regions and suppresses noise.
- Mechanism: PatchMatch-style propagation—using neighboring pixels and the Mueller matrix from the previous frame to "propose" initial values, with random perturbation searching for better solutions (10 iterations).
- Design Motivation: Event cameras do not generate events in regions without luminance change; the propagation mechanism fills these "holes" using spatiotemporal neighborhoods.

Loss & Training¶

Learning-free method—pure physical model + optimization. QWP rotation period is ~33ms \(\rightarrow\) 30fps frame rate. Processing 30 frames at \(500\times500\) resolution takes ~130 seconds.

Key Experimental Results¶

Main Results¶

Evaluation	MSE
SVD only	0.11
+ Cloude filtering	0.07
+ Reweighting	0.05
+ Spatiotemporal propagation	0.04
Validation on known sample	0.045

Application Demos¶

Application	Effect
Photoelasticity (stressed gelatin disk)	Real-time display of stress distribution changes
Transparent tape detection	Distinguish between areas with and without tape
Dynamic face/hair	Capture polarization features of hair
HDR Mueller matrix	Accurate in both highlights and shadows simultaneously

Key Findings¶

First realization of video-rate Mueller matrix imaging: 30fps processing of dynamic scenes.
Inherent HDR: Based on time differences instead of intensities, immune to illumination variations.
Accuracy comparable to traditional methods: Validation on known polarization devices yields an MSE of 0.045.

Highlights & Insights¶

The mapping of "event time difference \(\rightarrow\) Mueller matrix ratio" is the core physical insight—cleverly exploiting the event camera's characteristic of recording changes rather than absolute values.
Simple hardware design: Only a rotating QWP is added in front of a standard event camera, requiring no specialized sensors.
Opens up a new intersection of event cameras and polarization imaging.

Limitations & Future Work¶

Processing speed (130s/30 frames @ \(500\times500\)) is not yet real-time—GPU acceleration may improve this.
The mechanical precision of the rotating QWP limits measurement accuracy.
Only qualitative applications have been demonstrated; quantitative material characterization requires more calibration.

vs DoFP Polarization Cameras: Can only measure 4 Stokes parameters. The event ellipsometer measures the full 16-parameter Mueller matrix.
vs Traditional Mueller Matrix Polarimeters: Requires multiple static acquisitions. The event-based scheme achieves single-scan imaging.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ Pioneering event-based Mueller matrix imaging with deep physical insights.
Experimental Thoroughness: ⭐⭐⭐⭐ Simulation and real-world validation, calibration on known samples, and demos on multiple applications.
Writing Quality: ⭐⭐⭐⭐ Clear derivation of physical models.
Value: ⭐⭐⭐⭐ Opens up new directions for computational imaging and material detection.