Equivariant Symmetry-Aware Head Pose Estimation for Fetal MRI¶
Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/MedicalVisionGroup/E3-Pose
Area: Medical Imaging
Keywords: Fetal MRI, 6-DoF Pose Estimation, Rotation Equivariance, Anatomical Symmetry, Pseudovector Parametrization
TL;DR¶
E(3)-Pose explicitly embeds E(3)-equivariant convolutional filters and models the bilateral anatomical symmetry axis via an even-parity pseudovector, achieving robust 6-DoF fetal head pose estimation from highly artifact-corrupted clinical 3D MRI navigators in 0.3s for real-time adaptive slice prescription.
Background & Motivation¶
Fetal magnetic resonance imaging (MRI) provides indispensable diagnostic visualization for evaluating prenatal neurodevelopment and detecting congenital anomalies. Clinical diagnostic workflows rely on acquiring stacks of high-resolution 2D MRI slices. However, spontaneous, continuous, and unpredictable fetal movements during scanning cause significant inter-slice misalignments, spatial coverage gaps, and severe slice obliqueness, frequently forcing scan technicians to restart entire acquisitions and severely compromising radiological interpretations. A promising solution is prospective motion tracking: interleaving rapid, low-resolution, low signal-to-noise ratio (SNR) 3D "navigator" MRI volumes before each 2D diagnostic slice, estimating the 6-DoF rigid head pose from each navigator in real time, and dynamically adjusting the physical imaging plane prescription for the subsequent diagnostic slice.
Estimating head pose from these clinical navigator volumes is exceptionally difficult. Rapid interleaved sequences create severe spin history artifacts, visible as prominent dark shading bands along preceding excitation planes that routinely occlude facial landmarks such as the eyes. Furthermore, underdeveloped cortical folding in early-to-mid gestational ages produces an almost spherical brain shape with pronounced left-right anatomical symmetry, introducing severe pose ambiguities. Crucially, due to the scarcity of annotated clinical navigator volumes, models must be trained on artifact-free, high-SNR research scans (such as dHCP), creating a steep cross-domain generalization barrier. Existing landmark-based estimators fail when key landmarks are obstructed, optimization-based template registration tools (e.g., FireANTs) are corrupted by intensity perturbations and require 1.8s (prohibiting real-time deployment), and standard CNN pose regressors severely overfit and lack structural constraints against symmetry ambiguities.
This work addresses the issue from first principles: rather than relying on brittle data augmentation to implicitly learn geometric transformations, the network architecture is constrained by construction to respect rigid Euclidean symmetries. Core idea: develop an E(3)-equivariant CNN regressor that models the left-right anatomical symmetry axis as an even-parity pseudovector, ensuring that the predicted 6-DoF head pose is inherently rotation-equivariant and invariant to bilateral reflection, eliminating symmetry ambiguities and providing unprecedented out-of-distribution clinical generalization.
Method¶
Overall Architecture¶
E(3)-Pose decouples 6-DoF rigid head pose estimation \(T = t \circ R\) into independent translation \(t \in \mathbb{R}^3\) and 3D rotation \(R \in \text{SO}(3)\) predictions. In the first stage, a compact 3D convolutional network \(\psi\) segments the fetal brain from the full-uterus MRI volume, and the translation vector \(t\) is computed in closed form via the center-of-mass (CoM) of the predicted binary mask. The volume is subsequently cropped around the brain mask with a 40% margin. In the second stage, the cropped subvolume is fed into an E(3)-CNN regressor \(\phi\) that directly predicts the orthonormal basis vectors \((\hat{e}_x, \hat{e}_y, \hat{e}_z)\) of the canonical anatomical head frame, where the lateral left-right axis \(\hat{e}_x\) is parameterized as an even-parity pseudovector, while the posterior-anterior \(\hat{e}_y\) and inferior-superior \(\hat{e}_z\) axes are parameterized as regular odd-parity vectors. Finally, singular value decomposition (SVD) orthonormalization is applied, and the pseudovector sign is resolved by enforcing a positive determinant (\(\det(\hat{R}) = +1\)), yielding a continuous, non-reflective rotation matrix in \(\text{SO}(3)\).
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input Full-Uterus 3D MRI Volume<br/>Low resolution / noise / artifacts"] --> B["Brain Segmentation & Translation Estimation<br/>Network ฯ predicts mask & CoM t"]
B --> C["Margin-Adaptive Brain Cropping<br/>Extract brain ROI (40% margin)"]
C --> D["E(3)-Equivariant Feature Extraction<br/>Irreducible spherical tensor convolutions"]
D --> E["Symmetry-Aware Basis Regression<br/>Predict pseudovector ex & vectors ey, ez"]
E --> F["SVD Orthonormalization & Proper Rotation<br/>Enforce det(R)=1 to resolve chirality"]
F --> G["Real-Time Diagnostic Slice Prescription<br/>0.3s runtime for online scanner guidance"]
Key Designs¶
1. Brain Segmentation & Translation Estimation: Decoupling Translation to Eliminate Global Field-of-View Drift Full-uterus MRI scans exhibit large fields of view contaminated by maternal organs and surrounding tissues. Directly regressing coupled 6-DoF poses across large spaces leads to severe optimization instability. Addressing this issue, the origin of the canonical object coordinate frame is anchored at the brain center-of-mass. A compact 3D segmentation CNN \(\psi\) infers a voxel-wise occupancy probability \(\psi(x)\), and the translation vector \(t\) is extracted via a closed-form first-moment calculation: $\(t = \frac{\sum_{x \in X} x \psi(x)}{\sum_{x \in X} \psi(x)}\)$ The volume is then cropped with a 40% margin around the predicted mask. This decouples global positioning drift from angular orientation, allowing the rotation network to focus exclusively on local neuroanatomical structures.
2. E(3)-Equivariant Convolutional Representation: Enforcing Rigid Transformation Symmetries by Construction Standard CNNs are equivariant only to spatial translations; when subjected to arbitrary 3D head rotations, they must rely on exhaustive data augmentations and frequently overfit to limited training distributions. E(3)-Pose constructs an E(3)-CNN \(\phi\) operating on irreducible tensor fields. Convolutional kernels \(\kappa(x)\) are parameterized as steerable expansions over degree-\(j\) spherical harmonics \(Y_j\) and radial basis functions \(\varphi_m\). Under any Euclidean group transformation \(g = g_t \circ g_r \in \text{E}(3)\), network layers inherently satisfy the commutation relation: $\(\phi(f \circ g^{-1}) = \rho_h(g_r) \phi(f)\)$ By learning the linear combination coefficients \(w_{jm}\) rather than unconstrained 3D voxel weights, the network achieves exceptional sample efficiency, enabling zero-shot generalization from high-quality research scans to noisy, uncurated clinical navigators.
3. Symmetry-Aware Pseudovector Basis Regression: Eliminating Hemispheric Ambiguities via Parity Design In early gestational stages, fetal brains are largely spherical with near-perfect bilateral symmetry. When imaging artifacts or low SNR obliterate facial cues like the eyes, standard vector regressors struggle with conflicting gradient updates between mirrored poses. E(3)-Pose decomposes the target canonical basis \(h(R) = e_x \oplus e_y \oplus e_z\) into heterogeneous irreducible representations: the left-right axis \(e_x\) is parameterized as an order \(l=1\) even-parity pseudovector \(\rho^{l=1}_{\text{even}}(g_r) = (\det M(g_r)) M(g_r)\), while the orthogonal posterior-anterior \(e_y\) and inferior-superior \(e_z\) axes remain standard order \(l=1\) odd-parity vectors \(\rho^{l=1}_{\text{odd}}(g_r) = M(g_r)\). Under bilateral reflection \(g_\leftrightarrow\) across the sagittal plane (\(\det M(g_\leftrightarrow) = -1\) and spatial inversion \(-e_x\)), the double negation cancels out: $\(\rho_h(g_\leftrightarrow) h(R) = (\det M(g_\leftrightarrow))(-e_x) \oplus e_y \oplus e_z = e_x \oplus e_y \oplus e_z = h(R)\)$ This mathematically guarantees invariance to left-right reflection symmetry while preserving continuity across \(\text{SO}(3)\).
4. SVD Orthonormalization & Proper Rotation Alignment: Enforcing Physical SO(3) Manifold Geometry The unconstrained 9D tensor predicted by the neural network does not necessarily form an orthogonal matrix. At inference time, singular value decomposition \(U V^T = \text{SVD}(\hat{e}_x, \hat{e}_y, \hat{e}_z)\) is performed to project the predicted basis onto an orthonormal matrix. Because the pseudovector admits a dual solution space \(\{\hat{e}_x, -\hat{e}_x\}\), the proper rotation matrix is unambiguously resolved by enforcing a positive determinant (\(\det(M(\hat{R})) = +1\)): $\(\hat{R} = \det(U V^T) U V^T\)$ Unlike Gram-Schmidt orthogonalization which suffers from basis ordering bias, this SVD projection provides a continuous and robust mapping onto the proper rotation group \(\text{SO}(3)\) without additional computational burden.
Loss & Training¶
The network minimizes angular errors between predicted basis directions and ground-truth axes. To accommodate the inversion symmetry of the pseudovector \(\hat{e}_x\), its angular penalty is defined via a \(\pi\)-periodic absolute sine loss \(|\sin \theta_x|\), whereas normal vector predictions \(\hat{e}_y, \hat{e}_z\) utilize \(2\pi\)-periodic penalties \(|\sin(\theta/2)|\): $\(\mathcal{L}((e_x, e_y, e_z), (\hat{e}_x, \hat{e}_y, \hat{e}_z)) = \beta_x |\sin \theta_x| + \beta_y \left|\sin \frac{\theta_y}{2}\right| + \beta_z \left|\sin \frac{\theta_z}{2}\right|\)$ where \(\theta_k = \arccos(\hat{e}_k \cdot e_k)\). Furthermore, to bridge the domain gap between static research scans and interleaved clinical sequences, training volumes are augmented with synthetic spin history artifacts. The preceding excitation plane is modeled as a dark Gaussian attenuation band \(\tilde{f}(x) = f(x)(1 - \mathcal{N}((x - c_{\text{slice}})^T n_{\text{slice}}; 0, \sigma^2))\), exposing the model during training to realistic anatomical occlusions.
Key Experimental Results¶
Main Results¶
Models are trained on two research cohorts (Research-Fetal and publicly available dHCP) and evaluated across in-distribution test splits as well as two challenging clinical out-of-distribution test sets: Clinical-Young (60 younger fetuses, GA 18-23 weeks, spherical underdeveloped brains) and Navigators (47 time-series across 9 volunteers, 1210 total 3D volumes acquired with real interleaved 2D sequences, featuring 4-6mm voxels and real spin history artifacts). Evaluated metrics are geodesic rotation error (Rot. err., \(^\circ\)) and brain surface average absolute distance (AAD, mm).
| Model / Method | Research-Fetal Test | dHCP Test | Clinical-Young (Clinical OOD) | Navigators (Interleaved Navigator OOD) |
|---|---|---|---|---|
| No Training Required | ||||
| FireANTs (Registration) [40] | \(10.6 \pm 24.5^\circ\) / 5.7 mm | \(0.4 \pm 1.6^\circ\) / 0.6 mm | \(17.3 \pm 40.6^\circ\) / 4.1 mm | \(44.6 \pm 55.6^\circ\) / 19.1 mm |
| Fetal-Align (Landmarks) [37] | \(5.0 \pm 3.0^\circ\) / 3.0 mm | \(8.5 \pm 24.7^\circ\) / 3.5 mm | \(10.3 \pm 21.0^\circ\) / 3.2 mm | \(56.2 \pm 57.6^\circ\) / 24.1 mm |
| Trained on Research-Fetal | ||||
| EquiTrack [6] | \(11.3 \pm 13.1^\circ\) / 6.2 mm | \(7.1 \pm 5.0^\circ\) / 3.6 mm | \(18.9 \pm 29.0^\circ\) / 5.4 mm | \(44.7 \pm 48.7^\circ\) / 20.3 mm |
| 3DPose-Net [65] | \(19.8 \pm 15.0^\circ\) / 10.5 mm | \(24.0 \pm 19.2^\circ\) / 10.9 mm | \(39.2 \pm 36.8^\circ\) / 11.2 mm | \(65.0 \pm 47.8^\circ\) / 27.7 mm |
| 6DRep [20] | \(9.6 \pm 4.5^\circ\) / 5.3 mm | \(9.8 \pm 5.5^\circ\) / 4.9 mm | \(14.1 \pm 7.7^\circ\) / 4.6 mm | \(38.8 \pm 45.7^\circ\) / 17.2 mm |
| RbR [29] | \(7.6 \pm 3.9^\circ\) / 4.3 mm | \(8.6 \pm 5.0^\circ\) / 4.3 mm | \(11.2 \pm 4.6^\circ\) / 3.7 mm | \(22.6 \pm 24.1^\circ\) / 11.5 mm |
| E(3)-Pose (Ours) | \(\mathbf{5.1 \pm 2.6^\circ}\) / \(\mathbf{3.0 \pm 1.7\text{ mm}}\) | \(\mathbf{7.4 \pm 3.6^\circ}\) / \(\mathbf{3.7 \pm 1.8\text{ mm}}\) | \(\mathbf{9.1 \pm 4.7^\circ}\) / \(\mathbf{3.0 \pm 1.4\text{ mm}}\) | \(\mathbf{9.4 \pm 7.5^\circ}\) / \(\mathbf{6.3 \pm 4.0\text{ mm}}\) |
| Trained on dHCP | ||||
| EquiTrack [6] | \(29.5 \pm 43.5^\circ\) / 14.1 mm | \(13.6 \pm 36.7^\circ\) / 5.4 mm | \(48.3 \pm 62.1^\circ\) / 11.6 mm | \(59.0 \pm 57.5^\circ\) / 25.7 mm |
| 3DPose-Net [65] | \(33.5 \pm 26.6^\circ\) / 17.1 mm | \(21.2 \pm 27.4^\circ\) / 9.0 mm | \(51.5 \pm 37.8^\circ\) / 14.4 mm | \(93.9 \pm 55.8^\circ\) / 36.2 mm |
| 6DRep [20] | \(32.2 \pm 26.0^\circ\) / 17.3 mm | \(12.2 \pm 6.1^\circ\) / 5.9 mm | \(48.2 \pm 50.8^\circ\) / 12.4 mm | \(84.1 \pm 51.2^\circ\) / 34.0 mm |
| RbR [29] | \(15.4 \pm 18.3^\circ\) / 8.3 mm | \(8.0 \pm 4.0^\circ\) / 3.9 mm | \(50.9 \pm 48.4^\circ\) / 12.7 mm | \(77.2 \pm 53.6^\circ\) / 30.8 mm |
| E(3)-Pose (Ours) | \(\mathbf{5.7 \pm 3.6^\circ}\) / \(\mathbf{3.4 \pm 2.0\text{ mm}}\) | \(\mathbf{7.3 \pm 3.4^\circ}\) / \(\mathbf{3.7 \pm 1.7\text{ mm}}\) | \(\mathbf{12.1 \pm 6.8^\circ}\) / \(\mathbf{3.9 \pm 1.9\text{ mm}}\) | \(\mathbf{13.9 \pm 13.2^\circ}\) / \(\mathbf{7.9 \pm 5.6\text{ mm}}\) |
Ablation Study¶
The impact of equivariant convolutions, pseudovector symmetry modeling, basis completeness, loss formulations, and artifact augmentations are analyzed on Clinical-Young and Navigators.
| Ablation Configuration | Research-Fetal (Young) | Research-Fetal (Nav) | dHCP (Young) | dHCP (Nav) | Note |
|---|---|---|---|---|---|
| E(3)-Pose (Full model) | \(\mathbf{9.1 \pm 4.7^\circ}\) / 3.0 mm | \(\mathbf{9.4 \pm 7.5^\circ}\) / 6.3 mm | \(\mathbf{12.1 \pm 6.8^\circ}\) / 3.9 mm | \(\mathbf{13.9 \pm 13.2^\circ}\) / 7.9 mm | Full equivariant + pseudovector model |
| Standard CNN | \(10.3 \pm 5.4^\circ\) / 3.4 mm | \(18.0 \pm 19.3^\circ\) / 9.7 mm | \(53.3 \pm 55.8^\circ\) / 12.9 mm | \(80.2 \pm 52.7^\circ\) / 31.9 mm | Non-equivariant convolutions, severe overfitting |
| No pseudovector | \(10.4 \pm 4.0^\circ\) / 3.4 mm | \(10.9 \pm 11.1^\circ\) / 7.0 mm | \(12.1 \pm 7.7^\circ\) / 3.9 mm | \(20.6 \pm 26.1^\circ\) / 10.6 mm | Odd-parity vector along left-right axis |
| \(h(R) = e_y \oplus e_z\) | \(10.3 \pm 4.2^\circ\) / 3.4 mm | \(11.9 \pm 12.9^\circ\) / 7.3 mm | \(11.6 \pm 6.5^\circ\) / 3.7 mm | \(26.1 \pm 35.6^\circ\) / 12.3 mm | Regresses only 2 basis vectors |
| $ | \sin \frac{\theta_x}{2} | $ | \(8.8 \pm 3.6^\circ\) / 3.0 mm | \(12.1 \pm 11.1^\circ\) / 7.2 mm | \(11.1 \pm 5.3^\circ\) / 3.6 mm |
| Geodesic loss | \(9.2 \pm 4.7^\circ\) / 3.1 mm | \(13.5 \pm 20.0^\circ\) / 9.7 mm | \(30.9 \pm 53.4^\circ\) / 7.7 mm | \(27.8 \pm 42.5^\circ\) / 12.6 mm | Standard geodesic distance, unstable under ambiguity |
| No artifact augm. | \(12.0 \pm 21.1^\circ\) / 3.5 mm | \(16.2 \pm 18.6^\circ\) / 7.7 mm | \(18.4 \pm 21.1^\circ\) / 5.4 mm | \(31.9 \pm 36.7^\circ\) / 15.1 mm | Omits synthetic spin history artifact modeling |
Key Findings¶
- Unrivaled Cross-Domain Generalization: When trained on high-quality dHCP scans and transferred to clinical Navigators, all conventional regression and registration baselines catastrophically fail (rotation errors exceed \(77^\circ\)). In sharp contrast, E(3)-Pose maintains remarkable accuracy (\(13.9^\circ\)), demonstrating that hard geometric symmetries prevent distribution-shift collapse.
- Critical Synergy Between Equivariance and Symmetries: Ablating E(3) convolutions into standard CNNs increases dHCP-trained Navigators error from \(13.9^\circ\) to \(80.2^\circ\); similarly, omitting pseudovector symmetry modeling elevates error to \(20.6^\circ\), validating both inductive biases as necessary components for resolving symmetry ambiguities under occlusion.
- Fast Execution Enabling Prospective Slice Tracking: E(3)-Pose runs in 0.3s end-to-end (segmentation + pose estimation), compared to 1.8s for FireANTs. In closed-loop motion simulations under realistic trajectories, E(3)-Pose reduces slice obliqueness from \(31.2^\circ\) (motion-blind prescription) to \(9.8^\circ\), while reducing brain coverage gap ratios from 0.02 to 0.01.
Highlights & Insights¶
- Lie-Theoretic Anatomical Symmetry Formulation: Using an even-parity pseudovector to model bilateral reflection symmetry while maintaining a continuous representation on \(\text{SO}(3)\) represents an elegant unification of medical domain priors and group representation theory.
- Physics-Informed Spin History Augmentation: Synthesizing the specific dark-plane intensity attenuation of preceding slice excitations via Gaussian profiles successfully prepares the network for severe clinical landmark occlusions without requiring large clinical datasets.
- Real-Time Translation Feasibility: With a 0.3s runtime and demonstrated robustness across gestational ages and sensor resolutions, E(3)-Pose directly enables prospective slice plane tracking on clinical MRI scanner consoles.
Limitations & Future Work¶
- Breakdown Under Severe Asymmetrical Pathology: The framework assumes approximate bilateral brain symmetry. In sensitivity experiments simulating large unilateral lesions (e.g., 40-70mm brain tumors), the symmetry assumption degrades pose accuracy, requiring fallback to an unconstrained asymmetric model.
- Lack of Temporal Motion Priors and Uncertainty Estimation: Navigators are currently evaluated independently per volume. Future work could incorporate temporal filtering (e.g., SE(3) Kalman filtering or state-space models) and output pose uncertainty distributions over \(\text{SO}(3)\) to further stabilize clinical tracking.
Related Work & Insights¶
- vs Fetal-Align [37]: Fetal-Align relies on explicit landmark detection of the brain and eyes. When spin history artifacts intersect the eyes, landmark detection breaks down (Navigators error reaches \(56.2^\circ\)), whereas E(3)-Pose regresses pose globally from irreducible tensor fields without depending on landmark visibility.
- vs FireANTs [40] & EquiTrack [6]: Template-based registration methods require subject-specific templates and fail when artifact patterns distort image intensity correlations. Furthermore, FireANTs requires 1.8s runtime, which is too slow for real-time scanner steering. E(3)-Pose is template-free and runs in 0.3s.
- vs 6DRep [20] & RbR [29]: Standard unconstrained pose regression networks overfit to training distributions and collapse under out-of-distribution clinical shifts. E(3)-Pose regularizes the hypothesis space through E(3)-equivariance and pseudovector parameterization, ensuring stable generalization across domains.
Rating¶
- Novelty: โญโญโญโญโญ [Pioneering integration of E(3)-equivariant steerable convolutions and even-parity pseudovector symmetry parameterization for medical pose estimation]
- Experimental Thoroughness: โญโญโญโญโญ [Extensive cross-domain validation across research and clinical cohorts, rigorous ablations, lesion sensitivity tests, and prospective slice prescription simulations]
- Writing Quality: โญโญโญโญโญ [Exceptionally clear mathematical exposition, clean group representation proofs, and structured narrative linking clinical problems to geometric solutions]
- Value: โญโญโญโญโญ [Directly resolves a long-standing barrier in prospective motion compensation for fetal MRI, backed by open-source code and strong clinical translation potential]