UBone3D: Physics-Rectified Conditional Flow Matching for Anatomical 3D Shape Completion from Ultrasound¶
Conference: ECCV 2026
Paper: ECCV Official
Project Page: https://answerrtx.github.io/UBone3D-Proj/
Area: 3D Vision / Medical Imaging
Keywords: 3D Shape Completion, Conditional Flow Matching, Physics-Guided Generation, Ultrasound Bone Reconstruction, Neural Surrogates
TL;DR¶
UBone3D decouples anatomical plausibility from imaging physics consistency by combining a high-fidelity CT-trained conditional flow matching generative prior (BoneFM) with a differentiable acoustic physics proxy (USimNet), dynamically steering sampling trajectories at test time via explicit analytical gradients to achieve accurate anatomical 3D shape completion from severely artifact-laden and partial ultrasound point clouds.
Background & Motivation¶
Three-dimensional ultrasound (US) represents a safe, portable, and ionizing radiation-free modality holding tremendous potential for longitudinal clinical tracking, particularly in adolescent idiopathic scoliosis (AIS) where pediatric patients frequently undergo repetitive CT or X-ray examinations over multiple years. In contemporary clinical workflows, 3D skeletal reconstructions are typically compiled by aggregating tracked 2D B-mode ultrasound frames across spatial coordinates and interpolating empty space via Voxel Nearest Neighbor (VNN). However, due to strong acoustic reflections and severe acoustic impedance mismatch at cortical bone interfaces, acoustic waves cannot penetrate the cortical shell. Coupled with restricted probe apertures (FOV), beamwidth effects, reverberations, and acoustic shadowing, upstream bone segmentations yield point clouds that are extremely sparse, incomplete, and contaminated with deterministic acoustic artifacts.
Existing general-purpose 3D point cloud completion networks are overwhelmingly trained on synthetic, noise-free benchmarks such as ShapeNet, possessing zero domain awareness of ultrasound physics. Conversely, specialized ultrasound bone reconstruction methods either presume clean superficial boundaries—an assumption violated by in-vivo scans—or rely on variational autoencoders (VAEs) that directly force uncurated, chaotic physical artifacts into a compact latent bottleneck, triggering severe shape collapse when transferred from synthetic training distributions to in-vivo scans. The core challenge in reconstructing complete bone anatomy lies in the fact that the discrepancy between ultrasound observations and full anatomy is not merely stochastic noise, but structured, deterministic physics: cortical thickening, axial streak elongations, and deep signal dropouts systematically warp the point cloud geometry.
The critical insight of this work is to explicitly decouple global anatomical shape completion from acoustic physics constraints, harmonizing both forces during continuous generative flow integration. Core idea: parameterize an optimal-transport conditional flow matching anatomy prior (BoneFM) on clean CT data to safeguard global skeletal topology, while developing a differentiable neural physics proxy (USimNet) trained on realistic acoustic simulations to inject explicit observation-consistency rectification gradients into the flow trajectory during test-time inference.
Method¶
Overall Architecture¶
UBone3D formulates observation-guided shape completion from a partial, artifact-laden ultrasound point cloud \(y\) to a complete anatomical shape \(x^*\) as a probabilistic Bayesian decomposition \(p(x|y) \propto p(y|x)p(x)\). The architecture consists of two decoupled components: a generative shape prior BoneFM modeling the anatomical manifold \(p(x)\), and a differentiable physics proxy USimNet modeling the forward imaging degradation \(p(y|x)\). Starting from pure Gaussian noise, the continuous-time trajectory is numerically solved via second-order Heun integration, where BoneFM supplies the conditional velocity field and USimNet delivers analytical observation-consistency gradients evaluated on the predicted clean anatomy.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Artifact-Laden Partial US Point Cloud y and Noise x(0)"] --> B["Conditional Anatomy Shape Prior BoneFM<br/>FiLM-modulated PointNet++ predicts conditional velocity"]
B --> C["Differentiable Physics Proxy USimNet<br/>Forward projection onto US domain & directed CD gradients"]
C --> D["Physics-Rectified Flow Matching Inference<br/>Time-scheduled gradient steering + observation anchoring"]
D --> E["Anatomically Consistent Point Cloud PAC*"]
Key Designs¶
1. Conditional Anatomy Shape Prior BoneFM: Straight-Line Flow on Clean Skeletal Manifolds
Traditional generative diffusion frameworks demand hundreds of iterative steps along curved sampling trajectories, rendering online test-time optimization computationally prohibitive. BoneFM parameterizes a conditional velocity field under the Optimal Transport Flow Matching (OT-FM) framework, building straight interpolation trajectories \(x(t) = t x_1 + (1-t)x_0\) from base Gaussian noise \(p_0(x) = \mathcal{N}(0, \mathbf{I})\) to clean anatomical point clouds \(p_1(x) \approx p_{\text{AC}}(x)\) derived from CT. The velocity network utilizes a PointNet++ backbone modulated via Feature-wise Linear Modulation (FiLM), injecting global context vectors encoded from partial ultrasound observations \(y\) and sinusoidal time embeddings \(t\). The model is supervised by matching the constant transport velocity:
To unlock Classifier-Free Guidance (CFG) at inference, the conditional input \(y\) is randomly dropped with probability \(p_{\text{drop}} = 0.2\) during training, yielding an adjustable mechanism to calibrate condition strength versus generative realism.
2. Differentiable Physics Proxy USimNet: Neural Surrogate for Non-Differentiable Acoustic Formation
Accurate ultrasound simulation libraries (such as ray-based PyMUST) explicitly compute reflection physics, beam clipping, attenuation, and acoustic shadowing from mesh surfaces, but they are strictly non-differentiable and computationally too heavy to compute gradients inside ODE integration loops. The authors generate a large paired training set \(P_{\text{phys-full}}\) incorporating sector FOV clipping, acoustic shadowing (Shadow), exponential depth attenuation (Atten), cortical thickening (Thick), axial streak duplication (Streak), and anti-grid Poisson jitter. USimNet \(f_\phi\) is designed as a lightweight PointNet++ encoder-decoder surrogate that learns the forward operator mapping complete bone point cloud \(x\) to ultrasound-style point cloud \(\hat{x}_{\text{phys}} = f_\phi(x)\), supervised via symmetric Chamfer Distance:
By emulating complex wave-tissue interactions within a differentiable neural network, USimNet serves as a fast, exact gradient provider at test time.
3. Physics-Rectified Flow Matching Inference: Time-Scheduled Steering with Observation Anchoring
During inference given an unseen partial observation \(y\), Heun's second-order solver integrates the velocity field across \(K = 35\) discrete steps. At intermediate state \(x(t)\), the network first extrapolates the clean target shape \(\hat{x}_1(x(t), t) = x(t) + (1-t) v_{\text{cfg}}(x(t), t, y)\), and queries USimNet to evaluate the directed Chamfer Distance against real observation \(y\), thereby avoiding penalties on truly unobserved anatomical regions:
The rectified velocity incorporates observation gradients via automatic differentiation:
To safeguard the emergent global topology from noisy gradients during early diffusion phases, a truncated polynomial schedule is applied: \(\lambda(t) = 0\) for \(t < t_{\text{enable}} = 0.5\), and scales smoothly as \(\lambda_{\text{max}} t^\gamma\) (\(\gamma=3, \lambda_{\text{max}}=0.6\)) thereafter. Furthermore, an Observation Anchoring strategy fixes a subsampled core of \(N_{\text{vis}}\) observed points onto their deterministic OT-FM path \(x_{\text{vis}}(t) = t y_{\text{vis}} + (1-t) z_{\text{vis}}\), ensuring that completed structures perfectly respect known physical boundaries without drift.
Key Experimental Results¶
Main Results¶
UBone3D was evaluated on the Spine1K-PC simulation dataset (5,502 training vertebrae, 1,373 test vertebrae under unseen probe incident angles) and a clinical in-vivo dataset comprising 24 vertebrae (20 thoracic and 4 lumbar) scanned from healthy volunteers using a handheld Clarius ultrasound transducer under a zero-shot setting.
Quantitative comparisons on the full physical degradation test set (\(P_{\text{phys-full, test}}\)) under pure geometric training (\(P_{\text{geo}}\)) versus full physical simulation training (\(P_{\text{phys-full}}\)):
| Method | Training Setting | L1 CD \(\times 10^3\) ↓ | EMD \(\times 10^3\) ↓ | F-score@1% ↑ |
|---|---|---|---|---|
| PoinTr (Transformer) | Pure Geometry \(P_{\text{geo}}\) | 70.960 | 107.701 | 0.043 |
| SVDFormer (Transformer) | Pure Geometry \(P_{\text{geo}}\) | 42.106 | 207.075 | 0.033 |
| PCDreamer (Diffusion) | Pure Geometry \(P_{\text{geo}}\) | 74.281 | 194.532 | 0.032 |
| Gafencu et al. (VAE) | Pure Geometry \(P_{\text{geo}}\) | 41.648 | 250.939 | 0.056 |
| SSM-Net* (Statistical Shape Model) | Pure Geometry \(P_{\text{geo}}\) | 79.907 | 182.267 | 0.040 |
| UBone3D (Ours) | Pure Geometry \(P_{\text{geo}}\) | 56.294 | 105.292 | 0.065 |
| PoinTr (Transformer) | Full Physics \(P_{\text{phys-full}}\) | 32.936 | 78.404 | 0.055 |
| SVDFormer (Transformer) | Full Physics \(P_{\text{phys-full}}\) | 25.463 | 127.965 | 0.046 |
| PCDreamer (Diffusion) | Full Physics \(P_{\text{phys-full}}\) | 59.035 | 92.456 | 0.025 |
| Gafencu et al. (VAE) | Full Physics \(P_{\text{phys-full}}\) | 29.091 | 256.953 | 0.063 |
| UBone3D (Ours) | Full Physics \(P_{\text{phys-full}}\) | 28.161 | 74.298 | 0.079 |
Clinical fidelity evaluation on zero-shot in-vivo acquisitions reporting Laminae Distance Errors (LDE, mm) against raw partial observations and VNN-interpolated volumetric reference standards:
| Benchmark Reference | SVDFormer† | SSM-Net* | UBone3D (Ours) |
|---|---|---|---|
| Relative to Raw Input LDE (mm)¹ ↓ | 0.838 | 1.253 | 0.819 |
| Relative to VNN Reference LDE (mm)² ↓ | 3.937 | 3.382 | 1.293 |
Ablation Study¶
Ablations dissect the fidelity tiers of physics surrogate USimNet and compare the generative backbone against diffusion baselines.
Impact of physics proxy USimNet configurations on test-set completion:
| Physics Proxy Setting | L1 CD \(\times 10^3\) ↓ | EMD \(\times 10^3\) ↓ | F-score@1% ↑ |
|---|---|---|---|
| w/o USimNet (Pure Generative Prior) | 49.196 | 96.673 | 0.064 |
| Simple Proxy (Ray casting + spinous cut) | 44.287 | 87.271 | 0.065 |
| Full Proxy (Complete acoustic physics, Ours) | 28.161 | 74.298 | 0.079 |
Comparison of generative shape prior architectures (BoneDiff based on DDIM vs. BoneFM based on OT-FM):
| Generative Backbone | Training Input | L1 CD \(\times 10^3\) ↓ | EMD \(\times 10^3\) ↓ | F-score@1% ↑ |
|---|---|---|---|---|
| BoneDiff (DDIM, 2× integration steps) | \(P_{\text{geo}}\) | 59.274 | 105.920 | 0.068 |
| BoneFM (OT-FM, Ours) | \(P_{\text{geo}}\) | 56.294 | 105.292 | 0.065 |
| BoneDiff (DDIM, 2× integration steps) | \(P_{\text{phys-full}}\) | 42.597 | 75.987 | 0.071 |
| BoneFM (OT-FM, Ours) | \(P_{\text{phys-full}}\) | 28.161 | 74.298 | 0.079 |
Key Findings¶
- Differentiable physics rectification is critical for eliminating acoustic artifacts: Activating the full USimNet proxy slashes the Chamfer Distance by 42.8% (from 49.196 to 28.161) and Earth Mover's Distance by 23.1% relative to using the standalone unrectified generative prior, confirming that geometric shape priors alone cannot resolve deceptive artifact boundaries without an explicit forward forward model.
- Guidance weight exhibits a distinct U-shaped stability curve: Varying \(\lambda_{\text{max}}\) demonstrates that under-guidance (\(\lambda \to 0\)) fails to correct physical distortions, while over-guidance (\(\lambda > 1.0\)) disrupts anatomical validity due to gradient overshoot. An optimal equilibrium is achieved at \(\lambda_{\text{max}} = 0.6\).
- Flow matching surpasses diffusion in both accuracy and speed: BoneFM operates in 35 steps compared to 70 steps required by BoneDiff (DDIM), while substantially lowering CD under full physics conditions (28.161 vs. 42.597), proving the advantage of straight optimal-transport paths.
- Superior sim-to-real generalization: In zero-shot clinical in-vivo scans, baseline models produced severe anatomical collapses (SVDFormer VNN reference error reaching 3.937 mm), whereas UBone3D maintains high structural integrity and tight anatomical alignment (1.293 mm error).
Highlights & Insights¶
- Decoupled generative architecture: Rather than relying on fragile end-to-end black-box networks, UBone3D decouples global anatomy recovery (guaranteed by large-scale CT priors) from acoustic physics consistency (enforced via a neural surrogate), achieving a robust balance between generative completion and observation fidelity.
- Overcoming non-differentiability via neural surrogacy: Ray-tracing ultrasound simulators cannot provide backward analytical gradients. Training a lightweight PointNet++ surrogate network unlocks efficient, differentiable test-time guidance without sacrificing realistic wave interaction modeling.
- Safe gradient steering via time scheduling: Initiating physical guidance only after the coarse topology stabilizes (\(t \ge 0.5\)) and anchoring visible input points guarantees numerical stability and prevents early-stage topological fractures.
Limitations & Future Work¶
- Residual discrepancy on in-vivo scans: While outperforming baselines, in-vivo reconstructions still display minor structural softness compared to simulated testing, largely attributable to unmodeled clinical factors such as probe contact pressure, soft tissue acoustic heterogeneity, and patient respiratory motion.
- Extending isolated vertebrae to full-spine articulated graphs: Current inference processes individual vertebrae independently. Future research can incorporate kinematic constraints and inter-vertebral joint priors to enable simultaneous multi-vertebra tracking and global spinal curve estimation.
Related Work & Insights¶
- In contrast to linear parametric statistical shape models (SSM-Net) that lack expressive capacity for atypical deformities, and VAE-based models prone to posterior collapse under real-world acoustic noise, conditional flow matching provides a continuous, highly expressive dynamical system perfectly suited for test-time physical steering.
- The physics-rectified generative paradigm pioneered here offers an effective template for solving ill-posed inverse problems across other low-dose, high-artifact clinical modalities, including ultra-low-field MRI and sparse-view computed tomography.
Rating¶
- Novelty: 4.5 / 5.0 (First work integrating differentiable acoustic surrogates with conditional flow matching for 3D ultrasound bone completion)
- Experimental Thoroughness: 4.5 / 5.0 (Thorough evaluations spanning multi-tier acoustic ablations, unseen angle testing, and zero-shot in-vivo validation)
- Writing Quality: 4.5 / 5.0 (Clear formulation, elegant decoupling narrative, and well-structured empirical analysis)
- Value: 4.5 / 5.0 (Substantial clinical promise for reducing cumulative ionizing radiation in pediatric spinal monitoring)