XDen-1K: A Density Field Dataset of Real-World Objects

Conference: ECCV 2026
Paper: ECCV paper page
Area: Medical Imaging / X-Ray Density Field Dataset
Keywords: X-ray imaging, density field, real-world object dataset, physical property inference, volumetric segmentation

TL;DR

XDen-1K combines biplanar X-ray scans with part-level differentiable optimization to reconstruct volumetric density fields for 1,000 real-world objects (137 categories), paired with watertight meshes, part-level annotations, and real X-ray images, and defines three downstream tasks — a density-estimation benchmark, X-ray-conditioned volumetric segmentation, and center-of-mass-aware grasping, where adding its CoM prior raises AnyGrasp's success rate from 27.50% to 77.75%; reconstruction accuracy, however, is currently supported only by reprojection consistency and a small CT subset used as a partial reference.

Background & Motivation

Objects are more than their appearance. Beneath color and texture, internal composition — materials, thickness, voids, reinforcements, and mass distribution — determines how an object works, how it fails, and how it can be used. For robots these "invisible" properties matter most of all: where the mass concentrates (center of mass, moments of inertia), which regions are stiff or compliant, what forces and grasps are safe and effective, how an object will deform or fracture. The catch is that density is inherently volumetric and therefore "invisible" to ordinary RGB cameras; surface material estimation (SVBRDF, albedo, roughness) has advanced quickly on the back of large controlled visual datasets, while density has no real-world measured dataset at all.

Existing work consequently rests on synthetic assets or assumed physics. PIXIE regresses per-voxel material parameters from distilled feature volumes, GaussianProperty and PuGS assign physical attributes to 3D Gaussians via segmentation and vision-language priors, NeRF2Physics infers density from language-embedded feature fields, and PhysX-3D and 3DCoMPaT200 annotate synthetic assets with material categories — none of them carries real volumetric ground truth, so their predictions are hard to verify rigorously, and none exposes models to the multi-material mixtures and manufacturing artifacts inside real objects. The direct route to ground truth is CT, but a high-resolution CT scan typically takes 15–60 minutes and costs \(100–\)500 per object, so 1,000 objects imply hundreds to thousands of scanner-hours and a six-figure budget, with beam hardening further corrupting internal information in high-density materials. Cheaper options each have a hard flaw: a single X-ray integrates attenuation along the projection direction and loses depth entirely — a severely ill-posed inverse problem; sparse-view CT reconstruction methods such as SAX-NeRF and SCOPE still typically need dozens of projections; and methods that generate CT volumes from biplanar X-rays (e.g. X2CT-GAN) depend on large paired CT–X-ray datasets, which simply do not exist for non-medical objects.

XDen-1K's observation is that an object's 3D shape and part decomposition can already be recovered at high quality from multi-view RGB, and what is actually missing is one material parameter per part — exactly what two orthogonal X-ray projections can constrain. Core idea: take manually annotated part segmentation as a structural prior, assume each part is a single homogeneous material, build a differentiable X-ray forward model on the Beer-Lambert law, and thereby compress the ill-posed problem of recovering a dense density field from two X-rays into a low-dimensional optimization over one linear attenuation coefficient per part — yielding a physically grounded density-field dataset and benchmark for 1,000 real objects at low cost.

Method

Overall Architecture

XDen-1K provides three types of paired data for each object: a part-segmented watertight mesh, real captured biplanar X-ray images, and a reconstructed volumetric density field. The construction pipeline is standardized: calibrated multi-view RGB images first reconstruct a watertight mesh and measure real-world scale; a human-in-the-loop segmentation then decomposes the object into functionally meaningful components; a biplanar X-ray machine captures internal attenuation signals from two orthogonal directions; and the recovered shape priors finally enter an X-ray optimization process that estimates the volumetric density field. For a few objects with extremely complex interior geometry, the interior structure is manually modeled from the X-ray images, which clearly reveal it, as a supplementary prior for reconstruction.

The dataset covers 1,000 objects across 137 categories: from small hardware such as screws, flashlights, and keys, through medium household goods such as kettles, bicycle helmets, and power tools (about 20–30 cm), to larger items like cookware, sports equipment, and furniture; frequently encountered objects such as spoons, screwdrivers, scissors, and pliers are all present, with categories relatively evenly distributed. Acquisition uses a TaoImage Eagle Eye biplanar X-ray system at 100 kV tube voltage, 450 mA tube current, 280 ms exposure per shot, and 126 mAs total exposure. The project page and data live at xden-1k.github.io and Hugging Face (zhangjxx/XDen-1K).

Three premises are needed to read this pipeline correctly. First, the mesh and part segmentation are not open problems this paper tries to solve — they are used as structural priors, and reconstruction quality is directly bounded by segmentation quality, which is why manual annotation is kept. Second, the density field is piecewise constant: one density value per part, not a free per-voxel field — this is the representational price paid for reconstructing from two X-rays. Third, CT serves in this pipeline only as a partial reference, not ground truth: the relatively low tube voltage plus beam hardening produce streak artifacts in metallic parts (the air pump is one example) and deviations from the LAC-to-density linear conversion. On top of this, the paper defines three downstream subtasks: a density-estimation benchmark, X-ray-conditioned volumetric segmentation, and center-of-mass-aware robot manipulation.

Key Designs

1. Beer-Lambert forward model and LAC-to-density conversion: turning invisible density into an integrable, measurable physical quantity

X-rays penetrate matter, and beam intensity decays exponentially along the ray according to the Beer-Lambert law. For a ray \(r\) traversing an object, the detected intensity \(I\) relates to the incident intensity \(I_0\) as

\[I = I_0 \, e^{-\int_L \mu(x)\,dx},\]

and taking the logarithm gives the projection measurement \(p=\ln(I_0/I)=\int_L \mu(x)\,dx\) — the line integral of the linear attenuation coefficient (LAC) \(\mu\) along the ray. The log transform converts exponential attenuation into a linear integral form, the mathematical foundation of tomographic reconstruction. The link from LAC to density is the mass attenuation coefficient (MAC): \(\mu = \rho \cdot \frac{\mu}{\rho}(E)\), where the MAC is determined by atomic composition and photon energy. Per the NIST XCOM database, common low-atomic-number materials — water, polymers, biological tissues — converge to a MAC of about 0.17 cm²/g near 100 keV, so LAC is approximately linear in density and the reconstructed LAC converts directly to density. The paper's Table 1 lists representative values at 100 keV: water 1.00 g/cm³ corresponds to LAC 0.17 cm⁻¹, glass 2.40 to 0.432, aluminum 2.70 to 0.51, and air 0.0012 to 1.8×10⁻⁴. This conversion chain is the source of the dataset's physical meaning — and it also marks its boundary: the MAC varies by material, and materials departing from the "≈0.17" assumption (typically metals) need separate handling.

2. Piecewise-constant part-level field and differentiable biplanar optimization: collapsing a high-dimensional inverse problem into K scalars with a geometric prior

Two orthogonal X-rays are far from enough to densely invert a per-point LAC field, and XDen-1K's key reparameterization is to let known geometry carry all spatial variation: manual part segmentation partitions the object into K disjoint regions \(\{\Omega_k\}_{k=1}^K\), each assumed to be a single homogeneous material, so the attenuation field becomes

\[\mu(\mathbf{x}) = \sum_{k=1}^{K} \mu_k \, \mathbf{1}_{\Omega_k}(\mathbf{x}),\]

and, substituted into the forward model, the predicted intensity at detector pixel \(\mathbf{p}\) depends only on the ray's path length through each region:

\[\hat{I}_i(\mathbf{p}) = I_0 \exp\Big(\sum_{k=1}^{K} \mu_k \, \ell_{i,k}(\mathbf{p})\Big),\]

where the path length \(\ell_{i,k}(\mathbf{p})\) is precomputed from the part segmentation and projection geometry. Reconstruction then reduces to optimizing the K-dimensional vector \(\mu\) — jointly minimizing the discrepancy between simulated and measured projections from both biplanar views, with the background air fixed at \(\mu_{air}\); a differentiable forward-projection operator computes gradients automatically, and Adam iterates until convergence. The originally high-dimensional inverse problem of estimating a dense field \(\mu(x)\) becomes a low-dimensional problem with one parameter per part, the part masks supplying nearly all the regularization — with the side benefit that errors are localizable: when one part's density is wrong, it can be traced straight to its region.

3. Standardized multi-modal acquisition pipeline: putting 1,000 objects through one auditable pipeline

The dataset's reliability comes from process uniformity rather than per-object tuning. Every object follows the same order: calibrated multi-view RGB → watertight mesh reconstruction and real-scale measurement → human-in-the-loop part segmentation → biplanar X-ray capture → differentiable optimization for LAC → density conversion. Objects with very complex interiors get manually modeled internal geometry based on what the X-rays clearly reveal. Validation rests on two independent lines of evidence: reprojection consistency — the reconstructed field is re-rendered through the forward model and compared against real X-rays in the image domain via MAE and Pearson correlation — and high-resolution CT scans of a small subset as a partial volumetric reference. The trade-off is explicit: manual annotation secures current data quality, but the human in the loop also limits scalability, so the paper additionally reports a Fine variant that uses only external geometry and part segmentation with the optimization procedure unchanged, quantifying degradation under weaker priors.

Loss & Training

Density-field reconstruction itself is not a training problem: each object gets one independent low-dimensional optimization (K LAC parameters, Adam until convergence), with no learnable parameters shared across objects. The trained component is the downstream segmentation model X-Field, which extends PartField from surface segmentation to the volumetric domain, conditioning on biplanar X-rays to segment an object's 3D volume. On the backbone side, the input point cloud passes through PVCNN for per-point features, which are lifted into a triplane representation, and a transformer decoder predicts a queryable 3D feature field; on the X-ray side, an image encoder extracts features, and because X-ray appearance depends strongly on physical scale, the true scale \(\lambda\) is injected and FiLM modulates the encoded features into scale-aware ones, \(L_X^{\lambda}=\mathrm{FiLM}(L_X,\lambda)\), which are then fused with the triplane features through cross-attention to yield an X-ray-conditioned volumetric feature field. Query points are randomly sampled inside the object volume, a density MLP head regresses each point's density, and training is supervised by an \(\ell_2\) loss against synthetic ground-truth densities; to mitigate the missing-view ambiguity of having only two views, an auxiliary objective decodes an additional X-ray image from the features of the third plane and supervises it with an MSE loss, regularizing 3D feature learning. At inference, k-means clustering over the predicted densities at query points partitions the volume into regions of similar density, producing material-aware volumetric segmentation. X-Field is trained on a synthetic dataset built on PartNeXt with material properties synthesized to match the real-world collection, and never trains on XDen-1K's real data.

Key Experimental Results

Main Results

Dataset quality is validated from two complementary perspectives. Reprojection consistency (paper's Table 2): with the manual annotation prior, the LAT (lateral) reprojection MAE averages 0.778 (σ 0.191) and PA (frontal) 0.498 (σ 0.207), against a background-noise floor of 0.220 / 0.200, showing that the reconstructed field explains the real X-ray measurements; swapping the segmentation prior for X-Field predictions raises the errors to 1.121 / 0.724. Against CT references (paper's Table 3, small subset): the glue object reaches a CT density MAE of 0.181, the pencil 0.294, the razor 0.326, and the ballpoint pen 0.710, while the air pump, corrupted by metallic streak artifacts, is left without a CT number; on the reprojection side the razor is the worst (LAT MAE 1.443), and the ballpoint pen's LAT correlation falls as low as 0.065.

As a physical-reasoning benchmark (paper's Table 4), three representative methods are evaluated by voxel-wise density MAE, L1 center-of-mass distance (cm), and total-mass MAPE:

Method	Density MAE	CoM distance (cm)	Mass MAPE
NeRF2Physics	1.157	0.523	1.174
GaussianProperty	1.295	0.632	1.247
Pixie	1.228	0.547	0.713

Center-of-mass-aware grasping (paper's Table 6, Franka Emika Panda with a parallel gripper): the density-field prior guides grasp selection in AnyGrasp by picking the candidate whose gripper endpoints are closest to the predicted CoM, compared against no prior and against the NeRF2Physics CoM:

Config	Grasp success rate (%)	Joint-7 torque (N·m)	Mean torque (N·m)
AnyGrasp	27.50	0.521	9.645
+NeRF2Physics CoM	55.00	0.443	9.529
+XDen-1K CoM	77.75	0.435	9.497

The XDen-1K CoM prior lifts the success rate from 27.50% to 77.75% (+50.25 percentage points), 22.75 percentage points above the NeRF2Physics CoM, while wrist (Joint-7) and mean joint torques drop in step — grasps are both more stable and mechanically more efficient. The Pick/Place/Push qualitative experiments explain the mechanism: grasping away from the CoM enlarges the lever arm and torque until contact friction is overwhelmed; placement stability hinges on whether the CoM projection stays inside the support region; forces applied below the CoM favor translation, while forces above it generate a larger overturning moment.

Ablation Study

The contribution of X-ray conditioning to volumetric segmentation (paper's Table 5):

Method	Synthetic Data IoU	Real Data IoU
PartField	53.185	38.410
X-Field (X-ray conditioned)	72.645	49.251

Adding the biplanar X-ray condition raises IoU by 19.235 on synthetic data and 10.841 on real data, and the qualitative segmentations are visibly more consistent — the X-ray genuinely supplies the internal material structure that RGB cannot see.

An ablation over prior sources (paper's Table 2): replacing the manual segmentation prior with X-Field predictions degrades the LAT/PA reprojection MAE from 0.778 / 0.498 to 1.121 / 0.724, so the paper explicitly keeps manual labels as the current ground-truth pipeline and positions X-Field as a promising downstream application rather than a pipeline component. A scalability variant (Fine: removing the manually modeled interior structure, everything else unchanged, paper's Table 3) contradicts the paper's narrative: the text says removing the manual interior prior causes an expected accuracy drop, but taking LAT reprojection MAE as an example, 4 of the 5 objects are actually lower — air pump 1.064→0.741, razor 1.443→1.218, pencil 0.271→0.211, ballpoint pen 0.286→0.239, with only the glue degrading (0.298→0.341). This note keeps the original table numbers without correction, and the contradiction cannot be explained from the paper.

Key Findings

• Density-inference methods are still far from real measurements. All three methods sit at a density MAE of 1.15–1.30, while material densities in the dataset span from air at 0.0012 to aluminum at 2.70 g/cm³; even the best mass MAPE (Pixie, 0.713) leaves much to be desired. No method dominates: the LLM-based NeRF2Physics is best on density MAE and CoM distance, while the learning-based Pixie wins on mass MAPE — this is the first quantifiable gap after years without a real benchmark.
• X-ray is an immediate gain for segmentation but still immature as a reconstruction prior. The same modality, used as a segmentation condition, delivers 10–19 IoU points; yet when the X-Field-predicted prior feeds density reconstruction directly, the error is still clearly higher than manual labels (1.121 vs 0.778). "X-rays help" and "X-ray-driven segmentation can replace humans" are two different claims.
• The CoM predicts grasp success better than appearance. With the same AnyGrasp and only the CoM source changed, success rises from 27.50% (no prior) to 55.00% (NeRF2Physics CoM) to 77.75% (XDen-1K CoM) while torques fall monotonically — a low-dimensional physical quantity injected into a ready-made pipeline as candidate re-ranking pays off immediately.

Highlights & Insights

• A "one scalar per part" reparameterization replaces the dense inverse problem. The unknowns shrink from a per-voxel LAC field to a K-dimensional vector, with all spatial structure carried by the part masks; this is the fundamental reason two X-rays suffice for 3D density reconstruction, and it also makes reconstruction errors localizable to specific parts.
• The physical conversion chain is clear and auditable. Beer-Lambert → LAC → MAC → density, each step backed by NIST XCOM data; the density field is not a black-box regression output but a result traceable to measured physical quantities — with the failure regime for metals explicitly marked.
• X-ray is positioned as a conditioning modality for segmentation, not a CT substitute. X-Field does not try to generate a CT volume from two images; instead, X-ray features are FiLM-modulated by physical scale to condition the 3D feature field, with an auxiliary third-view decoding objective mitigating two-view ambiguity — a usage that avoids the paired-CT dependency of generative approaches.
• The CoM prior is plug-and-play. Adding one step to AnyGrasp's candidate ranking — pick the grasp whose endpoints are closest to the predicted CoM — nearly triples success without touching any network; low-dimensional physical quantities as priors offer exceptional cost-effectiveness, transferable directly to other manipulation tasks as candidate re-ranking.

Limitations & Future Work

• The piecewise-constant assumption limits expressiveness. One density per part cannot represent intra-part multi-material mixtures, gradients, or composite structures; the field's resolution is bounded by the granularity of the manual segmentation.
• The LAC-to-density linear conversion is unreliable for metals. Beam hardening and the relatively low tube voltage produce streak artifacts and deviations from the MAC≈0.17 cm²/g approximation in metallic parts (the air pump is the example); density fields for such objects should be used cautiously.
• The CT reference is not ground truth, and the subset is small. The authors state explicitly that CT serves only as a partial volumetric reference; the paper reports per-object CT comparison on only 5 objects, without explaining the subset's size criteria or selection standard, so overall reconstruction accuracy still leans on reprojection consistency as indirect support.
• Manual stages limit scalability. Human-in-the-loop segmentation and manual interior modeling are the source of current quality, and a clear gap remains between X-Field's automatic priors and manual labels (reprojection 1.121 vs 0.778); the authors envision that more robust segmentation models and standardized X-ray acquisition protocols can automate and scale the pipeline.
• Some objects reconstruct unstably. The ballpoint pen's LAT correlation is only 0.065, suggesting that for certain geometry/material combinations two views still under-constrain the problem; the paper offers no systematic analysis of failed objects.

Related Work & Insights

• vs CT / filtered backprojection: CT buys accurate reconstruction with dense angular projections at 15–60 minutes and \(100–\)500 per object; XDen-1K buys an approximate piecewise-constant density field with two exposures plus part-level geometric priors, trading per-voxel accuracy for thousand-object scalability.
• vs sparse-view CT reconstruction (SAX-NeRF, SCOPE): These methods use NeRF-style continuum modeling to fill in missing views but typically still need dozens of projections; XDen-1K pushes the count down to 2 at the price of depending on part-level geometric priors.
• vs biplanar generative CT (X2CT-GAN etc.): Generative and regression-based approaches need large paired CT–X-ray training data, which does not exist for non-medical objects; XDen-1K sidesteps the paired-data requirement with per-object physics optimization.
• vs visual physical-property inference (PIXIE, GaussianProperty, NeRF2Physics, PhysX-3D, 3DCoMPaT200): These methods predict materials or density from appearance, but both training and evaluation rest on synthetic assets and assumed properties; XDen-1K supplies the first real-measured volumetric ground truth, turning "looks plausible" into "can be scored".
• vs PartField: PartField segments parts from RGB point clouds alone; X-Field shows that adding X-ray conditioning consistently improves volumetric segmentation, and the X-ray dataset itself becomes a benchmark for future multimodal-guided segmentation.
• Reusable insight: When direct measurement is expensive, "strong structural prior + physical forward model + low-dimensional differentiable optimization" is a general recipe for substituting dense measurement; and low-dimensional physical quantities like the CoM can be injected into ready-made perception-grasping pipelines at low risk via candidate re-ranking.

Rating

• Novelty: ⭐⭐⭐⭐ (4/5). The first large-scale paired dataset of real biplanar X-rays and real-object density fields; the reconstruction framework is a pragmatic combination of established physics models and differentiable optimization rather than an algorithmic breakthrough.
• Experimental Thoroughness: ⭐⭐⭐ (3/5). There is dual validation by reprojection and CT, three downstream tasks, and real-robot grasping, but the CT subset is small and imperfect, only three benchmark methods are compared, and the Fine variant's numbers contradict the text.
• Writing Quality: ⭐⭐⭐ (3/5). The pipeline and acquisition parameters are fully documented and the three subtasks are clear, but Table 3's narrative conflicts with its numbers, and anomalous per-object metrics (the ballpoint pen's correlation) go undiscussed.
• Value: ⭐⭐⭐⭐ (4/5). It fills the missing real-world benchmark for physical property inference, with direct usability for embodied AI, physical simulation, and X-ray-based perception.