StructureGS: Structure-aware Gaussian Splatting for Articulated Object Reconstruction

Conference: ECCV 2026
Paper: ECCV Official
Area: 3D Vision
Keywords: Articulated Object Reconstruction, 3D Gaussian Splatting, Oriented Bounding Box, Structure-aware Loss, Separating Axis Theorem

TL;DR

StructureGS tackles the entanglement between geometry and motion caused by purely photometric supervision in articulated object reconstruction by introducing explicit oriented bounding boxes (OBBs) and canonical 3D Gaussian primitives, regularized via differentiable part fitting and SAT-based part contact losses to enable clean part boundaries, precise kinematics, and robust sparse-view reconstruction.

Background & Motivation

Reconstructing articulated objects with multiple movable parts is fundamental for embodied AI agents to understand spatial structures, recognize functional affordances, and plan physical interactions in everyday environments. Beyond static shape and appearance, articulated reconstruction requires recovering the underlying kinematic structure that governs relative rigid motions between components. However, this task involves jointly optimizing tightly coupled parameters: per-part geometry, view-dependent appearance, and inter-state rigid transformations.

Existing neural radiance field (NeRF) and 3D Gaussian Splatting (3DGS) pipelines rely predominantly on photometric reconstruction losses. Because observed image discrepancies between articulated states can stem either from part motions or local shape deformations, photometric supervision alone cannot resolve this inherent ambiguity. Consequently, optimizers frequently distort or scatter Gaussian primitives to compensate for motion errors, causing blurred part boundaries, floating geometric artifacts, and catastrophic part bleeding where moving parts merge into static bases.

To resolve this ambiguity, macroscopic physical constraints must be explicitly injected into the optimization process. A physically valid articulated object inherently exhibits two properties: spatial coherence (each part's geometry remains compact within its semantic boundary) and structural connectivity (adjacent articulated parts maintain persistent physical contact throughout motion). Core idea: equip each articulated part with an explicit oriented bounding box (OBB) enclosing canonical 3D Gaussians, and regularize the joint optimization via a part fitting loss for spatial compactness and a Separating Axis Theorem (SAT)-based contact loss for physical connectivity.

Method

Overall Architecture

StructureGS takes multi-view RGB images and calibrated camera parameters captured at an initial state (\(t=0\)) and an articulated state (\(t=1\)) to reconstruct \(K\) rigid parts \(\mathcal{P} = \{P_0, P_1, \dots, P_{K-1}\}\) (where \(P_0\) is the static base and \(P_{k \ge 1}\) are movable parts). Each part is parameterized as a set of 3D Gaussian primitives defined in a canonical coordinate frame \([-1, 1]^3\), which are transformed into world space via an affine transformation at \(t=0\) and subsequently mapped by a rigid transformation at \(t=1\). During training, differentiable photometric rendering loss, part fitting loss, SAT-based inter-part contact loss, and extent regularization jointly optimize the canonical Gaussians, OBB parameters, and articulation transformations.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Two-state Multi-view RGB Inputs<br/>t=0 Initial & t=1 Articulated"] --> B["Canonical Gaussian Primitive Parameterization<br/>[-1, 1]^3 Local Frame + OBB Scale Coupling"]
    B --> C["Two-state Spatial Transformations<br/>Initial Affine T0 & Articulation Rigid T1"]
    C --> D["Joint Structural and Photometric Optimization<br/>3DGS Rendering Loss + Structural Guidance Losses"]
    D --> E["Part Fitting Loss<br/>Coverage Enforcement + Margin Penalty"]
    D --> F["SAT Part Contact Loss<br/>15 Candidate Separating Axes Projections"]
    D --> G["Adaptive Extent Regularization<br/>Softmax Axis-weighted Expansion"]
    E & F & G --> H["Disentangled 3DGS & Kinematics Output<br/>Clean Part Boundaries + Accurate Joint Parameters"]

Key Designs

1. Canonical Gaussians and OBB Parameterization: Decoupling Intrinsic Geometry from Extrinsic Pose Conventional 3DGS optimizes Gaussian centers and covariances directly in world coordinates, leading to spatial dispersion across distinct parts. StructureGS decouples each part \(P_k\) into canonical Gaussian primitives \(\mathcal{G}_k = \{(\mathbf{x}_j, \alpha_j, \mathbf{q}_j, \mathbf{c}_j)\}_{j=1}^{N_k}\) bounded in \([-1, 1]^3\) via \(\boldsymbol{\mu}_j = \tanh(\mathbf{x}_j)\), paired with transformation parameters \(\{T_0^k, T_1^k\}\). The initial affine transformation \(T_0^k = (\mathbf{s}_0^k, R_0^k, \mathbf{t}_0^k)\) maps the canonical space to world space at \(t=0\):

\[\boldsymbol{\mu}_j^{k,0} = \mathbf{s}_0^k \odot (R_0^k \boldsymbol{\mu}_j) + \mathbf{t}_0^k\]

where \(\mathbf{s}_0^k \in \mathbb{R}_{>0}^3\) specifies the per-axis half-extents of the OBB, \(R_0^k \in SO(3)\) dictates its orientation, and \(\mathbf{t}_0^k \in \mathbb{R}^3\) denotes its center. The articulated state at \(t=1\) applies an additional rigid transformation \(T_1^k = (R_1^k, \mathbf{t}_1^k)\). By tying OBB dimensions directly to the affine scaling factors, the spatial volume and global pose of each part are explicitly parameterized, while Gaussian primitives focus on recovering high-frequency local surface geometry.

2. Part Fitting Loss: Enforcing Spatial Coherence via Differentiable Projections To ensure that each OBB tightly encloses its corresponding part without drifting or inflating, StructureGS introduces a differentiable part fitting loss based on binary structure images. Points uniformly sampled inside the canonical cube are transformed and rendered as binary masks \(\mathbf{I}_{\text{struct}}^k\) for individual parts and \(\mathbf{I}_{\text{struct}}^{\text{all}}\) for their union. The fitting objective combines two complementary terms:

\[\mathcal{L}_{\text{fit}} = \lambda_{\text{coverage}} \mathcal{L}_{\text{coverage}} + \lambda_{\text{margin}} \mathcal{L}_{\text{margin}}\]

The coverage loss \(\mathcal{L}_{\text{coverage}} = \mathbf{I}_{\text{struct}}^{\text{all}} \odot (1 - \mathbf{M})\) penalizes foreground object mask regions \(\mathbf{M}\) that are not covered by any part OBB. Conversely, the margin loss \(\mathcal{L}_{\text{margin}} = \frac{1}{K}\sum_{k=0}^{K-1} \mathbf{I}_{\text{struct}}^k \odot (1 - \mathbf{M})\) penalizes OBB regions projecting outside the actual object silhouette. The competition between coverage and margin drives OBBs to tightly fit the underlying geometry, effectively suppressing floating Gaussians and boundary bleeding.

3. Part Contact Loss via Separating Axis Theorem (SAT): Modeling Physical Connectivity In valid articulated assemblies, adjacent parts remain in persistent physical contact. StructureGS evaluates inter-part distances directly between OBBs using the Separating Axis Theorem (SAT). For two OBBs \(P_i\) and \(P_j\), SAT inspects 15 candidate separating axes (the 6 face normals and 9 cross products of their edge directions). Projecting both boxes onto normalized axis \(\hat{\mathbf{a}}\) yields center distance \(|c_i - c_j|\) and extent sum \((r_i + r_j)\), defining a signed separation distance \(d_{\mathbf{a}} = |c_i - c_j| - (r_i + r_j)\). The inter-part distance is the supremum over all candidate axes:

\[d(P_i, P_j) = \max_{\mathbf{a} \in \mathcal{A}} d_{\mathbf{a}}\]

A positive value indicates spatial separation, while non-positive values denote contact or penetration. The contact loss penalizes only separation across adjacent pairs \(\mathcal{E}\):

\[\mathcal{L}_{\text{contact}} = \lambda_{\text{contact}} \sum_{(i,j) \in \mathcal{E}} (\max(0, d(P_i, P_j)))^2\]

Because realistic articulated mechanisms (such as a drawer nested in a cabinet) allow bounding box overlapping, the one-sided hinge penalty enforces connectivity without prohibiting valid volumetric penetration.

4. Extent Regularization: Mitigating Under-coverage in Low-saliency Directions To prevent bounding box scales from getting trapped in local minima—where only high-gradient silhouette regions are bounded—the method incorporates an axis-weighted scale regularizer:

\[\mathcal{L}_{\text{ext}} = -\lambda_{\text{ext}} \sum_{k=0}^{K-1} \sum_{i=1}^{3} w_k^{(i)} s_{0,i}^k, \quad \text{where } \mathbf{w}_k = \text{softmax}(\boldsymbol{\beta}_k)\]

Learnable parameters \(\boldsymbol{\beta}_k \in \mathbb{R}^3\) adaptively assign weights to under-covered axes whose enlargement does not elevate the total objective, encouraging OBBs to expand along elongated or textureless components.

Loss & Training

The overall optimization objective spans both temporal states:

\[\mathcal{L}_{\text{total}} = \sum_{t \in \{0, 1\}} \left( \mathcal{L}_{\text{image}}^t + \mathcal{L}_{\text{fit}}^t + \mathcal{L}_{\text{contact}}^t \right) + \mathcal{L}_{\text{ext}}\]

Training runs for 10,000 iterations using the Adam optimizer with progressive scheduling. Structural losses (\(\mathcal{L}_{\text{fit}}, \mathcal{L}_{\text{contact}}, \mathcal{L}_{\text{ext}}\)) are active from step 0 to quickly establish coarse part separation. The photometric loss \(\mathcal{L}_{\text{image}}\) is activated at step 400. Once the structural layout has stabilized, \(\mathcal{L}_{\text{contact}}\) and \(\mathcal{L}_{\text{ext}}\) are disabled at step 1500 to allow unconstrained convergence of fine non-convex geometry.

Key Experimental Results

Main Results

On the PARIS benchmark (10 objects, single movable part, covering revolute and prismatic joints), StructureGS is evaluated across dense (100 views) and sparse (10 views) regimes against PARIS, ScrewSplat, and ArticulatedGS (Table 2 from the paper):

Setting	Metric	PARIS (ICCV'23)	ScrewSplat (2025)	ArticulatedGS (CVPR'25)	Ours (StructureGS)
PARIS Full Views (100)	Joint Angle Error Ang Err (°) ↓	5.635	7.787	0.091	0.088
	Joint Position Error Pos Err ↓	0.385	0.139	0.018	0.001
	Revolute Geodesic Dist Geo Dist (°) ↓	85.52	32.20	0.761	0.066
	Prismatic Motion Error Geo Dist (m) ↓	1.062	0.200	0.200	0.001
	Static Chamfer Dist CD-s (\(\times 10^3\)) ↓	5.528	38.948	2.119	1.558
	Movable Chamfer Dist CD-m (\(\times 10^3\)) ↓	67.935	32.202	1.585	1.400
	Whole-object Chamfer Dist CD-w (\(\times 10^3\)) ↓	5.895	13.935	1.908	0.919
	Novel View PSNR (dB) ↑	29.324	31.377	37.336	43.390
	Novel View SSIM ↑	0.948	0.965	0.985	0.995
	Novel View LPIPS ↓	0.097	0.046	0.039	0.014
PARIS Sparse Views (10)	Joint Angle Error Ang Err (°) ↓	32.840	11.537	8.553	0.701
	Joint Position Error Pos Err ↓	0.162	0.181	0.126	0.007
	Movable Chamfer Dist CD-m (\(\times 10^3\)) ↓	164.93	133.50	62.700	1.982
	Whole-object Chamfer Dist CD-w (\(\times 10^3\)) ↓	84.632	31.892	19.847	1.320
	Novel View PSNR (dB) ↑	22.363	28.135	23.320	33.264

On the multi-part DTA benchmark (Fridge and Storage scenes with 2 moving parts, Table 3 from the paper): - In the Fridge scene, StructureGS reduces whole-object CD-w from 8.369 (ScrewSplat) to 4.699, and part 0 angular error from 15.12° to 0.445°. - In the Storage scene, ScrewSplat merges the two moving parts together, leading to a disastrous 89.86° angular error on part 1, whereas StructureGS maintains a low 0.542° error while lowering CD-s from 136.1 to 3.336.

Ablation Study

Ablation on structural loss terms on the PARIS benchmark (Fig. 8 from the paper):

Configuration	Ang Err (°) ↓	Pos Err ↓	Geo Dist (Rot) ↓	CD-s (\(\times 10^3\)) ↓	CD-m (\(\times 10^3\)) ↓	CD-w (\(\times 10^3\)) ↓	Note
Photometric only \(\mathcal{L}_{\text{image}}\)	5.60	0.11	7.65	6.08	31.80	3.90	Severe geometry distortion & motion ambiguity
\(+ \mathcal{L}_{\text{fit}} + \mathcal{L}_{\text{contact}}\)	1.71	0.00	0.06	1.89	31.40	0.98	Drastic drop in joint & static errors
\(+ \mathcal{L}_{\text{ext}}\) (Full model)	0.08	0.00	0.06	1.56	1.40	0.91	Recovers full extent on Blade, drops CD-m to 1.40

Key Findings

• Robustness in Sparse Regimes: Under 10 training views, baseline ArticulatedGS degrades severely (CD-w surges from 1.908 to 19.847; angular error climbs from 0.091° to 8.553°). StructureGS remains remarkably stable, retaining a CD-w of 1.320 and angular error of 0.701°.
• Pitfall of Whole-object Metrics: Whole-object Chamfer distance frequently conceals massive part-level corruption. Baselines shift dynamic geometry into the static base, yielding acceptable CD-w while part CD-m exceeds 100 in sparse setups.
• Computational Efficiency: StructureGS converges in ~13 minutes per scene on full views, comparable to ArticulatedGS (13 min) and ScrewSplat (15 min), and significantly faster than PARIS (40 min).

Highlights & Insights

• OBB as an Effective Structural Proxy: Computing continuous distances and contact directly on dense meshes or implicit SDFs is computationally prohibitive and prone to local minima. OBBs provide a compact 9-DoF representation that seamlessly interacts with classical Separating Axis Theorem (SAT) for fast, differentiable physical contact regularization.
• Handling High-ratio Non-cuboidal Shapes: Even though OBBs are convex cuboids, Gaussian primitives are free to deform within the box. On the foldchair object where the OBB contains 97% empty space (\(r = 1 - V_{\text{mesh}}/V_{\text{OBB}} = 0.97\)), the method accurately reconstructs intricate thin tubular legs without geometric clipping.
• Stabilizing Sparse-view Ambiguity: In real-world robotic scenarios with sparse views, visual evidence is insufficient to constrain 3D geometry along camera rays. The geometric bounding and contact losses constrain the optimization search space to physically viable configurations.

Limitations & Future Work

• Geometric Collapse under Subtle Motion Cues: When moving parts induce very little visual or silhouette change (e.g., small rotations on the top door of an oven), the dynamic Gaussians can collapse into adjacent static structures.
• Convexity Assumption in Complex Contacts: SAT strictly applies to convex shapes. For articulated parts with concave interlocking joints or complex interlocking grooves, a single OBB cannot capture multi-surface contacts accurately. Future extensions could incorporate hierarchical OBB trees or convex decomposition.
• Integration with Visual Foundation Models: Incorporating 2D semantic feature fields (such as DINOv2 or SAM masks) could provide semantic segmentation priors to prevent part merging in textureless scenarios.

Related Work & Insights

• vs PARIS (ICCV 2023): PARIS models parts with implicit NeRF fields and explicit screw motion, requiring 40 minutes per scene and suffering from cloudy artifacts near part boundaries. StructureGS leverages 3DGS for 3× faster training and sharper physical boundaries.
• vs ArticulatedGS (CVPR 2025): ArticulatedGS uses deformation-aware spectral clustering of Gaussians. In sparse-view or low-texture setups, spectral clustering degrades severely (8.553° error). StructureGS enforces structural priors via OBBs and SAT contact, achieving 0.701° error under 10 views.
• vs ScrewSplat (2025): ScrewSplat performs soft clustering using screw motion parameters, which often merges multiple adjacent moving parts into one component in multi-part scenes. StructureGS enforces explicit geometric boundaries and separation penalties, isolating distinct kinematic parts cleanly.

Rating

• Novelty: ⭐⭐⭐⭐ [Elegant integration of classical Separating Axis Theorem (SAT) and OBB geometry into 3D Gaussian Splatting for articulated object reconstruction]
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive evaluations across PARIS, DTA-Multi, and challenging real-world smartphone captures across dense and sparse regimes]
• Writing Quality: ⭐⭐⭐⭐⭐ [Clear motivation, well-structured methodology, and crisp figures]
• Value: ⭐⭐⭐⭐⭐ [Highly valuable for embodied AI and digital twin pipelines requiring robust 3D articulation models from sparse visual observations]

Related Papers

• [ECCV 2026] AnchorSplat: Fast and Structure Consistent Detail Synthesis for Gaussian Splatting
• [ICLR 2026] Signal Structure-Aware Gaussian Splatting for Large-Scale Scene Reconstruction
• [CVPR 2026] Part\(^{2}\)GS: Part-aware Modeling of Articulated Objects using 3D Gaussian Splatting
• [ECCV 2026] Interaction-Aware 4D Gaussian Splatting for Dynamic Hand-Object Interaction Reconstruction
• [ECCV 2026] Geometry-Aware Style Transfer in 3D Gaussian Splatting