Scalable Cross-embodiment Dexterous Grasping via Morphology-Prior Diffusion¶

Conference: ECCV 2026
Paper: ECCV Original
Area: Robotics
Keywords: Cross-embodiment Dexterous Grasping, 3D Assembly, Morphology-Prior Diffusion, Zero-shot Transfer, Inverse Kinematics Optimization

TL;DR¶

By reformulating cross-embodiment dexterous grasping as a 3D assembly problem of rigid links, SOMO couples a shared 3D shape-aware geometric encoder with a classifier-free morphology-prior diffusion model, achieving scalable synthesis across diverse hands and annotation-free zero-shot transfer using only robot URDFs.

Background & Motivation¶

Multi-fingered dexterous grasping is a foundational capability for versatile robotic manipulation. However, real-world robotic hands exhibit vast morphological diversity, ranging from simple two-fingered parallel grippers to highly articulated anthropomorphic hands with more than twenty degrees of freedom. Building dedicated models for each specific embodiment quickly becomes unsustainable. While cross-embodiment solutions are urgently needed, current paradigms face severe scalability barriers. Object-oriented methods—relying on predicted contact maps or affordance fields—exhibit natural cross-hand transferability, yet recovering valid hand poses requires solving complex inverse kinematics (IK) under penetration and joint-limit constraints. This optimization is computationally expensive, struggles to converge on intricate geometries, and deteriorates rapidly when depth sensors produce partial, occluded point clouds. Conversely, interaction-oriented approaches such as D(R, O) and T(R, O) model joint spatial relationships via distance matrices or heterogeneous graphs. However, they either suffer quadratic memory scaling or bind generation to fixed-topology graph structures, while area-weighted link sampling inevitably washes out fine-grained geometric details on small yet critical parts like fingertips.

More fundamentally, existing frameworks universally depend on paired object-grasp interaction annotations for every single embodiment. Curating such datasets across numerous hands requires prohibitive simulation or teleoperation pipelines, creating a data collection bottleneck that scales linearly with the number of supported morphologies. Alternative attempts to map disparate hands onto a single canonical hand space (such as UniMorphGrasp's mapping onto a 24-DoF ShadowHand) rely on manual joint correspondence and full grasp supervision on every target hand, failing to accommodate non-anthropomorphic hands. How to construct a unified generative framework that natively handles hands with arbitrary topologies and joint numbers—without hand-specific branches or manual joint mappings, and enabling zero-shot synthesis for novel hands given only their URDF descriptions—remains a major open challenge in robotic grasping.

To address these limitations, this paper reformulates cross-embodiment grasping through the lens of conditional 3D part assembly. Because every rigid link of a robot hand can be treated as a constituent 3D piece to arrange around the target object, a grasp can be cleanly parameterized as a variable-length collection of link-level \(SE(3)\) spatial poses. By embedding both link meshes and object point clouds into a shared geometric latent space via a frozen 3D VAE, and repurposing classifier-free guidance so that the unconditional path models kinematically reachable configurations via forward-kinematic self-exploration, the model completely detaches generation from hand-specific kinematic trees. Core Idea: reformulate cross-embodiment dexterous grasping as 3D assembly of variable-length rigid links, leveraging a shared 3D geometric embedding and a classifier-free morphology-prior diffusion model to enable annotation-free zero-shot grasp synthesis directly from robot URDFs.

Method¶

Overall Architecture¶

The SOMO (Scalable Morphology-Prior Diffusion) framework disentangles grasp synthesis into two distinct phases: geometry-driven per-link \(SE(3)\) pose generation via a shared diffusion transformer, followed by physical configuration recovery through post-joint inverse kinematics optimization. Given an object surface point cloud \(O\) and a target robot hand \(e\) specified by its URDF, the robot is parsed into \(L_e\) rigid links. Both the object point cloud and each link's surface mesh are mapped into a common geometric latent space using a frozen pretrained 3D shape-aware encoder. A shared transformer denoiser predicts denoised \(SE(3)\) link poses conditioned on object geometry. Finally, a fast Levenberg-Marquardt IK optimizer projects the predicted link transformations onto kinematically feasible, joint-limited hand configurations.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Object Point Cloud & Hand URDF"] --> B["3D Shape-Aware Encoding<br/>Shared VAE for links & object"]
    B --> C["Rigid Link Tokenization<br/>Assemble SE(3) pose & morphology tokens"]
    C --> D["Morphology-Prior Diffusion Denoising<br/>Self-attn for kinematics + cross-attn for object"]
    D --> E["Predict Per-Link SE(3) Poses"]
    E --> F["Post Joint Optimization<br/>Pyroki IK & joint-limit projection"]
    F --> G["Output: Executable Hand Grasp Pose"]

Key Designs¶

1. 3D Assembly Formulation & Uniform Link Tokenization: Transcending Kinematic Topologies and Fingertip Dilution Conventional generative frameworks parameterize grasp actions in hand-specific joint spaces, which ties the network architecture to a fixed kinematic graph and prevents direct generalization to hands with different DoFs or link layouts. SOMO sidesteps this constraint by treating grasping as a 3D part reassembly task. Parsing any URDF into \(L_e\) rigid constituent links, SOMO directly predicts an \(SE(3)\) spatial pose \(T_l = [t_l, r_l] \in \mathbb{R}^6\) for each link \(l\), parameterized by a normalized 3D translation \(t_l\) and an axis-angle rotation \(r_l\). Furthermore, to rectify the flaw in prior works like T(R, O)—where area-weighted point sampling allocates minimal points to small yet functionally critical components like fingertips—SOMO adopts uniform surface sampling: exactly \(K=512\) points with surface normals are sampled from each link mesh. The resulting geometry is embedded into a fixed morphology vector \(e_l \in \mathbb{R}^{d_{link}}\) and concatenated with noisy poses at diffusion step \(m\):

\[V_R^{(m)}[l] = \left[ \tilde{\mathbf{t}}_l^{(m)},\; \tilde{\mathbf{r}}_l^{(m)},\; \mathbf{e}_l \right]\]

Because a standard transformer denoiser processes robot tokens as a variable-length sequence, SOMO accommodates diverse embodiments—from 5-link parallel grippers to 21-link multi-fingered hands—within a single weight-shared model without any structural changes.

2. Shared 3D Shape-Aware Geometric Encoding: Grounding Cross-Embodiment Surface Complementarity In physical assembly, parts fit together based on the local geometric complementarity of their contact surfaces. To allow cross-attention layers in the denoiser to reason about how a link's physical contour complements the local geometry of an object, both entities must reside in a unified geometric feature space. SOMO employs a frozen 3D VAE encoder \(\Phi\) from TripoSG, pretrained on millions of 3D assets. The object point cloud with normals is normalized and encoded into \(P\) patch tokens \(V_O \in \mathbb{R}^{P \times d_O}\). Concurrently, each robot link's normalized point cloud is processed by the exact same frozen encoder \(\Phi\) and projected via a two-layer MLP \(\phi_{link}\) into link embedding \(e_l\). Because robot meshes are constant, all link latents are pre-computed and cached at initialization. This unified embedding guarantees that novel robot links are seamlessly mapped into the same representational space as known embodiments, enabling cross-attention to transfer contact reasoning zero-shot across hands.

3. Morphology-Prior Learning via FK Exploration & Guided Diffusion: Eliminating Demonstration Bottlenecks To bypass the costly requirement of acquiring paired object-grasp annotations for every new robot hand, SOMO reinterprets classifier-free guidance (CFG). Rather than simply dropping condition tokens to model an uninformative marginal distribution, SOMO dedicates the unconditional branch to learning a pure "Morphology-Prior Path." For hands without grasp annotations, training configurations are synthesized through forward-kinematic (FK) self-exploration: a hand-wide coordinated open/close blend factor \(\alpha \sim (0.1, 0.9)\) establishes a base pose \(q_{base}\), followed by Gaussian jitter clamped strictly within the URDF joint limits \([q_{min}, q_{max}]\) and a randomized 3D root orientation. During training on this path, object tokens are zeroed (\(V_O = 0\)) and cross-attention is bypassed; the denoiser relies exclusively on self-attention across links to learn what valid, unscattered hand configurations look like for that specific morphology.

The joint training objective combines the object-conditioned loss \(\mathcal{L}_{GT}\) on annotated data with the morphology-prior loss \(\mathcal{L}_{no-GT}\):

\[\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{GT}} + \lambda \mathcal{L}_{\text{no-GT}}\]

At test time, an unseen hand with zero grasp demonstrations performs guided sampling:

\[\hat{\boldsymbol{\epsilon}}_{\text{cfg}} = (1 - s) \hat{\boldsymbol{\epsilon}}_\theta(\mathbf{p}^{(m)}, m \mid \emptyset) + s \hat{\boldsymbol{\epsilon}}_\theta(\mathbf{p}^{(m)}, m \mid \mathbf{V}_O)\]

The unconditional morphology prior anchors the predicted links to physically viable, kinematically realizable poses, while the cross-embodiment object conditional path steers the hand toward functional contact with the object. Scaling guidance to \(s=1.25\) provides the optimal trade-off, enabling zero-shot grasp synthesis using only a URDF file.

Loss & Training¶

The denoiser backbone consists of an 8-layer Transformer (\(d=384\)) modulated by AdaLN-Zero on sinusoidal diffusion timestep embeddings. The forward diffusion process applies a standard 1000-step linear noise schedule (\(\beta \in [10^{-4}, 0.02]\)) over the 6-DoF \(SE(3)\) pose components \(p_l\). The noise loss is averaged over valid links per embodiment:

\[\mathcal{L}_{GT} = \mathbb{E}_{m, \boldsymbol{\epsilon}} \left[ \frac{1}{L_e} \sum_{l=1}^{L_e} \|\boldsymbol{\epsilon}_l - \hat{\boldsymbol{\epsilon}}_{\theta,l}\|^2 \right]\]

Condition dropout is set to \(p_{uncond}=0.1\). The model is optimized using Adam with a learning rate of \(10^{-4}\) and gradient clipping at 1.0. At inference, a 100-step DDIM sampler generates \(SE(3)\) link poses, which are converted to joint angles via the GPU-accelerated Pyroki library running Levenberg-Marquardt IK optimization under joint limits within a few milliseconds.

Key Experimental Results¶

Main Results¶

Evaluations are conducted on the extended CMapDataset with 48 training objects and 10 held-out test objects from ContactDB and YCB. Physical grasp stability is validated inside Isaac Gym by applying six directional perturbation forces; a grasp is successful if object displacement remains under 2 cm. Comparisons span 3-hand, 4-hand, and 7-hand multi-embodiment training setups, measuring Success Rate (Suc. %), Grasp Diversity (Div.), and Inference Efficiency (Eff. in seconds per grasp).

Benchmark & Method	Allegro (Suc./Eff.)	Barrett (Suc./Eff.)	ShadowHand (Suc./Eff.)	LEAPHand (Suc./Eff.)	Robotiq (Suc./Eff.)	EZGripper (Suc./Eff.)	Avg. Suc.↑	Avg. Eff.↓
3-Hand Pool
D(R, O) [33]	92.10% / 0.19s	87.50% / 0.42s	83.00% / 0.78s	-	-	-	87.53%	0.46s
T(R, O) [6]	93.50% / 0.03s	91.90% / 0.03s	94.49% / 0.03s	-	-	-	93.90%	0.03s
SOMO (Ours)	94.80% / 0.02s	88.80% / 0.02s	93.50% / 0.01s	-	-	-	92.37%	0.02s
4-Hand Pool
D(R, O) [33]	89.90% / 0.20s	85.00% / 0.40s	83.70% / 0.73s	20.10% / 0.34s	-	-	69.69%	0.42s
T(R, O) [6]	97.20% / 0.03s	86.20% / 0.03s	94.00% / 0.03s	36.30% / 0.03s	-	-	78.42%	0.03s
SOMO (Ours)	94.30% / 0.02s	90.00% / 0.01s	91.30% / 0.02s	55.40% / 0.02s	-	-	82.75%	0.02s
7-Hand Pool (Base)
D(R, O) [33]	92.70% / 0.21s	87.20% / 0.41s	73.10% / 0.72s	42.00% / 0.34s	12.90% / 0.30s	78.80% / 0.32s	64.45%	0.39s
T(R, O) [6]	94.40% / 0.03s	91.30% / 0.03s	89.70% / 0.03s	59.00% / 0.03s	8.60% / 0.04s	39.30% / 0.04s	63.72%	0.03s
SOMO (Ours)	97.60% / 0.02s	92.80% / 0.01s	91.00% / 0.02s	65.30% / 0.02s	29.70% / 0.01s	97.20% / 0.01s	78.93%	0.02s
7-Hand Pool (Extended)
D(R, O) [33]	89.30% / 0.22s	85.50% / 0.40s	78.30% / 0.73s	40.77% / 0.34s	10.10% / 0.30s	79.50% / 0.31s	63.91%	0.38s
T(R, O) [6]	96.00% / 0.03s	84.60% / 0.03s	88.00% / 0.03s	62.00% / 0.03s	10.60% / 0.04s	54.00% / 0.03s	65.87%	0.03s
SOMO (Ours)	91.20% / 0.02s	90.80% / 0.01s	94.80% / 0.02s	67.70% / 0.02s	44.20% / 0.02s	97.40% / 0.01s	81.02% / 0.02s

Ablation Study¶

The ablation investigates architectural components on the 4-hand benchmark, alongside the sensitivity of guidance scale \(s\) on held-out annotation-free hands.

Config	Params	Variable Token	Link Enc.	Object Enc.	Avg. Suc.↑	Avg. Div.↑	Avg. Eff.↓
T(R, O) Baseline	28.06M	\(\times\)	BPS	VQ-VAE	78.42%	0.41	0.03s
Variant I (Fixed graph + Dual VAE)	23.80M	\(\times\)	3D VAE	3D VAE	80.58%	0.41	0.01s
Variant II (Variable tokens + BPS)	23.80M	\(\checkmark\)	BPS	VQ-VAE	79.38%	0.40	0.02s
Variant III (Variable tokens + Link VAE)	23.80M	\(\checkmark\)	3D VAE	VQ-VAE	81.63%	0.42	0.01s
SOMO Full Model	23.80M	\(\checkmark\)	3D VAE	3D VAE	82.75%	0.42	0.01s

In the guidance scale analysis for annotation-free hands (evaluating held-out Barrett): - At \(s=0.75\), success rate is 80.8% (insufficient object conditioning). - At \(s=1.00\), success rate reaches 81.4%. - At \(s=1.25\), success rate peaks at 83.1% (optimal amplification of transferred contact cues balanced by the morphology prior). - At \(s \ge 1.50\), performance degrades (\(s=1.5\) yields 81.7%; \(s=2.0\) drops to 79.9%), as excessive extrapolation perturbs physical validity.

Furthermore, leave-one-out experiments where a target hand receives zero grasp demonstrations reveal that SOMO achieves 93.30% on Allegro (vs. D(R, O)'s 43.20%), 80.40% on Barrett (vs. 60.80%), 78.00% on ShadowHand (vs. 30.80%), and 32.40% on the challenging LEAPHand (vs. 2.80%), outperforming the pretraining baseline by an average margin of over 36 percentage points.

Key Findings¶

Superior Cross-Embodiment Resilience: Expanding the hand pool from 3 to 7 hands causes baseline success rates to collapse by 30.18 percentage points for T(R, O) (93.90% \(\rightarrow\) 63.72%) and 23.08 pp for D(R, O). In contrast, SOMO drops by only 13.44 pp and recovers to 81.02% with extended data, proving that it transforms morphological diversity into scaling gains rather than negative cross-hand interference.
Unified Geometric Latent Space is Critical: Upgrading link embeddings from BPS to the pretrained 3D VAE provides a +2.25 pp improvement, and unifying the object encoding brings an additional +1.12 pp (+3.37 pp cumulative), highlighting the importance of measuring link-object complementarity within an identical feature manifold.
IK is Not the Failure Bottleneck: A 1000-sample breakdown demonstrates that Pyroki IK optimization budget exhaustion accounts for very few failures (2.2% on Allegro, 0.0% on Shadow/Barrett/EZGripper, and 10.9% on LEAP). The vast majority of failures stem from diffusion outputs generating kinematically solvable but physically weak grasp contacts.

Highlights & Insights¶

3D Assembly Perspective Simplifies Kinematics: By bypassing tangled hierarchical joint kinematics and framing grasping as rigid part assembly in \(SE(3)\), SOMO decouples generation from hand-specific topology, realizing an embodiment-agnostic architecture.
Repurposing CFG for Zero-Shot Morphology Priors: Reconceptualizing the unconditional diffusion path as an unsupervised morphology distribution fitted via forward-kinematic self-exploration allows novel hands to grasp stably without a single human or simulated demonstration.
High Efficiency for Real-Time Deployment: With only 23.8M parameters, 100-step DDIM, and fast Pyroki IK, SOMO generates a grasp in 0.02 seconds—10 to 40 times faster than distance-matrix baselines, making it well-suited for reactive closed-loop manipulation.

Limitations & Future Work¶

Isolated Object Assumption: Current experiments focus on isolated target objects. Dense clutter and severe self-occlusions introduce complex multi-body collision constraints that demand scene-level spatial reasoning.
Extremely Underactuated Hands: For hands with high mechanical joint coupling and minimal actuators, the mapping from unconstrained \(SE(3)\) link poses back to joint space can become ill-conditioned, occasionally causing IK convergence failures.
Sensitivity to URDF Link Granularity: Fragmenting a robot into an excessively fine mesh tree increases the token sequence length and introduces part-assignment ambiguities among interchangeable links (e.g., identical fingertips).

vs D(R, O) [33]: D(R, O) computes dense hand-object point-to-point distance matrices, resulting in quadratic memory costs and extreme sensitivity to pose initialization; SOMO tokenizes links into compact \(SE(3)\) representations, generating grasps 10-40\(\times\) faster with far better scaling resilience.
vs T(R, O) [6]: T(R, O) binds its denoiser to a fixed-node heterogeneous graph and area-weighted BPS link features, discarding fingertip geometry and failing across different link counts; SOMO leverages a variable-length transformer and uniform surface sampling, naturally accommodating arbitrary topologies.
vs UniMorphGrasp [36]: UniMorphGrasp relies on a manual mapping to a 24-DoF ShadowHand canonical space and requires full supervision on all hands; SOMO operates natively on URDF links with no canonical projection, supporting true zero-shot transfer via morphology-prior diffusion.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ [Brilliant framing of grasping as 3D link assembly combined with FK self-explored morphology-prior CFG]
Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive evaluations across 6+1 diverse hands, progressive scaling pools, leave-one-out transfer, and real-world robot trials]
Writing Quality: ⭐⭐⭐⭐⭐ [Clear motivation, self-contained mathematical formulation, and rigorous failure analysis]
Value: ⭐⭐⭐⭐⭐ [Eliminates the per-embodiment annotation bottleneck, providing a highly scalable paradigm for cross-embodiment robotics]