Kinematics-Agnostic 3D Human Motion Prediction via Equivariant Latent Diffusion¶

Conference: ECCV 2026
Paper: ECCV Official
Code: https://ceveloper.github.io/publications/equifusion/
Area: Human Understanding (Note: Currently located under image_generation; suggested reclassification: human_understanding or 3d_vision)
Keywords: Stochastic Human Motion Prediction, Kinematics-Agnostic, Permutation Equivariance, Latent Diffusion Model, Skeleton Generalization

TL;DR¶

EquiFusion introduces the first kinematics-agnostic stochastic human motion prediction framework by explicitly incorporating the skeleton adjacency matrix as input and establishing an end-to-end permutation-equivariant latent diffusion model, bypassing skeleton-specific hard-coding and retargeting errors to achieve zero-shot generalization across unseen body topologies and occluded limbs.

Background & Motivation¶

Forecasting plausible future human motions from brief historical observations is a fundamental prerequisite for embodied spatial AI and human-robot collaboration. Given the intrinsic ambiguity of human intent, the community has shifted from deterministic trajectory extrapolation toward Stochastic Human Motion Prediction (SHMP), which models probability distributions over diverse, physically valid futures. However, a pervasive bottleneck blocking foundation models in this domain is kinematics rigidity: motion capture ecosystems (such as marker-based Human3.6M, parametric SMPL/AMASS, and wearable ARIA/Nymeria) utilize vastly disparate skeletal topologies with distinct joint counts, naming conventions, and kinematic hierarchies. Existing diffusion and neural predictors hard-code the joint count $J$ directly into network weight matrices or collapse joints into fixed feature channels, binding every model instance exclusively to a single dataset.

Historically, reconciling heterogeneous datasets relied on motion retargeting as a preprocessing step. Because differing skeletons form non-homeomorphic graph structures (e.g., 17-joint H36M lacks the foot joints present in 22-joint AMASS), retargeting cannot establish bijective mappings. This leads to substantial cumulative geometric errors (often exceeding 20 mm) alongside severe distribution shifts that degrade downstream generative fidelity. Furthermore, in real-world computer vision pipelines, partial observations and limb occlusions (partial kinematics) are ubiquitous. Hard-coded networks completely collapse when encountering missing joints, while prior deterministic heuristics require cumbersome manual training for specific occlusion patterns.

To overcome this impasse, a model's learnable parameters must be strictly decoupled from the cardinality of the input joint set (i.e., $|\Theta|$ must remain constant regardless of $J$). The authors prove that enforcing joint-ordering permutation equivariance provides the formal mathematical foundation to satisfy this decoupling criterion. Core Idea: explicitly treat the skeleton connectivity adjacency matrix as an input parameter, construct an end-to-end permutation-equivariant latent diffusion model combining localized graph convolutions with joint-wise attention, and couple it with rigid bone-direction parameterization to enable zero-shot future prediction across arbitrary unseen kinematics and missing limbs without motion retargeting.

Method¶

Overall Architecture¶

EquiFusion reformulates stochastic motion forecasting as a permutation-equivariant latent diffusion process conditioned on the skeleton graph $K = (V_\tau, E)$ through its adjacency matrix $A \in \mathbb{R}^{J \times J}$. Given historical observations $X \in \mathbb{R}^{T_P \times J \times 3}$ across $T_P$ frames and the kinematic connectivity $A$, the objective is to predict diverse future motion trajectories $\tilde{Y} \in \mathbb{R}^{T_F \times J \times 3}$ spanning $T_F$ frames.

The pipeline comprises two primary stages: first, a deterministic permutation-equivariant Transformer autoencoder transforms the motion sequence into a compact latent space $z \in \mathbb{R}^{J \times L}$ where the first dimension preserves joint-level equivariance; second, an equivariant latent diffusion denoiser takes past motion latents $z_{\text{past}}$ and the skeleton adjacency matrix $A$ as structural conditions, iteratively reversing Gaussian noise into future motion latents before decoding back to Euclidean space. Every internal operation strictly satisfies permutation equivariance, rendering the network independent of joint ordering and cardinality.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    In["Input: Past Motion X and Adjacency Matrix A"] --> D1["Permutation-Equivariant Operators<br/>Degree-normalized graph conv + joint-wise self-attention"]
    D1 --> D2["Equivariant Latent Diffusion Architecture<br/>Topology-conditioned autoencoder and latent denoiser"]
    D2 --> D3["Bone Direction Motion Parametrization<br/>Relative parent-limb vectors with inference length rescaling"]
    D3 --> Out["Output: Diverse Plausible Future Trajectories Ŷ"]

Key Designs¶

1. Permutation-Equivariant Operators: Decoupling Joint Cardinality from Network Weights

Conventional motion forecasting networks allocate joint-dependent parameter blocks $W \in \mathbb{R}^{J \times J}$ or learn independent weight sets $W_j$ per joint, implicitly assuming fixed joint positions and counts. EquiFusion establishes that supporting arbitrary kinematic chains requires operations $o(X)$ to remain permutation-equivariant under any joint permutation matrix $P \in \mathbb{R}^{J \times J}$, satisfying $P \cdot o(X) = o(P X)$. Locally, the architecture implements a degree-normalized graph convolution:

\[c(Z, A) = D^{-1} A Z G_1 + Z G_0 + b\]

where $Z \in \mathbb{R}^{J \times F_i}$ denotes node representations, $D$ is the diagonal degree matrix of adjacency $A$, and $G_0, G_1 \in \mathbb{R}^{F_i \times F_o}$ represent shared weight transformations for the joint itself and its 1-hop graph neighbors, respectively, paired with shared bias $b \in \mathbb{R}^{F_o}$. This operation preserves equivariance with respect to both input trajectories and the adjacency matrix $A$. To incorporate global non-local skeletal correlations, multi-head self-attention treats joints as the token dimension, computing query, key, and value vectors through equivariant graph convolutions while omitting positional encodings, guaranteeing parameter invariance to $|K|$.

2. Equivariant Latent Diffusion Architecture: Explicit Topology Conditioning and Latent Denoising

Executing diffusion directly in high-dimensional coordinate spaces incurs heavy computational costs and risks structural drift. EquiFusion incorporates a deterministic latent autoencoder built with joint-equivariant Transformer blocks, encoding temporal motion sequences $M \in \mathbb{R}^{T \times J \times 3}$ into latent states $z \in \mathbb{R}^{J \times L}$. Within this latent domain, a diffusion denoiser conditions on the explicit adjacency matrix $A$ and past motion context $z_{\text{past}} = e(X)$. Formally, the complete generative pipeline satisfies:

\[f(P X, P A P^\top) = P f(X, A)\]

Because prior noise variables $\epsilon \sim \mathcal{N}(0, I)$ are sampled i.i.d. per joint token, permutation equivariance holds sample-wise under aligned noise seeds and distributionally across sampling runs. Crucially, the denoiser receives the graph structure dynamically, enabling immediate zero-shot evaluation on completely novel skeleton topologies or partial subgraphs without structural re-engineering.

3. Bone Direction Motion Parametrization: Relative Vectors Ensuring Constant Limb Lengths

Standard 3D Cartesian coordinates suffer from bone stretching and temporal jitter under multi-step diffusion sampling. Conversely, rotation angles (e.g., axis-angle or quaternions) require a canonical rest pose and suffer from rotational gimbal locks or optimization singularities. EquiFusion employs a robust middle-ground parameterization by representing joints via normalized bone direction vectors relative to their kinematic parent:

\[L_t^i = \frac{\text{joints}_t^i - \text{joints}_t^{\text{parent}(i)}}{\|\text{joints}_t^i - \text{joints}_t^{\text{parent}(i)}\|_2}\]

During diffusion generation, the network predicts the directional evolution of each limb. At test time, absolute joint positions are reconstructed by rescaling predicted directions using the subject's ground-truth bone lengths observed in the history frames $X$. This design mathematically guarantees 0.00% bone stretching artifacts. For occluded limbs missing from the past observation, the system gracefully defaults to dataset-wide average bone lengths as an anatomical prior.

Loss & Training¶

The framework is optimized in two disjoint stages: 1. Autoencoder Optimization: Minimizes trajectory reconstruction error alongside directional consistency: $$\mathcal{L}_{\text{AE}} = \|M - \hat{M}\|_2^2 + \lambda_{\text{dir}} \|L - \hat{L}\|_2^2$$ 2. Latent Diffusion Optimization: With the autoencoder frozen, the equivariant denoiser minimizes standard diffusion mean squared error: $$\mathcal{L}_{\text{diff}} = \mathbb{E}_{t, z_0, \epsilon \sim \mathcal{N}(0, I)} \left[ \|\epsilon - \epsilon_\theta(z_t, t, z_{\text{past}}, A)\|_2^2 \right]$$ Thanks to the kinematics-agnostic formulation, EquiFusion trains across heterogeneous datasets (e.g., AMASS and Nymeria simultaneously) within identical training batches without motion retargeting.

Key Experimental Results¶

Main Results¶

In the zero-shot kinematics benchmark on Human3.6M ($K_H$, 17 joints), models are trained exclusively on AMASS ($K_A$, 22 joints) or multi-dataset (AMASS + Nymeria). Baseline methods require preprocessing with Holden et al. motion retargeting, whereas EquiFusion evaluates natively out-of-the-box.

Method	Novel Kinematics	uADE (cm) ↓	uFDE (cm) ↓	MAE (deg) ↓	uMMA (cm) ↓	uMMF (cm) ↓	uAPD (m) ↑	CMD ↓	FID ↓	Stretch str (%) ↓	Jitter jit (%) ↓
ZeroVel	✓	11.77	17.88	6.753	13.74	18.56	0.000	22.822	-	0.00	0.00
TPK + Retargeting	✗	13.81	16.13	22.276	14.60	16.22	1.469	10.051	3.773	19.55	0.46
DLow + Retargeting	✗	12.71	14.80	21.887	13.60	14.97	2.060	9.204	2.875	20.26	0.53
GSPS + Retargeting	✗	9.29	11.91	8.107	10.85	12.35	2.069	7.409	1.735	11.51	0.38
DivSamp + Retargeting	✗	9.27	12.61	8.374	11.42	13.27	4.210	47.783	5.629	18.47	1.01
BeLFusion + Retargeting	✗	9.24	11.62	8.200	10.93	12.16	1.305	8.031	1.195	9.81	0.34
CoMusion + Retargeting	✗	10.07	12.15	21.066	12.49	12.96	2.070	8.587	1.426	15.98	0.51
SkelDiff + Retargeting	✗	10.81	14.99	14.947	12.75	15.42	0.992	7.616	5.252	11.25	0.28
EquiFusion (A)	✓	7.86	10.47	5.861	10.66	11.61	1.973	7.061	0.691	0.00	0.00
EquiFusion (A+N)	✓	7.71	10.21	5.683	10.58	11.36	1.797	7.349	0.504	0.00	0.00

Ablation Study¶

Table 1: Multi-Dataset Training and Equivariance Architecture Ablations | Model Variant | AMASS ADE ↓ | AMASS MAE ↓ | AMASS CMD ↓ | H36M Zero-Shot ADE ↓ | H36M Zero-Shot MAE ↓ | H36M Zero-Shot CMD ↓ | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | Modified SkelDiff (Multi-dataset) | 0.500 | 7.007 | 19.233 | 0.670 | 12.065 | 31.461 | | EquiFusion w/o Equivariance (+PE) | 0.539 | 6.974 | 22.294 | 0.467 | 7.245 | 9.559 | | EquiFusion (Full Model) | 0.512 | 6.525 | 19.699 | 0.380 | 5.685 | 8.086 | | Permuted Joints & Noise ($P + P_\epsilon$) | 0.512 | 6.525 | 19.699 | 0.380 | 5.685 | 8.086 | | Permuted Joints Only ($P$) | 0.513 | 6.529 | 19.686 | 0.380 | 5.680 | 8.088 | | Joint Semantics Tagging (+S) | 0.508 | 6.456 | 19.492 | 0.381 | 5.708 | 8.526 |

Table 2: Motion Parametrization Ablation (Bone Directions $L$ vs. 3D Coordinates $M$) | Model | Motion Repr. | ADE ↓ | FDE ↓ | MAE ↓ | APD ↑ | CMD ↓ | Stretch str (%) ↓ | Jitter jit (%) ↓ | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | SkelDiff | 3D Pos ($M$) | 0.480 | 0.545 | 6.124 | 9.456 | 11.417 | 3.15 | 0.20 | | SkelDiff | Direction ($L$) | 0.496 | 0.546 | 6.193 | 9.960 | 9.143 | 0.00 | 0.00 | | EquiFusion | 3D Pos ($M$) | 0.501 | 0.561 | 6.551 | 8.348 | 13.963 | 3.58 | 0.27 | | EquiFusion | Direction ($L$) | 0.498 | 0.559 | 6.173 | 8.413 | 12.530 | 0.00 | 0.00 |

Table 3: Zero-Shot Partial Kinematics with Random Limb Occlusion on AMASS | Method | Training Data | Occlusion Strategy | ADE ↓ | FDE ↓ | MAE ↓ | APD ↑ | Stretch str (%) ↓ | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | SkelDiff | AMASS | Rest-pose masking (+rp) | 0.574 | 0.727 | 6.996 | 8.890 | 8.15 | | SkelDiff | AMASS | Symmetric limb mirroring (+sl) | 0.567 | 0.683 | 7.162 | 9.274 | 5.64 | | EquiFusion | AMASS | Native graph adaptation | 0.553 | 0.618 | 10.190 | 10.152 | 0.00 | | EquiFusion | AMASS+Nymeria | Native graph adaptation | 0.499 | 0.553 | 8.777 | 9.099 | 0.00 |

Key Findings¶

Equivariance is Essential for Generalization: Simply scaling training data across multiple skeletons without structural equivariance (Modified SkelDiff) fails completely on novel kinematics (ADE rises to 0.670 on H36M). Adding positional encodings (Ours w/o Eq) breaks zero-shot performance, confirming that generalization stems from mathematical equivariance rather than data brute-forcing.
Bone Directions Eliminate Physical Distortions: Across both baseline and proposed architectures, relative bone direction parameterization brings limb stretching and jitter down to exactly 0.00% while improving Chamfer motion distance (CMD) realism without sacrificing accuracy.
75% Parameter Reduction with Superior Scalability: Sharing weights across permutation-equivariant operators allows EquiFusion to achieve SOTA accuracy with 75% fewer parameters than SkelDiff on single datasets, and a 90% parameter saving across three datasets, halving training and inference latencies.

Highlights & Insights¶

From Hard-Coded Chains to Kinematics-Agnosticism: Severing the hard-coded coupling between network weights and skeleton joints establishes the first architecture capable of processing arbitrary skeletal configurations via dynamic adjacency conditioning.
Permutation Equivariance as Inductive Bias: Demonstrates that permutation equivariance over joint ordering acts as a powerful regularizer, enabling compact parameter footprint while maintaining robust distribution-level diffusion sampling.
Native Partial Kinematics and Occlusion Robustness: Naturally handles missing joints, occluded limbs, or sensor dropouts at test time without requiring dedicated mask-training stages or synthetic data generation.

Limitations & Future Work¶

Internal Kinematic Chain Occlusions: The current relative bone direction parameterization relies on parent-child chains, functioning seamlessly when leaf nodes are absent, but cannot yet handle missing intermediate joints (e.g., missing elbow when the hand is tracked).
Heterogeneous Dynamic Distributions: Multi-dataset blending reveals distribution discrepancies between sedentary daily captures (Nymeria) and athletic high-dynamic motions (AMASS), demanding principled motion-distribution balancing.
Extension Beyond Human Morphology: Current evaluations focus on bipedal human motion; future work could investigate quadrupedal animal locomotion and robotic embodiments.

vs. Fixed-Skeleton Diffusion (SkelDiff, BeLFusion): Prior models treat joint positions as fixed feature channels, requiring independent models per dataset; EquiFusion achieves unified multi-dataset training and zero-shot novel topology prediction.
vs. Motion Retargeting Frameworks (Holden et al., G-DREAM): Non-homeomorphic retargeting inevitably injects cumulative geometric errors and distribution drifts; EquiFusion operates directly on the target skeleton without intermediate conversion.
vs. Equivariant Molecular Diffusion (EDM, PGDiff): While 3D molecular diffusion focuses primarily on spatial E(3)/SE(3) symmetries, EquiFusion brings permutation equivariance to temporal kinematic trees with variable bone-length constraints.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ First kinematics-agnostic formulation for stochastic motion prediction with rigorous permutation-equivariant latent diffusion.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive benchmarks spanning zero-shot kinematics, multi-dataset training, occlusion ablations, and motion representations.
Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical formulations, concise proofs, and structured experimental validation.
Value: ⭐⭐⭐⭐⭐ Addresses a critical bottleneck in computer animation and spatial AI, paving the way for human motion foundation models.