LaxMotion: Rethinking Supervision Granularity for 3D Human Motion Generation¶
Conference: ECCV 2026
Paper: ECCV Official
PDF: ECCV 2026 PDF
Code: None
Area: Human Understanding
Keywords: 3D Motion Generation / Relaxed Supervision / 2D Kinematics / View Invariance / Structural Consistency
TL;DR¶
LaxMotion challenges the prevailing reliance on coordinate-level 3D supervision in text-to-motion generation by decoupling sequences into global trajectories and relative limb vectors, supervising only with single-view 2D kinematics and structural consistency constraints to synthesize diverse and high-fidelity 3D human motions.
Background & Motivation¶
Text-driven 3D human motion generation has achieved rapid progress with diffusion and discrete token-based architectures. Prevailing frameworks such as MDM, T2M-GPT, MoMask, and MotionDiffuse report impressive reconstruction accuracy and low FID scores on standard benchmarks like HumanML3D and KIT-ML. However, high numerical fidelity on benchmark test sets frequently fails to translate into robust generalization. When exposed to novel action descriptions, distinct subject kinematics, or compositional prompts, existing models often undergo generative degradation. More critically, masked discrete token generators such as MoMask tend to collapse into repetitive and overly homogeneous skeletal patterns when repeatedly sampled with the same prompt, severely limiting the generative diversity.
The fundamental bottleneck does not merely stem from backbone parameter capacity, but rather from the excessive granularity of the supervision signal itself. Text-to-motion synthesis is intrinsically a one-to-many generative problem: a given natural language prompt naturally corresponds to a wide spectrum of plausible motion styles, execution tempos, and spatial orientations. Nonetheless, standard practices enforce rigid coordinate-level 3D supervision via direct regression over \(\mathbb{R}^{3J \times T}\), penalizing valid motion trajectories that deviate slightly in exact spatial coordinates from the single recorded ground truth. This rigid point-matching objective coerces models into memorizing dataset-specific low-level coordinate patterns and spurious artifacts rather than acquiring the underlying topological and semantic invariants essential for open-world generalization.
To resolve this core tension, this paper rethinks the necessity of explicit 3D pose coordinate targets. Monocular 2D kinematic projections naturally retain vital temporal articulation, relative limb dynamics, and overall motion semantics, while filtering out nuisance variations in absolute depth and camera placement. Crucially, a single 2D projection sequence does not define a unique 3D pose sequence, but rather corresponds to a bounded set of physically valid 3D explanations. The core idea is to decouple 3D motion into global trajectories and relative limb vectors, provide only partially observed single-view 2D kinematics during training, and enforce structural, cross-view, and anatomical consistency regularizations to induce complete 3D motions as self-consistent geometric explanations without direct 3D pose supervision.
Method¶
Overall Architecture¶
LaxMotion transitions the learning paradigm from rigid coordinate point-matching to structural consistency inference. The framework operates across three foundational pillars: First, a structural representation reformulation factorizes the motion space into independent global translation trajectories and local relative limb vectors that preserve invariant projection properties. Second, a reformulated training scheme under relaxed observability provides only partial monocular 2D kinematic observations and trajectory signals during training, prompting the generative backbone to reconstruct complete 3D articulations without seeing full 3D coordinates. Third, a suite of Relaxation Regularizations—comprising view-consistent reprojection, cross-view 2D distribution plausibility under virtual 3D rotations, anatomical orientation coupling, and feature-space alignment—guides the optimization end to end.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input: Text Prompt + Partial Observation mobs<br/>Global Trajectory τ + Monocular 2D Limbs v2D"] --> B["Decoupled Kinematic Factorization<br/>Isolate root trajectory from local limb topology"]
B --> C["Relaxed Observability Training<br/>Incomplete 2D cues supplied during training only"]
C --> D["3D Generative Backbone<br/>Predicts complete 3D trajectory and relative limb vectors"]
D --> E["Weak-Perspective Projection Φ & Virtual 3D Rotation R"]
E --> F["View-Consistent Alignment Lobs"]
E --> G["Cross-View Plausibility Regularization Lrec<br/>Frozen 2D VQ-VAE motion manifold prior"]
E --> H["Orientation & Feature Alignment Lori + Lfeat<br/>Coupled hip-foot geometry + latent embedding consistency"]
F --> I["Joint Relaxation Regularization Lrelax"]
G --> I
H --> I
I --> J["Output: Diverse & Structurally Consistent 3D Motion"]
Key Designs¶
1. Decoupled Kinematic Factorization: Isolating Topology from Global Drift Standard motion representations rely on root-relative absolute 3D joint coordinates, where small perturbations at root-adjacent joints propagate down kinematic chains, and bone-length variations confound general motion representations. LaxMotion replaces absolute positions with a skeletal topology tree representation of relative limb vectors. Given a skeletal graph with \(K\) limbs, each limb vector is defined by the difference between parent and child joints: $\(v^{3D} = \{j^{\text{parent}(n)} - j^{\text{child}(n)}\}_{n=1}^K\)$ A full 3D motion sequence \(m^{3D}_{1:T}\) over \(T\) frames is factorized into a global root trajectory \(\tau_{1:T}\) and relative limb dynamics \(v^{3D}_{1:T}\): \(m^{3D}_{1:T} = \{\tau_{1:T}, v^{3D}_{1:T}\}\). This formulation isolates global displacement from internal skeletal articulations, preserving linearity under weak-perspective projections. Symmetrically, the training observation is defined as \(m^{\text{obs}}_{1:T} = \{\tau_{1:T}, v^{2D}_{1:T}\}\), retaining the global trajectory while compressing 3D relative kinematics into single-view 2D projections. The model is thus guided to capture structural deformations rather than memorizing fixed Cartesian coordinates.
2. Relaxed Observability Training: Inferring 3D Realizations from Partial Cues During training, conventional motion synthesizers require full-rank 3D ground truth to calculate reconstruction or denoising losses. In contrast, LaxMotion feeds only the incomplete observation \(m^{\text{obs}}\) into the learning formulation while requiring the generator \(\mathcal{G}_\theta\) to synthesize complete 3D motions \(\hat{m}^{3D} = \{\hat{\tau}, \hat{v}^{3D}\}\). During test-time inference, the observation signal is omitted entirely, allowing unconditional or text-guided sampling. Withholding full 3D coordinates removes the shortcut of coordinate fitting and forces the model to infer unobserved dimensions via geometric constraints. The master objective is formulated as: $\(\mathcal{L} = \mathcal{L}_{\text{relax}} + \alpha \cdot \mathcal{L}_{\text{prior}}\)$ For diffusion-based backbones (such as MDM), forward noise scheduling inherently establishes a Gaussian structural prior, setting \(\alpha = 0\). For discrete token-based backbones (such as MoMask), \(\mathcal{L}_{\text{prior}}\) is instantiated as a commitment loss with \(\alpha = 2\) to anchor continuous latent states to codebook vectors.
3. Cross-View Plausibility Regularization: Mitigating Depth Ambiguity via Virtual Rotations While the basic reprojection loss \(\mathcal{L}_{\text{obs}} = \|\hat{\tau} - \tau\|_2^2 + \|\Phi(\hat{v}^{3D}) - v^{2D}\|_2^2\) anchors the generated motion to the observed view, it cannot resolve fundamental monocular depth ambiguity, risking degenerate 3D skeletal collapses that appear planar along the viewing axis. Rather than depending on calibrated multi-camera rigs, LaxMotion introduces a virtual rotation prior. The core premise posits that a physically plausible 3D motion must project to valid 2D kinematics under any arbitrary 3D rotation \(R \in SO(3)\). To score this 2D validity without manual supervision, the authors deploy a frozen 2D relative motion VQ-VAE \(\mathcal{V}(\cdot)\) pretrained on extensive 2D motion data. The generated 3D limbs are randomly rotated, projected onto 2D, and evaluated by their reconstruction error through the 2D VQ-VAE: $\(\mathcal{L}_{\text{rec}} = \|\mathcal{V}(\Phi(R \hat{v}^{3D})) - \Phi(R \hat{v}^{3D})\|_2^2\)$ If the synthesized 3D motion exhibits depth collapses or unphysical folding, its projected 2D kinematics under novel perspectives will violate the learned 2D motion manifold and trigger strong reconstruction penalties, enforcing out-of-plane 3D realism purely from monocular sources.
4. Orientation Coupling and Feature Consistency: Enforcing Physical and Latent Alignment To prevent severe biomechanical errors such as backward foot inversions, an orientation constraint couples pelvic normal direction with foot heading. The torso normal vector \(\hat{v}_{\text{ori}}\) is determined by the cross product of the left and right hip vectors: $\(\hat{v}_{\text{ori}} = \frac{\hat{v}_{\text{rhip}} \times \hat{v}_{\text{lhip}}}{\|\hat{v}_{\text{rhip}} \times \hat{v}_{\text{lhip}}\|_2}\)$ In natural bipedal locomotion, the forward projection of the foot vector \(\hat{v}_{\text{foot}}\) onto the torso normal must be non-negative, penalized via: $\(\mathcal{L}_{\text{ori}} = \max(0, -\hat{v}_{\text{foot}} \cdot \hat{v}_{\text{ori}})\)$ Furthermore, to stabilize latent representations against projection distortions, a feature-level loss matches the projected motion's encoded latent features with the original observation embeddings: \(\mathcal{L}_{\text{feat}} = \|\mathcal{E}_f(\hat{m}^{\text{obs}}) - \mathcal{E}_f(m^{\text{obs}})\|_2^2\), smoothing the manifold of the generated motions.
Loss & Training¶
The complete supervisory signal is governed by the composite relaxation loss: $\(\mathcal{L}_{\text{relax}} = \mathcal{L}_{\text{obs}} + \lambda_1 \mathcal{L}_{\text{rec}} + \lambda_2 \mathcal{L}_{\text{ori}} + \lambda_3 \mathcal{L}_{\text{feat}}\)$ with balancing hyperparameters fixed to \(\lambda_1 = \lambda_2 = \lambda_3 = 1.0\). All experiments are conducted on a single NVIDIA RTX 3090 GPU. For MoMask-based experiments, codebook commitment weight is \(\alpha = 2\); for MDM-based experiments, standard continuous diffusion scheduling is used without 3D coordinate supervision. In feature-fusion experiments (3D Finetune), representations learned under relaxed supervision are integrated into a 3D-supervised model to evaluate cross-paradigm synergy.
Key Experimental Results¶
Main Results¶
LaxMotion was benchmarked on the HumanML3D and KIT-ML test sets against top 3D-supervised baselines (T2M, MDM, MLD, MotionDiffuse, ReMoDiffuse, T2M-GPT, MotionLCM, MoMask, and Fg-T2M++). Alongside standard metrics, the Quality–MultiModality Score (\(\text{QM} = \text{MModality} / \sqrt{\text{FID}}\)) was evaluated to assess the trade-off between fidelity and diversity.
| Supervision | Method | R-Precision (Top-1) ↑ | R-Precision (Top-3) ↑ | FID ↓ | Diversity → | MultiModality ↑ | QM Score ↑ |
|---|---|---|---|---|---|---|---|
| Real Data | Real | 0.511 ± .003 | 0.797 ± .002 | 0.002 ± .000 | 9.503 ± .065 | — | — |
| 3D Supervised | T2M | 0.457 ± .002 | 0.740 ± .003 | 1.067 ± .002 | 9.188 ± .002 | 2.090 ± .083 | 2.023 |
| 3D Supervised | MDM | 0.320 ± .005 | 0.611 ± .007 | 0.544 ± .044 | 9.559 ± .086 | 2.799 ± .072 | 3.795 |
| 3D Supervised | MotionDiffuse | 0.491 ± .001 | 0.782 ± .001 | 0.630 ± .001 | 9.410 ± .049 | 1.553 ± .042 | 1.957 |
| 3D Supervised | T2M-GPT | 0.491 ± .003 | 0.775 ± .002 | 0.116 ± .004 | 9.761 ± .081 | 1.856 ± .011 | 5.449 |
| 3D Supervised | MoMask | 0.521 ± .002 | 0.807 ± .002 | 0.045 ± .002 | — | 1.241 ± .040 | 5.850 |
| 3D Supervised | Fg-T2M++ | 0.513 ± .002 | 0.801 ± .003 | 0.089 ± .004 | 9.223 ± .114 | 2.625 ± .084 | 8.799 |
| Relaxed Supervised | LaxMotion (MDM-Based) | 0.431 ± .004 | 0.749 ± .006 | 0.475 ± .041 | 9.523 ± .078 | 2.844 ± .083 | 4.127 |
| Relaxed Supervised | LaxMotion (MoMask-Based) | 0.487 ± .002 | 0.780 ± .002 | 0.054 ± .003 | 9.486 ± .048 | 2.046 ± .068 | 8.805 |
| Relaxed + 3D Finetune | LaxMotion (3D Finetune) | 0.526 ± .003 | 0.812 ± .004 | 0.034 ± .002 | 9.519 ± .079 | 2.529 ± .081 | 13.715 |
Ablation Study¶
On the HumanML3D dataset, component ablations evaluate the contribution of individual relaxation regularizers:
| Configuration | R-Precision (Top-1) ↑ | R-Precision (Top-3) ↑ | FID ↓ | MMDist ↓ | Diversity → | MultiModality ↑ | QM Score ↑ | Note |
|---|---|---|---|---|---|---|---|---|
| Ours (Full Model) | 0.487 | 0.780 | 0.054 | 3.155 | 9.486 | 2.046 | 8.805 | Complete relaxation regularization |
| w/o \(\mathcal{L}_{\text{rec}}\) | 0.368 | 0.678 | 2.588 | 3.912 | 8.449 | 2.127 | 1.322 | Depth ambiguity; FID degrades to 2.588 |
| w/o \(\mathcal{L}_{\text{ori}}\) | 0.465 | 0.756 | 0.124 | 3.282 | 9.299 | 2.225 | 6.319 | Foot inversion artifacts; FID increases |
| w/o \(\mathcal{L}_{\text{feat}}\) | 0.501 | 0.780 | 0.088 | 3.125 | 9.450 | 2.086 | 7.032 | Reduced latent space consistency |
Kinematic representation and prior architecture ablations further demonstrate: 1. Limb Vectors vs. Joint Coordinates: Replacing relative limb vectors with raw joint coordinates in the token-based pipeline increases MPJPE error from 56.4 mm to 106.9 mm and degrades FID from 0.054 to 3.902 (Table 7). 2. Prior Modeling Choice: Modeling the 2D distribution with VQ-VAE yields a QM Score of 8.805, significantly outperforming continuous VAE (6.233 QM, 0.121 FID) and standard autoencoders (4.016 QM, 0.258 FID), confirming the benefit of discrete codebooks in defining motion boundaries (Table 6).
Key Findings¶
- Cross-View Prior is Critical for 3D Lifting: Omitting \(\mathcal{L}_{\text{rec}}\) causes the FID to spike from 0.054 to 2.588 and slashes the QM Score from 8.805 down to 1.322. Monocular 2D projection alone cannot constrain out-of-plane articulation, whereas random 3D rotation paired with frozen 2D VQ-VAE verification resolves depth ambiguities without requiring calibrated multi-camera data.
- Overcoming the Fidelity-Diversity Dilemma: While fully supervised MoMask achieves low FID (0.045) through coordinate fitting, its MultiModality drops to 1.241. LaxMotion achieves a 64.9% increase in MultiModality (2.046) while maintaining comparable FID (0.054), yielding the highest standalone QM score (8.805).
- Synergy with 3D Models: Initializing and regularizing models with relaxed representations prior to 3D fine-tuning achieves state-of-the-art metrics (0.034 FID, 0.526 Top-1 R-Precision, and 13.715 QM Score), confirming that relaxed supervision captures structural invariants overlooked by pure coordinate regression.
Highlights & Insights¶
- Turning One-to-Many Nature into a Generative Asset: Instead of viewing the absence of 3D coordinates as a handicap, LaxMotion demonstrates that over-determined 3D coordinate supervision restricts generative diversity; relaxing the objective to structural consistency aligns training with the natural one-to-many properties of text-to-motion generation.
- Hardware-Free Multi-View Supervision: Rather than requiring synchronized multi-view capture facilities, applying virtual 3D rotations followed by distribution validation through a frozen 2D VQ-VAE acts as a lightweight surrogate for multi-camera physical setups.
- Applicability to In-The-Wild Motions: LaxMotion successfully synthesizes 3D motions for complex actions that cannot be recorded in motion capture labs (e.g., underwater swimming and microgravity gymnastics) by training directly on extracted 2D pose trajectories from single-view web videos.
Limitations & Future Work¶
- Dependence on 2D Prior Distribution Coverage: The quality of \(\mathcal{L}_{\text{rec}}\) hinges on the expressiveness of the pretrained 2D VQ-VAE. Unconventional acrobatics or non-standard motions underrepresented in the 2D dataset may receive suboptimal regularization.
- Static Skeletal Topology Assumption: The relative limb vectors presuppose a fixed human kinematic tree, limiting applicability to arbitrary non-human morphology or dynamic topology changes.
- Future Directions: Leveraging foundation video models (such as modern video diffusion backbones) as general-purpose 2D/3D kinematic evaluators could further eliminate the need for specialized 2D VQ-VAEs and scale training across unconstrained Internet video libraries.
Related Work & Insights¶
- vs. MoMask / T2M-GPT: Standard discrete token architectures supervise exact coordinate tokens, causing repetitive and deterministic outputs. LaxMotion shows that supervising topological limb dynamics and 2D projections retains quantization stability while expanding motion diversity.
- vs. MAS / Motion-2-to-3: Prior 2D-assisted frameworks rely on complex ancestral sampling (MAS) or multistage 2D-to-3D fine-tuning (Motion-2-to-3). LaxMotion trains end-to-end with relaxed monocular supervision, outperforming Motion-2-to-3 on HumanML3D global motion (0.054 vs. 0.321 FID; 0.780 vs. 0.697 Top-3 R-Precision).
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Rethinks the role of supervision granularity in text-to-motion generation, turning under-determined 2D cues into an advantage for generative diversity.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluations across two major benchmarks, two generative paradigms, detailed ablation studies, and in-the-wild video synthesis.
- Writing Quality: ⭐⭐⭐⭐⭐ Rigorous mathematical formulation, clear structural logic, and compelling analytical depth.
- Value: ⭐⭐⭐⭐⭐ Provides a viable path toward scaling 3D motion synthesis using uncalibrated monocular web video collections.