DPoser-X: Diffusion Model as Robust 3D Whole-Body Human Pose Prior

Conference: ICCV 2025 arXiv: 2508.00599 Code: https://dposer.github.io/ Area: Image Generation Keywords: Diffusion Model, Human Pose Prior, Whole-Body Modeling, Inverse Problems, Variational Diffusion Sampling

TL;DR

This paper proposes DPoser-X, a 3D whole-body human pose prior based on an unconditional diffusion model. It unifies various pose-related tasks as inverse problems and performs test-time optimization via a truncated timestep schedule for variational diffusion sampling. A hybrid training strategy is introduced to effectively combine whole-body and part-specific datasets. DPoser-X achieves up to 61% improvement across 8 benchmarks covering body, hand, face, and whole-body modeling.

Background & Motivation

Human pose prior modeling is a fundamental problem in 3D human modeling. The goal is to learn a plausible pose distribution from large-scale data to serve as regularization for downstream tasks such as human mesh recovery, motion capture, and pose completion.

Limitations of Prior Work: - GMM (e.g., SMPLify): Unboundedness can lead to implausible pose generation. - VAE (e.g., VPoser): Gaussian priors limit the expressiveness of the latent space, resulting in insufficient diversity. - NDF (e.g., Pose-NDF, NRDF): Difficult to generalize to the full space of high-dimensional human pose manifolds.

Key Challenge: The above methods primarily focus on body pose while neglecting whole-body modeling that includes hand gestures and facial expressions. Whole-body pose data is scarce (existing datasets mainly cover specific actions such as grasping and sign language), and complex inter-part dependencies exist (e.g., hand poses tend to be symmetric when standing).

Key Insight: The paper leverages the advantage of diffusion models in learning complex distributions by training an unconditional diffusion model to capture pose distributions and unifying downstream tasks into an inverse problem framework solved via variational diffusion sampling at test time. Key innovations include a truncated timestep schedule tailored for pose data and a hybrid training strategy that combines whole-body and part-specific datasets.

Method

Overall Architecture

DPoser-X consists of three levels: (1) part-level DPoser models (one unconditional diffusion model each for body, hand, and face); (2) a fusion module that combines the last-layer features of the three part models via fully connected networks to capture inter-part dependencies; and (3) a hybrid training strategy that integrates whole-body and part-specific datasets.

Key Designs

1. DPoser Regularization: The core idea is to use the single-step denoising of a diffusion model as a regularization term for the pose prior. Given the current optimization variable \(\mathbf{x}_0\) (i.e., SMPL pose parameters \(\theta\)), noise is added at timestep \(t\) to obtain \(\mathbf{x}_t\), and the trained noise predictor \(\epsilon_\phi\) performs a single denoising step to yield \(\hat{\mathbf{x}}_0(t)\). The regularization loss is:

   \(L_{\text{DPoser}} = w_t \|\mathbf{x}_0 - \text{sg}[\hat{\mathbf{x}}_0(t)]\|_2^2\)

where \(\hat{\mathbf{x}}_0(t) = \frac{\mathbf{x}_t - \sigma_t \epsilon_\phi(\mathbf{x}_t; t)}{\alpha_t}\) and \(\text{sg}\) denotes stop-gradient. The gradient direction of this loss is consistent with the regularization term in variational diffusion sampling (Eq. 4) (\(\propto \epsilon_\phi(\mathbf{x}_t; t) - \epsilon\)), while being more intuitive and naturally equivalent to Score Distillation Sampling.

Design Motivation: The stop-gradient operation eliminates the need for backpropagation through the trained diffusion network; only a single forward pass is required, incurring minimal computational overhead (approximately 10% increase over the no-prior baseline).

1. Truncated Timestep Schedule: Conventional image diffusion optimization employs a uniform timestep schedule \([1.0, 0.0]\), but the paper finds that pose data differs from images — critical pose information is concentrated in the small-\(t\) regime (\(t \leq 0.3\)). Empirical validation (Fig. 3) shows that concentrating DDIM sampling steps in the later stages (small \(t\)) yields better poses than uniform allocation under a limited step budget.

The truncated schedule is defined as: \(t = t_{\max} - \frac{(t_{\max} - t_{\min}) \times \text{iter}}{N-1}\). Typical intervals are: \([0.12, 0.08]\) for human mesh recovery, \([0.2, 0.05]\) for motion denoising, and \([0.15, 0.05]\) for pose completion.

Intuition: At small \(t\), the noising and denoising paths are short, so \(\hat{\mathbf{x}}_0(t)\) is close to \(\mathbf{x}_0\), providing weak but precise DPoser guidance. At large \(t\), guidance is stronger but may reduce the correlation between the denoised pose and the original. Selecting an appropriate range based on the noise level of each task is key.

1. Hybrid Training Strategy (DPoser-X-mixed): Addresses the scarcity of whole-body pose data.

   • Part-specific data (body-only/hand-only/face-only) is treated as incomplete whole-body data, with loss computed only on available parts.
   • For whole-body data, certain parts are randomly masked with a 20% probability, forcing the model to predict the masked parts (preventing excessive distribution gap between whole-body and part-specific data).
   • Data mixing ratio: approximately 65% whole-body + 14% body + 12% single hand + 4% both hands + 5% face.

This strategy enables the model to learn inter-part dependencies (e.g., bimanual coordination in whole-body data) while improving generalization through part-specific data augmentation.

Loss & Training

DPoser trains an unconditional diffusion model using sub-VP SDE parameterization, with noise prediction objective weighted as \(w(t) = \sigma_t^2\). The body model is trained on the AMASS dataset (approximately 55 million poses) using axis-angle representation (zero-mean, unit-variance normalized), with a fully connected network of approximately 8.28M parameters optimized with Adam for 800K iterations.

Key Experimental Results

Main Results

Human Mesh Recovery (EHF dataset, PA-MPJPE mm):

Initialization	No Prior	GMM	VPoser	Pose-NDF	NRDF	GAN-S	DPoser
From scratch	108.57	58.32	58.08	57.87	57.38	57.26	56.05
CLIFF	56.62	51.02	49.39	49.50	49.27	49.58	49.05

Whole-Body Pose Completion (one hand masked, min/mean MPVPE mm):

Method	ARCTIC	BEAT2
VPoser-X	37.34/43.24	27.49/35.46
DPoser-X	21.81/30.99	15.92/25.89

Whole-Body Mesh Recovery (ARCTIC dataset, PA-MPVPE mm):

Method	All	Hands	Face	Body
VPoser-X	66.74	17.44	10.99	79.88
DPoser-X	60.98	15.60	9.75	73.00

Ablation Study

Timestep Schedule Comparison:

Schedule	Whole-Body Mesh Recovery (All/Hands)	Motion Denoising (MPVPE/MPJPE)
Random	62.28 / 16.63	43.33 / 23.87
Fixed	61.69 / 15.71	45.69 / 22.54
Uniform	62.13 / 17.32	39.72 / 20.80
Truncated	60.98 / 15.60	38.21 / 19.87

The truncated schedule outperforms all existing strategies across tasks. Uniform performs poorly on mesh recovery (low noise), while Fixed underperforms on motion denoising (gradually varying noise).

Hybrid Training Strategy Comparison:

Model	ARCTIC Completion (min MPVPE)	Fit3D Recovery (All PA-MPVPE)
DPoser-X-base	25.49	72.79
DPoser-X-fused	21.51	72.06
DPoser-X-mixed	21.81	70.91

The mixed strategy significantly outperforms fused on zero-shot generalization (Fit3D motion scenarios) while achieving comparable completion accuracy.

Key Findings

• Hand Inverse Kinematics: DPoser-hand reduces MPJPE by over 50% compared to the second-best method under the sparse ReInterHand setting (3.21 vs. 8.25 mm).
• Motion Denoising: DPoser surpasses the dedicated motion prior HuMoR (19.87 vs. 22.69 MPJPE), despite not being designed for temporal tasks.
• Face Reconstruction: Combined with MICA initialization on the NOW benchmark, DPoser achieves a mean error of 8.76 mm (state of the art).
• Minimal Computational Overhead: DPoser regularization adds only approximately 10% to optimization time.

Highlights & Insights

• Unified Inverse Problem Framework: Tasks including pose completion, inverse kinematics, and human mesh recovery are unified as inverse problems, with DPoser serving as a plug-and-play general-purpose regularization term.
• Truncated Timestep Schedule is an important finding specific to pose data — pose information is concentrated in the low-noise regime, in contrast to image generation where structure is formed first and details are refined later.
• The hybrid training strategy is elegantly designed, treating part-specific data as a missing-value problem and applying 20% random masking of whole-body data as augmentation.
• The stop-gradient design ensures compatibility with any downstream optimizer without additional memory overhead.

Limitations & Future Work

• The axis-angle representation based on SMPL-X limits expressiveness to the degrees of freedom defined by the skeletal model.
• The truncated timestep interval \([t_{\max}, t_{\min}]\) requires manual tuning for each task.
• The weighting of data sources in the hybrid training strategy is currently determined empirically.
• Integration with conditional diffusion models (e.g., image-conditioned) has not been explored.
• Whole-body generation quality (Table 6) still has room for improvement due to limited training data.

Related Work & Insights

• VPoser (VAE prior) is the most widely used baseline; DPoser comprehensively surpasses it in expressiveness.
• BUDDI (human interaction prior) employs a similar SDS optimization paradigm, but DPoser is more general and introduces the truncated schedule.
• Score Distillation Sampling is widely used in 3D generation; DPoser extends it to the pose domain and provides an equivalent intuitive formulation.
• The hybrid training strategy is transferable to other data-scarce scenarios (e.g., multi-source learning from face, body, and hand data).

Rating

• Novelty: ⭐⭐⭐⭐ — Diffusion model as pose prior combined with truncated timestep scheduling is a novel contribution.
• Theoretical Depth: ⭐⭐⭐⭐ — The derivation of variational diffusion sampling is rigorous; the proof of equivalence between DPoser loss and SDS is textbook-quality.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ — 8 benchmarks with full coverage of body/hand/face/whole-body and comprehensive ablation studies.
• Practicality: ⭐⭐⭐⭐⭐ — Plug-and-play regularization with minimal computational overhead.

Related Papers

• [ICCV 2025] Learning Deblurring Texture Prior from Unpaired Data with Diffusion Model
• [ICCV 2025] DIIP: Diffusion Image Prior
• [ICCV 2025] Dual Recursive Feedback on Generation and Appearance Latents for Pose-Robust Text-to-Image Diffusion
• [CVPR 2026] Pose-dIVE: Pose-Diversified Augmentation with Diffusion Model for Person Re-Identification
• [ICCV 2025] DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Poses