FlowerDance: MeanFlow for Efficient and Refined 3D Dance Generation¶

Conference: ECCV2026
arXiv: 2511.21029
Code: https://sun-happy-ykx.github.io/FlowerDance/ (to be open-sourced upon acceptance)
Area: Human Understanding / 3D Motion Generation
Keywords: Music-to-Dance Generation, Flow Matching, MeanFlow, Bidirectional Mamba, Non-autoregressive Generation

TL;DR¶

FlowerDance combines MeanFlow (a flow matching variant that replaces instantaneous velocity prediction with interval-average velocity) with Physical Consistency Constraints (PCC), a BiMamba backbone, and channel-level cross-modal fusion. It generates high-quality 3D dance motions in only 5–20 sampling steps. While achieving state-of-the-art (SOTA) quality on FineDance and AIST++, its inference speed (2008 FPS) far exceeds the previous best method (MatchDance at 345 FPS), leaving ample computational budget for real-time 3D rendering.

Background & Motivation¶

Music-to-dance generation aims to convert audio signals into realistic human motion, serving as a key technology in virtual reality, automated choreography, and digital entertainment. Existing methods roughly follow three paradigms: GAN-based methods (CoheDancers, Choreography cGAN), though fast in generation, are prone to mode collapse and produce highly repetitive motions; autoregressive methods (Bailando, Bailando++, Duolando) predict frame-by-frame based on choreographic units, yielding biomechanically reasonable motions but suffering from high inference latency and exposure bias that compromises long-range consistency; diffusion-based methods (EDGE, Lodge, GCDance) achieve high fidelity through iterative denoising, but the curved denoising trajectories require a large number of sampling steps (typically over 50), leading to prohibitive computational costs. The common dilemma of these three paradigms is the lack of generation efficiency—leaving too little computational budget for downstream high-fidelity 3D rendering, which restricts the expressiveness of 3D characters in real-time interactive scenarios.

From the perspective of choreography, a dancer essentially "first initializes a high-entropy motion seed and then step-by-step refines the velocity, center of mass offset, and transitional motions"—a progressive sculpting process from coarse to fine. Flow Matching, which learns a continuous velocity field (ODE) from a simple prior to the data distribution, precisely mirrors this pipeline. However, standard Flow Matching estimates the instantaneous velocity at each time point. When generating high-dimensional curved trajectories, the prediction error of instantaneous velocity accumulates over integration steps. To compensate for the curved trajectories, one has to reduce the step size and increase the total step count. The deep contradiction lies in the mismatch between the training objective (instantaneous velocity matching) and the inference process (interval ODE integration). Meanwhile, due to its inherent sequential inductive bias and linear complexity, Mamba exhibits more natural advantages in fine-grained temporal modeling compared to Transformers (which suffer from weak positional encodings and quadratic complexity). However, whether the efficiency of Mamba and the progressive generation advantage of flow matching can be synergized remains an open question.

The core insight of FlowerDance is that if the generation strategy is changed from "predicting instantaneous velocity" to "predicting interval-average velocity", training and inference become unified—the model directly learns the average direction of change from the start point to the end point during training, and can traverse it in one or a few steps during inference without being forced to add steps to compensate for curved trajectories. However, directly applying MeanFlow faces new challenges unique to 3D human motion: the human motion manifold is highly complex and multi-modal. Relying solely on velocity field consistency constraints cannot anchor the generated trajectories, causing the model to drift into physically implausible regions (jittering, root drift, abnormal limbs). Therefore, a Physical Consistency Constraint (PCC) is introduced as an anchor. During training, the complete motion is additionally reconstructed from the velocity field, and a triple loss (reconstructed motion, joint position via Forward Kinematics (FK), and velocity) is imposed on the reconstructed motion. Concurrently, a bidirectional Mamba (BiMamba) is adopted to replace the Transformer as the backbone to capture bidirectional temporal dependencies between music and dance with \(O(n)\) complexity. Channel-level element-wise addition is used instead of cross-attention to achieve zero-parameter cross-modal fusion. Core Idea: Combine the MeanFlow interval-average velocity prediction with physical consistency constraints, coupled with a BiMamba linear complexity backbone and zero-parameter channel-level cross-modal fusion, to achieve SOTA dance quality and an inference efficiency of 2008 FPS (5.8x faster than the prior best method) with only 5–20 sampling steps.

Method¶

Overall Architecture¶

The overall pipeline of FlowerDance is an ODE-based generation framework driven by MeanFlow. The inputs consist of music features (a 35-dimensional vector including MFCC, Chroma, Peak, Beat, and Envelope extracted by Librosa), a dance style label (one-hot encoded), and current time-step information (start time \(r\) and end time \(t\)). Music features are first encoded by a multi-layer BiMamba and then fused with the style label embedding via a gating mechanism. Time information is fused via sinusoidal positional encoding and addition. The merged conditional features and the current flow state \(z_t\) are then fed into a multi-layer BiMamba module—each layer consists of BiMamba for temporal modeling, FiLM for incorporating time information, and channel-level addition for cross-modal fusion. It finally outputs the interval-average velocity \(u_\theta(z_t, r, t)\). Euler discretization is then applied to solve the ODE, recovering the complete dance sequence \(z_0\) in one or multiple steps. The entire sequence is generated non-autoregressively all at once, without frame-by-frame decoding.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Music + Style Label<br/>+ Time (r, t)"] --> BE["BiMamba<br/>Music Encoder"]
    Input --> GE["Style Embedding<br/>+ Gated Fusion"]
    TE["Time Embedding"] --> Blocks
    BE --> GF["Gated Fusion<br/>Music + Style"]
    GF --> Blocks["Multi-layer BiMamba Blocks<br/>BiMamba + FiLM + Channel Fusion"]
    Zt["Flow State z_t"] --> Blocks
    Blocks --> Vel["Interval-Average Velocity Prediction"]
    Vel --> ODE["Euler ODE Solver<br/>z_r = z_t − (t−r)·u"]
    ODE --> Output["Generated Dance z_0"]
    Vel -.-> PCC["Physical Consistency Constraint<br/>(Training Only)"]
    PCC --> L1["ℒ_rec + ℒ_pos + ℒ_vel"]
    Input --> BE

Key Designs¶

1. MeanFlow Generation Strategy: Aligning Training and Inference Mismatch with Interval-Average Velocity

Standard Flow Matching estimates the instantaneous velocity \(v(z_t, t) = \epsilon - x\) (along the linear interpolation path), and the training objective is to match this instantaneous velocity. However, in high-dimensional spaces, even if the conditional flow is designed linearly, the actual latent trajectory is still curved. When integrating with large steps, the instantaneous velocity model cannot accurately capture the curved trajectory, leading to degradation in generation quality. MeanFlow changes the prediction target from instantaneous velocity to interval-average velocity:

\[u(z_t, r, t) = \frac{1}{t-r} \int_r^t v(z_\tau, \tau) d\tau\]

Through the MeanFlow equation \(u(z_t, r, t) = v(z_t, t) - (t-r) \cdot \frac{d}{dt} u(z_t, r, t)\), the model can directly minimize the mean squared error between the predicted average velocity and the ground truth average velocity. During inference, the ODE update becomes \(z_r = z_t - (t-r) \cdot u_\theta(z_t, r, t)\), which aligns exactly with the training objective—there is no longer a mismatch of "training with instantaneous, inferring with interval". This allows achieving generation quality comparable to diffusion models with 50 steps using only 5–20 steps.

2. Physical Consistency Constraint (PCC): Anchoring Generated Trajectories to the Human Motion Manifold

The loss of MeanFlow, \(\mathcal{L}_{MF}\), only constrains the mathematical consistency of the velocity field itself and does not impose any constraints on the reconstructed concrete motion. In the high-dimensional and multimodal conditional mapping of 3D human motion, converging solely on velocity field consistency is extremely difficult—the model tends to drift into physically implausible regions, resulting in temporal jitter, global root drift, and abnormal limb movements. The core difficulty lies in the fact that the network directly outputs interval-average velocity rather than the original motion, preventing traditional physical losses (velocity, acceleration, joint losses) from being directly applied. FlowerDance solves this by additionally sampling a time \(t_1\) at each training iteration, letting the model predict the average velocity over the interval \((t_1, 0)\) as \(u_\theta(z_{t_1}, 0, t_1)\), and then recovering the predicted motion via \(\hat{z}_0 = z_{t_1} - t_1 \cdot u_\theta(z_{t_1}, 0, t_1)\). Then, a triple loss is applied to \(\hat{z}_0\): reconstruction loss \(\mathcal{L}_{rec} = \| \hat{z}_0 - z_0 \|^2\), joint position loss \(\mathcal{L}_{pos} = \| FK(\hat{z}_0) - FK(z_0) \|^2\) (where FK stands for Forward Kinematics, mapping joint angles to 3D positions), and velocity loss \(\mathcal{L}_{vel} = \| FK(\hat{z}_0)' - FK(z_0)' \|^2\). Ablation studies show that training directly diverges to NaN without PCC, demonstrating its critical role.

3. BiMamba Backbone Network: Linear-Complexity Bidirectional Temporal Modeling

There are three major reasons to choose Mamba over Transformer. First, music and dance require strong local continuity; the positional encoding of Transformer is essentially a weak inductive bias, making it less effective at modeling fine-grained local dependencies than Mamba's sequential state space mechanism. Mamba naturally possesses high sequential inductive bias through its selection mechanism (S6) and recurrent state updates. Second, Mamba's computational complexity is \(O(n)\), which is significantly lower than Transformer's \(O(n^2)\), showing a clear advantage for long sequences (e.g., 1024 frames of 34-second dance). Third, Mamba supports non-autoregressively generating the entire sequence at once, avoiding the exposure bias of autoregressive methods and the boundary artifacts of window-based stitching methods. However, standard Mamba is unidirectional, whereas the temporal dependencies of music and dance are bidirectional—the current action is jointly affected by past and future music. FlowerDance adopts BiMamba: the sequence is processed by forward and backward Mamba branches simultaneously, the outputs are fused through addition, and a multiplicative residual connection is utilized to enhance gradient flow and feature preservation.

4. Channel-level Cross-modal Fusion: Zero-parameter Element-wise Addition Over Cross-Attention

Music and dance are frame-aligned in time, and the temporal modeling has already been handled by BiMamba, making complex interactions like cross-attention redundant. FlowerDance directly fuses music features and dance features along the channel dimension using element-wise addition—requiring zero parameters and adding no inference burden, while achieving better generalization than cross-attention on small-scale 3D dance datasets (which are prone to overfitting due to excessive parameters). Experiments show that replacing addition with cross-attention slightly degrades quality (FIDg rises from 19.59 to 24.45) and drops inference speed from 2008 FPS to 1463 FPS, validating the dual superiority of addition in both efficiency and quality.

Loss & Training¶

The total loss is \(\mathcal{L} = \lambda_{mf} \cdot \mathcal{L}_{MF} + \lambda_{rec} \cdot \mathcal{L}_{rec} + \lambda_{pos} \cdot \mathcal{L}_{pos} + \lambda_{vel} \cdot \mathcal{L}_{vel}\), where \(\mathcal{L}_{MF}\) is the MeanFlow velocity field consistency loss, and \(\mathcal{L}_{rec} / \mathcal{L}_{pos} / \mathcal{L}_{vel}\) are the three sub-losses of the physical consistency constraint. \(\lambda\) is balanced based on the magnitude of each loss at the beginning of training. During inference, only the ODE forward solver is required, and all calculations related to PCC are stripped away, incurring no additional inference cost.

Key Experimental Results¶

Main Results¶

Dataset	Metric	FlowerDance	Prev. SOTA	Gain
FineDance	FIDk ↓	29.73	Match 36.68	Reduced by 18.9%
FineDance	FIDg ↓	19.59	Lodge 35.52	Reduced by 44.9%
FineDance	Diversity DIVk ↑	8.42	Bailando 7.74	+8.8%
FineDance	Beat Alignment BAS ↑	0.232	Lodge 0.226	+2.7%
FineDance	FPS ↑	2008	Match 345	5.8× Faster
FineDance	Parameters	63M	Lodge 235M	Reduced by 73%
AIST++	FIDk ↓	20.50	Bailando 28.16	Reduced by 27.2%
AIST++	Geometric Diversity DIVg ↑	6.52	FACT 6.18	+5.5%

On the FineDance dataset, a user study (40 dancers, 5-point double-blind questionnaire) showed that FlowerDance outperforms existing methods in dance quality (4.18 vs MEGA 4.12), beat synchronization (4.41 vs 4.23), and creativity (4.33 vs 4.27).

Ablation Study¶

Configuration	FIDk ↓	FIDg ↓	FSR ↓	FPS ↑	Description
FlowerDance (Full)	29.73	19.59	0.147	2008	Full model
BiMamba → Mamba (Unidirectional)	39.40	38.93	0.197	2387	Quality drops significantly after removing bidirectionality
BiMamba → Transformer (NAR)	NaN	NaN	NaN	1829	Non-autoregressive Transformer fails to generalize to long sequences
BiMamba → Transformer (II)	23.29	21.02	0.191	218	Inpainting-based Transformer achieves decent metrics but is extremely slow
Addition → Cross-Attention	32.40	24.45	0.194	1463	Cross-attention leads to slightly worse quality and slower speed
W/o PCC	NaN	NaN	NaN	2008	Training diverges, showing PCC is indispensable

Key Findings¶

Substantial step-count advantage of MeanFlow: Generating with 10 steps achieves quality comparable to RectFlow with 50 steps (FIDk 26.17 vs 33.47), and fully surpasses it at 20 steps. Generating with 5 steps is still reasonable (but with a high FSR of 1.206, indicating foot sliding). Single-step generation remains an open challenge.
PCC is an indispensable anchor: Removing PCC leads to complete training divergence (NaN), demonstrating that for complex, high-dimensional tasks like 3D human motion, the velocity field consistency loss alone is far from sufficient to constrain trajectories onto the plausible motion manifold.
BiMamba is the sweet spot between efficiency and quality: Unidirectional Mamba is more efficient but suffers from poor quality; Transformer (II) achieves comparable quality but at only 218 FPS (1/9 of BiMamba's speed); Transformer (NAR) completely fails on long sequences.
Channel-level addition matches cross-attention in quality: It not only generalizes better (providing higher stability under the non-autoregressive setting) but is also 37% faster with fewer parameters.

Highlights & Insights¶

Training-inference consistency design philosophy: MeanFlow aligns "training to predict average velocity" with "inference using ODE integration with average velocity", resolving the fundamental mismatch between the training objective and the inference process in diffusion and standard FM models. This simple algebraic formulation yields a massive reduction in sampling steps.
PCC resolves the dilemma of imposing physical constraints on velocity field models: Since the network predicts average velocity instead of raw motion, direct joint-level losses cannot be applied. The strategy of sampling an extra \(t_1\), recovering the completed motion, and then applying joint position losses via FK is elegant and highly reproducible.
System-level synergy of BiMamba and addition fusion: While individual components (Mamba, addition fusion) are not entirely novel on their own, combining them accomplishes an order-of-magnitude efficiency leap to 2008 FPS (vs 345 FPS previously), proving that in efficiency-oriented system design, component synergy outweighs isolated innovations.
Time-decay soft-mask editing strategy: Recognizing that few-step sampling lacks the capability of step-by-step correction, the authors employ a gradually decaying soft constraint instead of hard masking. This avoids mutation artifacts at hard boundaries and is a highly practical design for real-world applications.

Limitations & Future Work¶

One-step generation is not yet realized. At 5 steps, the FSR jumps to 1.206, indicating noticeable foot-sliding artifacts, which is still a step away from real-time interaction.
It relies on hand-crafted Librosa features (MFCC, Chroma, etc.) and does not explore end-to-end audio feature learning (such as Jukebox or MERT encoders), which might lose high-level semantics.
It was validated only on the SMPL parametric body model. Its generalization to non-standard body types and multi-person interaction remains unverified. Additionally, it evaluates only at a fine-grained 30 FPS without exploring variable frame-rate generation for varying compute budgets.
In the ablation study, FIDk slightly increases as MeanFlow steps rise (from 29.05 at 5 steps to 29.73 at 20 steps). The paper leaves this non-monotonic phenomenon unexplained—it might be due to over-smoothing of fine details when step count increases.

Rating¶

Novelty: ⭐⭐⭐⭐ First application of MeanFlow to motion generation + PCC elegantly solves the lack of motion constraints in velocity field models, but component-level innovations (BiMamba backbone, addition fusion) are not entirely novel.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Covers two datasets + three types of metrics (quality, efficiency, human) + four groups of ablations (generative model, backbone, fusion, PCC), overall comprehensive and complete.
Writing Quality: ⭐⭐⭐⭐ Clear motivational chain (efficiency bottleneck → MeanFlow alignment → PCC anchoring → BiMamba acceleration) with solid mathematical derivations, though the method section is slightly wordy.
Value: ⭐⭐⭐⭐⭐ 2008 FPS vs the previous best of 345 FPS represents an almost 6x efficiency improvement, unlocking possibilities for real-time 3D interactive scenarios, making it highly practical.