Equivariant Flow Matching for Symmetry-Breaking Bifurcation Problems

Conference: NeurIPS 2025
arXiv: 2509.03340
Code: FHendriks11/bifurcationML
Area: Image Generation
Keywords: flow matching, equivariance, symmetry breaking, bifurcation, generative modeling

TL;DR

This paper proposes an equivariant flow matching framework combined with a symmetric coupling strategy to model multimodal probability distributions arising in symmetry-breaking bifurcation problems via generative AI, significantly outperforming deterministic models and VAEs on physical systems (buckling beam, Allen-Cahn equation).

Background & Motivation

Bifurcation phenomena in nonlinear dynamical systems give rise to multiple coexisting stable solutions, particularly under symmetry breaking. This is critical across many physical domains:

• Fluid mechanics: Flow oscillations at high Reynolds numbers break time-translation symmetry
• Structural mechanics: Symmetric structures under compression can buckle in multiple directions
• Phase separation: Homogeneous mixtures can separate into distinct regions
• Mechanical metamaterials: Programmable shape transformations via controlled symmetry breaking

Fundamental limitation of deterministic ML models: They can only output the average of multiple solutions, producing nonphysical predictions. For example, a beam that may buckle left or right will be predicted as unbuckled (the average of both), which does not correspond to any physical solution.

The dilemma of equivariant models: Equivariant models that preserve system symmetry cannot "choose" an asymmetric outcome, and thus fail to capture post-bifurcation low-symmetry behavior.

Limitations of VAEs: The target distribution after bifurcation is singular (supported on a low-dimensional subspace, e.g., multiple Dirac deltas); VAEs struggle to learn such concentrated multimodal distributions and tend to produce blurry predictions.

Advantages of flow matching: By progressively approximating complex mappings through iterative integration steps, the nonlinearity is distributed across multiple stages, making flow matching better suited for modeling singular and multimodal distributions.

Method

Overall Architecture

The bifurcation problem is formulated as learning a conditional probability distribution \(p(y|x)\), where \(x\) is the input parameter (control parameter) and \(y\) is the output (system state). Flow matching is used to learn the transformation from a simple prior \(p(y_0)\) (Gaussian noise) to the target distribution \(p(y|x)\).

The key equivariance condition is: \(p(y|x) = p(g \cdot y \mid g \cdot x)\) for all group elements \(g\). Satisfying this requires: (i) parameterizing the probability distribution with a \(G\)-equivariant model, and (ii) using a \(G\)-invariant prior \(p(y_0)\).

Key Designs

Equivariant Flow Matching: A vector field \(u(y_t, t, x)\) is learned to transform prior samples \(y_0\) into target samples \(y_1\) over pseudo-time \(t \in [0,1]\). Training minimizes the mean squared error between the model's predicted vector field and the target vector field along linearly interpolated paths.

Symmetric Coupling: Symmetry-equivalent solutions are leveraged to improve training. The core idea is: for each noise sample \(y_0\) and target \(y_1\), find the nearest equivalent representation of \(y_1\) under the input symmetry group \(G_x\):

\[\tilde{g} = \arg\min_{g_x \in G_x} c(y_0,\, g_x \cdot y_1)\]

where \(c\) is a cost function (e.g., squared Euclidean distance). The original target is then replaced by \(y_1' = \tilde{g} \cdot y_1\) for training.

This is equivalent to "straightening" flow paths (analogous to mini-batch optimal transport), but optimizes the coupling only between a single sample and its symmetry-equivalent counterparts. Specific implementations include: - Permutation symmetry: Hungarian algorithm - Rotation symmetry: Kabsch algorithm - Reflection symmetry: Enumeration of all possible reflections - Periodic translation: FFT cross-correlation

Probabilistic perspective on equivariance: Although individual post-bifurcation solutions break symmetry, the set of solutions (the orbit) remains symmetric. The generative model need not produce a single equivariant output, but rather an equivariant output distribution.

Architecture choices: For the buckling beam problem, EGNN (E(n)-Equivariant Graph Neural Network) is used for node interactions combined with a UNet for temporal evolution; random-walk priors are used for general problems.

Loss & Training

The standard flow matching loss minimizes the mean squared error between the predicted vector field and the target vector field (the direction of linear interpolation from prior to target). Symmetric coupling dynamically selects the optimal symmetry-equivalent target at each training step without introducing any additional loss terms.

Key Experimental Results

Main Results

Performance comparison across systems (Wasserstein distance, lower is better):

Test System	Deterministic Model	VAE	FM	FM+SymCoupling
Two Delta Peaks	1.0	0.25	0.091	0.0041
Heads or Tails	56.2	33.0	8.30	-
Three Roads	21.4	17.3	3.88	-
Four Node Graph	10.0	9.89	2.02	1.19
Buckling Beam	-	-	23.1	9.6
Allen-Cahn	-	-	255	244

• The theoretical optimum of the deterministic model on Two Delta Peaks is predicting the mean at 0, yielding Wasserstein = 1.0
• Flow matching comprehensively outperforms deterministic models and VAEs on all systems

Ablation Study

• Two Delta Peaks: Even with a non-equivariant model, symmetric coupling can exploit the sign-flip symmetry of the target to improve results (0.091 → 0.0041, a 95% reduction)
• Four Node Graph: With only two valid permutations (identity and node swap), symmetric coupling reduces the error from 2.02 to 1.19 (41% reduction)
• Buckling Beam: Symmetric coupling (selecting the nearest reflected target trajectory) reduces the error from 23.1 to 9.6 (58% reduction)
• Allen-Cahn: Combining FFT cross-correlation (finding the nearest cyclic translation) with reflection and sign-flip reduces the error from 255 to 244

Key Findings

1. Deterministic models are fundamentally incapable of addressing bifurcation problems: They can only predict the average of solutions, which is nonphysical
2. VAEs outperform deterministic models but fall far short of flow matching: VAEs have limited capacity for multimodal modeling
3. Flow matching is naturally suited for singular distributions: It can precisely concentrate probability mass onto low-dimensional manifolds
4. Symmetric coupling is the critical improvement: Straightening flow paths reduces training difficulty
5. Trained models successfully reproduce the pitchfork bifurcation diagram as parameters vary

Highlights & Insights

• New paradigm: The first systematic application of flow matching to symmetry-breaking bifurcation problems
• Theoretical consistency: The equivariance condition holds at the distributional level (individual outputs may break symmetry, but the distribution remains equivariant)
• Elegant symmetric coupling: Integrates optimal transport coupling with group theory, exploiting the problem's inherent symmetry to improve training
• Physical significance: Successfully models multi-solution coexistence in the buckling beam and complex bifurcation behavior of the Allen-Cahn equation
• Open-source and reproducible

Limitations & Future Work

• Symmetric coupling for discrete groups requires enumerating all group elements, making extension to continuous symmetry groups (e.g., SO(3)) difficult
• Absolute errors on the Allen-Cahn problem remain large (244), as small perturbations in the Laplacian term lead to sensitive residuals
• Experiments are relatively small-scale (toy problems and simplified physical systems); validation on high-dimensional complex PDEs is absent
• No comparison with other generative models (e.g., score-based diffusion, consistency models)
• Inference speed is not discussed (flow matching requires multi-step integration)

Related Work & Insights

• Equivariant Flow Matching (Klein et al., 2023; Song et al., 2023): Optimizes coupling over the full group; this work considers only the input-invariant subgroup \(G_x\)
• Mini-batch OT (Tong et al., 2023): Optimizes prior-target coupling within mini-batches; this work optimizes coupling among symmetry-equivalent targets
• EGNN (Satorras et al., 2021): E(n)-Equivariant Graph Neural Network used for node interactions
• Insight: Generative models can be applied to multi-solution prediction in physical simulation, beyond image generation

Rating

• Novelty: 5/5 — Uniquely integrates flow matching, equivariance, and bifurcation theory
• Value: 3/5 — Primarily targets computational mechanics/physical simulation; direct applicability to image generation is limited
• Experimental Thoroughness: 3/5 — Multiple systems tested but at small scale; comparisons with additional baselines are lacking
• Writing Quality: 5/5 — Concepts are clearly presented with a well-structured progression from toy examples to physical systems

Related Papers

• [NeurIPS 2025] High-order Equivariant Flow Matching for Density Functional Theory Hamiltonian Prediction
• [NeurIPS 2025] Flow Matching Neural Processes
• [NeurIPS 2025] Value Gradient Guidance for Flow Matching Alignment
• [ICCV 2025] EC-Flow: Enabling Versatile Robotic Manipulation from Action-Unlabeled Videos via Equivariant Flow Matching
• [NeurIPS 2025] A Gradient Flow Approach to Solving Inverse Problems with Latent Diffusion Models