Skip to content

Follow-Your-Mind: Towards Inversion-Free Brain-Driven Visual Context Synthesis and Editing

Conference: ECCV 2026
Paper: ECCV Official
Area: Medical Imaging
Keywords: Brain-Computer Interface, Brain Decoding, Visual Image Editing, Flow Matching, Rectified Flow

TL;DR

MinD-Flow is a pioneering inversion-free framework for brain-driven visual context synthesis and editing based on Rectified Flow, achieving structure-preserving and semantically aligned neural editing in just 5 Euler steps via decoupled flow guidance and neuro-adaptive soft masking.

Background & Motivation

Humans perceive and comprehend the world through vision, fusing raw sensory input with intrinsic cognition to produce creative artifacts. In neuro-generative AI and brain-computer interfaces (BCIs), directly deciphering brain activity to synthesize and edit visual content marks a paradigm shift from passive neural decoding tools to proactive human-AI collaborative partners. While recent advances in latent diffusion models (LDMs) and diffusion transformers (DiTs) have demonstrated impressive fidelity in fMRI and EEG visual reconstruction, extending brain-guided manipulation to localized visual editing and contextual composition has remained severely constrained by fidelity degradation and structural distortion.

Existing brain-driven visual manipulation approaches face two fundamental bottlenecks. First, the conventional diffusion-based "inversion-generation" pipeline (such as DDIM Inversion or Null-Text Inversion) relies on iterative, stochastic back-projection into the noise latent space. This process introduces significant cumulative approximation errors across discrete sampling steps, leading to blurred backgrounds, degraded fine-grained details, and catastrophic loss of unedited reference structures. Second, reconciling heterogeneous brain signals with external multi-modal anchors poses a severe challenge. fMRI provides rich spatial topology at low temporal resolution, whereas EEG/fNIRS offers millisecond-level temporal dynamics burdened by extreme noise. When combining brain signals with explicit visual and textual prompts, implicit neural intents are easily overshadowed by dominant modalities, resulting in modality dominance and severe semantic conflict.

To break free from the inversion error accumulation and multi-modal dominance inherent in diffusion models, this paper introduces a deterministic flow paradigm governed by Rectified Flow. Core idea: MinD-Flow unifies heterogeneous brain signals with generative latent spaces via a hybrid neural encoder and Perceiver bridge, leveraging Decoupled Flow Guidance and Neuro-Adaptive Soft Masking to achieve high-fidelity localized editing directly along the source trajectory in a single forward pass without noise inversion.

Method

Overall Architecture

The MinD-Flow framework maps heterogeneous physical brain signals into a unified generative latent space, executing deterministic visual context synthesis and editing without iterative noise inversion loops. The pipeline consists of three core components: first, the Neuro-aware Hybrid Encoder disentangles spatial topology (fMRI) and temporal dynamics (EEG/fNIRS) through specialized dual-stream encoders; second, the Neuro-Perceiver Bridge (NPB) leverages learnable queries and adaptive gating to filter physiological noise and compress neural semantics into a unified condition vector; third, the Inversion-Free Neuro-Flow Solver employs Decoupled Flow Guidance (DFG) and Neuro-Adaptive Soft Masking on a Rectified Flow model, executing precise localized editing and synthesis in 5 Euler function evaluations.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multi-Modal Brain Signal Input<br/>fMRI Voxels / EEG / fNIRS Series"] --> B["Neuro-aware Hybrid Encoder & NPB<br/>Spatial MAE + RoPE 1D-CNN + Gated Fusion"]
    C["Visual Anchor & Text Guidance<br/>VAE Latent + Frozen T5-XXL"] --> D["Decoupled Flow Guidance<br/>Isolating Reference Flow & Delta Velocities"]
    B --> D
    D --> E["Neuro-Adaptive Soft Masking & Trajectory Mixing<br/>Cross-Attention Soft Threshold + Euler Solver"]
    E --> F["High-Fidelity Synthesized & Edited Output<br/>5-Step Deterministic Generation"]

Key Designs

1. Neuro-aware Hybrid Encoder and Neuro-Perceiver Bridge: Disentangling Spatiotemporal Dynamics and Filtering Neural Noise

To tackle the high heterogeneity and poor signal-to-noise ratio of neural recordings, the encoder establishes two dedicated paths. For the spatial stream (fMRI voxel volume \(\mathbf{x}^f \in \mathbb{R}^V\)), a Masked Volumetric Autoencoder applies a high random masking ratio \(\rho = 0.75\) before processing visible patches with a Vision Transformer, compelling the network to infer global contextual semantics from sparse activations and producing spatial tokens \(\mathbf{F}_{spa} \in \mathbb{R}^{N_s \times D}\). For the temporal stream (EEG and fNIRS time-series \(\mathbf{x}^\tau \in \mathbb{R}^{C \times T}\)), a 1D-CNN equipped with Rotary Position Embedding (RoPE) preserves the relative phase of rapid electrical oscillations and slower hemodynamic trends, generating temporal features \(\mathbf{F}_{temp} \in \mathbb{R}^{T \times D}\). Subsequently, the Neuro-Perceiver Bridge deploys \(K=64\) learnable spatial queries \(\mathbf{Q}_S\) and temporal queries \(\mathbf{Q}_T\) to extract static entities and dynamic editing directions via cross-attention, functioning as an information bottleneck to suppress physiological noise. Finally, an adaptive gated fusion synthesizes the streams via a learnable scalar \(\alpha\):

\[\mathbf{c}_{brain} = \sigma(\alpha) \cdot \mathbf{H}_S + (1 - \sigma(\alpha)) \cdot \mathbf{H}_T\]

producing a compact, noise-free neural condition token sequence \(\mathbf{c}_{brain}\) ready to condition the generative backbone.

2. Decoupled Flow Guidance: Harmonizing Multi-Modal Guidance and Eliminating Modality Dominance

In multi-modal context generation and editing, explicit visual anchors, text instructions, and implicit neural intents often exert conflicting directional forces on the generative latent field. Direct joint conditioning causes semantic drift and structural degradation. MinD-Flow resolves this via Decoupled Flow Guidance (DFG). Let the reference flow driven purely by the source image latent \(\mathbf{c}_{img}\) be \(\mathbf{v}_{ref}(\mathbf{z}_t, t) = \mathbf{v}_\theta(\mathbf{z}_t, t, \mathbf{c}_{img})\). The editing forces of brain signals and text prompts are explicitly isolated as velocity deviations relative to this reference baseline:

\[\Delta \mathbf{v}_{brain} = \mathbf{v}_\theta(\mathbf{z}_t, t, \mathbf{c}_{brain}) - \mathbf{v}_{ref}(\mathbf{z}_t, t), \quad \Delta \mathbf{v}_{text} = \mathbf{v}_\theta(\mathbf{z}_t, t, \mathbf{c}_{text}) - \mathbf{v}_{ref}(\mathbf{z}_t, t)\]

The global composed velocity field \(\mathbf{v}_{composed}\) is formed as a linear composition of the reference flow and scaled delta deviations:

\[\mathbf{v}_{composed}(\mathbf{z}_t, t) = \mathbf{v}_{ref}(\mathbf{z}_t, t) + \gamma_b \cdot \Delta \mathbf{v}_{brain} + \gamma_t \cdot \Delta \mathbf{v}_{text}\]

where guidance scales \(\gamma_b = 3.5\) and \(\gamma_t = 4.0\) independently govern neural and textual strengths. This decoupling ensures that background layout remains anchored to the reference while neural and linguistic signals dictate contextual shifts without mutually washing out.

3. Neuro-Adaptive Soft Masking and Inversion-Free Trajectory Mixing: Deterministic Editing in a Single Forward Pass

Traditional diffusion editing relies on inverting the source image back to Gaussian noise via DDIM or Null-Text inversion, which is computationally sluggish and degrades high-frequency textural fidelity. Taking advantage of the straight ODE trajectories in Rectified Flow, MinD-Flow initiates the editing trajectory directly at the source latent: \(\mathbf{z}_{edit}^{t=0} = \mathbf{z}_{src}\). Because the conditioned model naturally targets the source image, the baseline velocity is near zero. To restrict manipulation strictly to intended regions, cross-attention maps between intermediate latents \(\mathbf{z}_t\) and conditioning tokens are soft-thresholded at the 60th percentile to yield a continuous Neuro-Adaptive Soft Mask \(\mathbf{M}_t\). The resulting editing ODE is formulated as:

\[d\mathbf{z}_t^{edit} = \left[ \mathbf{v}_{ref}(\mathbf{z}_t, t) + \mathbf{M}_t \odot (\mathbf{v}_{composed}(\mathbf{z}_t, t) - \mathbf{v}_{ref}(\mathbf{z}_t, t)) \right] dt\]

For unedited background regions (\(\mathbf{M}_t \approx 0\)), the trajectory locks onto \(\mathbf{v}_{ref}\), guaranteeing pixel-perfect background preservation. In target foreground regions (\(\mathbf{M}_t \approx 1\)), the trajectory transitions to \(\mathbf{v}_{composed}\), executing brain-driven semantic modifications. A straightforward Euler solver integrates this ODE from \(t=0\) to \(1\) in only 5 steps, completely bypassing iterative inversion loops.

Loss & Training

During training, the pre-trained FLUX.1-dev MM-DiT backbone and T5-XXL linguistic encoder remain frozen, optimizing only the neural encoder, NPB, and cross-attention adaptation layers. To empower Decoupled Flow Guidance at inference time, stochastic Condition Dropout randomly replaces brain, visual, or text conditions with empty tokens \(\emptyset\) with probability \(p=0.1\). The multi-modal training objective combines flow matching with semantic CLIP regularization:

\[\mathcal{L} = \mathbb{E}_{t, \mathbf{z}_0, \mathbf{z}_1} \left[ \| \mathbf{v}_\theta(\mathbf{z}_t, t, \mathcal{C} \odot \mathbf{b}) - (\mathbf{z}_1 - \mathbf{z}_0) \|_2^2 \right] + \lambda_{sem} \mathcal{L}_{sem}(\mathbf{c}_{brain}, \mathbf{e}_{clip})\]

where \(\lambda_{sem} = 0.1\), and \(\mathcal{L}_{sem}\) forces the pooled brain tokens to align with the CLIP embedding of the target visual image. Training is conducted across 8 NVIDIA A800 GPUs for 50,000 steps using AdamW optimization.

Key Experimental Results

Main Results

The authors evaluate MinD-Flow on the newly established Brain-Gen Benchmark, comprising Neuro-Context (dual context fusion of reference image and fMRI without text), Neuro-Creation (tri-modal synthesis combining image, text, and fMRI), and Neuro-Editing (localized intent modification).

Table 1: Quantitative benchmark comparisons on Neuro-Context and Neuro-Creation

Task Method / Setting L1 ↓ L2 ↓ LPIPS ↓ PDist ↓ FID ↓ CLIP-I ↑ DINO ↑ CLIP-T ↑
Neuro-Context FLUX.1-R/MindEye 0.1956 0.0782 0.4215 0.4389 44.83 0.5312 0.5734 -
Neuro-Context FLUX.1-R/MindTuner 0.1742 0.0695 0.3581 0.3924 39.12 0.5586 0.6189 -
Neuro-Context MindCustomer 0.1583 0.0617 0.3294 0.3418 36.57 0.5630 0.6540 -
Neuro-Context MinD-Flow (Ours) 0.1427 0.0543 0.3068 0.2905 32.18 0.5914 0.6937 -
Neuro-Context Relative Gain vs. MindCustomer ↓9.9% ↓12.0% ↓6.9% ↓15.0% ↓12.0% ↑5.0% ↑6.1% -
Neuro-Creation FLUX.1-K/MindEye 0.1843 0.0712 0.3921 0.4215 39.84 0.6842 0.5813 0.2945
Neuro-Creation MindCustomer 0.1512 0.0589 0.3156 0.3384 33.18 0.7342 0.6615 0.3124
Neuro-Creation MinD-Flow (Ours) 0.1387 0.0512 0.2943 0.2916 29.74 0.7618 0.7042 0.3258
Neuro-Creation Relative Gain vs. MindCustomer ↓8.3% ↓13.1% ↓6.7% ↓13.8% ↓10.4% ↑3.8% ↑6.5% ↑4.3%

In the fine-grained Neuro-Editing benchmark, MinD-Flow sets new SOTA standards across all settings. Under Pure Neural inputs without text, MinD-Flow (EEG+fNIRS) achieves a 24.5% reduction in L1 error (0.1894 vs. 0.2509) and a 19.4% drop in LPIPS (0.4416 vs. 0.5479) over LoongX (All Neural). When augmented with text guidance (EEG+fNIRS+Text), MinD-Flow reaches peak performance (L1: 0.1356, FID: 29.13, CLIP-I: 0.7638), outperforming LoongX (Neural+Speech) with a 47.7% reduction in pixel reconstruction error.

Ablation Study

Systematic ablations inspect neural encoding, fusion paradigms, guidance formulations, loss designs, and inversion mechanisms.

Table 2: Ablation of key architectural and algorithmic components

Neuro Enc. Fusion Guidance Loss Creation LPIPS ↓ Creation CLIP-I ↑ Editing LPIPS ↓ Editing CLIP-I ↑
Abs. Pos. Concat Standard CFG MSE 0.4102 0.6415 0.4215 0.6482
Abs. Pos. Concat Standard CFG Flow Matching 0.3654 0.6723 0.3842 0.6814
Abs. Pos. Gated Decoupled FM + \(\mathcal{L}_{sem}\) 0.3105 0.7214 0.3489 0.7315
RoPE Concat Decoupled FM + \(\mathcal{L}_{sem}\) 0.3082 0.7356 0.3452 0.7402
RoPE Cross-Attn Decoupled FM + \(\mathcal{L}_{sem}\) 0.3015 0.7489 0.3415 0.7510
RoPE Gated Standard CFG FM + \(\mathcal{L}_{sem}\) 0.3342 0.7105 0.3685 0.7015
RoPE Gated Decoupled Plain FM 0.2998 0.7512 0.3392 0.7556
RoPE (Full) Gated (Full) Decoupled (Full) FM + \(\mathcal{L}_{sem}\) 0.2943 0.7618 0.3195 0.7638

Table 3: Generation and inversion policy efficiency comparison in Neuro-Editing

Method Steps L1 ↓ LPIPS ↓ CLIP-I ↑ DINO ↑ CLIP-T ↑
DDIM Inversion 10 0.2147 0.4413 0.6158 0.4824 0.2249
DDIM Inversion 50 0.1853 0.4027 0.6514 0.5138 0.2352
Null-Text Inversion 10 0.1917 0.4108 0.6342 0.5019 0.2316
Null-Text Inversion 50 0.1394 0.3287 0.7453 0.6082 0.2558
Flow ODE Inversion 10 0.1683 0.3846 0.6917 0.5528 0.2414
Flow ODE Inversion 50 0.1529 0.3562 0.7154 0.5847 0.2483
MinD-Flow (Inversion-Free) 5 0.1356 0.3195 0.7638 0.6241 0.2645

Key Findings

  • Superiority of Inversion-Free Solver: With only 5 Euler steps, MinD-Flow outperforms 50-step Null-Text Inversion across all metrics (L1: 0.1356 vs. 0.1394, CLIP-I: 0.7638 vs. 0.7453), slashing function evaluations by 90% while preventing cumulative quantization drift.
  • Critical Role of Decoupled Flow Guidance: Replacing traditional CFG with DFG slashes Editing LPIPS from 0.3685 to 0.3195, effectively preventing the multi-modal overshadowing of subtle neural signals by explicit background layouts.
  • Neurobiological Cognitive Grounding: ROI importance mapping highlights early visual areas correlating with low-level structure, while LOC and PPA dictate high-level semantic layout and scene context, and the retrosplenial cortex (RSC) anchors contextual alignment. Spectrally, low-frequency oscillations (\(\theta, \alpha\)) govern global context updates, whereas high-frequency rhythms (\(\beta, \gamma\)) drive localized object manipulation.

Highlights & Insights

  • Pioneers the use of Rectified Flow for brain-driven visual editing, establishing an inversion-free trajectory mixing paradigm that circumvents the error-prone noise reversal cycles of diffusion models.
  • The Decoupled Flow Guidance decomposes velocity fields into reference structural baselines and delta semantic forces, eliminating multi-modal competition.
  • Neuro-Adaptive Soft Masking dynamically determines soft editing boundaries from cross-attention statistics, outperforming both hard cutoffs and unmasked diffusion.
  • Introduces the Brain-Gen Benchmark, formalizing evaluation standards across Neuro-Context, Neuro-Creation, and Neuro-Editing tasks.

Limitations & Future Work

  • Evaluation remains based on offline fMRI and EEG datasets; deploying real-time interactive BCI systems requires addressing streaming wireless data transfer and motion artifacts.
  • Cross-subject generalization still benefits from subject-specific calibration data; developing adaptive functional topology networks would further improve zero-shot transfer.
  • The current scope focuses on 2D image synthesis; extending rectified flow formulations to continuous temporal video editing and dynamic 3D neural generation represents an exciting frontier.
  • vs MindEye / NeuroPictor / UMBRAE: Prior works focused on decoding fMRI signals to reconstruct static images without interactive manipulation; MinD-Flow enables multi-modal contextual synthesis and fine-grained localized editing.
  • vs Null-Text Inversion / FlowEdit: Existing editing methods demand expensive inversion steps to reconstruct initial latents; MinD-Flow starts directly at source image latents and mixes trajectories in 5 forward steps.
  • vs LoongX / Uni-Neur2Img: Earlier brain-guided editing tools suffered from background distortion and modality competition under diffusion architectures; MinD-Flow achieves pixel-level background preservation via Rectified Flow ODEs and DFG.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Pioneering inversion-free rectified flow architecture for brain-driven image editing, featuring decoupled velocity fields and adaptive soft masking.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive benchmarks spanning three multi-modal tasks, four neural modalities, in-depth ablation studies, cognitive neuroscience mapping, and user studies.
  • Writing Quality: ⭐⭐⭐⭐⭐ Mathematically clear, systematically structured, and well-grounded in cognitive neuroscience principles.
  • Value: ⭐⭐⭐⭐⭐ Establishes a highly efficient, real-time algorithmic foundation for next-generation interactive brain-computer interfaces.