Vision Bridge Transformer at Scale

Conference: ECCV2026
Paper: ECCV Original Paper
Area: Image Generation
Keywords: Brownian bridge, stabilized velocity matching, image editing, video stylization, variance-corrected sampling

TL;DR

ViBT replaces noise-to-target conditional generation with source-to-target Brownian bridge transport, using stabilized velocity matching to adapt large models; its 20B image-editing experiment reports a 2.3-fold inference speedup while the average ImgEdit score decreases from 3.90 to 3.76.

Background & Motivation

Image editing often changes only part of an existing image, yet conditional diffusion models generate the entire target from noise while receiving the original image as an additional condition. For edits that preserve the subject and change only the background or style, the model must reconstruct substantial content that already exists. Transformers also process extra visual conditioning tokens, an especially expensive requirement for video.

A Brownian bridge offers a more direct formulation: the source image or video becomes the starting point, and the model learns a stochastic path to the target. However, moving small bridge models to large Transformers requires more than replacing the input. Ordinary velocity targets diverge near the endpoint, whereas displacement targets vanish there. These objectives emphasize different time regions and make stable adaptation of pretrained generators difficult.

The paper first runs a controlled comparison from the same SD3.5-Large backbone, then extends the approach to Wan 2.1 1.3B for video and a 20B Qwen editing model. Core Idea: retain pretrained visual priors, represent the visual condition as the transport starting point instead of an extra input, and stabilize target scales and sampling noise so that large models can learn and execute source-to-target bridges reliably.

Method

Overall Architecture

Training uses paired source and target images or videos with corresponding text conditions, with a pretrained VAE encoding the visual inputs into latents. ViBT constructs noisy bridge states between paired latents, trains a Transformer to predict target-directed velocity, and stabilizes the prediction error. Inference has no target image available: it starts from the source latent, applies variance-corrected stochastic updates, and decodes the result with the VAE.

Text instructions still specify the requested edit. The removed input is the additional visual condition, not all conditioning information. Stabilization applies only when computing the training loss; the network continues to output the raw velocity required by the sampler.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Source image or video<br/>Text instruction"] --> Bridge["Latent Data Bridge"]
    Target["Paired target<br/>Training only"] --> Bridge
    Bridge --> Stable["Stabilized Velocity Matching"]
    Stable -->|Trained velocity network| Sample["Variance-Corrected Sampling"]
    Input -->|Source latent initialization| Sample
    Sample --> Output["VAE decoding<br/>Edited image or video"]

Key Designs

1. Latent Data Bridge: start from existing visual structure

Let \(x_0\) and \(x_1\) denote source and target latents. At the default noise scale, Eq. (7) constructs a training state through linear interpolation plus Brownian bridge noise:

\[ x_t=(1-t)x_0+t x_1+\sqrt{t(1-t)}\,\epsilon, \qquad \epsilon\sim\mathcal N(0,I). \]

The noise vanishes at both endpoints and grows in the middle of the path. The start therefore preserves the input, the end corresponds to the training target, and intermediate states still allow stochastic variation. Unlike a noise state accompanied by source-image conditioning, the source already participates in the generation state, eliminating the extra visual token sequence. The controlled efficiency result follows from this conditioning change; it does not establish fewer sampling steps for every task.

The Transformer inherits semantic and synthesis capabilities from a pretrained generative model. The image-editing variant removes the Qwen backbone's visual conditioning input channel and adapts it through post-training. Accordingly, scaling to 20B means adapting an existing 20B backbone, not training a 20B bridge model from scratch.

2. Stabilized Velocity Matching: correct target-scale imbalance over time

The conditional Brownian bridge velocity is \((x_1-x_t)/(1-t)\). Substituting the bridge state reveals a noise component that grows as \(t\) approaches 1, so ordinary velocity mean squared error is dominated by near-terminal samples. Predicting the remaining displacement avoids divergence, but that displacement vanishes near the endpoint and instead emphasizes earlier times. The problem is not simply insufficient model capacity: supervision has incompatible scales across time.

Eqs. (12)-(15) normalize both prediction and target using the endpoint distance and latent dimension \(D\). For endpoint pairs with \(\|x_1-x_0\|>0\), the following expresses the original objective as an equivalent weighted velocity loss:

\[ \alpha^2=1+\frac{tD}{(1-t)\|x_1-x_0\|^2}, \qquad \mathcal L=\mathbb E\left[\frac{1}{\alpha^2} \left\|v_\theta(x_t,t)-\frac{x_1-x_t}{1-t}\right\|^2\right]. \]

The normalized target satisfies \(\mathbb E_\epsilon\|\tilde u_t\|^2=\|x_1-x_0\|^2\): its second moment no longer varies with time for a fixed endpoint pair. This does not equalize all sample difficulties or guarantee a constant empirical optimization loss. It removes a specific time-dependent scale bias. Since the weight also depends on the endpoint distance, it is not merely a function of time.

An important implementation detail is that the network still outputs \(v_\theta\); normalization occurs only in the loss. Inference does not require the true target to compute this factor. Although the input header of Algorithm 2 lists an endpoint pair, the actual updates use only the source state, time, and predicted velocity. That header should not be interpreted as requiring a target image at test time.

3. Variance-Corrected Sampling: reduce excessive noise near the endpoint

Standard Euler-Maruyama updates scale noise by the square root of the step size, whereas a bridge should progressively reduce uncertainty near its target. With finite steps, the ordinary update injects excessive late-stage noise. Eq. (17) corrects the stochastic term using the remaining time:

\[ x_{k+1}=x_k+\Delta t_k\,v_\theta(x_k,t_k) +\sqrt{\Delta t_k\frac{1-t_{k+1}}{1-t_k}}\,\epsilon_k, \qquad \epsilon_k\sim\mathcal N(0,I). \]

When the final step reaches \(t_{k+1}=1\), the stochastic term becomes zero. This corrects discrete sampling noise rather than training an additional artifact-removal module. Figure 10 shows visible artifacts without correction. The main text also mentions a paired evaluation on 200 editing samples, but the numerical metrics are in Table S5, which is absent from the cache; this note does not invent those values.

All formulas above use the default \(s=1\). The paper also introduces a global diffusion coefficient \(s\) to control path stochasticity and refers to Appendix C for the complete training and inference modifications. Since the appendix is absent from the cache, this note reports only the verifiable main-text noise ablations rather than reconstructing unverified implementation details.

Loss & Training

Algorithm 1 uniformly samples time, draws Gaussian noise, constructs a bridge state, and updates parameters using stabilized velocity error. Tasks share this formulation but use different backbones and datasets; their results should not be interpreted as a single model undergoing one unified capability evaluation.

• Controlled image editing: both models start from SD3.5-Large with the same dataset, optimization strategy, and training iterations; Table 1 isolates the paradigm change.
• Video tasks: initialization uses Wan 2.1 1.3B; video stylization trains on 10,000 samples from Ditto-1M.
• 20B image editing: rank-128 LoRA, 8 NVIDIA H100 GPUs, 20,000 iterations, and per-GPU batch size 1; the authors report 128 GPU-hours in total.
• 20B adaptation data: 5,000 synthetic editing pairs constructed from Open Images, evaluated on ImgEdit-Bench; the best image-editing configuration uses \(s=0.5\).

Key Experimental Results

Main Results

The table selects representative results from Tables 1, 2, 3, and 5. Each row supports a comparison only within its corresponding task and setup. ImgEdit is an editing benchmark score, not classification accuracy; metric directions are stated explicitly.

Setup and source table	Metric	Comparison method	ViBT / Bridge	Interpretation
Controlled SD3.5-Large comparison, Table 1	Average ImgEdit, higher is better	Conditional DiT: 2.43	3.18	0.75 points higher at equal training iterations
Same setup, Table 1	Human preference	Conditional DiT: 40.7%	59.3%	Preference for the bridge under this setup
Same setup, Table 1	Training GPU-hours / inference seconds	162 / 14.5	96 / 6.2	Not absolute timings for the 20B model
Video stylization, Table 2	MUSIQ, higher is better	Ins.V2V: 60.621	64.045	Improved perceptual quality
Video stylization, Table 2	CLIP Score, higher is better	Ins.V2V: 0.827	0.782	Text alignment is not the best
Depth-to-video, Table 3	PSNR / DISTS	Wan Fun Control: 10.899 / 0.281	11.403 / 0.230	Higher PSNR and lower DISTS are better
Depth-to-video, Table 3	VBench, higher is better	Wan Fun Control: 0.69	0.71	Reported to two decimal places in the source
20B image editing, Table 5	Average ImgEdit, higher is better	Qwen-Image-Edit: 3.90	3.76	Competitive, but 0.14 points lower

The stylization test uses 100 videos generated by Wan 2.2 14B from MovieGen Bench prompts, each with a random style instruction. A separate user study involves 26 participants, 5 dimensions, and ratings from 1 to 3. Depth-to-video training uses 1,003 generated videos paired with Depth Anything V2 maps; testing uses 946 videos generated from VBench prompts.

The main text also reports evaluation on 500 real OpenVid clips, but the results are in Table S3, absent from the cache. The 20B model's 2.3-fold speedup is the authors' report in Section 5.3; precise timing details are delegated to Appendix A. Table 1's seconds must not be reused to fill that gap.

Ablation Study

Table 8 compares training objectives. Stabilized velocity obtains an ImgEdit score of 3.55 under the default noise setting, which must not be conflated with the 3.76 obtained after selecting \(s=0.5\).

Training objective, Table 8	Depth-to-video PSNR	DISTS, lower is better	VBench	ImgEdit
Displacement matching	11.04	0.26	0.695	3.50
Ordinary velocity matching	10.81	0.27	0.698	3.36
Stabilized velocity matching	11.40	0.23	0.71	3.55

The noise ablations in Tables 6 and 7 show that stabilizing the objective does not produce one optimal noise scale for every task.

Noise scale	Depth-to-video VBench, Table 6	Average image-editing score, Table 7
\(s=0\)	0.604	3.10
\(s=0.1\)	0.536	3.03
\(s=0.5\)	0.666	3.76
\(s=1\)	0.709	3.55
\(s=2\)	0.711	3.44
\(s=4\)	0.482	2.97

Key Findings

• In Table 8, stabilized velocity improves ImgEdit by 0.19 points over ordinary velocity and 0.05 over displacement. Normalization helps, but does not account for every gain.
• Image editing favors the smaller \(s=0.5\), whereas depth-to-video VBench peaks at \(s=2\). This does not make \(s=2\) optimal for every depth-related metric.
• The 20B variant scores 4.87 on Style versus Qwen's 4.00, but 3.03 versus 3.90 on Remove and 3.95 versus 4.51 on Action. Preserving structure and substantially changing structure involve a real trade-off.

Highlights & Insights

• Visual conditioning need not always be an additional repeated input; it can be encoded in the initial generation state. For editing, this changes both computational cost and the transformation the model must learn.
• Stabilization controls the conditional target's second moment rather than heuristically clipping large gradients. It exposes a direct relationship between optimization behavior and path geometry.
• Large-model adaptation reuses generative priors instead of introducing a complicated new backbone. The contribution is a transferable post-training approach, not evidence that bridges replace diffusion on every metric.

Limitations & Future Work

• The authors explicitly identify task-dependent noise tuning and propose automatic selection as future work. Current results still require ablations rather than blindly adopting extremely small noise.
• Object removal and large action changes remain weaker than the Qwen backbone. Eliminating extra visual conditioning is not equivalent to improving every editing capability.
• Most primary video tests use model-generated inputs. The main text mentions real OpenVid validation, but Table S3 is absent from the local cache, preventing independent verification of its numbers.
• The normalization formula assumes non-degenerate endpoint pairs, and the main text does not specify protection for zero endpoint distance. Exact optimizer settings, sampling steps, time schedules, and 20B timing details cannot be fully recovered from this cache either.
• Source inconsistencies deserve caution: ViBT's Add score is 4.14 in Table 5 but 4.15 in Table 7, and the FLUX row in Table 5 has sub-scores that are difficult to reconcile with its average. This note does not derive additional rankings from those disputed entries.

Related Work & Insights

• vs Conditional DiT / Qwen-Image-Edit: these models receive the source as an additional condition, whereas ViBT starts sampling from the source latent. Table 1 supports controlled-setting gains; Table 5 shows the speed-quality trade-off after backbone adaptation.
• vs BBDM: both use Brownian bridges for image translation. ViBT focuses on stabilized velocity targets and adaptation of large pretrained Transformers, not the first introduction of bridge-based generation.
• vs LBM: the paper contrasts earlier preference for very small noise, \(s=0.005\), with its own task-dependent optima. A transferable lesson is to treat stochasticity as a task-specific design variable rather than a fixed constant.
• Future directions: adapt noise to the editing region or instruction strength and evaluate substantial structural edits, especially removal and action changes, separately. World and action models are proposed extensions, not demonstrated capabilities.

Rating

• Novelty: 4/5, interpretable objective stabilization and large-model bridge adaptation are the main contributions.
• Experimental Thoroughness: 4/5, controlled comparisons, multiple tasks, and objective and noise ablations are present, but appendix evidence cannot be checked in this cache.
• Writing Quality: 3/5, the main argument is clear, but algorithm inputs and some table entries are misleading or inconsistent.
• Value: 4/5, a practical modeling alternative with efficiency benefits for structure-preserving editing.