Transport Discrepancy as a Reliability Signal for Vision-Language-Action Models

Conference: ECCV 2026
Paper: ECCV official page
Project: DiG
Area: Robotics & Embodied AI
Keywords: vision-language-action models, transport discrepancy, reliability gating, flow matching, out-of-distribution generalization

TL;DR

DiG uses sliced Wasserstein transport cost between observation features and action representations as an internal reliability signal to gate feature residuals, training sample weights, and inference-time refinement, improving \(\pi_{0.5}\) from 92.4% to 96.4% on LIBERO-Long and from 41.4% to 52.6% on low-data RoboCasa.

Background & Motivation

Vision-language-action (VLA) models typically encode images, instructions, and robot state into context features, then generate continuous action chunks through a flow-matching action expert. Producing a smooth control sequence does not establish that the expert can reliably interpret its current context: the pretrained backbone spans a large representation space, whereas the region that the action head can decode correctly may be much smaller. Background changes, unfamiliar objects, and interventions can move features outside that region; long-horizon execution also feeds early errors into subsequent observations, accumulating drift.

Scaling data or changing the action generator does not directly answer whether a particular prediction should be trusted. The standard flow-matching loss supervises a velocity field with demonstration actions during training, but the corresponding ground-truth actions are unavailable at deployment. This paper builds a low-cost comparison interface from the action expert's existing input projection: when observation features are incompatible with the projected action chunk, that action representation should not strongly drive feature correction, and similar training pairs should not dominate gradient updates.

The approach does not train an additional success classifier. Instead, it extracts a signal related to observation-action compatibility from the policy itself and shares it between training and inference. Core Idea: turn transport discrepancy in a shared feature space into a monotonically decreasing trust gate, allowing compatible pairs to contribute more strongly to residual refinement and learning while treating incompatible pairs conservatively, then recompute the gate using current action predictions to refine the chunk.

Method

Overall Architecture

DiG is inserted between the backbone and action expert without changing the flow-matching target velocity or replacing action chunks with discrete labels. Its inputs are the context features derived from the current observation and an action chunk used to assess compatibility; its output is a context representation modulated by a gated residual, which conditions the original continuous action generator.

During training, the comparison branch uses the demonstrated ground-truth action. During inference, the initial comparison uses the previous control step's predicted chunk, and subsequent comparisons use the current prediction. The refinement loop below runs only at inference time, while the weighted-loss branch operates only during training; they are not a single shared forward computation.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    OBS["Current observation<br/>and instruction"] --> H["Backbone context features"]
    ACT["Training: ground-truth actions<br/>Initial inference: previous chunk"] --> SPACE["Shared-Space Discrepancy"]
    H --> SPACE
    SPACE --> GATE["Conservative Reliability Gating"]
    H --> GATE
    GATE --> HEAD["Action expert<br/>Current action chunk"]
    GATE -->|Training only: stop gate gradients| LOSS["Weighted flow-matching loss"]
    HEAD -->|Inference only| REFINE["Prediction-Based Refinement"]
    REFINE -->|Next round: current prediction| SPACE
    REFINE -->|Round budget reached| EXEC["Execute first action<br/>Replan at next control step"]

Key Designs

1. Shared-Space Discrepancy: compare observation tokens and action chunks in common coordinates

The backbone outputs a sequence of context features \(H\), and the action expert already contains an input projection \(f\) that maps action vectors into its internal feature dimension. DiG reuses this projection, maps each time step of an action chunk, and mean-pools the results into an action centroid. It does not train a separate action encoder, so the comparison coordinates remain connected to the actual action-generation pathway; the projection continues to learn through the normal flow-matching loss rather than through additional alignment labels.

The comparison is neither between two action sequences nor a Euclidean distance between current and previous actions. The observation side is the empirical distribution of context tokens, while the action side broadcasts the same centroid to the same sequence length, effectively forming a point mass at that centroid. The method projects both distributions along \(M\) random unit directions, computes the one-dimensional squared Wasserstein cost for each direction, and averages these costs into discrepancy \(D\). Because the action distribution contains only one centroid, each one-dimensional cost is the mean squared distance from the projected observation tokens to the projected centroid; general multipoint distribution matching is unnecessary.

This definition reflects both displacement and dispersion of observation features around the action centroid, unlike cosine similarity between two pooled vectors. It also creates an information bottleneck: the centroid discards action order, so temporally different chunks with similar average representations can produce similar discrepancies. This compression applies only to the reliability branch; the flow-matching head still handles the full action chunk. Multiple centroids or temporal bins are future directions mentioned by the authors, not implemented and validated components of the current method.

2. Conservative Reliability Gating: higher discrepancy means less trust in action-guided residuals

An exponential mapping converts discrepancy into a gate, with a lower bound preventing training weights from reaching zero. The mapping explicitly described in the text is:

\[ g=\max\{g_{\min},\exp(-\tau D)\},\qquad g_{\min}=0.05. \]

A larger \(\tau\) makes the gate decay more rapidly with discrepancy. Here it controls sensitivity, so the inverse intuition associated with some softmax temperatures does not apply. Low discrepancy produces a larger gate and high discrepancy a smaller one, but this scalar is not a success probability calibrated against success/failure labels. The theoretical motivation is that, under a local Gaussian residual model, large squared deviations increasingly weaken the explanation that a pair lies near a decodable region, making an exponential function a proxy for compatibility.

The gate enters two pathways. The feature pathway applies a lightweight linear residual operator \(R\) to the normalized context consumed by the action expert and scales the correction with the gate and strength \(\lambda\); the loss pathway uses the same gate to weight the sample's flow-matching loss. High discrepancy reduces the action-guided residual, rather than directly stopping the robot, rejecting the action, or invoking a safety controller. The important direction is conservative: DiG does not apply a stronger correction simply because a prediction is less reliable; it avoids letting an unreliable action representation drive a large correction.

During training, stop-gradient applies only to the gate. Its forward value remains unchanged, but the model cannot exploit the branch that directly changes discrepancy to manipulate its loss weight; the backbone, residual operator, and action expert, including the reused projection \(f\), still learn through the original action-learning task. This does not freeze discrepancy: normal updates to representations can change it indirectly. Figure 8b shows discrepancy declining and remaining nonzero, which supports the absence of obvious collapse in that run rather than guaranteeing it under all training conditions.

3. Prediction-Based Refinement: break the circular dependency with an old chunk, then reassess trust using new predictions

At deployment, ground-truth actions are unavailable. The gate depends on an action, while action generation depends on the gate, creating a circular dependency. DiG-Refine first compares the previous control step's predicted chunk with the current context to obtain an initial gate and generate a new chunk. It then feeds this prediction back through the same projection and discrepancy branch, updates the gate, and reruns the action expert. After \(N\) refinement rounds, it executes the first action of the final chunk and replans at the next control step.

Thus, \(N=0\) already includes one action-generation pass; it simply does not update the gate using the current prediction. With \(N=3\), there are 3 additional action-expert passes after the initial prediction, not 3 passes in total. The previous chunk can be stale, especially after an abrupt observation change, which explains why the first current-prediction update can be particularly valuable. Each round reuses the current control step's backbone features and residual-operator output rather than rerunning the vision-language backbone. This saves backbone computation, not the cost of repeated action generation.

The text reports saturation after roughly 3 rounds and approximately 8 ms of additional latency per round for \(\pi_{0.5}\) on a single A100, or about 24 ms for 3 rounds. These are incremental costs, not total policy latency; meeting the control deadline also depends on the original model and system overhead. The supplied main text does not specify how the first control step is initialized when no previous predicted chunk exists. Reproduction requires checking this detail rather than attributing a zero-action initialization to the authors.

A Worked Example

Consider a task of sorting objects into a drawer, as evaluated in the paper. A background change during execution makes the current observation context less compatible with the previous predicted action chunk. The resulting high initial discrepancy keeps the residual small, and the policy generates its current action from this more conservatively conditioned representation instead of letting the old chunk force a large feature correction.

The current prediction is then projected into an action centroid and compared with the same current observation. If compatibility improves, the gate increases and allows the residual to contribute more strongly to the next prediction; otherwise, the correction remains suppressed. This is a self-consistency iteration between representations and predictions. The robot does not execute an action or receive a new environment observation after every internal refinement round.

With \(N=3\), one initial prediction plus 3 updates calls the action expert 4 times before executing the first action. This example illustrates the mechanism rather than reproducing a recorded rollout. It does not imply that discrepancy decreases monotonically at every round or that an internally consistent prediction must complete the task.

Loss & Training

The base policy retains flow matching: it interpolates Gaussian noise and a demonstrated action according to time and trains the expert to predict the corresponding conditional velocity. DiG does not change that target velocity; it changes the conditioning representation and each sample's contribution to the loss. The reliability branch computes the gate from the ground-truth action, while the flow-matching branch continues to receive its noised action and time condition. These action inputs serve different purposes and should not be conflated.

After stopping gate gradients, training treats the current gate value as a constant sample weight. The authors' mixture-contamination model separates coherent pairs from shortcut pairs and assumes larger gates for the former; under that assumption, reweighting reduces the shortcut component's mass. This supports the design rationale, but it does not prove that high discrepancy identifies shortcuts in real data or that rare valid actions will avoid downweighting. The gate floor mitigates this risk without eliminating it.

The hyperparameter discussion states that \(M=16\) captures most gains, with diminishing returns beyond \(M=32\); an excessively large \(\lambda\) amplifies residuals too strongly, while a very small \(\tau\) makes the gate nearly constant. The prose gives \(\lambda\approx0.1,\tau\approx1.0\) as useful moderate values, but Figure 9 displays a residual-strength axis of approximately 0.20–0.60. This inconsistency is preserved rather than treating those values as verified defaults. The conclusion also says temperature and projection count are tuned per backbone: a common gate floor across tasks does not establish that every hyperparameter transfers without tuning.

Several displayed equations are incompletely extracted in the local full text, and the referenced supplementary material is absent. The descriptions above therefore use readable textual definitions for discrepancy, residual refinement, and the training objective, retaining only the exponential gate equation confirmed by the prose. They do not reconstruct damaged equation numbering, the exact supplementary estimator, proofs, or missing training configurations.

Key Experimental Results

Main Results

LIBERO contains 40 tasks in four suites, Spatial, Object, Goal, and Long, with 50 evaluation rollouts per task. RoboCasa uses a low-data setting with 24 tasks and 50 demonstrations per task, also evaluated with 50 rollouts per task. The table below selects results from Tables 1 and 2. Gains are success-rate percentage points; absolute success rates across the two benchmarks are not directly comparable.

Benchmark and metric	Backbone	Baseline success (%)	With DiG (%)	Gain (percentage points)
LIBERO Avg	GR00T-N1	93.9	95.3	+1.4
LIBERO Avg	\(\pi_{0.5}\)	96.9	98.3	+1.4
LIBERO Long	\(\pi_{0.5}\)	92.4	96.4	+4.0
RoboCasa Avg	GR00T-N1	36.0	43.2	+7.2
RoboCasa Avg	\(\pi_{0.5}\)	41.4	52.6	+11.2
RoboCasa Doors/Drawers	\(\pi_{0.5}\)	57.8	73.4	+15.6

The real-robot results below are selected from Tables 4 and 5 and report only whole-task success, not sub-task success. Each condition has 20 trials, so one additional success represents 5 percentage points. These are small-sample estimates, and the original tables do not include confidence intervals.

Platform and condition	Task	\(\pi_{0.5}\) (%)	\(\pi_{0.5}\) + DiG (%)	Gain (percentage points)
Franka OOD	Stack-Bowls	15	40	+25
Franka OOD	Spray-Plant	10	30	+20
Franka OOD	Wipe-Whiteboard	20	30	+10
Franka OOD	Sort-Into-Drawer	20	35	+15
Adam-U Unseen	3-object clean-up	25	40	+15

Ablation Study

Table 6 reports LIBERO success rates. Metric ablations retain the exponential gate form, while gate ablations retain the corresponding comparison pathway; the MLP variant instead replaces transport-based gating with a learned scalar gate on pooled features. The full model below is Ours in the source table, and average drops are relative to its Avg=98.3.

Config	Avg (%)	Long (%)	Avg drop (percentage points)	Note
Full DiG	98.3	96.4	0.0	Sliced Wasserstein + adaptive gating
Cosine	96.5	92.6	1.8	Alternative discrepancy metric
MMD	96.8	93.4	1.5	Kernel distribution discrepancy
Sinkhorn	97.9	95.8	0.4	Alternative transport metric
Fixed gate	92.3	84.6	6.0	Fixed \(g=0.5\)
Random gate	89.8	80.8	8.5	Random modulation
\(\lambda=0\)	97.3	94.2	1.0	No residual; retain loss reweighting
MLP gate	97.5	94.8	0.8	Learned scalar gate instead of transport gate

Key Findings

• Gains are associated with long horizons, scarce data, and distribution shift: LIBERO Avg improves by only 1.4 percentage points, compared with 4.0 on Long and 11.2 for \(\pi_{0.5}\) on RoboCasa Avg. These settings support a common trend without isolating every confounding factor.
• Benefits remain without residual refinement: \(\lambda=0\) reaches Avg=97.3, above the original \(\pi_{0.5}\) at 96.9 but below full DiG at 98.3. Both weighting and conditioning refinement contribute; a fixed gate even underperforms the original baseline, so merely adding a residual is not inherently beneficial.
• Not every result improves: under cosine+sine perturbations in Table 3, Spatial falls from 87.8 to 86.4, while Long rises from 67.8 to 79.2. Average robustness gains do not establish regression-free performance on every task.

Highlights & Insights

• Reuse the action expert's own projection. The reliability comparison attaches to an action interface already trained by the policy, avoiding extra success/failure labels. Common coordinates support interpretable comparison but do not automatically calibrate cross-modal distances.
• Connect training and inference with one signal. Training reduces the influence of incompatible pairs, while deployment limits corrections driven by untrusted actions. The transferable idea is to estimate trust before choosing correction strength, rather than simply expanding network capacity.
• Iterate only the action expert. Current-observation backbone features can be cached, concentrating extra computation in the smaller action branch. A related design opportunity is short self-consistency refinement of downstream conditional generation while expensive representations remain fixed.

Limitations & Future Work

• Evaluation covers only two flow-matching VLA architectures. Discrete-action policies, 3D representations, and other embodiments require further experiments; interface compatibility does not establish universal applicability.
• Action centroids discard temporal structure and can miss incorrect action order when average representations remain similar. Temporal bins and multiple centroids deserve testing but introduce computation and hyperparameters.
• High discrepancy can arise from rare valid demonstrations, while internally consistent but physically incorrect predictions can receive high gates. No calibrated failure-detection rate or safety guarantee is established here, so the signal should not be treated as a safety certificate.
• The theory relies on local Gaussian noise and an idealized mixture, assuming that coherent pairs receive larger gates. Independent reliability diagnostics are needed to establish whether real distribution shifts satisfy this condition.
• Inference adds \(N\) action-expert passes, potentially exceeding high-frequency control budgets. Real-robot cells contain only 20 trials; statistical robustness and repeatability across random seeds need more evidence.
• The supplied main text does not fully specify first-step initialization, the supplementary estimator, or training reproduction settings, and Figure 9 differs from the hyperparameter prose as noted above. These details require checking the typeset paper and supplement rather than filling gaps with customary values.

Related Work & Insights

• vs \(\pi_{0.5}\) / GR00T-N1: these provide the backbone and action-expert architectures; DiG is an incremental module at their representation interface, not a replacement policy. Same-backbone comparisons in Tables 1 and 2 most directly support its contribution.
• vs OT-CFM: OT-CFM uses optimal transport to improve probability paths or training couplings, whereas DiG treats transport cost as a compatibility signal. It acts at a different location and does not redefine the flow-matching target velocity.
• vs domain randomization / domain adaptation: those approaches broaden training coverage or align source and target distributions. DiG adjusts trust using an internal signal at each step and may complement them, but the paper does not show that it replaces data coverage.
• Research direction: with the action head and compute budget controlled, compare centroid, temporal-bin, and multi-centroid discrepancies on failures with similar mean actions but incorrect order. This directly tests the current signal's information bottleneck and is a proposed experiment, not a result reported in the paper.

Rating

• Novelty: 4/5. Observation-action transport discrepancy unifies trust gating, loss reweighting, and inference refinement in a targeted design.
• Experimental Thoroughness: 4/5. Two backbones, simulation, real robots, perturbations, and mechanism ablations provide broad evidence, but real-robot samples are small and reproduction details depend on an unavailable supplement.
• Writing Quality: 3/5. The main argument is clear, but hyperparameter text and plots disagree, and reliability interpretation must be distinguished from probability calibration and safety guarantees.
• Value: 4/5. A practical robustness module for continuous-action policies with notable long-horizon gains, subject to deployment latency and reliability-signal failure modes.