title: >- [Paper Note] Generating Multi-view Adversarial Examples for Visual Geometry Grounded Transformer description: >- [ECCV2026][3d_vision][Adversarial Attack] A real-time feed-forward multi-view adversarial perturbation generator (MVAP-G) that degrades VGGT 3D reconstruction in a single pass without test-time optimization. tags: - ECCV2026 - 3d_vision - Adversarial Attack - VGGT - 3D Foundation Model date: 2026-09-19 content_hash: a06578feb43ff578
Generating Multi-view Adversarial Examples for Visual Geometry Grounded Transformer¶
Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/qsong2001/mvap-g
Area: 3D Vision
Keywords: Adversarial Examples, 3D Vision Foundation Models, Multi-view Perturbations, VGGT, Geometric Consistency
TL;DR¶
Addressing the security vulnerabilities of the 3D vision foundation model VGGT in multi-view reconstruction, this paper introduces MVAP-G, the first feed-forward multi-view adversarial perturbation generator that produces visually imperceptible, cross-view consistent perturbations in a single forward pass without test-time iterative optimization, collapsing reconstructed 3D point clouds and disrupting downstream geometric poses in milliseconds.
Background & Motivation¶
With the advent of 3D vision foundation models such as the Visual Geometry Grounded Transformer (VGGT), feed-forward multi-view inference can directly predict high-quality 3D point clouds, depth maps, and camera poses from arbitrary image sequences. This unified paradigm provides substantial computational acceleration and powerful geometric representations for safety-critical systems, including autonomous driving, spatial tracking, and robot navigation. However, as foundation models scale in parameter size and real-world deployment, understanding their security against imperceptible adversarial perturbations has become an urgent priority. Traditional adversarial attacks, such as PGD and FGSM extensions, rely heavily on per-scene gradient-based iterative optimization. Such optimization incurs massive computational overhead—often requiring minutes per input scene—rendering them impractical in dynamic environments where multi-view scenes stream continuously.
To bypass per-scene optimization, Universal Adversarial Perturbations (UAPs) offer a precomputed, input-agnostic noise pattern that operates in a single step. However, static UAPs fundamentally fail when applied to VGGT because a fixed 2D pattern lacks content adaptiveness and cannot model the parallax, depth variation, and cross-view geometric correspondences inherent in 3D multi-view scenes. Consequently, VGGT's internal cross-view attention layers easily dilute and smooth away such unstructured noise. Furthermore, existing generative adversarial methods (such as ATN or GAP) are primarily designed for single-view architectures; they cannot handle multi-view inputs and lack mechanisms to coordinate perturbations across viewpoints.
The key challenge lies in synthesizing adversarial perturbations that preserve cross-view geometric coherence and strong destructive power in a single feed-forward step without test-time iterative searching. The core idea is to build a feed-forward multi-view perturbation generator (MVAP-G) equipped with a Cross-view Adversarial Alignment (CAA) module and a DPT dense prediction head, guided by a point-cloud collapse objective and progressive perceptual regularization to achieve real-time, imperceptible, and geometrically coordinated multi-view adversarial attacks.
Method¶
Overall Architecture¶
Given an input sequence of \(T\) multi-view images \(\mathcal{S} = \{I_t\}_{t=1}^T\), MVAP-G directly generates a set of frame-specific perturbations \(\{\delta_t\}_{t=1}^T\) in a single feed-forward pass under the \(L_\infty\) constraint \(\|\delta_t\|_\infty \le \epsilon\). The perturbed frames \(\hat{I}_t = I_t + \delta_t\) are then fed into the frozen, pre-trained VGGT model \(\mathcal{F}\), causing catastrophic failures in its reconstructed 3D point clouds \(\hat{P}\), estimated depth maps, and predicted camera poses.
The generator architecture couples a Transformer encoder with a Dense Prediction Transformer (DPT) decoding head. Each input frame is partitioned into \(16 \times 16\) non-overlapping patches and projected into latent feature tokens together with learnable camera positional embeddings. These tokens pass through multiple Transformer layers enhanced by Cross-view Adversarial Alignment (CAA), which models both intra-view spatial relationships and inter-view geometric correspondences. Multi-scale feature representations extracted from intermediate layers are subsequently assembled and progressively upsampled by the DPT head, outputting full-resolution unbounded perturbations that are finally clipped within the predefined perturbation budget.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Multi-view Sequence S = {I_t}<br/>Patch Embedding + Camera Positional Tokens"] --> B["Cross-view Adversarial Alignment (CAA)<br/>Joint Self-Attention and Cross-View Attention"]
B --> C["DPT Dense Prediction Head<br/>Multi-scale Readout from Layers 3, 6, 9, 12"]
C --> D["Dynamic Perceptual Regularization & Clipping<br/>LPIPS Visual Stealth Control + L_inf Projection"]
D --> E["Perturbed Image Sequence {I_t + δ_t}<br/>Feed-forward Attack on Frozen VGGT Model"]
E --> F["Point Cloud Collapse Objective<br/>Penalize Average Coordinate Norm towards Origin"]
Key Designs¶
1. Cross-view Adversarial Alignment: Modeling Global Inter-View Correlations to Breach Multi-View Fusion
The robustness of VGGT against naive single-view noise stems from its cross-view attention mechanism, which aggregates redundant spatial visual cues across multiple viewpoints to correct localized geometric errors. To disrupt such an architecture, the adversarial perturbation cannot be composed of disconnected per-frame artifacts; it must form a geometrically coordinated adversarial field that deceives cross-view correspondence matching.
MVAP-G introduces a Cross-view Adversarial Alignment (CAA) mechanism within each Transformer layer. For layer \(l\) and view \(t\), given the token embeddings \(\mathbf{z}_t^{(l-1)} \in \mathbb{R}^{N_p \times D}\) (with \(N_p = \frac{H}{16} \times \frac{W}{16}\) and embedding dimension \(D=384\)), CAA computes the aggregated attention contribution from all views \(k \in \{1, \dots, T\}\): $\(A_t^{(l)} = \sum_{k=1}^T \mathbf{A}_{tk}^{(l)} = \sum_{k=1}^T \text{softmax}\left(\frac{\mathbf{z}_t^{(l-1)} (\mathbf{z}_k^{(l-1)})^\top}{\sqrt{D}}\right) \mathbf{z}_k^{(l-1)}\)$ When \(t = k\), this operation calculates intra-view self-attention, encoding spatial textures and object semantics. When \(t \neq k\), it computes inter-view cross-attention, capturing epipolar geometry, parallax shifts, and shared scene context across frames. Layer representations are updated via \(\mathbf{z}_t^{(l)} = \text{LayerNorm}(\mathbf{z}_t^{(l-1)} + \mathbf{A}_t^{(l)})\). This allows the network to synthesize view-consistent perturbations that attack VGGT precisely where it relies on inter-frame consistency.
2. DPT Dense Prediction Head: Multi-Scale Hierarchical Decoding for High-Fidelity Perturbations
Converting high-level latent tokens into imperceptible pixel-level noise requires capturing both fine local textures and global spatial structures. Simple bilinear interpolation or shallow convolutional decoders fail to generate high-frequency adversarial patterns.
MVAP-G employs a Dense Prediction Transformer (DPT) head that extracts feature representations from the 3rd, 6th, 9th, and 12th Transformer layers via learned readout projections \(W_{\text{readout}}^{(l)} \in \mathbb{R}^{D \times D}\), yielding feature tokens \(\mathbf{h}_t^{(l)} \in \mathbb{R}^{N_p \times D}\). These multi-scale representations are processed through four progressive Reassemble stages (\(s = 1 \sim 4\)), concatenating features with learnable fusion tokens (\(K_s \in \{1, 4, 16, 64\}\)) and projecting them to upsample spatial resolutions to \(H/2^s \times W/2^s\). A lightweight refinement block with \(3 \times 3\) depth-wise convolutions and \(1 \times 1\) point-wise convolutions outputs full-resolution unbounded perturbations \(\tilde{\delta}_t \in \mathbb{R}^{H \times W \times 3}\). During inference, strict clipping \(\delta_t = \text{clip}(\tilde{\delta}_t, -\epsilon, \epsilon)\) enforces the \(L_\infty\) budget while preserving effective high-frequency attack vectors.
3. Point Cloud Collapse Objective and Progressive Regularization: Balancing Attack Efficacy with Perceptual Invisibility
Directly optimizing complex geometric divergence metrics like Chamfer Distance during generative training leads to high computational cost and gradient instability. The authors discover that the most effective and universal way to dismantle 3D point cloud reconstruction is to collapse all 3D coordinates toward the spatial origin \((0,0,0)\).
The adversarial objective is therefore formulated as minimizing the mean \(L_2\) norm of the predicted point coordinates: $\(\mathcal{L}_{\text{adv}} = \frac{1}{|\hat{P}|} \sum_{p \in \hat{P}} \|p\|_2\)$ This loss aggressively strips the reconstructed point cloud of spatial dispersion and bounding box volume. To prevent this strong objective from introducing noticeable visual distortion into the input frames, MVAP-G incorporates a multi-component regularization loss: $\(\mathcal{L}_{\text{reg}} = \sum_{t=1}^T \left( \text{LPIPS}(\hat{I}_t, I_t) + \text{ReLU}(\|\delta_t\|_\infty - \epsilon) \right)\)$ where LPIPS penalizes perceptual divergence in deep feature space and the ReLU term provides soft boundary regularization. The total training objective is \(\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{adv}} + \lambda \mathcal{L}_{\text{reg}}\).
Loss & Training¶
The generator is trained via a progressive two-stage curriculum: 1. Single-View Pre-training: MVAP-G is first pre-trained on single-view RGB images from the COCO dataset for 4.0M iterations using the AdamW optimizer with a learning rate of \(1 \times 10^{-5}\). To allow the model to first acquire robust low-level adversarial image priors, \(\mathcal{L}_{\text{reg}}\) is introduced only after 2.4M steps, stabilizing feature learning before multi-view fine-tuning. 2. Multi-View Fine-Tuning: The model is subsequently fine-tuned on multi-view sequences sampled from seven 3D datasets (CO3Dv2, BlendMVS, ScanNet, Virtual KITTI, DTU, ETH3D, FlyingThings3D). Each batch randomly draws 2 to 10 views per scene, resized to a maximum dimension of 518 pixels. The entire training pipeline completes in approximately 20 days on a single NVIDIA RTX 4090 GPU, producing a single model that generalizes across arbitrary unseen scenes.
Key Experimental Results¶
Main Results¶
Attack effectiveness is evaluated by the Shifted Chamfer Distance (Shifted CD) between 3D point clouds reconstructed from clean versus perturbed multi-view inputs. Higher CD indicates superior attack performance and greater geometric degradation. Baselines include Gaussian random noise, Universal Adversarial Perturbations (UAP), and iterative adversarial attacks (AP with 10 and 20 steps).
| Method | Forward Steps | COCO (v=1) | ImageNet (v=1) | LLFF (v=1) | LLFF (v=4) | LLFF (v=8) | LLFF (v=16) | LLFF (v=25) | CO3D (v=1) | CO3D (v=4) | CO3D (v=8) | CO3D (v=16) | CO3D (v=25) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gaus. noise | 1 | 0.534 | 0.505 | 0.912 | 0.900 | 0.941 | 0.949 | 0.982 | 0.678 | 0.666 | 0.671 | 0.632 | 0.672 |
| UAP | 1 | 0.565 | 0.556 | 0.940 | 0.964 | 0.980 | 1.010 | 1.043 | 0.673 | 0.675 | 0.671 | 0.673 | 0.659 |
| AP (iter=10) | 10 | 1.361 | 1.349 | 1.841 | 1.769 | 1.803 | 1.870 | 1.834 | 1.442 | 1.718 | 1.890 | 1.643 | 1.913 |
| AP (iter=20) | 20 | 1.535 | 1.584 | 1.842 | 1.811 | 1.817 | 1.882 | 1.925 | 1.889 | 1.892 | 1.958 | 1.875 | 2.000 |
| MVAP-G (Ours) | 1 | 1.566 | 1.611 | 1.975 | 1.885 | 1.857 | 1.922 | 1.910 | 1.995 | 2.016 | 2.011 | 1.999 | 2.047 |
Ablation Study¶
Incremental ablation evaluates the individual contributions of single-view pre-training, the DPT head, Cross-view Adversarial Alignment (CAA), and the perceptual regularization loss \(\mathcal{L}_{\text{reg}}\) (metrics averaged over 1, 2, and 4 views):
| Config | CD ↑ | \(\mathcal{L}_{\text{adv}}\) ↓ | PSNR ↑ | LPIPS ↓ | Note |
|---|---|---|---|---|---|
| Baseline (Conv+Linear Decoder) | 0.58 | 0.400 | 21.1 | 0.400 | No pre-training/CAA/reg; weak attack capability |
| + Pre-training (COCO dataset) | 1.40 | 0.250 | 20.8 | 0.435 | Delivers largest gain in attack strength (CD +0.82) |
| + DPT head (Dense Reassembly) | 1.68 | 0.120 | 20.5 | 0.478 | Improves multi-scale pixel-level perturbation modeling |
| + CAA (Cross-view Alignment) | 1.75 | 0.008 | 20.3 | 0.512 | Models inter-view geometry; reduces \(\mathcal{L}_{\text{adv}}\) to 0.008 |
| + \(\mathcal{L}_{\text{reg}}\) (Full Model) | 1.78 | 0.008 | 29.8 | 0.107 | Boosts PSNR to 29.8 dB and slashes LPIPS to 0.107 |
Key Findings¶
- Order-of-Magnitude Speedup and Memory Reduction: Iterative AP requires 25 to 200 seconds per scene and exceeds 30 GB of peak GPU memory as view counts scale from 1 to 25. In contrast, MVAP-G executes in a single forward pass within 50 to 500 milliseconds (real-time speed) while keeping peak memory under 14 GB.
- Overcoming Multi-View Fusion Defenses: When the input view count reaches \(v=32\), removing CAA allows VGGT's multi-view attention to partially recover recognizable though fragmented scene geometry. With CAA enabled, cross-view perturbations disrupt joint matching, causing total geometric collapse.
- Cross-Task Degradation of Geometric Outputs: Supervised solely on point cloud collapse, MVAP-G induces catastrophic collateral degradation across VGGT's shared visual geometry backbone. On CO3D (\(v=8\), averaged over 100 scenes), camera rotation error increases from \(2.3^\circ\) to \(44.4^\circ\), translation error rises from 0.05 to 0.43, and focal length shifts by 18.7 pixels.
Highlights & Insights¶
- Transitioning 3D Attacks from Optimization to Feed-Forward Generation: Eliminates test-time per-scene gradient backpropagation, pioneering real-time input-conditioned multi-view perturbation synthesis against 3D foundation transformers.
- Cross-View Adversarial Alignment Prevents Geometric Self-Healing: Leverages cross-view attention within the generator to inject geometrically interlocking perturbations, neutralizing the intrinsic self-correcting mechanisms of multi-view models.
- Progressive Regularization Resolves the Stealth-Efficacy Dilemma: Demonstrates that high attack efficacy in 3D vision stems from targeted structural precision rather than large perturbation magnitude, maintaining high PSNR (29.8 dB) and low LPIPS (0.107).
Limitations & Future Work¶
- Critical Safety Risks in Physical Deployment: The instant millisecond disruption of camera poses and depth maps poses severe security threats to autonomous vehicles and robotic navigation systems if inputs are spoofed.
- Evaluation Confined to Digital Space: The perturbations are currently validated under digital \(L_\infty\) constraints; real-world physical transmission through printing, lens distortion, variable illumination, and motion blur remains unexplored.
- Urgent Need for 3D Defense Benchmarks: Calls for future research into cross-view consistency checks, robust adversarial fine-tuning on point cloud collapse objectives, and geometric attention purification layers.
Related Work & Insights¶
- vs Universal Adversarial Perturbations (UAPs): UAPs rely on static image-agnostic patterns that cannot adapt to scene parallax and multi-view geometry, achieving a meager CD of ~0.66 on CO3D; MVAP-G leverages input-conditioned generation and CAA to achieve CD scores surpassing 2.0.
- vs NeRFool / NerFail / Poison-splat: Prior 3D adversarial methods target implicit neural representations (NeRFs) or 3D Gaussian Splatting (3DGS) via iterative scene fitting or parameter poisoning; MVAP-G is the first to evaluate and attack unified feed-forward 3D foundation transformers.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Pioneering study uncovering the adversarial vulnerabilities of feed-forward 3D foundation transformers like VGGT with a real-time generator.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive cross-dataset evaluations (COCO, ImageNet, LLFF, CO3D) analyzing view scalability, runtime efficiency, GPU memory footprint, and geometric pose degradation.
- Writing Quality: ⭐⭐⭐⭐⭐ Clear motivation, rigorous architectural breakdown, well-defined mathematical formulations, and compelling qualitative visualizations.
- Value: ⭐⭐⭐⭐⭐ Unveils vital security blind spots in emerging 3D vision foundation models, providing an indispensable foundation for defensive 3D representation research.