Scaling Laws for Black-box Adversarial Attacks

Conference: ECCV2026
Paper: ECCV Paper Page
Authors: Chuan Liu, Huanran Chen, Yichi Zhang, Jun Zhu, Yinpeng Dong
Area: AI Safety
Keywords: Black-box transfer, adversarial robustness, model ensembling, scaling laws, gradient conflict

TL;DR

After mitigating gradient conflict among surrogate models, visual adversarial transfer success grows approximately linearly with the logarithm of ensemble size within a finite regime, a conditional empirical relationship examined on classifiers, defended models, and multimodal models.

Background & Motivation

Transfer-based black-box adversarial research optimizes against accessible surrogate models and measures whether the resulting image perturbations transfer to held-out targets. Prior work improves transfer through momentum, input transformations, or small ensembles, but usually fixes a relatively small surrogate set. This supports algorithm comparisons while leaving unclear whether the main bottleneck is the algorithm or the limited coverage of features and architectures in the surrogate set.

The paper treats surrogate count as a form of sample scale: perturbations are optimized on a finite ensemble, whereas evaluation concerns generalization to models outside that ensemble. However, enlarging the set does not automatically improve generalization, because gradients from different models can cancel before optimization finds a shared direction. Idealized empirical risk minimization does not settle the issue either: real perturbations obey pixel-space constraints, and pretrained surrogates share data and architectural biases rather than satisfying independent sampling assumptions.

The contribution is therefore primarily a controlled scaling study, not the invention of SSA or CWA. Its core subject is visual adversarial transfer, subsequently extended to vision encoders and multimodal outputs, rather than textual jailbreaking or instruction safety. Core Idea: first make ensemble optimization preserve shared cross-model directions, then vary surrogate count systematically and measure success, loss, and semantic behavior, making ensemble scale an explicit axis of robustness evaluation.

Method

Overall Architecture

The study takes natural images, designated target classes, and a pool of pretrained surrogates, and produces success-rate curves on held-out models at different ensemble sizes. Its analytical sequence comprises ensemble risk modeling, gradient conflict mitigation, log-linear scaling assessment, and cross-modal validation with vision encoders. These are stages of an empirical investigation, not a new end-to-end network requiring training.

The classifier experiments select differently sized ensembles from 64 Torchvision/Timm models, hold the optimization method and perturbation budget fixed, and compare seven scales. A separate vision-encoder experiment uses 12 pretrained OpenCLIP models and changes both the objective and the output evaluation rule. The two experiments address related but distinct questions; their success rates do not form a single curve under a shared budget and metric.

Key Designs

1. Ensemble risk modeling: formulate transfer as cross-model generalization

For targeted evaluation, where the designated class differs from the original class, the average target-class loss across surrogates approximates an expectation over an unknown model population. The following restates the meaning of Eq. (2) from Section 3.1; the cached equation is corrupted, so this is a reconstruction from the prose rather than a verbatim transcription:

\[ \min_{\|\mathbf{x}-\mathbf{x}_{\mathrm{nat}}\|_\infty\leq\epsilon} \widehat L_T(\mathbf{x}) =\frac{1}{T}\sum_{i=1}^{T}\mathcal L(f_i(\mathbf{x}),y). \]

Here \(T\) denotes surrogate count, \(y\) is the designated target class, and \(\epsilon\) bounds the maximum per-pixel change relative to the natural image. The formulation separates a low loss on the surrogate ensemble from successful target-class prediction on an unseen model; the latter is the generalization property measured by transfer evaluation. Untargeted evaluation only requires departing from the true class, without selecting a particular alternative, and therefore reaches success-rate saturation more easily.

Section 4.1 presents an asymptotic analysis under unconstrained optimization and independently sampled surrogates, suggesting an \(O(1/T)\) loss-gap rate. The authors also explain that a constrained optimum can lie on the perturbation boundary, invalidating unconstrained stationarity, while correlated pretrained models violate independent sampling. The analysis consequently motivates scaling qualitatively; it does not derive the observed log-linear law for real-world ASR.

2. Gradient conflict mitigation: optimization must exploit shared directions

Naive logit averaging aggregates predictions, but input gradients from different models can cancel as the ensemble grows. Figure 2 diagnoses this through the gradient \(L_1\) norm at iteration 10 for 1 to 30 surrogates: the naive ensemble loses gradient magnitude, whereas CWA maintains a comparatively stable norm. This explains why an optimizer that performs well on individual models does not necessarily resolve directional conflict within a larger ensemble.

Common Weakness Attack (CWA) is an existing method that encourages gradient alignment and flatter shared loss regions to identify vulnerabilities common to several models. The paper selects SSA-CWA, combining frequency-domain input transformations from Spectrum Simulation Attack (SSA) with CWA's shared-direction optimization to reduce dependence on one surrogate or one input appearance. SSA supplies input variation, whereas CWA addresses cross-model conflict; attributing the scaling effect entirely to SSA would conflate their roles.

Section 6.1 further separates stronger optimization components from a better ensemble strategy: MI-FGSM, DI-MI-FGSM, and VMI-FGSM stagnate with naive logit averaging, while MI-CWA, VMI-CWA, and MI-SVRE exhibit scaling. SVRE chiefly reduces gradient variance, so its internal mechanism should not be treated as identical to CWA's. The main text points to Appendix C for algorithmic details, but the local cache omits the appendices; this note does not invent full update rules or missing implementation hyperparameters.

3. Log-linear scaling assessment: identify a valid regime, not unlimited growth

Section 4.2 fixes 40 iterations and an \(\ell_\infty\) budget of \(8/255\), testing \(T\in\{1,2,4,8,16,32,64\}\). Each of 1,000 ImageNet images from the NIPS 2017 dataset is paired with 8 target classes, giving 8,000 source-target pairs per scale and 56,000 generated adversarial examples in total. At each scale, the source images are partitioned into 8 batches of 125, with a newly sampled surrogate ensemble for each batch to reduce dependence on one model selection. This is not eight independent repetitions for every image, and at \(T=64\) the ensemble always contains the entire pool.

An interpretable notation for the empirical relationship described in Section 4.3 is:

\[ \operatorname{ASR}(T)\approx a+b\log_2 T. \]

This restates the fitted relationship in the prose and is not an additional theoretical result. With ASR expressed as a fraction in \([0,1]\), \(a\) represents the single-surrogate baseline and \(b\) the fractional gain per doubling of surrogate count; multiplying by 100 converts the gain to percentage points. Different targets and target classes can have different intercepts and slopes, and the paper does not provide one universal value of \(b\). The relationship describes a regime after initial variability stabilizes and before success approaches 1; indefinite extrapolation would violate the probability bound.

The authors also observe decreasing target-class cross-entropy, supplementing a discrete success event with a continuous loss measure. The coefficients of determination in Table 1 describe fit quality, not success rates, and do not imply an equally high probability of misleading the target. The FLOPs analysis in Figure 4c additionally shows that doubling surrogate count and doubling computation are different interventions because the surrogates have unequal computational costs.

4. Cross-modal validation: change both the visual objective and the success rule

Section 5 replaces classification loss with a representation-similarity objective that brings surrogate image embeddings closer to a target-class text embedding. This uses CLIP's aligned image-text space rather than directly optimizing a target MLLM's language output, and it does not establish knowledge of the target's internal vision encoder. The 12 surrogates come from OpenCLIP, while the targets comprise 4 commercial and 3 open-source models.

This experiment randomly samples only 100 NIPS17 images and retains 8 target concepts, producing 800 pairs per ensemble configuration, while changing the perturbation budget to \(16/255\). The main text reports 8 NVIDIA A100 GPUs and less than 24 hours for each experimental group; this is a group-level resource statement, not per-image latency or total project cost. The SSA-CWA protocol is retained, but the missing appendices prevent verification of all model configurations and evaluation prompts from the current cache.

The metrics must remain distinct: Section 4 uses relaxed keyword matching for its MLLM validation, whereas Section 5 consistently uses GPT-4o as a semantic judge. In Section 5, a description must mention the target concept without indicating low image quality, noise, or modification to count as a success. ASR is the fraction of evaluated pairs satisfying this joint condition; it measures visual semantic misdirection in a particular description task, not harmful-content generation or real-system compromise. The increasingly class-like textures in Figure 6c offer a qualitative account involving shared semantic features, but are not a complete causal demonstration.

Key Experimental Results

Main Results

The table below selects the Average column of original Table 3 and converts fractions to percentages: 12 CLIP surrogates, \(\epsilon=16/255\), 100 images paired with 8 concepts, and GPT-4o judging. These are paper-reported results, not fresh measurements of current services.

Target model	Average ASR	Interpretation
GPT-4o	85.0%	High transfer to visual descriptions under this protocol
Claude-3.5-Sonnet	31.0%	Lowest in the table, but not immune
Gemini-2.5-Pro	69.0%	Substantial variation across targets
Doubao-Seed-1.6	72.0%	Evaluated with the same judging rule
Llama-3.2-90B-Vision	58.0%	Parameter count does not directly determine robustness
Qwen3-VL-235B	90.0%	Highest in the table
DeepSeek-VL2	59.0%	Not a statement about all its multimodal tasks

In the Average column of original Table 2, GPT-4o and Qwen3-VL-235B each have \(R^2=0.97\), while Claude-3.5-Sonnet has \(R^2=0.94\). A lower ASR can therefore coexist with a good fit: goodness of fit and robustness level are different quantities. The Average column of original Table 1 reports \(R^2\) values from 0.96 to 0.98 for standard classifiers; these are quoted column values, not newly calculated averages of the label-specific entries.

Ablation Study

Figure 6 does not provide a complete numerical point table in the main text, so the following preserves exact settings and qualitative findings without guessing ASR decimals from plots or corrupted text.

Analysis and source	Controlled setting	Reported observation
Untargeted transfer, Fig. 6a	\(\epsilon=2/255\); 64-model pool; 3 target classifiers	Log-linear behavior persists when a small budget avoids early saturation
Naive ensembling, Fig. 6b	Target sea lion; MI-FGSM, DI-MI-FGSM, VMI-FGSM	Stronger base components do not produce clear scaling
Conflict mitigation, Fig. 6b	MI-CWA, VMI-CWA, MI-SVRE	Scaling trends agree with the main experiment
Gradient diagnosis, Fig. 2	1 to 30 surrogates; iteration 10	CWA maintains a more stable gradient norm than naive averaging
Semantic visualization, Fig. 6c	\(T=1,8,64\); 5 CIFAR10 classes	Unstructured noise develops recognizable class patterns

Key Findings

• Conflict mitigation is an important condition for the reported scale benefit; the conclusion is not that every ensemble improves predictably with size.
• In Figure 4a, defenses such as DiffPure lower ASR without removing the trend over the tested regime; the text's claim of an improvement exceeding 30% does not clearly specify a percentage-point convention, so it is not treated here as an exact difference.
• Budgets, model versions, and judging rules are not harmonized here with earlier commercial-model studies; the cited prior 40% to 50% success rates are not a matched-condition SOTA baseline.

Highlights & Insights

• Surrogate count becomes an independent evaluation axis, distinguishing a defense that lowers baseline success from one that reduces the scaling slope. A single small-ensemble result can miss the latter.
• The paper links scale benefits to optimization feasibility: more surrogates help only when shared directions are not canceled. This is more explanatory than reporting one stronger ensemble's endpoint.
• Transfer from classifiers to MLLMs brings shared visual representations into the analysis. Semantic visualizations are useful clues, but further interventions are needed instead of treating class-like appearance as a complete explanation.

Limitations & Future Work

• The authors explicitly acknowledge the gap between unconstrained independent-sampling theory and bounded optimization over correlated surrogates; finite-range fits neither prove universality nor justify extrapolation beyond saturation.
• The 1,000 main-study images and 100 extension-study images still come from NIPS17, with only 8 target concepts; they do not represent all visual tasks, open-vocabulary distributions, or input conditions.
• The available main text does not provide slope confidence intervals, full multi-seed variance, or a comprehensive factorial experiment controlling computation and diversity; eight-batch sampling does not replace these analyses.
• GPT-4o is both one evaluated target and the common judge, creating a possible judging bias; blinded human annotations and alternative judges could test robustness. This metric also differs from Section 4's keyword metric.
• A small \(\ell_\infty\) distance does not guarantee human imperceptibility, particularly when comparing \(16/255\) with \(8/255\); the evidence about semantic texture remains primarily qualitative.
• Appendices B through E are absent from the cache, preventing verification of the complete surrogate lists, pseudocode, and judging prompts; conclusions are restricted to the available main text rather than filling these gaps with assumptions.

Related Work & Insights

• Compared with CWA (ICLR 2024): CWA supplies the common-weakness optimization principle; this paper studies whether it enables systematic ensemble scaling rather than claiming the optimizer as a new invention.
• Compared with SSA and SVRE: SSA changes frequency-domain input appearance, whereas SVRE reduces ensemble gradient variance; combinations and substitutions test whether the scaling observation depends on one specific algorithm.
• Compared with conventional neural scaling laws: the independent variable is surrogate ensemble size, not language-model parameters or text tokens; idealized loss convergence and empirical ASR curves are also distinct mathematical objects.
• Implication for robustness evaluation: authorized benchmarks should report perturbation budget, surrogate count, computation, judging rules, and uncertainty together to distinguish model differences from evaluation-condition differences.

Rating

• Novelty: 4/5. Jointly studies scale, conflict mitigation, and transfer, while reusing existing optimization algorithms.
• Experimental Thoroughness: 4/5. Covers three target families and alternative optimizers, but lacks complete uncertainty and strictly controlled cost analysis.
• Writing Quality: 4/5. Distinguishes theoretical motivation from empirical results, although some strong wording exceeds the finite experimental evidence.
• Value: 4/5. Useful for more comprehensive visual robustness benchmarks, but insufficient for general claims about real-service security.