Proteus: Model Leakage-Induced Adversarial Attack in Federated Learning¶

Conference: ECCV 2026
Paper: ECCV 2026
Code: https://github.com/HKU-TASR/Proteus
Area: AI Safety
Keywords: Federated Learning, Adversarial Attack, Temporal Transferability, Model Leakage, Feature Drift

TL;DR¶

Addressing the overlooked vulnerability stemming from federated learning's requirement to broadcast intermediate models in plaintext, this paper proposes Proteus to generate temporally persistent adversarial examples that bypass subsequent training dynamics, achieving an 85.40% attack success rate on the final deployed model even with a model leaked after only 30% of training rounds.

Background & Motivation¶

Deep neural networks are inherently fragile against imperceptible input perturbations, presenting substantial safety hazards for computer vision systems. However, after over a decade of research, the practical threat of adversarial attacks remains heavily debated. White-box attacks require access to the exact parameters of the deployed model, whereas black-box transfer attacks built on independently trained surrogates often suffer from low transferability or demand query-intensive interactions that trigger detection. The rapid expansion of privacy-sensitive vision applications has propelled federated learning (FL) as the standard paradigm: distributed clients collaboratively train a shared model under the coordination of a central server, which in every round broadcasts the current global model in plaintext to selected participants for local optimization.

This collaborative protocol introduces an unavoidable yet neglected attack surface: any participant selected in a given round obtains a plaintext snapshot of the intermediate model. Conventional wisdom suggests that models exposed early during training cannot serve as effective surrogates because subsequent training rounds will substantially alter the parameter space and decision boundaries. However, empirical findings reveal a striking contrast: an adversary training a surrogate on purely local private data yields a transfer attack success rate (ASR) of merely 8.35%, whereas directly attacking an intermediate model leaked after only 30% of FL training rounds achieves a moderate transfer ASR of 59.75%, demonstrating substantial intrinsic temporal transferability along the optimization trajectory.

The failure of standard adversarial examples across successive rounds originates primarily from temporal feature drift during continued parameter optimization. Conventional PGD perturbations optimized against a single intermediate checkpoint tend to settle into sharp local optima with high parameter sensitivity. As the global model absorbs client updates in subsequent rounds, the latent feature representation shifts drastically, driving the adversarial point back to the correct side of the decision boundary. Core idea: leverage the leaked intermediate model as a temporal anchor to systematically discover perturbation directions with low parameter sensitivity and explicit safety margins, ensuring adversarial persistence across subsequent training rounds.

Method¶

Overall Architecture¶

Proteus targets a strict threat model where a malicious client obtains a single intermediate checkpoint \(\theta_S\) at an early round \(S \ll T\), without access to private client data, subsequent query channels, or future model snapshots. Under a first-order Taylor expansion, the magnitude of feature drift between the leaked checkpoint and the final model \(\theta_T\) is bounded by the Frobenius norm of parameter sensitivity, \(\|\nabla_{\theta_S} h_{\theta_S}(x_{\text{adv}})\|_F\). Proteus guides adversarial optimization toward parameter-insensitive basins by iteratively executing three interconnected phases.

In each iteration, Proteus first discovers a primary persistent direction by suppressing brittle local sensitivities via stochastic transformation ensembling. Next, it performs lookahead extrapolation along this direction to construct an explicit safety margin against cumulative feature drift. Finally, it blends the primary and margin gradients through dimension-averaged normalization and momentum accumulation, advancing the perturbation via projected \(L_\infty\) ascent.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input x and Current Perturbation &delta;k"] --> B["Persistent Direction Discovery<br/>Ensemble transformation suppresses parameter sensitivity"]
    B --> C["Adversarial Margin Expansion<br/>Lookahead sign shift and margin gradient re-estimation"]
    C --> D["Momentum Fusion and Projected Ascent<br/>Dual-gradient normalization and momentum update"]
    D -->|k < Max Iterations| A
    D -->|Finished| E["Output Adversarial Example x_adv<br/>Persistently deceives final deployed model &theta;T"]

Key Designs¶

1. Persistent Direction Discovery: Suppressing Parameter Sensitivity via Transformation Ensembling

Standard gradient ascent overfits to idiosyncratic, brittle features of the intermediate checkpoint that quickly dissolve under subsequent updates. To suppress this high parameter sensitivity, Proteus evaluates the model across a stochastic transformation ensemble \(\mathcal{T}\) containing geometric scaling, Gaussian perturbations, and localized transformations. Leveraging the triangle inequality over the ensembled parameter sensitivity norm, the primary persistent gradient is obtained by averaging across transformed views:

\[\mathcal{G}(\delta; x) = \frac{1}{|\mathcal{T}|} \sum_{\tau \in \mathcal{T}} \nabla_{\delta} \mathcal{L}(F_{\theta_S}(\tau(x + \delta)); y)\]

In iteration \(k\), the primary gradient \(g_k^{\text{primary}} = \mathcal{G}(\delta_k; x)\) cancels out components tied to sharp, fragile local optima during averaging, leaving stable adversarial directions that persist across subsequent FL rounds.

2. Adversarial Margin Expansion: Lookahead Shift for Drift Buffering

Even if the primary direction avoids sharp parameter optima, perturbations situated too close to the current decision boundary remain susceptible to minor feature shifts during subsequent training rounds. Proteus introduces a lookahead extrapolation mechanism to expand the adversarial margin. Using a drift step scale \(\rho \cdot \alpha\), it projects \(\delta_k\) along the sign of the primary gradient:

\[\delta_k^+ = \Pi_{\epsilon} \left( \delta_k + \rho \cdot \alpha \cdot \text{sign}(g_k^{\text{primary}}) \right)\]

Proteus then evaluates the margin gradient at this extrapolated point, \(g_k^{\text{margin}} = \mathcal{G}(\delta_k^+; x)\). If \(\delta_k\) lies in an unstable boundary region, the discrepancy between \(g_k^{\text{primary}}\) and \(g_k^{\text{margin}}\) provides corrective force, steering the perturbation deeper into a broad, parameter-insensitive adversarial region.

3. Momentum Fusion and Projected Ascent: Dual-Gradient Normalization and Smooth Updates

To synthesize the local guidance and the expanded margin, Proteus fuses the two estimates into a composite gradient \(g_k = g_k^{\text{primary}} + g_k^{\text{margin}}\). To prevent magnitude fluctuations across iterations from destabilizing the ascent trajectory, \(g_k\) is normalized by the mean absolute value across all \(d\) dimensions and accumulated with a momentum factor \(\mu\):

\[m_{k+1} = \mu \cdot m_k + \frac{g_k}{\frac{1}{d} \|g_k\|_1}\]

The adversarial perturbation is then updated via bounded step size \(\alpha\) along the momentum sign direction within the \(L_\infty\) budget \(\epsilon\):

\[\delta_{k+1} = \Pi_{\epsilon} \left( \delta_k + \alpha \cdot \text{sign}(m_{k+1}) \right)\]

This momentum-guided fusion filters out high-frequency gradient noise and ensures stable progression toward robust adversarial manifolds on the future deployed model.

Loss & Training¶

The attack optimizes an untargeted classification loss under the standard cross-entropy objective \(\mathcal{L}\). Default attack hyperparameters are: perturbation budget \(\epsilon = 8/255\), total iterations \(K = 20\), attack step size \(\alpha = 2/255\), drift step multiplier \(\rho = 2\), momentum decay \(\mu = 0.5\), and \(|\mathcal{T}| = 20\) stochastic transformations per iteration. The attack runs entirely offline without client data or server interaction.

Key Experimental Results¶

Main Results¶

Experiments simulate an FL ecosystem of 100 clients collaboratively training a ResNet-18 model using FedAvg (10 clients per round, 5 local epochs with SGD). Intermediate checkpoints are captured at 30% training progress (RAF-DB: round 44/145, CIFAR-10: round 55/183, TinyImageNet: round 46/153). Transfer ASR (%) evaluates adversarial efficacy against the final converged model:

Dataset	Metric	Proteus (Ours)	Leaked PGD Baseline	Strongest Transfer Baseline	Gain
RAF-DB (Facial Emotion, default)	Transfer ASR (%)	85.40	59.75	59.75 (FGSM)	+25.65%
CIFAR-10 (Standard benchmark)	Transfer ASR (%)	90.00	~68.00	~71.00 (OPS)	+19.00%
TinyImageNet (200-class complex vision)	Transfer ASR (%)	79.61	~55.00	64.72 (MuMoDIG)	+14.89%

Robustness across diverse neural network architectures (RAF-DB, 30% progress leakage):

Backbone Architecture	Metric	Proteus (Ours)	Runner-up Baseline	Gain
ResNet-18	Transfer ASR (%)	85.40	59.75 (FGSM)	+25.65%
MobileNetV3	Transfer ASR (%)	77.38	57.33 (OPS)	+20.05%
EfficientNet	Transfer ASR (%)	82.99	67.44 (FGSM)	+15.55%

Ablation Study¶

Ablation of Proteus components on RAF-DB under 30% progress leakage:

Config	Transfer ASR (%)	Note
Full Proteus	85.40	Full pipeline with ensemble discovery, margin expansion, and momentum fusion
Without Phase 1	75.82	ASR drops by 9.58%; transformation ensembling is the primary pillar for suppressing parameter sensitivity
Without Phase 2	82.23	ASR drops by 3.17%; omitting lookahead margin leaves perturbations vulnerable to long-term drift
Without Phase 3	83.17	ASR drops by 2.23%; normalization and momentum stabilization provide valuable refinement

Key Findings¶

Severe Hazard from Early Leaks: Leaking a model at round 44 (over 100 rounds prior to convergence) allows Proteus to attain an 85.40% attack success rate. Over 20 attack iterations, the misclassification alignment between the leaked model and the final model surges from 8.96% to 66.67%.
Defenses Face Severe Trade-offs: In evaluating five post-deployment defenses, adversarial purifiers (DiffPure, ScoreOpt, ADBM) collapse clean model accuracy from 71.58% to 14.60%–22.27% due to the lack of extensive in-distribution data at the server. Differential privacy maintains clean accuracy (67.87%) but Proteus retains an ASR of 59.64%, far exceeding baselines.
Near Real-Time Computational Efficiency: Operating at 47.78 ms per image on an NVIDIA RTX 4090 GPU, Proteus runs significantly faster than prior advanced transfer attacks (OPS at 121.90 ms) and operates within real-time vision latency budgets (~50 ms).

Highlights & Insights¶

Novel Perspective on FL Threat Surfaces: Overturns the common belief that intermediate, non-converged checkpoints carry limited adversarial utility, rigorously demonstrating that legitimate model-sharing protocols inherently leak temporally persistent vulnerabilities.
Formal Connection between Parameter Sensitivity and Feature Drift: Employs first-order expansion to establish parameter sensitivity as a quantifiable upper bound for temporal feature drift, providing a rigorous mathematical foundation for temporal transferability.
Transferable Principles for Dynamic Systems: The strategy of ensembling transformations to filter out fragile local gradients and using lookahead extrapolation to secure margins generalizes naturally to other evolving model settings, such as continual learning and online fine-tuning.

Limitations & Future Work¶

Author-Acknowledged Limitations: Absence of dedicated, non-disruptive defenses; temporary heuristics such as premature early stopping or restricting late-stage participation to verified clients violate the inclusive design of cross-device FL.
Additional Identified Limitations: The evaluation focuses on synchronous FedAvg and FedSGD schemes; the impact of asynchronous aggregations or aggressive adaptive optimizers (e.g., FedAdam) with large parameter jumps across rounds remains to be investigated.
Future Directions: Exploring low-rank subspace projections that constrain adversarial perturbations away from primary global update directions to withstand extreme non-IID client distributions.

vs Conventional Transfer Attacks (OPS / DeCoWa / FoolMix / MuMoDIG): Prior methods optimize for cross-architecture or cross-genus transferability on static models but struggle with temporal parameter drift over optimization trajectories; Proteus achieves at least 14.89% higher ASR on TinyImageNet by addressing the temporal dimension directly.
vs FL Privacy & Integrity Threats (Deep Leakage / Imperio): Existing FL threat models focus on data reconstruction via gradient inversion or model corruption via backdoors; Proteus introduces an evasion vector where a compliant participant passively receives a legitimate model, models future training trajectories offline, and attacks the future deployed model.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ First systematic formulation and exploration of model leakage-induced temporal adversarial attacks in federated learning.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous evaluation across 3 benchmark datasets, 3 backbones, various non-IID Dirichlet splits, and 5 defense baselines.
Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear narrative, elegant mathematical motivation, and tight alignment between diagrams and design principles.
Value: ⭐⭐⭐⭐⭐ Exposes a critical structural vulnerability in distributed learning protocols, providing an urgent impetus for defense research.