🤝 Federated Learning¶

📷 CVPR2026 · 19 paper notes

🔥 Top topics: Federated Learning ×17 · Personalized Generation ×7 · Alignment/RLHF ×3

Domain Sensitive Federated Learning with Fisher-Informed Pruning: FEDFIP estimates channel importance using domain-specific Fisher information to assemble a globally shared pruning mask on the server, while clients "reactivate" a small number of locally critical channels. Combined with domain-prototype structural contrastive regularization and a "shared-channel-only" aggregation strategy, it significantly compresses models while achieving higher accuracy and stability than mainstream FL baselines in multi-domain federated scenarios.
FedAdamom: Adaptive Momentum for Improved Generalization in Federated Optimization: This paper uses diffusion theory to explain the root cause of why "FedAdam converges fast but generalizes poorly"—the adaptive learning rate weakens the preference for flat minima. Based on this, FedAdamom is proposed: shifting the adaptive mechanism from the learning rate to the momentum coefficient. This preserves the ability to quickly escape saddle points while restoring the selection of flat minima, simultaneously achieving faster convergence and higher accuracy on CIFAR-10/100, Tiny-ImageNet, and LEAF.
FedAlign: Differentially Private Distribution Alignment for Non-IID Federated Learning: FedAlign requires each client to upload noisy versions of the first four statistical moments (mean, variance, skewness, kurtosis) of their local data. The server aggregates these into a global reference distribution and broadcasts it back. Clients then align the distribution of their locally sampled data accordingly—mitigating both Non-IID heterogeneity and privacy leakage under differential privacy constraints, achieving a ~4% accuracy gain over strong baselines on CIFAR-10.
FedARA: Resource-adaptive Low-rank Personalized Federated Learning via Anchor-driven Representation Alignment on Heterogeneous Edge Devices: FedARA designs the shared feature extractor into a low-rank structure that can be adaptively decomposed and reconstructed by the server based on client-specific resources, allowing heterogeneous edge devices to utilize their desired ranks. Concurrently, it computes "consistency anchors" using the aggregated global features from the server to regularize local representations, mitigating feature drift and global knowledge forgetting under non-IID settings. It outperforms 17 SOTA baselines on three datasets with lower communication and computation overhead.
FedHarmony: Harmonizing Heterogeneous Label Correlations in Federated Multi-Label Learning: To address the issue in Federated Multi-Label Learning (FedMLL) where clients observe only local label spaces and generate conflicting label correlations (Label Correlation Drift), FedHarmony utilizes "Consensus Correlation" from the majority of clients as a global teacher to correct local training biases. Furthermore, it weights clients during server aggregation based on both data volume and correlation quality. It consistently outperforms existing SOTA on three non-IID federated benchmarks: FLAIR, COCO-80, and VOC2007 (e.g., +11.4 mAP on FLAIR).
FedRAC: Rolling Submodel Allocation for Collaborative Fairness in Federated Learning: FedRAC introduces a "dynamic reputation calculation that evolves with training" alongside "submodel construction via historical frequency rotation followed by reputation-based allocation." This dual-module approach ensures high-contribution clients receive superior submodels (fairness) while maintaining uniform training for every neuron in the global model (accuracy). It outperforms existing collaborative fairness methods in both fairness and accuracy.
FedRG: Unleashing the Representation Geometry for Federated Learning with Noisy Clients: To address the dual challenges of "noisy client annotations + Non-IID data" in Federated Learning, FedRG abandons the unreliable small-loss heuristic. Instead, it identifies clean/noisy samples based on representation geometry. Specifically, it first learns label-agnostic representations on a hypersphere through self-supervision, then uses a vMF mixture model to compare "geometric evidence" with "annotated label evidence" in a shared space for noise detection. Finally, it employs a personalized noise absorption matrix for robust optimization, achieving SOTA across multiple datasets and four noise scenarios.
Fine-Tuning Impairs the Balancedness of Foundation Models in Long-tailed Personalized Federated Learning: This paper empirically reveals that fine-tuning CLIP in long-tailed federated scenarios destroys its inherent class balance, even falling below zero-shot performance. It proposes FedPuReL: using zero-shot predictions to "purify" local gradients into directions that preserve balance for a global model, and reframing personalization as "residual correction" atop a frozen global model. FedPuReL outperforms existing SOTA in both global and personalized models across 8 long-tailed datasets.
From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneity: Addressing the pain point in Federated Continual Learning (FCL) where "selecting samples is easy, but utilizing them is difficult," FEAT does not modify the replay strategy itself. Instead, it employs a set of fixed ETF prototypes shared across all clients. It uses geometric structure distillation during training to align feature angles across clients and applies energy-based geometric correction during inference to "pull back" tail-class features from head-class subspaces. As a plug-and-play module layered on Re-Fed+ or FedCBDR, it yields stable performance gains.
Fully Decentralized Certified Unlearning: Addressing the neglected scenario of "decentralized networks without a central coordinator," this paper proposes RR-DU—a random-walk-based certified unlearning algorithm. It performs noisy projected gradient ascent on the forgetting set only at the client initiating the deletion, while other clients continue with noise-free descent. By incorporating sub-sampled Gaussian noise and trust region projections, the authors prove \((\varepsilon,\delta)\) network unlearning certificates, convergence, and deletion capacity bounds. Notably, the noise does not scale with the size of the forgetting set \(m\), successfully reducing backdoor attack success rates to random-guess levels while maintaining clean accuracy on image classification tasks.
GDFA: Geometry-Driven Federated Unlearning with Directional Task Vector Alignment: GDFA reinterprets "Federated Unlearning" as a loss surface geometry problem: it first migrates the global model to a flat minima region via perturbations, then has relevant clients generate task vectors on unlearning data, retaining only components with directional consensus (sign consensus) for reverse aggregation. This achieves precise erasure of target client knowledge in Non-IID scenarios with almost no loss in retention task accuracy.
Generalized and Personalized Federated Learning with Black-Box Foundation Models via Orthogonal Transformations: FEDOT treats frozen black-box foundation models (FM) as pure feature extractors. Each client stacks a local orthogonal transformation on the output embeddings for personalization, while all clients share and aggregate a global classifier for generalization. The authors prove that the orthogonal constraint (condition number \(\kappa=1\)) minimizes the upper bound of cross-client gradient conflicts, achieving SOTA generalization and personalization under severe non-IID conditions without accessing FM internal parameters.
HiLoRA: Hierarchical Low-Rank Adaptation for Personalized Federated Learning: HiLoRA decomposes the LoRA update of each client into a three-layer orthogonal subspace structure consisting of "root-cluster-leaf," which respectively capture global consensus, subgroup commonalities, and client personalization. Combined with an adaptive clustering mechanism based on LoRA subspace similarity, it achieves SOTA performance in both personalization and generalization to new clients on CIFAR-100 and DomainNet.
OS-FED: One Snapshot Is All You Need: OS-FED encodes the cumulative gradients of a client's entire local training period into a "synthetic snapshot"—comprising one 224×224 synthetic image and a set of learnable labels. In each round, only this snapshot is transmitted. The server recovers and aggregates global updates through a single feedforward and backpropagation step, reducing communication overhead by 1.5–16× and accelerating convergence by 18–45% compared to SOTA methods.
Personalized Federated Training of Diffusion Models with Privacy Guarantees: PFDM decomposes the reverse denoising process of diffusion models into two components: a "client-private denoiser + server-shared denoiser." Clients only upload data that has been clipped and subjected to forward noise, providing formal Local Differential Privacy (LDP) guarantees for each data point. The shared model only observes noised data and cannot reproduce any client samples in isolation, while collaboration significantly enhances generation quality for minority or underrepresented classes.
Single-Round Scalable Analytic Federated Learning: SAFLe constructs a deterministic non-linear classification head using "feature bucketing + shuffling & grouping + sparse embedding summation" and proves its mathematical equivalence to a high-dimensional sparse linear regression. This allows the direct application of the single-round closed-form aggregation law of Analytic Federated Learning (AFL)—retaining the expressivity of non-linearity while preserving AFL's advantages of "single-round communication" and "complete invariance to data heterogeneity." It outperforms both linear AFL and multi-round DeepAFL across three visual federated benchmarks.
SubFLOT: Efficient Personalized Federated Learning via Optimal Transport-based Submodel Extraction: SubFLOT shifts "personalized pruning" from the client to the server: it uses the client's historical models as proxies for local data distributions and employs Optimal Transport (Wasserstein distance minimization) at the server to extract heterogeneous submodels tailored to each device's data. Combined with an adaptive regularization term that scales with the pruning rate to stabilize local training and an OT-aligned aggregation module to mitigate parametric drift, SubFLOT significantly outperforms 9 SOTA federated pruning methods across 8 datasets.
Taming Noise-Induced Prototype Degradation for Privacy-Preserving Personalized Federated Fine-Tuning: Addressing the issue where prototype-based personalized federated learning (ProtoPFL) injects isotropic Gaussian noise to satisfy local differential privacy (LDP) when sharing class prototypes—consequently drowning out discriminative dimensions—this paper proposes a client-side plugin, VPDR. It uses variance-adaptive VPP to shift noise budget from discriminative subspaces to redundant ones, and distillation-guided DCR to actively push feature norms toward the clipping threshold, significantly improving the privacy-utility trade-off under the same LDP guarantees.
Towards Stable Federated Continual Test-Time Adaptation in Wild World: This paper proposes BPFedCTTA, which unifies "Federated Continual Test-Time Adaptation (FedCTTA)" from a Bayesian perspective: treating the global model as a Gaussian prior, stabilizing the local adaptation of each unlabeled client through MAP estimation (BPA), and selectively fusing client updates via an uncertainty gate calculated from output entropy (UGSA). This approach achieves adaptation to new domains while preserving the global model and mitigating catastrophic forgetting in extreme heterogeneous scenarios where clients arrive sequentially with completely unrelated distributions.