AnaPFL: When Closed-Form Solutions Meet Generalization and Personalization in Personalized Federated Learning¶
Conference: ECCV 2026
Paper: Official paper page · Paper PDF
Area: Optimization & Theory
Keywords: personalized federated learning, analytic learning, closed-form solution, non-IID data, residual refinement
The title follows the PDF title page, restoring the space in “Generalization and”; the official listing joins these words. This is not a different paper.
TL;DR¶
AnaPFL analytically aggregates a global primary stream and then fits a local residual refinement stream on frozen visual features, requiring one aggregation round and improving accuracy over the strongest baseline in each of 18 benchmark settings by 1.57–16.71 percentage points.
Background & Motivation¶
Personalized federated learning must give clients access to collective knowledge while producing models suited to their own data distributions. When clients have different class proportions, local gradient updates can point in conflicting directions, making the aggregated model less useful both globally and locally. FedAvg emphasizes a shared model, whereas methods such as Ditto, FedALA, and FedSelect introduce personalization but usually retain iterative local optimization and communication. The relevant setting here is narrower than federated learning in general: a pretrained feature extractor is already available and can remain frozen.
Analytic Federated Learning, or AFL, replaces iterative classifier updates with least-squares solutions and obtains a global classifier through one aggregation round. However, avoiding optimization-path bias does not remove differences in what clients need from their predictions. A classifier fitted to the aggregate population may still be a poor match for a client with strongly skewed class proportions. The paper illustrates this on CIFAR-100: AFL's global model is preferable under milder heterogeneity, but local models become preferable when heterogeneity is severe. The missing mechanism is therefore local correction after analytic aggregation, rather than simply additional rounds of global training.
AnaPFL separates shared prediction from client-specific compensation: it first obtains the global solution, freezes it, and then solves a residual prediction problem for each client. Core idea: retain an analytically aggregated global primary stream, then fit its remaining local errors in a second random feature space, enabling personalization without iterative gradient updates.
Method¶
Overall Architecture¶
The inputs are labeled images distributed across clients; raw images are not collected by the server. All clients extract features with the same frozen backbone, implemented as self-supervised ViT-MAE-Base using its CLS token in the experiments. The pipeline then performs “Dual-space analytic representation,” “Global analytic aggregation,” and “Local residual refinement,” producing a shared classifier and a separate correction classifier for every client. During training, the primary stream communicates with the server, whereas refinement is solved locally after the global model arrives. At inference, the two streams produce class scores that are added with a configurable refinement weight. “Gradient-free” describes this federated procedure on top of the frozen backbone, not the historical pretraining of ViT-MAE.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Client images and labels"] --> B["Frozen ViT-MAE<br/>CLS features"]
B --> C["Dual-space analytic<br/>representation"]
C --> D["Global analytic<br/>aggregation"]
D --> E["Local residual<br/>refinement"]
E --> F["Global scores plus local correction<br/>Personalized prediction"]
The cached extraction corrupts symbols in Equations (1)–(17), including inverses, signs, and regularization terms. The explanation below therefore follows the verifiable prose in Section 3 rather than presenting reconstructed equations as originals; the exact recursive coefficients require consulting the PDF. The cache contains the complete main method and experiments but not the separately referenced Appendices A and B, so their full proofs cannot be independently checked here.
Key Designs¶
1. Dual-space analytic representation: nonlinear features without giving up a solvable classifier
A linear classifier fitted directly to frozen CLS features is restricted by separability in that representation. AnaPFL applies random projections and nonlinear activations to the backbone output, creating separate primary-stream and refinement-stream feature spaces. The backbone, Gaussian random projection, and activation for the primary stream are shared across clients. This common coordinate system matters: otherwise the server would receive local classifier weights referring to incompatible feature dimensions, undermining analytic aggregation. The refinement stream uses a different projection from the primary stream and can use client-specific projections and activations because its parameters are not globally aggregated.
The random projections are fixed rather than learned through backpropagation; the fitted parameters are the classifier weights after the activated features. Nonlinearity is thus supplied by the feature map while the remaining optimization stays a least-squares problem with a closed-form solution. The two streams reuse the same backbone output instead of duplicating trainable backbones. Both streams use 1024-dimensional features and Tanh in the reported experiments, but their different projections provide different bases for prediction and residual correction. This is an explicit capacity trade-off: richer fixed features are available, but the representation itself cannot adapt to the downstream task through gradient learning.
2. Global analytic aggregation: combine statistical structure rather than average optimization trajectories
Each client first fits a regularized least-squares classifier from its primary-stream features and labels, and computes the corresponding autocorrelation matrix. It uploads this matrix and its local classifier weights, not raw examples or successive gradients. The server recursively maintains an aggregated autocorrelation matrix and a fused knowledge matrix as client information arrives. Unlike a simple FedAvg weight average, this procedure uses information about local feature geometry to recover the classifier associated with the aggregate data. The final computation also accounts for the difference between accumulated local regularization and the regularization in the global objective before distributing the shared primary stream.
Theorem 1 in Section 3.4 states that this classifier equals the optimum obtained by centrally solving the same fixed-feature objective on all participating data. Theorem 3 gives the precise meaning of heterogeneity invariance: if the complete dataset, shared feature mapping, and objective remain fixed, repartitioning examples among clients does not change the global classifier. This does not imply that every client's test accuracy remains constant, or that its personalized stream is independent of its own data. The result is also independent of client arrival order, allowing recursive processing as clients arrive. However, when dropout changes the union of participating data, the fixed-dataset invariance statement no longer compares the same problem.
3. Local residual refinement: preserve shared prediction and learn only its local shortfall
After receiving the global primary stream, each client evaluates its class scores on local training images and subtracts those scores from the labels to obtain residual targets. The refinement stream does not independently relearn the original labels: it fits these residuals with a separately regularized least-squares classifier in its own random feature space. The primary stream therefore carries what collective training already explains, while the private classifier concentrates on errors left under the client's distribution. Keeping the global classifier fixed makes the refinement objectives independent across clients, so solving them does not trigger another server aggregation round.
At inference, both feature maps process the same image, and the final class scores add the global scores to the refinement scores scaled by \(\lambda\). The paper uses \(\lambda\in(0,1]\), with larger values assigning greater influence to local correction. This is additive residual prediction at the score level, not a convex mixture of two independently normalized probability distributions. The mechanism explains why stronger heterogeneity can make refinement useful: global and local objectives leave more disagreement to correct. It also explains why refinement should not always receive maximum weight, since a small client's residual estimates may be unreliable.
Loss & Training¶
Both stages use squared prediction error and weight regularization rather than cross-entropy; preserving least-squares structure enables direct solution. The primary regularization coefficient is \(\gamma=10^{-2}\), while the refinement coefficient is \(\beta=1\). Section 4.1 selects \(\lambda\) by randomly holding out 10% of the training data for validation without accessing the test set. After selection, the held-out examples are restored and the final model is fitted on the complete training set. The selected values are \(\lambda=0.5,0.3,0.2\) for CIFAR-100, Tiny-ImageNet, and ImageNet-R, respectively. The other hyperparameters above are empirical choices rather than the outcome of an exhaustive search. Baselines use the same validation protocol to select recommended configurations; gradient-based methods run 200 communication rounds with 3 local training epochs per round.
Key Experimental Results¶
Main Results¶
Section 4.1 and Table 1 cover CIFAR-100, Tiny-ImageNet, and ImageNet-R, with either 50 or 100 clients and Dirichlet concentration \(\alpha\in\{0.1,0.5,1.0\}\). The selected rows below show high-heterogeneity settings. The metric is average client accuracy on each client's local test set, in percent and higher is better—not accuracy on a single shared global test set. All methods use the same frozen ViT-MAE-Base backbone, so this controlled comparison does not establish rankings when representations are unfrozen.
| Dataset | Clients / \(\alpha\) | AnaPFL ↑ | Strongest baseline in setting ↑ | Gain (percentage points) |
|---|---|---|---|---|
| CIFAR-100 | 50 / 0.1 | 80.64 | FedALA: 72.89 | +7.75 |
| CIFAR-100 | 100 / 0.1 | 79.22 | FedALA: 70.63 | +8.59 |
| Tiny-ImageNet | 100 / 0.1 | 68.57 | FedALA: 61.16 | +7.41 |
| ImageNet-R | 100 / 0.1 | 44.91 | FedALA: 28.20 | +16.71 |
All entries come from Table 1; gains are absolute accuracy differences, not relative percentage improvements. The smallest advantage across all 18 settings is 1.57 percentage points on ImageNet-R with 100 clients and \(\alpha=1.0\), so double-digit gains are not universal.
Ablation Study¶
The available main paper does not provide a standalone component-ablation table or verifiable numerical sweeps for projection sharing, refinement removal, or \(\lambda\). Instead, the following genuine analysis uses Table 1 to compare heterogeneity levels on CIFAR-100 with 50 clients. Values are average local-test accuracy in percent, higher is better; AFL is an independent baseline, not a strictly matched “AnaPFL without refinement” ablation.
| Method | \(\alpha=0.1\) ↑ | \(\alpha=0.5\) ↑ | \(\alpha=1.0\) ↑ |
|---|---|---|---|
| FedALA | 72.89 | 52.60 | 46.42 |
| AFL | 56.63 | 56.78 | 56.66 |
| AnaPFL | 80.64 | 66.96 | 62.63 |
As class distributions become more uniform, FedALA's personalization benefit weakens, AFL remains relatively stable, and AnaPFL exceeds both across the tested range. This supports complementarity between global sharing and local correction, but does not isolate the individual contributions of different random projections and residual targets.
Key Findings¶
- One round need not trade accuracy for speed: Section 4.3 reports one aggregation round per client for AnaPFL and AFL, versus up to 200 rounds for gradient baselines; AnaPFL remains more accurate in this protocol.
- The efficiency claim has a specific reference: Section 4.3 and Figure 5 report over 99% reductions in computation and communication against representative FedAvg and FedPCL comparisons, not against the slightly faster AFL.
- Costs remain nonzero: the prose gives ranges of 100–300 seconds versus 45,000–66,000 seconds and 200–400 MB versus 20,000–40,000 MB. These are ranges across comparisons, not endpoints that can be arbitrarily paired into exact dataset-specific ratios.
- Feature extraction is included: efficiency experiments use 50 clients, \(\alpha=0.1\), and 1024-dimensional streams, counting both feature extraction and classifier training. AFL is slightly faster because it omits personalization.
Highlights & Insights¶
- Closed-form learning specifies the aggregation target: its value extends beyond removing iterations to recovering the centralized fixed-feature solution. That guarantee concerns model parameters, not uniform test performance across clients.
- Personalization can be a residual problem: local classifiers complement shared predictions instead of competing to fit all labels independently. This division of responsibility also removes the need for another communication round.
- Nonlinear representation and solvable fitting can be separated: random projections and Tanh enrich the features while squared loss preserves analytic classifier fitting. The corresponding limitation is the inability to learn task-specific representations within this procedure.
Limitations & Future Work¶
- Author-stated scope: the method targets efficient personalized federated learning with pretrained frozen features, not every gradient-based approach with trainable representations. Stronger self-supervised backbones are an extension discussed by the authors.
- Author-proposed efficiency direction: autocorrelation matrices and their associated computations still incur costs. Low-rank compression or sketching is suggested, but compressed variants are not evaluated here.
- Reader assessment—privacy is not automatic: retaining raw images locally and using self-supervised pretraining do not establish formal privacy guarantees for uploaded statistics or classifier weights. The available main paper provides neither a differential-privacy budget nor a statistical-leakage evaluation.
- Reader assessment—component attribution remains limited: standalone ablations, random-seed error bars, and larger-scale client-system measurements are missing, leaving the separate contributions of feature spaces, residual fitting, and parameter selection insufficiently resolved.
- Evidence boundary: corrupted equations and unavailable appendices limit verification of recursive coefficients and full proofs. The mechanisms and results described here are grounded in the available main text rather than reconstructed derivations.
Related Work & Insights¶
- vs AFL: AFL supplies the foundation for single-round analytic aggregation but produces a global model. AnaPFL adds an analytic residual stream to adapt to client distributions without another aggregation round.
- vs FedALA / FedSelect: these methods personalize through adaptive local aggregation or customized parameter selection while retaining iterative optimization. AnaPFL restricts fitting to analytic classifiers on fixed nonlinear features, trading representation-learning flexibility for efficiency.
- vs AFCL / DeepAFL: the paper positions federated continual learning and deeper analytic networks as adjacent directions. AnaPFL focuses on reconciling sharing with personalization and does not experimentally establish the performance of combinations with those methods.
Rating¶
- Novelty: 4/5. A coherent connection between analytic global aggregation and local residual fitting, although random features, least squares, and dual streams are established tools.
- Experimental Thoroughness: 3/5. Three datasets and multiple heterogeneity settings provide useful coverage, but independent component ablations and uncertainty estimates are missing.
- Writing Quality: 4/5. The motivation, stream responsibilities, and frozen-feature protocol are clear; complete theoretical details require the appendices.
- Value: 4/5. Relevant when reliable frozen visual backbones are available and communication rounds are constrained, rather than a universal replacement for end-to-end federated learning.