Skip to content

title: >- [Paper Note] Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions description: >- [ECCV 2026][self_supervised][Continual Learning] Identifies spectral collapse in recursive least squares analytic continual learning under long-tailed distributions, proposing Geometry-Spectral Rectification (GSR) with spherical geodesic interpolation to achieve anisotropic spectral regularization and provable stable rank improvements. tags: - ECCV 2026 - self_supervised - Continual Learning - Class-Incremental Learning - Long-Tailed Distribution - Spectral Regularization date: 2026-09-19 content_hash: 35da0c6ef7a91a22

Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions

Conference: ECCV 2026
Paper: ECCV 2026 Official
Full-text Cache: /Users/zy/workspace/paper_cache/ECCV2026/eccv-4012.txt
Area: Self-Supervised Learning
Keywords: Analytic Continual Learning, Class-Incremental Learning, Long-Tailed Distributions, Spectral Collapse, Geodesic Interpolation

TL;DR

Addressing catastrophic performance drops of recursive least squares (RLS) analytic continual learning under long-tailed distributions caused by autocorrelation matrix spectral collapse, this paper proposes Geometry-Spectral Rectification (GSR), which employs hyperspherical geodesic interpolation to inject structured anisotropic spectral perturbations, provably improving Gram matrix stable rank with \(O(D)\) linear complexity.

Background & Motivation

Analytic Continual Learning (ACL) has recently emerged as an appealing "Green AI" alternative to gradient-based continual learning paradigms. By reformulating incremental learning objectives into a recursive least squares (RLS) framework, ACL methods freeze pre-trained foundation models and iteratively accumulate the autocorrelation (Gram) matrix \(G = Z^T Z\) alongside the cross-correlation term \(Q = Z^T Y\). This formulation enables instantaneous closed-form classifier updates via regularized matrix inversion \((G + \tau I)^{-1} Q\) without iterative backpropagation, matching offline joint training performance on class-balanced streams. However, this theoretical elegance hinges on the numerical stability of the RLS inversion. In realistic scenarios characterized by long-tailed class distributions, state-of-the-art ACL methods such as GACL, RanPAC, and AnaCP suffer devastating performance collapse, frequently being outperformed by elementary prototype-based baselines like SimpleCIL.

While previous continual learning literature typically attributes class-imbalance failures to standard sample scarcity and responds with heuristic re-sampling or loss re-weighting, the authors reveal that the root vulnerability in RLS-based ACL is an algebraic pathology: Spectral Collapse. In imbalanced streams, head classes overwhelmingly dominate the principal eigenspaces (large eigenvalues), while tail classes span subspaces with vanishing variance where eigenvalues collapse near zero (\(\lambda_{\text{tail}} \approx 0\)). Consequently, standard isotropic ridge regularization (\(\tau I\)) falls into an unavoidable dilemma: if \(\tau\) is set small to preserve fidelity, inverted tail components \((\lambda_{\text{tail}} + \tau)^{-1}\) explode, amplifying representation noise by orders of magnitude; if \(\tau\) is enlarged to dampen tail variance, inverted head components \((\lambda_{\text{head}} + \tau)^{-1}\) are over-shrunk, causing severe underfitting and erasure of high-discriminability features. Existing scalar re-weighting approaches merely rescale existing vectors without recovering collapsed geometric directions.

The core idea of this paper is to reformulate long-tailed analytic continual learning as an anisotropic spectral regularization problem, proposing Geometry-Spectral Rectification (GSR) to synthesize unit-norm support representations via spherical geodesic interpolation, constructing a class-covariance-aligned spectral perturbation matrix that selectively inflates collapsed tail eigenvalues and provably guarantees improved stable rank for the Gram matrix.

Method

Overall Architecture

GSR operates on representations extracted from frozen pre-trained backbones across sequential incremental tasks. The overall pipeline proceeds through four distinct stages: first, assessing the spectral health of each class to compute an adaptive mixing intensity \(\alpha_c\); second, applying geodesic interpolation on the unit hypersphere between intra-class sample pairs to produce synthetic support features \(Z_{\text{aug}}\); third, executing a dual-stream block-wise update that decouples true data fidelity from geometric spectral rectification to update \(G_t\) and \(Q_t\); and fourth, solving the closed-form regularized least squares problem to analytically produce the updated classification head \(W_t\).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Data Stream<br/>Frozen backbone extracts features Z_t and labels Y_t"] --> B["Class-Adaptive Mixing<br/>Compute mixing intensity α_c based on sample count N_c"]
    B --> C["Hyperspherical Geodesic Interpolation<br/>Synthesize arc-projected support features Z_aug"]
    C --> D["Dual-Stream Block-wise Update<br/>Unify fidelity and rectification terms for G_t, Q_t"]
    D --> E["Analytic Closed-Form Inversion<br/>Solve for optimal classification head W_t"]

Key Designs

1. Class-Adaptive Mixing Strategy: Sample-aware spectral injection control To prevent indiscriminate variance injection across all classes, GSR assigns each class \(c\) an adaptive mixing intensity \(\alpha_c\) that scales inversely with its empirical sample count \(N_c\): $\(\alpha_c = \alpha_{\text{base}} + (1 - \alpha_{\text{base}}) \cdot e^{-\xi \cdot N_c}\)$ where \(\alpha_{\text{base}}\) provides a minimal baseline and \(\xi\) controls the exponential decay rate. For severely starved tail classes (\(N_c \ll D\)), the decay term \(e^{-\xi N_c}\) approaches 1, driving \(\alpha_c\) toward its maximum and broadening the dispersion of the sampling distribution \(\text{Beta}(\alpha_c, \alpha_c)\) to generate diverse synthetic supports. Conversely, for abundant head classes, the exponential decay vanishes rapidly, collapsing \(\alpha_c\) down to \(\alpha_{\text{base}}\) to strictly preserve the clean empirical statistics and crisp decision boundaries of majority classes.

2. Hyperspherical Geodesic Interpolation: Arc-preserving anisotropic spectral perturbation Standard linear interpolation (e.g., Manifold Mixup) traverses along the Euclidean chord connecting two features, causing severe norm shrinkage (\(\|\gamma z_i + (1-\gamma) z_j\|_2 < 1\)) and energy attenuation that deviates from the compact hyperspherical embedding manifolds typical of modern self-supervised models (e.g., DINO-v2). GSR introduces spherical mixup, interpolating strictly along the geodesic arc connecting intra-class pairs \((z_i, z_j)\) and renormalizing back onto the unit sphere: $\(\tilde{z} = \frac{\gamma z_i + (1 - \gamma) z_j}{\|\gamma z_i + (1 - \gamma) z_j\|_2}, \quad \gamma \sim \text{Beta}(\alpha_c, \alpha_c)\)$ This nonlinear normalization prevents synthetic samples from collapsing onto the linear convex hull. From a Vicinal Risk Minimization perspective, the second moment of this augmented stream induces an empirical covariance shift \(\mathbb{E}[\tilde{z}\tilde{z}^T] \approx G_{\text{original}} + \Delta\). Because \(\Delta\) is proportional to the intra-class covariance \(\Sigma_{\text{class}}\), it acts as an anisotropic spectral filter that selectively inflates the collapsed eigenvalues \(\lambda_{\text{tail}}\) along valid semantic principal directions rather than injecting isotropic white noise into null-spaces. Under Theorem 1, this structured perturbation provably guarantees a strictly increased stable rank \(\text{sr}(\tilde{G} + \tau I) = \frac{\sum (\lambda_i + \tau)^2}{\lambda_{\max}^2}\), preventing numerical instability during inversion.

3. Dual-Stream Block-wise Update: Balancing statistical fidelity and spectral rectification To ensure unbiased learning from real observations while preventing rank deficiency, GSR formulates a dual-stream recursive accumulation mechanism for both the cross-correlation matrix \(Q_t\) and the Gram matrix \(G_t\): $\(Q_t = Q_{t-1} + Z_t^T Y_t + \beta Z_{\text{aug}}^T Y_{\text{aug}}\)$ $\(G_t = G_{t-1} + \underbrace{Z_t^T Z_t}_{\text{Fidelity Term}} + \underbrace{\beta Z_{\text{aug}}^T Z_{\text{aug}}}_{\text{Rectification Term}}\)$ where \(\beta\) is a balancing coefficient. The fidelity term guarantees accurate alignment with true empirical class prototypes, whereas the rectification term densifies the tail covariance spectrum along geodesic trajectories. The resulting regularized weight matrix is computed via closed-form inversion \(W_t = (G_t + \tau I)^{-1} Q_t\). Because synthetic samples are generated via simple element-wise normalization, this entire rectification operates in \(O(D)\) linear complexity, bypassing the computationally prohibitive \(O(D^3)\) covariance estimation and Cholesky decomposition required by Gaussian sampling methods.

Loss & Training

The feature extractor remains frozen throughout all incremental stages. Classifier weights are solved non-iteratively at each incremental task boundary via the regularized least squares closed-form solution. Key hyperparameters include the base mixing intensity \(\alpha_{\text{base}}\), decay coefficient \(\xi\), rectification stream weight \(\beta\), and the baseline ridge parameter \(\tau\).

Key Experimental Results

Main Results

The authors evaluate GSR across four recognized class-incremental benchmark datasets under long-tailed distributions: Split CIFAR-100, Split ImageNet-R, Split Tiny-ImageNet, and Split CUB-200. Evaluation metrics include Last Accuracy (\(A_{\text{last}}\)) and Average Incremental Accuracy (\(A_{\text{avg}}\)). Representative results using a frozen DINO-v2 backbone (extracted from Table 1 of the paper) are summarized below:

Dataset Metric GACL Baseline GACL + GSR (Ours) RanPAC + GSR (Ours) AnaCP + GSR (Ours)
Split CIFAR-100 \(A_{\text{last}}\) (%) 48.78 ± 1.32 65.51 ± 1.25 66.68 ± 1.25 82.51 ± 0.01
Split CIFAR-100 \(A_{\text{avg}}\) (%) 58.84 ± 2.97 71.79 ± 2.69 71.98 ± 2.65 88.29 ± 0.73
Split ImageNet-R \(A_{\text{last}}\) (%) 47.84 ± 2.18 61.58 ± 0.58 62.08 ± 0.62 72.13 ± 0.84
Split ImageNet-R \(A_{\text{avg}}\) (%) 51.53 ± 1.34 65.90 ± 0.72 66.65 ± 1.07 77.96 ± 0.47
Split Tiny-ImageNet \(A_{\text{last}}\) (%) 42.98 ± 0.37 68.88 ± 0.29 69.57 ± 0.29 78.81 ± 0.16
Split Tiny-ImageNet \(A_{\text{avg}}\) (%) 53.65 ± 1.32 70.70 ± 1.27 71.45 ± 1.25 83.71 ± 0.96
Split CUB-200 \(A_{\text{last}}\) (%) 80.73 ± 1.07 82.46 ± 0.58 83.03 ± 0.62 85.85 ± 0.43
Split CUB-200 \(A_{\text{avg}}\) (%) 87.20 ± 1.17 87.98 ± 1.00 88.64 ± 1.02 90.53 ± 0.85

Ablation Study

The ablation study on Split CUB-200 (extracted from Table 3 of the paper) validates the step-by-step contribution of each design component over the standard ridge regression baseline:

Variant Configuration Mixup Spherical Projection Adaptive (\(\alpha_c\)) DINO-v2 \(A_{\text{last}}\) (%) MoCo-v3 \(A_{\text{last}}\) (%)
Standard Ridge (GACL) - - - 80.73 ± 1.07 34.48 ± 1.27
+ Linear Mixup ✓ - - 77.85 ± 1.63 45.90 ± 0.83
+ Spherical Mixup ✓ ✓ - 82.01 ± 0.47 47.45 ± 0.67
+ Full GSR Framework ✓ ✓ ✓ 82.46 ± 0.58 48.33 ± 0.80

Key Findings

  • Euclidean linear interpolation degrades high-dimensional representations: In DINO-v2's normalized embedding space, conventional linear mixup drops performance from 80.73% to 77.85% (-2.88%) due to chordal norm shrinkage. Only geodesic hyperspherical projection maintains feature energy and improves accuracy to 82.01%.
  • Resilience under severe imbalance ratios: Under varying imbalance ratios \(\rho\) from 10 to 150 on Split CIFAR-100 (Table 4), the baseline GACL collapses from 81.23% to 38.60% (a 42.63% drop), whereas GACL + GSR drops only to 55.50%, widening the margin from +3.99% to +16.90%.
  • Resurrecting tail classes without sacrificing heads: Frequency breakdown (Table 5) shows that on Split CIFAR-100, tail accuracy surges from 12.50% to 38.00% (a 3x improvement) with a negligible 0.13% change in head accuracy (95.38% vs 95.25%). On ImageNet-R, GSR simultaneously enhances both head (+5.62%) and tail (+26.29%) accuracy.
  • Superior efficiency over Gaussian sampling: Compared to covariance shrinkage or transfer methods that require \(O(D^3)\) Cholesky factorization, GSR executes in \(O(D)\) time while achieving superior final accuracy (78.81% vs 75.21% on Tiny-ImageNet when paired with AnaCP).

Highlights & Insights

  • Root-cause spectral diagnosis of long-tailed analytic continual learning: Instead of treating class imbalance as an empirical data-counting issue, the paper formalizes the failure mode as spectral collapse of the empirical Gram matrix and highlights the fundamental inadequacy of isotropic ridge dampening.
  • Riemannian geodesic interpolation for feature regularization: Cleverly aligns vicinal risk minimization with the hyperspherical geometry of self-supervised embeddings, demonstrating that unit-norm preservation avoids energy suppression and creates valid anisotropic covariance mass.
  • Geometric synergy with contrastive projection in AnaCP: Elegantly explains why GSR unlocks AnaCP's performance (reaching 82.51% SOTA on CIFAR-100): contrastive projection layers depend on Neural Collapse cluster geometry, which is prone to dimensional collapse under imbalanced data; GSR restores the requisite stable rank.

Limitations & Future Work

  • Reliance on pre-trained embedding quality: The method assumes that the pre-trained feature extractor preserves basic semantic clustering. If out-of-distribution shifts severely corrupt the representation geometry, geodesic interpolation cannot synthesize genuinely novel orthogonal semantics.
  • Fixed hyperparameter scheduling: The exponential decay factor \(\xi\) and dual-stream balance \(\beta\) are currently set as static parameters across all tasks, which may not dynamically adapt if task-specific imbalance profiles fluctuate unpredictably.
  • Future directions: Developing an online, closed-form adaptive estimation for \(\beta_t\) guided directly by real-time stable rank metrics \(\text{sr}(G_t)\) to achieve fully parameter-free spectral rectification.
  • vs AIR (Analytic Imbalance Rectifier): AIR applies inverse-frequency scalar weighting directly to the Gram matrix, which merely inflates existing variance without recovering missing orthogonal subspaces. GSR injects a structured anisotropic tensor perturbation \(\Delta\) along geodesic arcs.
  • vs Gaussian Feature Augmentation (Covariance Transfer / Shrinkage): Gaussian methods incur an unscalable \(O(D^3)\) computational cost, making them impractical for edge devices; GSR achieves comparable or superior rectification in \(O(D)\) linear complexity.
  • vs Manifold Mixup: Traditional Manifold Mixup is designed for gradient-based training and causes norm shrinkage via Euclidean interpolation; GSR is explicitly constructed for closed-form RLS updates via unit hypersphere projection.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Formulates the failure of RLS-ACL under long-tailed distributions as spectral collapse and provides rigorous stable rank improvement proofs.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation across 4 benchmark datasets, multiple backbones, varying imbalance ratios, stable rank trajectories, and computational complexity benchmarks.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear theoretical exposition, cohesive mathematical transitions from problem diagnosis to geometric solution.
  • Value: ⭐⭐⭐⭐⭐ Provides a highly practical, computationally lightweight \(O(D)\) framework for edge-friendly, backpropagation-free continual learning.