📐 Optimization & Theory

🎞️ ECCV2026 · 13 paper notes

📌 Same area in other venues: 🧠 NeurIPS2026 (4) · 📷 CVPR2026 (22) · 🔬 ICLR2026 (222) · 🧪 ICML2026 (88) · 🤖 AAAI2026 (21) · 🧠 NeurIPS2025 (126)

🔥 Top topics: Federated Learning ×4

LDC-MTL: Balancing Multi-Task Learning through Scalable Loss Discrepancy Control

LDC-MTL explicitly controls task-loss discrepancies through bilevel optimization and removes an empirically small gradient term to enable single-loop first-order training, achieving an aggregate relative performance-drop score of -1.31 on CelebA at 1.07 times the training time of linear scalarization.

HSFM: Hard-Set-Guided Feature-Space Meta-Learning for Robust Classification under Spurious Correlations

HSFM freezes an ERM-pretrained backbone and treats the feature-space embeddings of a small support set as learnable parameters, optimizing them through a bilevel meta-objective driven by the model's own loss on hard examples (inner loop adapts the linear head, outer loop minimizes hard-set loss); in a few minutes on a single GPU it reaches worst-group accuracy on par with methods that use group labels on Waterbirds and Dominoes, and the resulting feature edits can be decoded back into images via unCLIP to expose the model's spurious dependencies.

Why Can Accurate Models Be Learned from Inaccurate Annotations?

The paper turns "why can accurate models be learned from inaccurate annotations?" into an analyzable question: label inaccuracy only deposits noise in the lower singular components of the classification weight matrix, and at moderate inaccuracy rates the principal subspace barely rotates, so task-relevant knowledge survives. This is argued empirically through singular-value spectra and Grassmann subspace similarity and theoretically through a Davis-Kahan angle bound, and it motivates LIP — a training-free post-processing plug-in that keeps the top-\(k\) singular components and re-estimates the flattened tail singular values from the training set — which improves 11 baselines on 6 real-world partial-label datasets and on CIFAR/CUB noisy settings.

Coordinate Singularities Break Conformal Coverage for Gaze and Head Pose

Reveals that using flat chart coordinates (yaw-pitch or Euler angles) for conformal prediction nonconformity scores causes catastrophic 30–50 pp slice-conditional coverage collapse near singularities on \(\mathcal{S}^2\) and \(\mathrm{SO}(3)\) despite nominal 90% marginal coverage, proves that scalar adaptive calibration cannot resolve this shape distortion, and introduces a retraining-free, zero-overhead geodesic scoring remedy.

Dive into the implicit biases of low-rank vision-language alignment

Challenging the prevailing dogma that vision-language alignment demands full-parameter fine-tuning, this paper demonstrates that low-rank alignment significantly slashes training overhead while outperforming full-parameter baselines, and theoretically as well as geometrically proves that low-rank adaptation preserves per-token linear separability (averting the LS-curse) and biases optimization toward flat, perturbation-robust parameter subspaces.

RBE-Flow: Recurrent Bayesian Estimation on Feature Manifolds for Cross-Modal Registration

Reformulates cross-modal dense optical flow estimation as a closed-loop recurrent Bayesian state estimation problem on feature manifolds. Relying on Recurrent Manifold Optimization (RMO) to generate flow observations with uncertainty, and Uncertainty-Adaptive Probabilistic Update (UAPU) to perform MMSE-optimal Bayesian fusion via Sigma-point projections while feeding back the posterior covariance to regulate the next optimization step. It consistently achieves SOTA on three cross-modal datasets (OSdataset, WHU-OPT-SAR, and RoadScene), reducing the AEPE by up to 45.4% compared to previous best methods.

SUM: Unified Geometric Surgery on Spatio-Temporal Adaptation Vectors for Federated Class Incremental Learning

Reinterpreting Federated Class Incremental Learning (FCIL) as a unified multi-task learning problem, SUM performs purely server-side geometric surgery on spatio-temporal adaptation vectors during aggregation—eliminating cross-client and cross-task directional interference without any client-side computational or communication overhead.

AnaPFL: When Closed-Form Solutions Meet Generalization and Personalization in Personalized Federated Learning

AnaPFL analytically aggregates a global primary stream and then fits a local residual refinement stream on frozen visual features, requiring one aggregation round and improving accuracy over the strongest baseline in each of 18 benchmark settings by 1.57–16.71 percentage points.

Can Vision Models Truly Forget? Mirage: Representation-Level Certification of Visual Unlearning

Addressing the critical vulnerability that existing visual unlearning evaluations rely solely on output-level metrics and mask persistent internal traces, this paper proposes Mirage, a representation-level certification framework combining linear probe recovery, centered kernel alignment, feature separability, and layer-wise tracking, revealing the widespread "forgetting illusion" and an unlearning trilemma across utility, output forgetting, and representational erasure.

H-SFP: Hierarchical Federated Learning with Decoupled Split-Model Prototyping

Addressing the bandwidth bottlenecks and synchronous locks of transmitting high-dimensional activations and backward gradients in split federated learning, H-SFP decouples cross-tier backpropagation by exchanging compact class-wise first- and second-order feature statistics, synthesizing Gaussian feature distributions at upper tiers to cut communication overhead by over two orders of magnitude while substantially curbing client drift.

HEM: a margin-based loss for visual categorisation tasks

Addressing the issues of overconfidence and perpetual weight overwriting in cross-entropy (CE) loss as well as gradient vanishing in traditional multi-class margin (MM) losses, this paper introduces High Error Margin (HEM) loss, which adaptively filters low errors and incorporates class-frequency adjusted margins, substantially outperforming CE and specialized losses across unknown class rejection, adversarial robustness, continual learning, and semantic segmentation.

Prototype Normalization: Optimizing Prototype Separation for Heterogeneous Federated Learning

Addressing poor prototype separation from weighted averaging and high client computational burdens in heterogeneous federated learning, ProtoNorm models global prototypes as charged particles on a hypersphere and solves the Thomson problem entirely server-side, achieving maximal inter-class angular separation and Euclidean magnitude upscaling with minimal computational overhead.

SON-GOKU: Conflict-Free Scheduling for Multi-Task Learning via Graph Coloring

SON-GOKU consistently improves various MTL baselines across six datasets by measuring real-time gradient conflicts between tasks, constructing a conflict graph, and applying a greedy graph coloring algorithm to dynamically partition tasks into low-conflict groups. Only one task group is activated per training step, with periodic recoloring to adapt to gradient evolution.