title: >- [Paper Note] Why Feature Magnitude Deceives OOD Detectors: An Angular Separation Perspective description: >- [ECCV 2026][AI Safety][OOD Detection] Proposes Angular Separation Learning (ASL) via feature l2-normalization to overcome feature magnitude deception in distance-based OOD detection, achieving SOTA near-OOD performance. tags: - ECCV 2026 - AI Safety - OOD Detection - Feature Normalization - Angular Separation date: 2026-09-19 content_hash: f50aef31b1b3d28e
Why Feature Magnitude Deceives OOD Detectors: An Angular Separation Perspective¶
Conference: ECCV 2026
Paper: ECCV 2026
Code: https://github.com/HAIV-Lab/ASL
Area: AI Safety
Keywords: Out-of-Distribution Detection, Feature Normalization, Angular Separation Learning, Mahalanobis Distance, Neural Collapse
TL;DR¶
Uncovering a critical failure mode where distance-based OOD detectors are deceived by small feature magnitudes, this paper proposes Angular Separation Learning (ASL)—applying scaled \(\ell_2\)-normalization directly before a bias-free linear classifier to induce hyperspherical contrastive learning with minimal computational overhead.
Background & Motivation¶
Deploying deep neural networks in open-world, safety-critical applications demands robust Out-of-Distribution (OOD) detection, ensuring models know when they do not know rather than making catastrophically overconfident predictions on unfamiliar inputs. A prominent line of post-hoc detection methods analyzes the geometry of pre-trained feature spaces, most notably through distance metrics such as the Mahalanobis Distance Score (MDS). However, the representations inherited from standard cross-entropy training are optimized solely to be discriminative among closed-set in-distribution (ID) classes rather than modeling true underlying data distributions. This optimization dynamic often leaves feature spaces loose and fragile, frequently projecting OOD samples into deceptively high-confidence regions.
To address these shortcomings, recent research has increasingly shifted toward training-based methods, incorporating synthetic outlier generation, complex contrastive objectives, or specialized loss functions. Yet, these approaches introduce substantial computational burden, hyperparameter sensitivity, and intricate pipelines, often failing to translate into clear performance gains in challenging near-OOD regimes. A deeper inspection of the Mahalanobis distance formulation reveals an overlooked structural flaw: the quadratic distance computation fundamentally conflates feature magnitude (\(\ell_2\)-norm) with angular orientation. Under standard cross-entropy, networks aggressively exploit feature magnitude inflation as an optimization shortcut to drive down training loss rapidly. Consequently, feature magnitudes exhibit vast variance, resulting in a pathological failure mode where low-norm OOD samples—despite severe angular deviations from class prototypes—yield smaller Mahalanobis distances than genuine high-norm ID samples, completely evading detection.
Rather than passively wrestling with magnitude-induced noise during post-hoc scoring, the authors propose a proactive paradigm shift: enforcing the network during training to optimize exclusively for angular separation on a hypersphere. Core idea: introduce Angular Separation Learning (ASL), which enforces \(\ell_2\)-normalization on feature representations prior to a bias-free linear classifier, transforming cross-entropy into an implicit prototype-based contrastive learning objective that achieves high intra-class compactness and large inter-class angular margins without explicit negative mining.
Method¶
Overall Architecture¶
ASL disentangles feature magnitude from directional alignment by forcing the feature representation onto a hypersphere. Given an input sample \(x\), the feature extractor \(f(\cdot)\) outputs a \(d\)-dimensional embedding. Instead of passing this embedding directly to a standard linear layer with bias, ASL applies a scaled \(\ell_2\)-normalization to obtain normalized feature \(\tilde{f}\). The logit vector is then computed via inner product with the unconstrained, bias-free weight matrix \(W\) (whose columns serve as class prototypes), followed by the cross-entropy loss regularized by weight decay. The overall end-to-end flow is summarized below:
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input Sample x"] --> B["Feature Encoder f(x)"]
B --> C["Feature Normalization & Scaling<br/>f~ = s * f(x) / ||f(x)||2"]
C --> D["Bias-Free Linear Classifier<br/>logits = W^T * f~"]
D --> E["Cross-Entropy & Weight Decay<br/>LASL = LCE + lambda * ||W||_F^2"]
E --> F["Hyperspherical Embeddings & Scoring<br/>Evaluate via MDS or Cosine Similarity"]
At test time, the exact same normalized feature mapping is applied, and OOD detection scores are computed directly using either the Mahalanobis distance or nearest class-mean cosine similarity (FeatCos) in the resulting angular space.
Key Designs¶
1. Feature Normalization & Scaling: Blocking Optimization Shortcuts and Stabilizing Gradients
Standard cross-entropy allows networks to minimize training loss by continuously scaling up feature magnitudes rather than tightening intra-class variance. ASL suppresses this magnitude shortcut by applying \(\ell_2\)-normalization along with a fixed scalar scaling factor \(s\) (set to 10 by default):
Directly mapping features onto the unit sphere would drastically diminish back-propagated gradient magnitudes, impairing convergence in deep networks. The scaling factor \(s\) restores appropriate gradient magnitude dynamics, ensuring stable optimization while constraining all sample representations strictly to a hypersphere of radius \(s\).
2. Implicit Hyperspherical Contrastive Learning: Margin Separation Without Pair Mining
ASL treats the final classifier weight matrix \(W = [w_1, w_2, \dots, w_K] \in \mathbb{R}^{d \times K}\) as learnable class prototypes and removes the bias term. The logit for class \(j\) becomes \(w_j^\top \tilde{f} = \|w_j\|_2 \|\tilde{f}\|_2 \cos(\theta_j) = s \|w_j\|_2 \cos(\theta_j)\), where \(\theta_j\) is the angle between feature \(\tilde{f}\) and prototype \(w_j\). Under standard weight decay \(\lambda\), the objective is:
This formulation implicitly performs prototype-based contrastive learning in angular space, minimizing the angle \(\theta_y\) to the target prototype while maximizing the angular separation \(\theta_j (j \neq y)\) from all negative prototypes. Crucially, the authors prove in Proposition 1 that at convergence, prototype norms are bounded by \(\|w_j\|_2 \le \frac{s}{2\lambda}\). Here, weight norms function as dynamic inverse temperatures, facilitating coarse-grained exploration early in training and strict angular clustering in later stages.
3. Theoretical OOD Separability Lower Bound and Weight Decay Regularization
To formalize the performance leap on near-OOD samples, the authors establish a theoretical lower bound on distribution separability \(\Delta\) using the Integral Probability Metric (IPM) across ID, covariate shift, and OOD distributions:
This analytical lower bound reveals that increasing the weight decay coefficient \(\lambda\) restricts the hypothesis space, effectively suppressing the statistical penalty term scaling with \(1/\lambda\). At the same time, it drives the network to extract finer-grained, more discriminative representations that push the empirical OOD mean \(\hat{\mu}_{ood}\) geometrically further from the ID mean \(\hat{\mu}_P\). This directly explains why tuning \(\lambda\) reliably sharpens near-OOD separability while far-OOD separation remains naturally preserved.
Loss & Training¶
The overall training objective of ASL consists of standard cross-entropy calculated on the normalized logits, supplemented only by standard Frobenius-norm weight decay:
The implementation requires only a one-line modification after feature extraction and disabling bias in the linear classification layer. With scaling factor \(s = 10\) and standard weight decay \(\lambda \in [1 \times 10^{-4}, 5 \times 10^{-4}]\), ASL eliminates the need for negative pair mining, memory banks, or synthetic outlier generators, keeping per-epoch training time nearly identical to vanilla cross-entropy.
Key Experimental Results¶
Main Results¶
Evaluated on the standardized OpenOOD benchmark, ASL was benchmarked on ResNet-18 trained from scratch on ImageNet-200 (Table 1), as well as on CIFAR-10 and CIFAR-100 (Table 2).
Table 1: OOD detection performance on ImageNet-200 with ResNet-18 trained from scratch (FPR95↓ / AUROC↑)
| Method | Near-OOD: SSB-hard | Near-OOD: NINCO | Near-OOD: Avg. | Far-OOD: Avg. | In-Distribution Acc.↑ |
|---|---|---|---|---|---|
| CE (Standard Cross-Entropy) | 72.5 / 78.8 | 46.1 / 86.9 | 59.3 / 82.8 | 24.4 / 94.0 | 86.3% |
| CIDER (ICLR 2023) | 71.0 / 75.5 | 45.3 / 85.3 | 58.1 / 80.4 | 21.5 / 94.4 | 84.2% |
| LogitNorm (ICML 2022) | 66.7 / 78.4 | 46.6 / 86.4 | 56.7 / 82.4 | 26.1 / 93.1 | 86.4% |
| PALM (ICLR 2024) | 70.1 / 75.7 | 50.3 / 84.8 | 60.2 / 80.2 | 21.3 / 94.5 | 83.8% |
| ReweightOOD (CVPR 2024) | 67.2 / 80.5 | 41.8 / 86.8 | 54.5 / 83.7 | 21.9 / 94.5 | 84.0% |
| T2FNorm (CVPR 2024) | 66.1 / 79.2 | 46.2 / 86.8 | 56.1 / 83.0 | 25.3 / 93.5 | 86.5% |
| ASL (Ours) | 65.7 / 81.5 | 39.1 / 88.6 | 52.4 / 85.1 | 20.4 / 95.0 | 86.8% |
Table 2: OOD detection performance on CIFAR-10 and CIFAR-100 benchmarks (FPR95↓ / AUROC↑)
| Dataset / Method | Near-OOD: FPR95↓ | Near-OOD: AUROC↑ | Far-OOD: FPR95↓ | Far-OOD: AUROC↑ | Overall Avg.: AUROC↑ | In-Distribution Acc.↑ |
|---|---|---|---|---|---|---|
| CIFAR-10 | ||||||
| CE | 40.2 | 88.8 | 30.6 | 91.0 | 89.9 | 95.2% |
| CIDER | 31.4 | 90.1 | 22.5 | 93.4 | 91.7 | 92.8% |
| LogitNorm | 28.9 | 92.5 | 15.8 | 96.3 | 94.4 | 94.5% |
| T2FNorm | 25.8 | 93.0 | 15.2 | 96.5 | 94.8 | 94.7% |
| ASL (Ours) | 25.1 | 93.3 | 12.0 | 97.2 | 95.2 | 94.8% |
| CIFAR-100 | ||||||
| CE | 67.8 | 74.8 | 51.9 | 82.6 | 78.7 | 77.4% |
| NPOS | 67.1 | 77.7 | 50.2 | 83.6 | 80.7 | 72.9% |
| LogitNorm | 62.4 | 78.5 | 50.4 | 82.1 | 80.3 | 76.1% |
| PALM | 64.7 | 77.2 | 37.8 | 86.7 | 82.0 | 75.7% |
| T2FNorm | 58.2 | 79.9 | 49.5 | 83.3 | 81.6 | 76.3% |
| ASL (Ours) | 56.2 | 80.9 | 47.8 | 85.0 | 83.0 | 76.7% |
Ablation Study¶
The computational efficiency and compatibility of ASL were analyzed in detail across training overhead and post-hoc detector pairings.
Table 3: Training time per epoch (seconds) on ImageNet-200 across training-based methods
| Method | Time (s/epoch)↓ | Overhead relative to CE | Complexity / Mechanism |
|---|---|---|---|
| Standard CE | \(252 \pm 1.63\) | Baseline | Standard Cross-Entropy |
| LogitNorm | \(294 \pm 1.25\) | +16.7% | Output logit normalization |
| T2FNorm | \(293 \pm 2.05\) | +16.3% | Train-time feature normalization |
| PALM | \(381 \pm 2.36\) | +51.2% | Mixture of prototypes |
| ReweightOOD | \(406 \pm 2.36\) | +61.1% | Dynamic pairwise loss reweighting |
| CIDER | \(657 \pm 4.55\) | +160.7% | Hyperspherical mutual information clustering |
| NPOS | \(874 \pm 2.16\) | +246.8% | Non-parametric outlier synthesis |
| HamOS | \(3399 \pm 7.76\) | +1248.8% | Hamiltonian Monte Carlo outlier generation |
| ASL (Ours) | \(277 \pm 1.70\) | +9.9% | Feature \(\ell_2\)-normalization only |
Table 4: Performance of post-hoc detectors with and without ASL features on CIFAR-10 (FPR95↓ / AUROC↑)
| Post-hoc Detector | Vanilla CE: Near-OOD | + ASL Features: Near-OOD | Vanilla CE: Far-OOD | + ASL Features: Far-OOD |
|---|---|---|---|---|
| MDS | 49.2 / 85.2 | 25.1 / 93.3 | 33.3 / 89.0 | 12.0 / 97.2 |
| RMDS | 38.6 / 89.8 | 33.7 / 90.6 | 25.9 / 92.4 | 18.9 / 94.4 |
| VIM | 45.6 / 88.4 | 26.2 / 93.0 | 28.1 / 92.4 | 13.1 / 96.9 |
| KNN | 34.4 / 90.7 | 26.6 / 92.9 | 24.4 / 93.1 | 15.6 / 96.5 |
| fDBD | 35.5 / 90.6 | 30.3 / 92.1 | 24.3 / 93.4 | 15.5 / 96.3 |
| NCI | 48.7 / 88.6 | 34.1 / 91.3 | 33.8 / 90.8 | 14.7 / 96.4 |
| MD++ | 40.2 / 88.8 | 25.1 / 93.3 | 30.6 / 91.0 | 12.0 / 97.2 |
Key Findings¶
- Substantial Leap in Near-OOD Benchmarks: On challenging ImageNet-200 near-OOD tasks (SSB-hard and NINCO), ASL delivers an average AUROC of 85.1% and suppresses FPR95 to 52.4%, outperforming the second-best training method by 1.4% AUROC without hurting closed-set accuracy (86.8% vs. 86.3% CE).
- Superior Training Efficiency: ASL requires only 277 seconds per epoch on ImageNet-200, which is just 9.9% slower than vanilla CE, while running over \(12\times\) faster than Hamiltonian Monte Carlo outlier synthesis (HamOS at 3399s).
- Universal Enhancement for Distance Detectors: When paired with ASL features, traditional distance-based detectors experience massive improvements. Classical MDS sees its near-OOD FPR95 drop dramatically from 49.2% to 25.1% on CIFAR-10, demonstrating that decoupling feature magnitude resolves the primary vulnerability of quadratic distance scoring.
- Progressive Depth Representation: Evaluating intermediate layers on ResNet-18 reveals that ASL representations become significantly more discriminative than standard CE starting as early as Block 4 (AUROC of 81.3% vs. ~65%), establishing clean angular boundaries well before the final classifier.
Highlights & Insights¶
- Geometric Diagnosis of Mahalanobis Sensitivity: Formally exposes how standard cross-entropy encourages norm inflation shortcuts, causing low-norm OOD samples to deceptively achieve smaller Mahalanobis distances than ID samples.
- Minimalist Design Beating High Complexity: Achieves state-of-the-art OOD separation without synthetic outliers, external memory banks, or pairwise contrastive mining, fundamentally challenging the assumption that OOD robustness requires complex architectures.
- Broad Versatility Across Backbones and Domains: Validates consistent gains across ResNet-18/50, ViT-B/16, covariate shift scenarios, and real-world safety-critical medical imaging (BIMCV COVID-19/RSNA bone age datasets, reducing average FPR95 from 23.0% to 12.4%).
Limitations & Future Work¶
- Static Scaling Factor: The scalar factor \(s\) is empirically fixed at 10; extending this to learnable or dynamic curvature scaling for large-vocabulary open-world scenarios remains to be explored.
- Long-Tailed Class Imbalance: Under severe class distribution skew, minority class prototypes on the hypersphere may risk geometric crowding, warranting further investigation into margin-aware angular regularizers.
- Extension to Multimodal VLMs: While evaluated on unimodal vision backbones, adapting explicit angular separation directly into vision-language alignment projection heads represents a promising avenue.
Related Work & Insights¶
- vs LogitNorm / T2FNorm: LogitNorm regularizes logit outputs to suppress confidence overestimation but fails to reshape internal angular geometry; T2FNorm treats feature normalization merely as a regularizer, whereas ASL formalizes it as implicit contrastive learning with bias removal and provable prototype norm bounds.
- vs CIDER / PALM / ReweightOOD: These contrastive training methods incur high compute costs from pairwise mining or mixture-prototype clustering (\(1.5\times\) to \(2.4\times\) slower); ASL matches or surpasses their performance at nearly the speed of standard cross-entropy.
- vs MDS / MD++: MDS fails due to norm distortion; MD++ normalizes features only at test-time post-hoc, leaving the underlying training space unconstrained; ASL fundamentally reshapes the representation space during training.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Elegant geometric decomposition of Mahalanobis failure modes coupled with an extraordinarily clean implicit contrastive formulation.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive coverage across CIFAR, ImageNet-200/1K, ViT/CNN backbones, covariate shifts, and clinical chest X-ray benchmarks.
- Writing Quality: ⭐⭐⭐⭐⭐ Rigorous logical progression connecting motivation, mathematical propositions, and empirical confirmations seamlessly.
- Value: ⭐⭐⭐⭐⭐ Delivers an exceptionally efficient, high-performance SOTA baseline that redefines the complexity-to-performance trade-off in OOD detection.