Skip to content

Exploiting Local Flatness for Efficient Out-of-Distribution Detection

Conference: ECCV 2026
arXiv: 2606.29952
Code: https://github.com/shpark97/Fold
Area: AI Safety / OOD Detection
Keywords: OOD Detection, Loss Landscape Curvature, Feature Hessian, Partial Feature Normalization, Self-Supervised Calibration

TL;DR

This paper provides the first systematic analysis of the difference in loss landscape curvature between OOD and ID samples, discovering that OOD inputs exhibit larger Hessian curvature which increases as the distribution shift intensifies. Based on this, a lightweight OOD detector, Fold, is proposed, which replaces expensive parameter-space curvature approximations with a feature-space Hessian. This is paired with partial feature normalization to enhance ID-OOD separability, and a self-supervised logit masking scheme (AutoFold) is introduced to automatically calibrate the normalization parameter. Fold achieves an average AUROC improvement of 1.63% and a 2.30% reduction in FPR95 across multiple benchmarks, with a computational overhead comparable to a single forward pass.

Background & Motivation

Background: One of the mainstream paradigms of OOD detection is post-hoc methods—extracting discriminative signals directly from pre-trained models without retraining. These methods mainly develop along three directions: uncertainty estimation based on the output layer (e.g., MSP, ODIN, Energy), threshold truncation based on intermediate layer activations (e.g., ReAct, ASH), and methods based on gradient or neuron statistics (e.g., GradNorm). Among them, utilizing the loss landscape curvature to estimate the Bayesian posterior via Laplace approximation is a theoretically appealing direction, but computing the full Hessian entails a cost quadratic to the number of parameters.

Limitations of Prior Work: Although parameter-space curvature-based OOD detection methods (such as Laplace-approximation-based methods) are theoretically elegant, their computational cost remains prohibitively high even with approximation techniques like Hutchinson's randomized estimator, making them unusable in real-time deployment scenarios. Furthermore, these methods implicitly rely on an unverified assumption: that the loss landscape induced by ID data is flatter than that induced by OOD data.

Key Challenge: Curvature signals are rich in ID/OOD discriminative information, but the cost of acquiring them is too high—there is a fundamental conflict between the high computational cost of parameter-space Hessians and the lightweight efficiency pursued by post-hoc methods. Meanwhile, there is a lack of systematic empirical validation in the academic community regarding this "curvature discrepancy" assumption itself, leaving its magnitude and the conditions under which it holds unknown.

Goal: (1) To conduct the first systematic empirical analysis of the ID-OOD curvature discrepancy; (2) to design a lightweight OOD detection method that leverages curvature signals without resorting to parameter-space Hessians; (3) to resolve hyperparameters' sensitivity to datasets, achieving automatic calibration without external OOD data.

Key Insight: The authors observe that the Hessian in logit space has a strong spectral correspondence with the full parameter Hessian (Lee et al. 2023). Moreover, the linear mapping matrix of the classifier itself encodes the discriminative geometry of ID features. If the logit-space curvature is "routed" back to the feature space through the classifier, it can simultaneously capture both predictive uncertainty and the geometric structure of the feature space while requiring far less computation than the parameter-space Hessian.

Core Idea: Use the feature Hessian instead of the parameter Hessian to measure curvature, then apply partial feature normalization to amplify the ID-OOD curvature gap, and finally employ self-supervised logit masking to automatically select hyperparameters without relying on external OOD data.

Method

Overall Architecture

The overall pipeline of Fold consists of three steps: given an input, it first obtains the hidden representation via a feature encoder and computes its Hessian curvature in the feature space (projecting the logit-space curvature back to the feature space through the classifier weight matrix). Next, it performs partial feature normalization (dividing by the norm raised to the power of \(\alpha\), where \(0 < \alpha \leq 1\)) to suppress the degrading impact of magnitude on curvature estimation. Finally, the trace of the normalized feature Hessian is used as the OOD detection score—the larger the trace, the more likely the input is OOD. Building upon the first two steps, the AutoFold variant uses logit masking on the ID validation set to generate pseudo-OOD samples, automatically searching for the optimal \(\alpha\) without relying on any external OOD data.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Image"] --> B["Feature Encoder<br/>Get hidden representation h"]
    B --> C["Compute Feature Hessian<br/>Project logit curvature to feature space"]
    C --> D["Partial Feature Normalization<br/>h / ||h||^α"]
    D --> E["Fold Score<br/>Larger tr(∇²ℒ) trace indicates OOD"]
    F["ID Validation Set + Logit Masking<br/>Generate pseudo-OOD"] -.->|"AutoFold: Search optimal α"| D

Key Designs

1. Feature Hessian: Projecting logit-space curvature to feature space, replacing expensive parameter Hessian with the discriminative geometry of linear classifiers

The pain point is straightforward: parameter Hessians capture the curvature differences between ID/OOD, but even with Hutchinson's approximation, they are still too slow for networks like ResNet-50. The authors' key insight is that the logit-space Hessian preserves spectral properties similar to the full Hessian (Lee et al. 2023), but has a dimension equal only to the number of classes \(C\) (such as 1000 for ImageNet), which is far smaller than the parameter count and much lighter to compute.

However, measuring curvature purely in logit space discards the discriminative geometry encoded by the classifier itself—the Jacobian of the classifier weight matrix characterizes the orientation of decision boundaries for each class in the feature space, and these geometric structures are critical for OOD detection (as exploited by methods like VIM). Therefore, the authors "transport" the logit curvature back to the feature space using the second-order chain rule:

\[\nabla_{\mathbf{h}}^2 \mathcal{L} = (\nabla_{\mathbf{h}}g)(\nabla_{\mathbf{z}}^2 \mathcal{L})(\nabla_{\mathbf{h}}g)^\top\]

Since the classifier \(g\) is a linear layer, its second-order derivative is zero, causing the second term on the right-hand side to vanish. Here, \(\nabla_{\mathbf{h}}g \in \mathbb{R}^{C \times d}\) is the classifier weight matrix, and \(\nabla_{\mathbf{z}}^2 \mathcal{L}\) is the Hessian in the logit space with respect to the energy function \(\mathcal{L}(\mathbf{z}) = \log(\sum \exp(z_i))\). Consequently, the feature Hessian projects prediction uncertainty from the logit space along the category discriminative directions, aligning the curvature signal with the feature manifold learned by the classifier. Ablation studies (Table 4) confirm that the feature Hessian significantly outperforms the pure logit Hessian on complex datasets like ImageNet-200/1K (average AUROC 86.81% vs. 86.02%).

2. Partial Feature Normalization: Suppressing magnitude degradation with \(h / \|h\|^\alpha\), balancing directional and magnitude information with different alpha across datasets

The feature Hessian has a critical vulnerability: it is highly sensitive to the feature norm. A larger feature norm leads to larger logits, making the softmax more saturated and the loss landscape flatter, which causes the Hessian to approach zero—rendering the curvature estimation meaningless noise. However, complete normalization (setting \(\alpha=1\), i.e., pure directional vectors) is also problematic: for complex datasets (like ImageNet), the feature magnitude itself carries useful fine-grained semantic information, and discarding it completely severely harms detection performance (on ImageNet-200, when \(\alpha=1\), the AUROC drops from 87.64% to 79.02%).

The partial normalization formula proposed by the authors is \(\widetilde{\bm{h}} = \bm{h} / \|\bm{h}\|^\alpha\), where \(0 < \alpha \leq 1\). This is equivalent to applying a sample-dependent temperature scaling to logits, continuously adjusting between fully preserving magnitude (\(\alpha=0\)) and complete normalization (\(\alpha=1\)). Experiments show that the optimal \(\alpha\) is related to dataset complexity: on CIFAR-10, larger \(\alpha\) is better (~1.0), whereas on ImageNet, a smaller \(\alpha\) is preferred (~0.2), since complex data requires retaining fine-grained information in the magnitude.

Spectral analysis (Figure 6) offers a deeper mechanistic explanation: without normalization, the spectral differences between ID and OOD are concentrated in the top 10 largest eigenvalues (i.e., worst-case curvature); complete normalization causes the two spectra to overlap completely, making them indistinguishable; partial normalization, on the other hand, maintains the gap in the largest eigenvalues while widening spectral separation in the mid-range eigenvalues. This indicates that it utilizes more distributed "average-case" curvature information rather than relying on only a few extreme directions.

3. AutoFold: Generating pseudo-OOD signals with ID logit masking for self-supervised optimal alpha search, completely eliminating dependency on external OOD data

Hyperparameter tuning in traditional post-hoc methods requires an auxiliary OOD validation set, which implies knowing the OOD distribution in advance during practical deployment—an unrealistic assumption. AutoFold's solution is elegant: given an ID validation sample (with known class \(k\)), its \(k\)-th logit is set to negative infinity (\(\tilde{z}_k = -\infty\)), forcing the model to rely on the remaining classes' logits to make judgements. This mimics the predictive ambiguity of encountering an "unknown class", generating a pseudo-OOD signal without requiring any additional training or retraining.

Specifically, the search objective is to perform logit masking across all \(K\) classes, maximizing the AUROC between the Fold scores of original ID samples and masked samples (i.e., making them as separable as possible). Unlike training-time leave-one-out methods (which require \(K\) rounds of retraining), AutoFold operates entirely at inference time with negligible overhead. The search resolution is a step size of 0.01 (100 candidate \(\alpha\) values), whereas manual Fold only searches 10 points, allowing AutoFold to occasionally yield even better results than manual Fold. Crucially, its setup time is several orders of magnitude lower than competing methods that require external OOD validation sets (such as KNN, VIM, RMDS, etc.).

Loss & Training

Fold is a purely post-hoc method requiring no training. The only hyperparameter to be determined is the normalization coefficient \(\alpha\). For standard Fold, \(\alpha\) is grid-searched over \(\{0.1, 0.2, \dots, 1.0\}\) using the AUROC on an auxiliary OOD validation set as the selection criterion. For AutoFold, \(\alpha\) is searched over \(\{0.01, 0.02, \dots, 1.00\}\) based on the self-supervised AUROC from Equation (7). All pre-trained models remain frozen, leaving ID classification performance unaffected.

Key Experimental Results

Main Results

The table below compares Fold and its variants against representative baselines on four CIFAR and ImageNet benchmarks (complete results are in Table 2):

Method CIFAR-10 AUROC / FPR95 CIFAR-100 AUROC / FPR95 ImageNet-200 AUROC / FPR95 ImageNet-1K AUROC / FPR95 Average AUROC / FPR95
MSP 89.83 / 37.21 78.60 / 57.40 87.41 / 43.19 81.55 / 57.14 84.35 / 48.73
EBO 90.00 / 48.24 80.15 / 56.26 87.51 / 45.01 84.03 / 50.46 85.42 / 49.99
ReAct 89.31 / 51.12 80.52 / 54.93 88.13 / 42.10 87.15 / 42.46 86.28 / 47.65
VIM 91.88 / 31.65 79.46 / 54.70 86.23 / 39.99 84.44 / 43.34 85.50 / 42.42
KNN 92.19 / 27.52 81.66 / 56.17 88.52 / 40.43 82.55 / 48.83 86.23 / 43.24
ASH 77.42 / 81.61 79.79 / 61.37 89.29 / 42.33 88.71 / 37.02 83.80 / 55.58
Fold 92.46 / 29.43 81.94 / 51.62 88.65 / 40.30 85.74 / 48.35 87.20 / 42.42
Fold-r 92.30 / 30.65 82.22 / 50.55 89.07 / 38.92 88.04 / 40.36 87.91 / 40.12
Fold-a 89.43 / 49.18 81.98 / 54.26 89.68 / 37.06 88.28 / 37.03 87.34 / 44.38

Fold-r (combined with ReAct) achieves the best average results, with 87.91% AUROC and 40.12% FPR95, improving upon the previous best baselines by 1.63% and 2.30%, respectively. Fold-a (combined with ASH) is particularly prominent on ImageNet. Crucially, the Fold family of methods maintains consistent and stable performance across both CIFAR and large-scale ImageNet scenarios, unlike some baselines that fluctuate drastically between different benchmarks.

Ablation Study

Ablation Configuration CIFAR-10 AUROC ImageNet-200 AUROC Description
Logit Hessian (Without feature projection) 92.52 86.35 Removes feature-space projection, pure logit curvature
Feature Hessian (Full Fold) 92.46 88.65 Full feature Hessian; shows a clear advantage on complex datasets
\(\alpha=0\) (Without normalization) 89.72 87.81 No normalization, almost no impact on CIFAR
\(\alpha=1\) (Complete normalization) 92.46 62.23 Complete normalization on ImageNet results in severe degradation
Partial normalization (Optimal \(\alpha\)) 92.46 88.65 Retains an appropriate amount of magnitude information, yielding the best result
Average gain of partial normalization on various baselines +4.00 AUROC (CIFAR-10) +2.27 AUROC (ImageNet-200) Partial normalization is universally effective as a plug-and-play component

Key Findings

  • The contribution of partial normalization is far greater on complex datasets than on simpler ones: on CIFAR-10, the gap between \(\alpha=0\), partial, and \(\alpha=1\) is minimal, whereas on ImageNet-200, \(\alpha=1\) causes the average AUROC to collapse from 87.64% to 79.02%, indicating that semantic information in feature magnitudes cannot be discarded in complex scenarios.
  • Cohen's \(d\) effect size analysis (Figure 5) indicates that partial normalization enhances the separability of ID-OOD score distributions from 1.605 to 1.689 while significantly compressing the OOD variance, whereas complete normalization collapses Cohen's \(d\) to 0.520, causing the two distributions to almost entirely overlap.
  • AutoFold performs slightly worse than manual tuning on CIFAR-100 (79.76% vs 81.94% AUROC) but matches or closely approaches the optimum on the other three benchmarks, completely without external OOD data. On ImageNet-1K, the setup time of AutoFold is far lower than KNN (which requires collecting all training features) and VIM (which requires estimating the PCA subspace), especially since these baselines do not account for the OOD search overhead in their reported setup times.
  • Generalization across architectures (Table 5): Fold consistently outperforms baselines across four different backbones (RegNet, DenseNet, WRN, ResNeXt), with Fold-a achieving the highest average AUROC of 87.08%.

Highlights & Insights

  • The derivation of the feature Hessian is exceptionally clean: By exploiting the linear property of the classifier, the complex terms in the second-order chain rule disappear, yielding a concise formulation of \((\nabla_{\mathbf{h}}g)(\nabla_{\mathbf{z}}^2 \mathcal{L})(\nabla_{\mathbf{h}}g)^\top\)—essentially measuring logit curvature within the discriminative subspace defined by the classifier's weight matrix. This "simplifying complex formulations using specific structures of linear layers" is a technique worth noting.
  • Partial normalization serves as a universal enhancement component: Table 3 shows that incorporating partial normalization into MSP, EBO, ReAct, and ASH consistently yields performance gains and outperforms complete normalization. This means it is not limited to Fold itself and can be applied plug-and-play to any post-hoc method that relies on feature representation. It provides an elegant answer to the long-standing question in the OOD field of whether feature magnitude should be retained: adaptively keep a portion based on dataset complexity.
  • Logit masking as pseudo-OOD generation: Neither training, external data, nor generative models are required; simply setting the ground-truth class logit to negative infinity produces high-quality pseudo-OOD signals. This simple yet clever idea essentially leverages the classifier's behavior when deprived of the "correct answer" to simulate encountering an unknown class.
  • Self-consistent theoretical analysis despite simplifications: Under a binary Gaussian mixture model setting, the authors prove that OOD samples indeed exhibit a larger expected feature Hessian trace, provided that \(c|\mu^\top \Sigma^{-1} R\mu| \leq \mu^\top \Sigma^{-1} \mu\). Intuitively, this condition implies that the distribution shift should not be so extreme as to reverse the relative direction of the classification boundaries.

Limitations & Future Work

  • Simplified assumptions in theoretical analysis: Theorem 6.1 is built under the setting of binary classification, Gaussian mixtures, and a linear classifier, and does not incorporate the impact of partial normalization. In practice, deep non-linear networks are applied to multi-class scenarios, posing a substantial gap between theory and experiments. However, the authors honestly note this, positioning the theory as "intuitive support" rather than a rigorous guarantee.
  • Unclear physical interpretation of alpha: Although experiments clearly demonstrate that the optimal \(\alpha\) varies across datasets, the explanation of "why CIFAR needs \(\sim 1.0\) and ImageNet needs \(\sim 0.2\)" remains at a qualitative level of dataset complexity, lacking a quantitative analysis of feature magnitude distributions to guide the selection of \(\alpha\).
  • Performance degradation of AutoFold on CIFAR-100: The self-supervised \(\alpha\) selected in this scenario is significantly inferior to manual tuning on OOD validation sets (an AUROC gap of ~2.2%), which indicates that the pseudo-OOD generated by logit masking cannot fully simulate the curvature characteristics of some real-world OOD data. A likely reason is that classes in CIFAR-100 share more fine-grained semantics, meaning that the residual logit patterns after masking one class still differ from real OOD.
  • Implicit dependency on network architectures: The derivation of the feature Hessian relies on the classifier being a linear layer—which holds for standard architectures like ResNet, but may require re-derivation for models employing non-linear classification heads (e.g., certain ViT variants).
  • Future research directions: (1) Replace the global \(\alpha\) with per-class statistics of feature magnitudes to achieve class-adaptive partial normalization; (2) ensemble the Fold score with other complementary signals (such as distance-based methods like VIM/KNN), since curvature signals and distance signals may capture different types of distribution shifts; (3) explore more fine-grained logit masking strategies (such as masking the top-\(k\) logits instead of only the ground-truth class) to generate more realistic pseudo-OOD signals.
  • vs. Parameter-space Laplace approximation methods (Ritter 2018, Kristiadi 2020): These methods estimate the Bayesian posterior using Laplace approximations of the parameter Hessian, offering strong theoretical backing at the cost of high computation even under KFAC approximations. Fold's contribution lies in pointing out that "you don't need parameter Hessians—logit Hessians with classifier projection are sufficient," reducing the computational complexity from \(O(\text{number of parameters}^2)\) to \(O(\text{feature dimension}^2)\). The latter is typically only 512–2048 in ResNets, far smaller than the total parameter count.
  • vs. ReAct / ASH (Sun 2021, ASH): These methods enhance ID-OOD separation by truncating or pruning abnormal activation values, focusing on the magnitude distribution of feature values. Fold approaches the problem from a curvature perspective, focusing on the flatness of the loss landscape. They are complementary: joint experiments on Fold-r and Fold-a directly demonstrate that curvature signals can be combined with activation shaping for cumulative gains. However, while ReAct/ASH require manual thresholding, Fold's hyperparameter \(\alpha\) can be automatically selected via AutoFold.
  • vs. VIM / KNN (Wang 2022, Sun 2022): VIM uses the deviation of feature residuals from the PCA principal space for detection, while KNN uses nearest neighbor distances in feature space. These methods essentially measure the "typicality of features under the training distribution," whereas Fold measures "model sensitivity to inputs"—a completely different source of signal. Experiments show that Fold enjoys a distinct efficiency advantage over them, especially on ImageNet, as it avoids maintaining a training feature database or estimating a PCA basis.
  • vs. Gradient-based methods (GradNorm): GradNorm uses the gradient norm as the OOD score, which, like Fold, is a "model sensitivity" signal but computes parameter gradients rather than the feature Hessian. The advantage of the feature Hessian is that it naturally factors out the influence of classifier weights (since the second derivative of the linear layer is zero), thereby focusing more directly on the feature manifold itself.

Rating

  • Novelty: ⭐⭐⭐⭐ First to systematically validate the empirical rule of ID-OOD curvature discrepancy; the combination of feature Hessian + partial normalization is elegant and effective, and self-supervised calibration via logit masking is a neat new approach. However, the basic components (energy score, feature Hessian derivation, normalization) have been built upon by prior works.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Covers 4 benchmarks (CIFAR-10/100, ImageNet-200/1K), compares against over 10 baselines, with ablations covering feature Hessian vs. logit Hessian, three normalization schemes, the full spectrum of \(\alpha\) sensitivity, cross-architecture generalization, inference time measurements, spectral analysis, and Cohen's \(d\) effect sizes—nearly every design choice is backed by a corresponding ablation.
  • Writing Quality: ⭐⭐⭐⭐ Clear structure, presenting a smooth linear narrative from observation to methodology, experiments, and theory. The derivation steps in the method section are comprehensive, though the Section 7 discussion feels slightly disjointed, and the connection between the theoretical section and the actual method could be tighter.
  • Value: ⭐⭐⭐⭐ Introduces a new signal dimension (curvature) for post-hoc OOD detection that is computationally efficient enough for practical use; partial normalization acts as a versatile plugin, and AutoFold addresses a key deployment pain point (requiring OOD data for tuning). Ideal as a foundation method for real-time OOD detection systems.