Skip to content

Ranked Activation Shift for Post-hoc Out-of-Distribution Detection

Conference: ECCV 2026
Paper: ECCV 2026
Code: https://github.com/gigug/RAS
Area: AI Safety
Keywords: Out-of-Distribution Detection, Post-hoc Detection, Activation Editing, Histogram Matching, Hyperparameter-free

TL;DR

Addressing the failure modes of existing scaling- and pruning-based post-hoc activation editing methods on non-rectified architectures (ConvNeXt, ViT) and their dependence on hyperparameter tuning, RAS proposes a hyperparameter-free plug-and-play method that replaces sorted penultimate-layer activations with a fixed in-distribution reference profile, aligning global feature histograms and compressing score variance while preserving in-distribution accuracy.

Background & Motivation

Deep neural networks deployed in open-world scenarios inevitably encounter inputs that deviate from their training distributions. Reliable out-of-distribution (OoD) detection serves as an indispensable prerequisite for the safety and reliability of artificial intelligence systems in safety-critical domains such as autonomous driving, medical diagnostics, and financial risk assessment. In recent years, post-hoc detection methods have gained substantial traction because they operate without retraining the original classifiers, impose negligible computational overhead, and seamlessly integrate into existing inference pipelines. Among these, activation editing techniquesβ€”most notably ReAct, ASH, and SCALEβ€”have achieved state-of-the-art results by suppressing extreme outliers, sparsifying representations, or dynamically scaling penultimate-layer activations.

However, prevailing activation editing methods suffer from two critical limitations. First, their theoretical and empirical premises are fragile: methods such as SCALE hinge on the assumption that the mean-to-standard-deviation ratio of in-distribution (ID) activations strictly exceeds that of OoD inputs. An empirical audit across primary OpenOOD benchmark setups reveals that this condition is violated in three of four primary configurations, rendering the relative efficacy of pruning versus scaling highly vulnerable to arbitrary threshold hyperparameter tuning. Second, modern architectures induce negative-activation breakdown: contemporary vision backbones like ConvNeXt and Vision Transformers (ViTs) utilize GELU activations or lack rectification at the penultimate layer. The presence of negative feature values disrupts the non-negativity and monotonic ratio properties relied upon by scaling formulas, causing methods like SCALE and ASH-S to experience catastrophic performance collapse.

The key to overcoming this dilemma lies in abandoning local threshold clipping and sign assumptions, moving instead toward global statistical profile matching. Core Idea: eliminate threshold-dependent pruning and scaling by replacing sorted activation magnitudes with a fixed in-distribution reference profile derived from unlabeled training data, thereby achieving robust OoD separation through bidirectional activation shifting and variance compression without hyperparameter tuning.

Method

Overall Architecture

RAS (Ranked Activation Shift) consists of a one-time offline reference construction phase and an inference-time feature substitution stage. In the setup phase, an in-distribution reference profile \(\boldsymbol{\mu}\) is computed by averaging sorted activation intensities across a small subset of unlabeled ID samples. At inference time, for each test input, penultimate-layer activations are extracted and sorted in ascending order; their values are then directly substituted with the corresponding values from the reference profile \(\boldsymbol{\mu}\) before being mapped back to their original channel coordinates and forwarded to the classifier head for scoring (e.g., via EBO, GEN, or ViM).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input test sample x<br/>Extract penultimate activation a"] --> B["Offline Reference Profile Construction<br/>Compute mean ranked profile ΞΌ on ID data"]
    B --> C["Rank-Preserving Activation Substitution<br/>Map sorted activations in a to ΞΌ"]
    C --> D["Bidirectional Activation Shift & Variance Compression<br/>Simultaneously inhibit & excite while reducing variance"]
    D --> E["Classification Head Forwarding & Scoring<br/>Evaluate EBO / GEN / ViM score"]

Key Designs

1. Offline Reference Profile Construction: capturing global intensity profiles via ranked means

Existing activation shaping approaches rely on auxiliary outlier validation data to tune crucial hyperparameters (such as pruning percentiles \(p\) or truncation thresholds \(c\)), intertwining detection performance with specific validation sets. To establish a universal and hyperparameter-free standard, RAS extracts penultimate-layer activation vectors \(\mathbf{a}_i \in \mathbb{R}^d\) across \(N\) unlabeled ID samples (from either the training split or a held-out validation set). A sorting operator \(\mathrm{r}(\cdot)\) rearranges each vector in ascending order, after which an element-wise mean profile is calculated: $\(\boldsymbol{\mu} = \frac{1}{N} \sum_{i=1}^N \mathrm{r}(\mathbf{a}_i)\)$ This vector captures the canonical monotonic intensity progression of the penultimate representation under in-distribution inputs. It requires neither ground-truth labels nor non-negative constraints. Furthermore, empirical analysis demonstrates that the reference profile converges stably with as few as 10 to 100 samples, keeping the calibration overhead negligible.

2. Rank-Preserving Activation Substitution: maintaining semantic orientation while preventing negative-value breakdown

To resolve the breakdown of scaling ratios (\(r = Q/Q_p\)) caused by negative values in unrectified architectures like ConvNeXt and ViT, RAS operates directly on the geometric distribution of sorted magnitudes. During inference, for an incoming feature vector \(\mathbf{a} \in \mathbb{R}^d\) (or the class token representation for ViT), RAS computes the permutation index \(\pi\) that sorts the activations in ascending order such that \(a_{\pi(1)} \le a_{\pi(2)} \le \dots \le a_{\pi(d)}\). The modified activation vector \(\bar{\mathbf{a}}\) is then constructed by assigning the reference values back to the permuted indices: $\(\bar{a}_{\pi(j)} = \mu_j, \quad \forall j \in \{1, \dots, d\}\)$ This operation functions as an exact histogram matching step: it strictly preserves the directional and spatial ranking of the original activations (which channel responds more strongly than another), but replaces their absolute scalar magnitudes with the expected ID statistical profile. Consequently, numerical instability and negative-denominator collapse are completely avoided.

3. Bidirectional Activation Shift & Variance Compression: dual inhibition-excitation gains with compressed score variance

Prior literature, spearheaded by ReAct, operated under the premise that OoD samples primarily manifest as abnormally high activations that require downward suppression (inhibition). RAS reveals the equal importance of bidirectional calibration: pushing abnormally suppressed activations upward toward \(\boldsymbol{\mu}\) (excitation) independently aids OoD discrimination just as effectively as downward inhibition. By replacing the entire vector with \(\boldsymbol{\mu}\), RAS simultaneously curtails peaked positive noise and elevates depressed background features. This bidirectional shift compresses the variance \(\sigma\) of the downstream energy scores (EBO) across both ID and OoD distributions, substantially boosting the distribution separation and signal-to-noise ratio.

Key Experimental Results

Main Results

On the OpenOOD benchmark across diverse ID datasets (CIFAR-10/100, ImageNet-200, ImageNet) and model architectures (ResNet18/50, EfficientNet-B0, ConvNeXt-T/B, ViT-B/16), RAS was evaluated with EBO as the underlying scoring function against leading post-hoc enhancement techniques (competing methods used fully optimized hyperparameters, whereas RAS is entirely parameter-free):

Backbone Dataset / Architecture EBO (Baseline) ReAct ASH-S SCALE RAS (Ours)
ResNet-18 CIFAR-10 (AUROC / FPR95) 90.00 / 48.24 89.31 / 51.12 82.90 / 77.47 85.18 / 71.77 90.24 / 40.16
ResNet-18 CIFAR-100 (AUROC / FPR95) 80.15 / 56.26 80.52 / 54.93 81.52 / 54.29 81.29 / 54.60 82.09 / 52.31
ResNet-18 ImageNet-200 (AUROC / FPR95) 87.51 / 45.01 88.13 / 42.10 90.51 / 38.15 90.29 / 38.89 89.51 / 36.70
ResNet-50 ImageNet (AUROC / FPR95) 84.03 / 50.46 87.15 / 42.46 89.68 / 35.05 90.42 / 34.02 86.55 / 40.92
EffNet-B0 ImageNet (AUROC / FPR95) 76.76 / 72.29 84.27 / 49.01 56.18 / 81.21 59.05 / 79.70 83.78 / 46.87
ConvNeXt-T ImageNet (AUROC / FPR95) 69.36 / 83.03 77.87 / 69.45 32.59 / 97.64 60.10 / 84.32 83.31 / 46.28
ConvNeXt-B ImageNet (AUROC / FPR95) 59.36 / 93.29 80.85 / 64.06 26.86 / 99.67 65.22 / 76.53 84.86 / 45.29
ViT-B/16 ImageNet (AUROC / FPR95) 72.35 / 88.48 79.12 / 66.15 22.68 / 98.71 68.64 / 89.52 81.48 / 55.19

Ablation Study

The paper dissects RAS into RAS-inhibit (shifting only activations above \(\boldsymbol{\mu}\) downward) and RAS-excite (shifting only activations below \(\boldsymbol{\mu}\) upward), reporting their energy score statistics and AUROC gains over baseline EBO:

Benchmark Split Metric EBO Baseline EBO + RAS-excite EBO + RAS-inhibit EBO + RAS (Full Model)
CIFAR-100 (ResNet-18) Energy Score Distribution \(\mu \pm \sigma\) \(7.60 \pm 1.90\) \(8.66 \pm 1.66\) \(7.41 \pm 1.54\) \(8.50 \pm 1.37\)
CIFAR-100 (ResNet-18) Average \(\Delta \text{AUROC}\) Gain β€” \(+0.88\) \(+1.16\) \(+1.53\)
ImageNet-200 (ResNet-18) Energy Score Distribution \(\mu \pm \sigma\) \(8.51 \pm 2.23\) \(9.97 \pm 1.97\) \(8.29 \pm 1.75\) \(9.73 \pm 1.58\)
ImageNet-200 (ResNet-18) Average \(\Delta \text{AUROC}\) Gain β€” \(+1.86\) \(+1.01\) \(+2.45\)
ImageNet (ResNet-50) Energy Score Distribution \(\mu \pm \sigma\) \(12.31 \pm 2.48\) \(13.00 \pm 2.20\) \(11.85 \pm 1.96\) \(12.55 \pm 1.87\)
ImageNet (ResNet-50) Average \(\Delta \text{AUROC}\) Gain β€” \(+2.17\) \(+2.01\) \(+2.86\)

Key Findings

  • Robustness on unrectified modern backbones: On ConvNeXt-B and ViT-B/16, ASH-S collapses to 26.86% and 22.68% AUROC due to negative activation values breaking the scaling ratio. In contrast, RAS achieves 84.86% and 81.48%, improving over baseline EBO by 10 to 25 percentage points without hyperparameter tuning.
  • Bidirectional synergy via variance compression: Both RAS-inhibit and RAS-excite independently yield performance gains across benchmarks, and their combination delivers the greatest separation. Furthermore, RAS systematically compresses the standard deviation \(\sigma\) of energy scores across datasets, indicating that variance reduction is the primary mechanism driving detection gains.
  • Strict preservation of in-distribution accuracy: Across all eight evaluated model-dataset pairs, the perturbation to top-1 classification accuracy is within \(\pm 0.1\%\) (e.g., CIFAR-10 accuracy moves from 95.06% to 95.03%, and ViT-B/16 stays exactly at 81.14%), ensuring cost-free post-hoc safety enhancement.
  • Data sample efficiency: The reference vector \(\boldsymbol{\mu}\) can be computed with as few as 10 to 100 samples and works equally well on an unseen validation set, requiring no retention of the full training set.

Highlights & Insights

  • Global histogram matching over local pruning thresholds: Instead of tuning arbitrary percentile cuts, RAS reframes activation shaping as a rank-based histogram matching operation, preserving semantic feature direction while eliminating all threshold search.
  • Disproving the one-sided suppression myth: Demonstrates that below-mean activation shifts contribute just as much discriminative information as suppressing abnormally high activations.
  • Versatile plug-and-play compatibility: Universally enhances logit-based scoring functions (EBO, GEN, ViM) and outperforms standard \(\ell_2\)-normalization variants in feature distance approaches like Mahalanobis distance (MDS++).

Limitations & Future Work

  • Slight gap behind tuned SCALE on ResNet50: On ImageNet with ResNet50, RAS reaches 86.55% AUROC, lagging behind peak-tuned SCALE (90.42%) where strictly rectified activations favor customized scaling ratios.
  • Layer selection constraints: While applicable to any layer, applying RAS to non-penultimate intermediate layers generally yields minor or inconsistent gains across diverse architectures.
  • Future directions: Exploring class-conditional reference vectors or channel-importance reweighting, and extending the methodology to multimodal vision-language models (VLMs).
  • vs ReAct: ReAct clips abnormally high activations using a scalar threshold that requires validation set tuning, addressing only the upper tail; RAS requires zero hyperparameter tuning and applies bidirectional shifts.
  • vs ASH / SCALE: ASH and SCALE rely on non-negative activation assumptions and scaling ratios that fail in modern unrectified architectures; RAS is sign-agnostic and maintains stable performance across ConvNeXts and Transformers.
  • vs MDS++ / Feature Normalization: MDS++ applies radial \(\ell_2\)-normalization to align vector lengths; RAS aligns both vector length and the entire rank-ordered activation distribution, yielding superior FPR95 reductions.

Rating

  • Novelty: β­β­β­β­β˜† Reframes activation editing into a hyperparameter-free rank-based histogram alignment method, providing clear theoretical and empirical insights.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Evaluated across 8 distinct architectures, 4 standardized benchmarks, multiple scoring frameworks, and granular ablation of directional shifts.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clearly identifies failure modes of existing scaling literature, with structured experiments and concise theoretical analysis.
  • Value: ⭐⭐⭐⭐⭐ Plug-and-play, zero-hyperparameter, architecture-agnostic, and zero accuracy cost make it highly valuable for practical deployment.