AracNet: Revealing Debiasing Signals across Layers with Shallow Monitors

Conference: ECCV 2026
Paper: Official paper page · Paper PDF
Code: https://github.com/Malga-Vision/AracNet
Area: AI Safety
Keywords: Visual Debiasing, Spurious Correlations, Shallow Monitors, Sample Reweighting, Group Robustness

TL;DR

AracNet trains shallow monitors on a frozen biased classifier, selects the layer whose ground-truth confidence distribution best exposes bias, and reweights training of a reinitialized classifier, reaching \(91.27\pm0.56\%\) worst-group accuracy on Waterbirds without relying on a narrowly timed early-stopping point for bias mining.

Background & Motivation

Image classifiers can use water backgrounds to recognize waterbirds or image corruption types to predict CIFAR-10 classes. When these attributes correlate strongly with training labels, empirical risk minimization can reward shortcuts that fail once the correlation changes at test time. Bias-aligned examples follow the dominant attribute–class relationship; bias-conflicting examples violate it. Explicit attribute annotations can help identify these groups, but many datasets provide only task labels. Accordingly, “unsupervised debiasing” here means learning without bias labels, not learning without image-class labels.

Methods such as LfF and JTT exploit the tendency to learn simple shortcuts before memorizing difficult exceptions. They use an auxiliary model's losses or mistakes to identify examples deserving additional training emphasis. This creates a timing problem: an undertrained model is uninformative, while an overtrained model may correctly classify both aligned and conflicting training examples. Selecting the right stopping point can require heuristics or annotated validation data; generating aligned examples instead introduces a separate computational pipeline. AracNet asks whether a memorizing final classifier necessarily means that every internal representation has also lost its useful bias signal.

The paper argues that internal layers can preserve a useful separation even when the final prediction head has memorized the training set. The appropriate depth can depend on the shortcut: backgrounds may be accessible early, whereas finer semantic attributes may require later features. A fixed first-layer or last-layer strategy is therefore not sufficient. Core idea: replace the search for when to stop with a search for where bias remains visible, then use the selected shallow monitor to weight the training of a fresh classifier.

Method

Overall Architecture

The input is a biased image dataset with target-class labels but without bias-attribute annotations for training; the output is a conventional image classifier. A backbone is first trained on the biased task and frozen, followed by Frozen-Feature Monitoring, Bimodality-Based Selection, and Reweighted Self-Mitigation. “Self-mitigation” means using signals from the backbone's own architecture to train a reinitialized target model, not updating the frozen biased parameters in place. Training retains the biased backbone as a signal source alongside a separate trainable target; deployment requires only the target classifier.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Images and class labels"] --> B["Train biased backbone"]
    B --> C["Frozen-Feature Monitoring"]
    C --> D["Bimodality-Based Selection"]
    D --> E["Reweighted Self-Mitigation"]
    A --> F["Reinitialize target model"]
    F --> E
    E --> G["Debiased classification"]

Key Designs

1. Frozen-Feature Monitoring: expose bias without rewriting the representation

The initial backbone may already achieve very high training accuracy, including on conflicting examples. Rather than trusting its final output, AracNet attaches a monitor to each network block: pooling, a single dense layer, and a softmax classifier. Each monitor predicts the original target class using cross-entropy; it is not supervised to predict a background attribute or a bias-conflict indicator. The backbone remains frozen during monitor training, so the probes read an existing representation instead of changing the object being measured. A linear monitor also has less opportunity than a new deep auxiliary model to introduce additional computation that memorizes every exception.

The relevant confidence is the probability assigned to the ground-truth class, not the maximum predicted probability. A shortcut-sensitive monitor may be very confident in a wrong class on a conflicting example, while still assigning that example low ground-truth confidence. The desired monitor is consequently not necessarily the most accurate one: it is the one that provides useful separation between training subpopulations. Figure 2 illustrates why depth matters: early Waterbirds blocks expose background-related separation, whereas a later block is more informative for the finer attribute bias in BFFHQ. This is empirical evidence about the evaluated models, not a universal law assigning each bias type to a fixed depth.

2. Bimodality-Based Selection: choose an informative confidence distribution

A monitor that confidently classifies nearly every training example has little remaining ability to distinguish examples for debiasing. A uniformly uncertain monitor may simply be undertrained. AracNet instead looks for a high-confidence majority accompanied by a low-confidence tail, rather than selecting the layer with the largest classification error. For each monitor, it computes confidence quantiles over the training set and multiplies a confidence span by the median. Writing \(Q_p(c_\ell)\) for the \(p\)th percentile, the score described in Section 4.2 is:

\[ B(\ell)=\bigl[Q_{99}(c_\ell)-Q_1(c_\ell)\bigr]Q_{50}(c_\ell). \]

The percentile span favors separation between low- and high-confidence examples, while the median factor favors distributions whose main population is already recognized confidently. The highest-scoring layer is retained, and the other monitors are discarded before the subsequent mitigation phase. Selection uses training images and class labels, without bias-group labels or an unbiased validation set to choose the bias-mining stopping point. However, the “bimodality score” is a heuristic summary, not a statistical test proving that a distribution has two modes. Low confidence does not establish that an example is bias-conflicting, and exceptionally rare conflicts may also be poorly represented by the chosen tail percentiles.

3. Reweighted Self-Mitigation: emphasize examples on which the monitor fails

AracNet preserves the frozen biased backbone and selected monitor while reinitializing the target classifier. The stated rationale is to avoid continuing directly from parameters already strongly dependent on shortcuts. Training multiplies each target cross-entropy loss by two weighting factors, increasing the influence of examples that the monitor struggles to classify. The first factor is a power of the monitor's surprisal, explicitly given in Algorithm 1:

\[ w_{\mathrm{surprisal}}=(-\log c_{\ell^*}(x))^\lambda. \]

Here, \(c_{\ell^*}(x)\) is the selected monitor's ground-truth class probability, and \(\lambda\) controls the contrast between high- and low-confidence examples. The second factor is inspired by LfF and uses monitor and target-model losses as a relative difficulty signal. The prose describes this factor as close to zero for aligned examples and tending toward one for conflicting examples. Together, the factors multiply the target cross-entropy rather than introducing a supervised bias-attribute prediction objective. Equations (5) and (6) are corrupted in the local text extraction, so the second factor's exact algebra and numerical stabilization cannot be verified here; this note does not fill them in from a familiar LfF formula. The supported mechanism is the product of two weights and the target loss, together with the exact surprisal definition preserved in Algorithm 1.

Mitigation alternates monitor training on frozen source features with target training while the monitor is held fixed. Updating the monitor does not unfreeze the source backbone, and the two classifiers do not share trainable parameters. Section 3.4 and Algorithm 1 differ slightly in which update they present first within an epoch, although both clearly specify alternating updates and a fixed monitor during target updates. The available text establishes these training responsibilities but should not be treated as definitive evidence of the implementation's precise loop ordering.

A Worked Example

Consider a waterbird photographed against a land background, contrary to the dominant training relationship. The biased backbone's final layer may have memorized this training image, making its final prediction an unhelpful indicator of the shortcut. Frozen-Feature Monitoring asks shallow classifiers at different blocks to predict the class independently. A monitor that strongly reflects the background can still favor “landbird,” assigning low probability to the true waterbird label. Bimodality-Based Selection chooses a single signal source from the full training distribution, not a different layer for this individual image. When the target is updated, the image receives a larger surprisal weight, encouraging learning that cannot succeed solely by matching the usual background. An aligned image typically receives a smaller weight, and test images later pass directly through the debiased target without monitor selection. This example describes the intended effect; it does not imply that every heavily weighted difficult example fails for the same reason.

Loss & Training

Section 4.2 uses ResNet50 for Waterbirds and UrbanCars and ResNet18 for BAR and BFFHQ, with ImageNet initialization for these biased-backbone configurations. Corrupted CIFAR-10 uses a randomly initialized ResNet20 at \(32\times32\) resolution; other datasets are resized to \(224\times224\). Monitor warmup lasts 30 epochs except on Corrupted CIFAR-10, where it lasts 50; this is distinct from the biased-backbone pretraining duration varied in Table 6c. The backbone and monitors use SGD, while target debiasing uses AdamW; the target learning rate is \(5\times10^{-5}\) on Waterbirds, BFFHQ, and BAR. Monitor learning rates are 0.005 on Waterbirds and UrbanCars and 0.05 elsewhere; batch sizes are 64 except for Corrupted CIFAR-10, which uses 256. Section 3.4 explicitly describes random reinitialization of the target, so the earlier ImageNet initialization should not be read as retaining the biased backbone for direct fine-tuning. Warmup duration, learning rates, and \(\lambda\) remain hyperparameters: avoiding an early-stopping criterion for bias mining does not make the entire method tuning-free.

Key Experimental Results

Main Results

The following selection from Tables 1, 2, 3, and 5 highlights both benefits and costs rather than reproducing the full benchmark tables. Accuracies are percentages; accuracy differences are percentage points. AracNet reports means and standard deviations over three independent runs; baseline uncertainty is retained only where the original table provides it. Waterbirds uses test worst-group accuracy (WGA), the minimum accuracy across class–background groups; BFFHQ evaluates only bias-conflicting test examples. Corrupted CIFAR-10 reports average accuracy on an unbiased test set with a 90:10 conflicting-to-aligned ratio. UrbanCars gaps are I.D. accuracy minus the accuracy of the corresponding conflicting subgroup: negative values mean that subgroup performs better, so closeness to zero is more informative than seeking increasingly negative values.

Dataset / setting	Metric	AracNet	Selected comparator	Source and interpretation
Waterbirds	WGA ↑	91.27 ± 0.56	DDB: 90.81 ± 0.49	Table 2; +0.46 points
BFFHQ, correlation 0.995	Conflicting-example accuracy ↑	75.93 ± 0.76	DeNetDM: 75.70	Table 3; +0.23 points
Corrupted CIFAR-10, correlation 0.995	Average accuracy ↑	33.23 ± 1.93	DeNetDM: 38.93 ± 1.16	Table 1; −5.70 points
Corrupted CIFAR-10, correlation 0.950	Average accuracy ↑	56.80 ± 0.78	DeNetDM: 56.30 ± 0.42	Table 1; +0.50 points
UrbanCars	I.D. accuracy ↑	85.95 ± 1.01	ERM: 97.30	Table 5; lower in-distribution accuracy
UrbanCars	Joint-conflict gap, closer to 0 is better	0.35 ± 1.40	ERM: 69.20	Table 5; substantially reduced subgroup disparity

These are not fully resource-controlled, same-architecture reruns of every comparator: methods can use generative models, external models, or different training procedures. The Waterbirds and BFFHQ leads are small, and differences between means alone do not establish statistical superiority over close competitors. The stronger overall case is useful debiasing relative to ERM, applicability across datasets, and competitiveness without a generative pipeline—not universal leadership.

Ablation Study

All entries below come from Table 6 and measure Waterbirds WGA in percent, higher being better. Differences are calculated from the reported means relative to the full configuration. Changing backbone pretraining duration and changing the weighting factors are separate experiments, not a combined configuration.

Configuration	WGA ↑	Difference from full configuration	Evidence
Full configuration, surprisal exponent 2	91.27 ± 0.56	0.00	Tables 6a and 6b
Without surprisal weight	84.94 ± 1.41	−6.33	Table 6a
Without monitor relative-loss weight	88.78 ± 1.36	−2.49	Table 6a
Surprisal exponent 5	84.47 ± 2.58	−6.80	Table 6b
Biased-backbone pretraining for 200 epochs	90.85 ± 0.74	−0.42	Table 6c; 50-epoch reference is 91.27 ± 0.56

Key Findings

• Both weights help, with the larger drop following removal of the surprisal factor. This supports emphasizing difficult examples but does not isolate every source of improvement.
• Extending backbone pretraining from 50 to 200 epochs reduces WGA by only 0.42 points on Waterbirds, supporting resistance to memorization in this setting rather than proving it for all datasets.
• The poorer result at \(\lambda=5\) than at \(\lambda=2\) shows that aggressively concentrating weight on low-confidence examples can be harmful.
• UrbanCars exposes a real trade-off: reducing reliance on both shortcuts need not improve accuracy on the dominant original distribution.

Highlights & Insights

• A monitor is a bias sensor, not merely an auxiliary classifier whose accuracy should be maximized. Its value lies in producing differentiated training signals.
• Moving from training time to representation depth decouples bias mining from the final head's memorization state. Frozen source features and shallow probes jointly matter; adding a linear layer alone does not capture the contribution.
• Sample-level weights are independent of the target architecture. Section 4.5 also tests transferring monitor signals to other backbones without requiring layer-to-layer correspondence.

Limitations & Future Work

• The authors propose multimodal biases and dynamic monitors as future directions; current evidence primarily concerns image-classification benchmarks, not general fairness guarantees.
• Reader assessment: low-confidence tails can contain label noise, rare classes, or ordinary difficult examples. Removing bias annotations also removes a direct semantic check on why an example is difficult.
• Reader assessment: the percentile score assumes a useful majority-aligned confidence pattern, and the paper does not provide a complete quantitative fixed-layer versus automatic-selection ablation to isolate selection itself.
• Reader assessment: small probe overhead is not the same as free training. The full procedure still includes biased-backbone training, monitor warmup, and target-model training.
• Source caveats: corrupted Equations (5)–(6) prevent exact reconstruction of the second weight; BAR Table 4 reports 76.22 ± 1.47 at correlation 0.990, while Section 4.3 states 76.65 ± 1.47, so this note avoids drawing a numerical conclusion from that result.

Related Work & Insights

• vs JTT: JTT identifies examples for upsampling from an early model's mistakes. AracNet first trains a backbone and then selects an internal layer that preserves useful bias separation; the main distinction is where the signal is extracted, not simply whether training happens twice.
• vs LfF: Both convert a biased model's failures into training emphasis. AracNet adds frozen internal probes, automatic layer selection, and surprisal weighting; the relative-loss weighting idea itself is not entirely new.
• vs DDB: DDB generates aligned examples to limit the auxiliary model's exposure to conflicting examples. AracNet searches existing representations instead, avoiding image generation while introducing assumptions about useful layerwise separation.
• vs SIFER and OccamNets: SIFER modifies representation learning through feature sieving, while OccamNets uses early exits to favor simpler hypotheses. AracNet's monitors primarily guide sample weighting rather than serving as deployment-time exits.

Rating

• Novelty: 4/5. Internal probes, confidence-based layer selection, and self-mitigation form a coherent combination built on established reweighting ideas.
• Experimental Thoroughness: 4/5. Multiple bias types, architectures, and meaningful ablations are covered, but detailed ablations concentrate on Waterbirds and some gains are small.
• Writing Quality: 3/5. The conceptual flow is clear, but extraction-damaged formulas, update-order differences, and the BAR numerical inconsistency hinder exact reproduction.
• Value: 4/5. A practical direction for visual shortcut robustness without a generative model, especially relevant to internal representations and training-group imbalance.