Skip to content

On the Faithfulness of Post-Hoc Concept Bottleneck Models

Conference: ECCV 2026
arXiv: 2606.30498
Project Page: https://posthoc-cbm-faithfulness.github.io/
Area: Interpretability / Concept Bottleneck Models
Keywords: Post-Hoc Concept Bottleneck Models, Concept Faithfulness, Covariate Shift, Surrogate Label Noise, Explainable AI

TL;DR

This paper systematically analyzes the faithfulness of concept projection in post-hoc CBMs. It demonstrates that classifier accuracy cannot serve as a proxy metric for the quality of the bottleneck layer (since random projections can also achieve competitive performance), identifies two fundamental causes of unfaithfulness—covariate shift in auxiliary datasets and systematic errors in VLM surrogate labels—and proposes corresponding diagnostic metrics (\(\mathcal{H}\Delta\mathcal{H}\)-divergence and error-activation correlation).

Background & Motivation

Background: Post-hoc CBMs freeze a pretrained backbone and insert a concept bottleneck layer in the feature space to achieve interpretable classification. To bypass the lack of concept annotations in the target domain, two mainstream approaches are: (1) training a concept projection \(\pi_\theta\) on an auxiliary concept dataset (e.g., Broden); (2) using VLMs (CLIP/DINO/Grounding DINO) to generate surrogate concept labels.

Limitations of Prior Work: It is widely common in the field to evaluate the quality of post-hoc CBMs based on downstream classification accuracy. However, existing works (Midavaine et al., Makonnen et al.) have found that random concept projections can also achieve high accuracy—implying that accuracy does not reflect whether the concept bottleneck has truly learned meaningful semantic concepts.

Key Challenge: If the classifier \(h\) is sufficiently complex (e.g., an MLP), it can learn to reconstruct the original features from the random projection for classification—essentially a form of information leakage, which occurs when the upstream \(\pi_\theta\) is unfaithful. This means the assumption "high accuracy = good concepts" might be entirely incorrect, yet there is a lack of systematic frameworks for analyzing faithfulness.

Goal: (1) To prove the theoretical foundation of why random projections can reach competitive performance (manifold JL lemma + constructive linear reconstruction); (2) to identify and formalize two sources of unfaithfulness in post-hoc CBMs; (3) to propose diagnostic metrics for faithfulness that do not rely on ground-truth concept annotations.

Key Insight: To formalize "faithfulness" as whether \(\pi_\theta\) can minimize the expected risk \(\mathbb{E}_{P_{\text{task}}}[\mathcal{L}(\pi_\theta(f(x)), c)]\) over the true task distribution, and then analyze how two types of distribution mismatch violate this definition.

Core Idea: Covariate shift in the auxiliary dataset can be quantified through an \(\mathcal{H}\Delta\mathcal{H}\)-divergence upper bound; the unfaithfulness of VLM surrogate labels does not stem from the magnitude of the noise, but rather from the systematic correlation between the noise and the backbone features.

Method

Overall Architecture

The analysis framework of this paper consists of three steps: first, proving that downstream accuracy does not reflect concept faithfulness through random projection experiments and a constructive linear reconstruction (based on the Moore-Penrose pseudoinverse); second, analyzing the two forms of distribution mismatch during the training of \(\pi_\theta\); and finally, proposing the corresponding diagnostic metrics.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input x"] --> B["Frozen Backbone f"]
    B --> C["Activation a"]
    C --> D["Concept Projection \pi_\theta"]
    D --> E["Concept Scores c"]
    E --> F["Classifier h"]
    F --> G["Prediction"]
    D -.-> H["Two Sources of Unfaithfulness"]
    H --> I["① Auxiliary Dataset<br/>Covariate Shift"]
    H --> J["② VLM Surrogate Labels<br/>Systematic Error"]
    I --> K["Metric: \mathcal{H}\Delta\mathcal{H}-divergence<br/>(domain discriminator)"]
    J --> L["Metric: |\rho(\Delta_k, \alpha_j)|<br/>(error-activation correlation)"]

Key Designs

1. Constructive Proof of Random Projections Achieving Competitive Performance

Under three assumptions (activations lie in an \(m\)-dimensional linear subspace \(\mathcal{S}\), the backbone classifier \(g\) is linear, and \(\pi_\theta\) is a random Gaussian projection), the paper constructs a classifier \(h(c) = g(U(PU)^\dagger \sigma^{-1}(c))\). Using the Moore-Penrose pseudoinverse \((PU)^\dagger\), the classifier precisely reconstructs the original activation \(a\) from the random projection \(c\), which is then fed into the original classifier \(g\). Under more general manifold assumptions, the manifold JL lemma guarantees that \(K\) does not need to be close to \(d\) to maintain the geometric structure. This implies that as long as \(h\) has sufficient capacity, it can "see through" the random projection—meaning accuracy does not reflect concept quality.

2. Covariate Shift in Auxiliary Datasets: \(\mathcal{H}\Delta\mathcal{H}\)-Divergence Bound

When \(\pi_\theta\) is trained on an auxiliary distribution \(P_{\text{aux}}\) but applied to the target distribution \(P_{\text{task}}\), an upper bound can be derived from domain adaptation theory (Ben-David et al.):

\[J_{\text{task}}(\theta; L_1) \leq J_{\text{aux}}(\theta; L_1) + \frac{1}{2}\hat{d}_{\mathcal{H}\Delta\mathcal{H}}(\mathbb{X}_{\text{task}}, \mathbb{X}_{\text{aux}}) + \Omega + \lambda_{\text{ideal}}\]

Under the covariate shift assumption (where concept semantics remain invariant across domains, and only the input marginal distribution shifts), \(\lambda_{\text{ideal}}\) can be neglected. \(\hat{d}_{\mathcal{H}\Delta\mathcal{H}}\) can be approximately estimated from the classification error of a domain discriminator—without requiring ground-truth concept annotations in the target domain.

3. Systematic Errors in VLM Surrogate Labels: Error-Activation Orthogonality Condition

By modeling \(\pi_\theta\) as a multivariate GLM, the true target gradient at the optimal surrogate label parameters \(\tilde{\theta}\) is derived to be proportional to:

\[\nabla J^*_{\text{task}}(\tilde{\theta}; \mathcal{L}) \propto \mathbb{E}_{P_{\text{task}}}[(\tilde{c}(x) - c^*(x)) \otimes f(x)]\]

This implies that even if there is error in the surrogate labels, as long as the error \(\delta(x) = \tilde{c}(x) - c^*(x)\) is orthogonal in expectation to the backbone activations \(f(x)\) (i.e., random noise), the gradient remains zero—meaning the model is still faithful. However, systematic errors (e.g., when the VLM always predicts the "boat" concept whenever a "water" texture is detected) produce non-orthogonal error components, leading to unfaithfulness. The proposed metric is the absolute Pearson correlation coefficient \(|\rho_{k,j}|\) between the surrogate error \(\mathbf{\Delta}_k\) and activation \(\boldsymbol{\alpha}_j\).

Key Experimental Results

Faithfulness vs. Performance Comparison on the Elements Dataset

Method Training Domain sim∢↑ Acc Paux↑ Acc Ptask↑ Acc h↑ Acc h*↑
PCBM Ptask 0.96 1.00 1.00 1.00 1.00
PCBM Pnear∀ 0.97 1.00 0.70 1.00 0.99
PCBM Pnear∃ 0.57 0.82 0.89 1.00 0.85
PCBM POOD 0.47 0.98 0.77 1.00 0.21
LFCBM 25% Random Noise 0.02 0.75 0.98 1.00 0.98
LFCBM CLIP 0.05 0.79 0.56 0.99 0.54
VLG-CBM Grounding DINO 0.02 0.74 0.85 1.00 0.04

Key Findings: - \(h\) (the learnable classifier) is near-perfect across almost all methods—accuracy absolutely does not reflect concept faithfulness. - \(h^*\) (the oracle rule mapping ground-truth concepts \(\to\) classes) reveals the true gap: POOD drops to 21%, and VLG-CBM is only slightly better than random (\(1/36 \approx 0.028\)). - 25% random label noise barely affects faithfulness (\(h^*=0.98\))—validating the theoretical prediction that "random noise is orthogonal, whereas systematic error is fatal."

\(\hat{d}_{\mathcal{H}\Delta\mathcal{H}}\) Diagnostics for Covariate Shift

The correlation coefficient between estimated \(\hat{d}_{\mathcal{H}\Delta\mathcal{H}}\) and the actual cross-domain generalization error is \(>0.9\). OOD domains yield the largest divergence and highest generalization error, while near-domain (Pnear) divergence is significantly lower. This proves that \(\hat{d}_{\mathcal{H}\Delta\mathcal{H}}\) is an effective proxy metric for faithfulness that does not require target domain ground-truth concept annotations.

Systematic Analysis of Surrogate Label Errors

The error-activation correlation for random noise is close to zero (\(|\rho| \approx 0\)), whereas the surrogate labels from CLIP/DINO/Grounding DINO yield significantly non-zero correlations. This directly links systematic error with an unfaithful \(\pi_\theta\).

Highlights & Insights

  • The constructive proof of random projection + pseudoinverse reconstruction is extremely strong: It proves that "accuracy \(\neq\) faithfulness" using a concrete, operational construction (without relying on asymptotic analysis)—which is much more convincing than merely showing empirical results.
  • The error orthogonality condition is an elegant theoretical insight: It reveals that the "structure" of surrogate label noise is more important than its "magnitude"—25% random noise causes less harm than 15% systematic noise. This provides direct guidance for choosing VLMs for surrogate annotation.
  • \(\hat{d}_{\mathcal{H}\Delta\mathcal{H}}\) is a practitioner-friendly metric: It can be calculated without target domain ground-truth concept annotations—merely requiring the training of a small domain discriminator. This resolves the chicken-and-egg problem of "how to evaluate faithfulness without ground-truth concepts."
  • The oracle classifier \(h^*\) is a necessary component for evaluating post-hoc CBMs: This paper clearly demonstrates the massive gap between \(h\) and \(h^*\), suggesting that future post-hoc CBM works should report both metrics.

Limitations & Future Work

  • The assumptions of linear subspaces and linear backbones only approximately hold in real deep networks; reconstruction conclusions in non-linear settings are stronger but much harder to prove.
  • The estimation of \(\hat{d}_{\mathcal{H}\Delta\mathcal{H}}\) depends on the quality of the domain discriminator, and training discriminators in high-dimensional activation spaces is inherently challenging.
  • The error-activation correlation metric requires ground-truth concept labels—rendering it inapplicable in unannotated scenarios.
  • Only two types of post-hoc CBMs (PCBM + LFCBM + VLG-CBM) were analyzed, leaving other variants (such as Concept Embedding Models, CEM) uncovered.
  • vs Makonnen et al. (2025): They use the classical JL lemma to show that random projections preserve distance. This paper utilizes the manifold JL lemma to extend the conclusion from finite sets to the entire activation manifold and provides a constructive proof of linear reconstruction.
  • vs Schoen et al. (2025): They connect JL with information leakage. This paper further demonstrates that even in the absence of leakage, the concept projection itself can still be unfaithful.
  • vs Huang et al. (2024): They evaluate faithfulness by matching input regions with concepts. The error-activation correlation proposed in this paper provides a more direct mathematical formulation.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ (Systematically formalizes the faithfulness problem of post-hoc CBMs for the first time; the theoretical depth is impressive)
  • Experimental Thoroughness: ⭐⭐⭐⭐ (Validated across Elements+CUB+CIFAR, but ground-truth concepts are only available on Elements)
  • Writing Quality: ⭐⭐⭐⭐⭐ (The argumentative chain is clear; flows seamlessly from "accuracy is insufficient" \(\to\) "why it is insufficient" \(\to\) "two unfaithfulness mechanisms" \(\to\) "diagnostic metrics")
  • Value: ⭐⭐⭐⭐⭐ (Provides direct guidance for evaluation practices in the post-hoc CBM field; the proposed diagnostic metrics can be immediately adopted by the community)