Seeing Through Circuits: Faithful Mechanistic Interpretability for Vision Transformers¶

Conference: ECCV2026
Paper: ECCV
Area: Interpretability
Keywords: mechanistic interpretability / visual circuits / activation patching / typographic attacks / activation steering

TL;DR¶

The paper formalizes a ViT as a directed graph over residual-stream connections, builds clean/corrupted input pairs via semantic segmentation plus inpainting, and prunes edges sequentially with target logit difference as the criterion (Vi-CD) to automatically recover class-specific circuits in ViT-B and OpenCLIP that are faithful yet roughly 10x sparser than the strongest baseline, and it shows these circuits act as causal switches that defend CLIP against typographic attacks and lower the danger retrieval score on RoCOCO.

Background & Motivation¶

Mechanistic interpretability aims to reverse-engineer the internal computation of neural networks, and its central object is the circuit: a sparse subgraph of the model's computation that implements a specific behavior. In transformer language models, circuit discovery has converged on a fairly uniform edge-based formulation — the model is a directed graph over residual-stream connections between attention heads and MLP blocks, and the goal is to find the minimal subgraph responsible for a behavior, validated by faithfulness (restricting computation to that subgraph preserves the original task performance). Sequential activation-patching pruning (ACDC) and gradient-based approximations (EAP) both live on this line.

Vision took a different path: circuit discovery there has stayed at the neuron, channel, or feature level — from the Distill Circuits thread, through work that disentangles polysemantic neurons into concept-specific circuits, prunes for feature-preserving subnetworks, traces connectome-style inter-layer feature interactions, or finds influential neuron paths in ViTs. All of them answer "which components matter" and sidestep "how information flows between them." That is precisely the crux: node-based circuits cannot disentangle polysemantic neurons that participate in several circuits at once, and they cannot support fine-grained intervention without ablating whole components. The edge-based formulation solves both, and it also admits a sharper notion of faithfulness, since individual connections can be included or excluded on their own. Yet every existing edge-based method was developed and evaluated on language models only, and transferring to vision is far from trivial: vision transformers operate over spatially distributed patch embeddings rather than discrete tokens, so corruption cannot be localized to a single representational unit, and per-edge intervention is even harder to afford on the much larger visual computation graph. Whether edge-based faithful circuits can be meaningfully recovered in vision architectures was an open question.

This paper's angle is that the corruption problem in vision should not be solved with noise but with semantics — segment out the foreground object of the target class and inpaint it away, so that the resulting performance drop reflects the absence of class evidence rather than out-of-distribution artifacts. On such clean/corrupted pairs, the language-side per-edge patching machinery applies almost as-is. Core idea: push circuits from the neuron level down to the residual-stream edge level, use segmentation-plus-inpainting semantic corruption plus per-attention-block attention-input aggregation to make the search tractable, prune edges with target logit difference while validating faithfulness with classification accuracy as a separate metric, and treat the recovered circuits as causal valves for projection-based steering that corrects harmful model behavior.

Method¶

Overall Architecture¶

The question Vi-CD asks is concrete: given a target class \(A\) (or an attack word), can we find a small set of edges in a ViT's residual-stream computation graph such that letting signal flow only along those edges reproduces the model's judgment about that class? The pipeline runs in four serial stages: first build clean/corrupted input pairs per target class (corruption = segment away the foreground object and inpaint), then formalize the model as a directed graph and reduce its size, then iteratively substitute non-circuit connections with corrupted activations and decide per edge whether it should stay, and finally validate faithfulness with a metric that deliberately differs from the pruning criterion (classification accuracy). The resulting class circuits are not just an analysis artifact: at inference time the sender outputs of edges inside the circuit can be directionally ablated to suppress the pathway an attack introduced.

The paper's formalization of "circuit" and "faithfulness" is the foundation everything else rests on. The model is written as a directed graph \(G=(V,E)\) over the residual stream, where nodes are components (attention heads and MLP blocks) and edges are the additive contributions they transmit to later components through the residual stream; a circuit is simply a subset of edges \(E_C \subseteq E\), equivalently a 0/1 indicator \(i_e\) per edge. Faithfulness follows the established definition: for a task \(T\) (image classification here), a circuit \(E_C\) is faithful if restricting computation to it loses at most \(\epsilon\) of task performance,

\[\bigl|M_T(E) - M_T(E_C)\bigr| \le \epsilon\]

where \(M_T\) is the task fidelity metric, which the paper stresses must differ from the pruning criterion — otherwise faithfulness would merely be self-certifying. In practice pruning uses target logit difference and validation uses classification accuracy.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["target-class image"] --> B["segment + inpaint<br/>clean/corrupted pairs"]
    B --> C["attention-input node<br/>simplified graph"]
    C --> D["sequential edge pruning<br/>criterion: target logit diff"]
    D --> E["class circuit<br/>sparse faithful subgraph"]
    E --> F["corruption-aligned direction<br/>projection removal"]
    F --> G["typographic attack defense<br/>lower RSMS"]

Key Designs¶

1. Segmentation-plus-inpainting semantic corruption: making the performance drop reflect only the absence of class evidence

The entire activation-patching logic rests on the premise that a corrupted input removes task-relevant information while leaving the rest of the input statistics largely unchanged, and that premise is hardest to satisfy in vision. Language models can swap a token; vision models take spatially distributed patch embeddings, and no single representational unit can be "swapped out." If one instead used noise or black-box occlusion, the damage would extend beyond class evidence to low-level statistics and background context, the performance drop would be contaminated by out-of-distribution artifacts, and the faithfulness numbers would stop being trustworthy. Vi-CD attacks this at the semantic level: segment the foreground object relevant to the target task to obtain a mask, then inpaint the masked region with an inpainting model, producing a visually plausible image that preserves global background statistics and low-level structure. The paper uses the ForAug dataset directly, which already supplies ImageNet images processed through exactly this segmentation-and-inpainting pipeline, saving the cost of training its own segmentation and inpainting models.

The most direct consequence is that edges encoding background correlations — the shortcut edges — show almost no activation difference between the clean and corrupted forward passes, so patching does not flag them as important and the circuit converges on object-level pathways instead of spurious background texture. Appendix H ablates the quality of the segmentation masks, and the paper notes that the corruption strategy is not tied to classification: swapping in a synthetic inpainting method such as the one used by SUB extends it to discovering concept-level circuits. Notably, in the typographic-attack application the roles of clean and corrupted are reversed — the typographic image serves as the clean input and its uncorrupted counterpart as the corrupted input — because there the information to be removed is exactly the pathway the attack text injected.

2. The attention-input node: shrinking the \(O(L^2H^2)\) candidate edge set by a factor of \(H\)

The cost of per-edge intervention scales with the number of candidate edges, and a full residual-stream dependency graph for an \(L\)-layer, \(H\)-head transformer contains \(O(L^2H^2)\) edges — exhaustive search at that scale is infeasible, so the edge set must shrink first, but it cannot shrink in a way that changes the model's functionality, or the recovered circuit would no longer be a circuit of the original model. Vi-CD simplifies only the receiver side: it introduces one attention-input node per attention block, which aggregates the block's residual input and serves as the shared receiver for all incoming edges, while attention heads remain independent senders. Attention heads no longer each own \(H\) receiving slots but share a single entry point, cutting the number of candidate edges by roughly a factor of \(H\) without modifying the forward computation at all.

On the simplified graph the edge types are fixed to nine categories, which together form the complete search space: input→attn_in, input→mlp, input→logits; attn→attn_in, attn→mlp, attn→logits; and mlp→attn_in, mlp→mlp, mlp→logits. The paper illustrates the collapse on a toy 2-layer transformer (Figure 3 of the original).

3. Sequential activation-patching pruning: keeping the pruning criterion and the fidelity metric deliberately separate

With the graph and the paired samples in place, what remains is to rank and discard unimportant edges. Vi-CD inherits the sequential pruning framework of ACDC, whose technical precondition is the additivity of the residual stream: because component contributions are additive, an edge-level intervention can substitute the output of a single sender while leaving all other prior computation unchanged. Concretely, for the predecessor set \(P(v)\) of a node \(v\), the residual connections outside the candidate circuit are replaced by their contributions from the corrupted run, giving the patched input to \(v\):

\[\mathrm{in}^v_C(\tilde{x}) = \sum_{u \in P(v)} \bigl(i_e\, r_u(x) + (1-i_e)\, r_u(\tilde{x})\bigr)\]

where \(i_e = \mathbb{1}[(u \to v) \in E_C]\) indicates whether the edge currently remains in the candidate circuit. Pruning is iterative: tentatively remove an edge, and if the loss measured by target logit difference is small enough after removal, drop it from the candidate circuit and update the model, repeating until no edge can be removed. Target logit difference is chosen over a distribution-level criterion such as KL divergence for a practical reason — most vision classifiers do not operate over a fixed token dictionary, so distribution-level comparison is not directly aligned with "getting class A right"; the comparison between the two criteria is in Appendix F.

The crucial point is the separation of criterion and validation: search uses target logit difference, while the reported faithfulness uses classification accuracy. The two are not the same quantity, so "the circuit is very sparse and accuracy barely moves" cannot be self-certifying. This also defines the paper's class circuit directly: a triple \(C_A = (D_A, M, E_C)\) consisting of a set of diverse inputs correctly classified as the target class \(A\), a scalar performance metric \(M\) (e.g. accuracy), and a circuit \(E_C\) validated by activation patching as faithful under that metric. For models like ViT-B the logits come straight from the classification head, while for contrastive models like OpenCLIP they are the dot product between the image embedding and the corresponding text embedding, so one pipeline covers two very different model families.

4. Projection-based steering along the corruption-aligned direction: using circuits as causal valves

Faithfulness is inherently a passive metric: a subgraph that merely correlates with the behavior, but happens to be compensated by other pathways under ablation, can still post a flattering faithfulness curve. Demonstrating that a circuit carries a genuine mechanism requires a predictable causal intervention. Vi-CD first discovers a set of faithful circuits associated with an "attack" (in the typographic setting, attack circuits are defined exactly analogously to class circuits), then estimates a corruption-aligned direction for every component \(j\) in the circuit: the mean activation difference across paired examples, where the two activations of each example are normalized to unit norm before differencing, so the direction is insensitive to activation magnitude and characterizes only the orientation toward the corrupted side.

At inference time the intervention touches only the sender outputs \(h\) corresponding to edges in \(E_C\), and it is patch-wise. For each patch \(p\) the projection coefficient is

\[c_p = \frac{\langle h_p, v_{j,p}\rangle}{\lVert v_{j,p}\rVert_2^2 + \varepsilon}\]

followed by directional ablation (projection removal), \(h^{\text{steered}}_p = h_p - \alpha\,\mathrm{ReLU}(c_p)\,v_{j,p}\), where \(\alpha \ge 0\) controls steering strength; on the RoCOCO experiments the paper uses \(\alpha = 0.4\). Geometrically the operation rotates the activation within the two-dimensional subspace spanned by the current activation \(h\) and the feature direction \(v\), and the ReLU ensures that only positively aligned projections are removed, preventing amplification of anti-correlated features. The intervention works precisely because the circuit is faithful: suppressing the corruption-aligned direction at upstream edges keeps it from propagating downstream, while computations outside the circuit are largely unaffected — which is why steering helps while barely costing clean accuracy.

Loss & Training¶

Vi-CD trains no parameters; everything is post-hoc on a pretrained model, so there is no loss function. The closest analogue to an objective is the pruning criterion — the drop in target logit difference, together with the threshold \(\epsilon\) that decides whether an edge can be dropped; circuit search is greedy iteration under that criterion. The only extra hyper-parameters appear in the steering stage: the strength \(\alpha\) (0.4 on RoCOCO) and the maximal receiver layer that is intervened on. In the typographic experiments the steering strength is tuned so that ASR drops by roughly 90% in relative terms before results are reported (the table caption records this as ASR ↓90%), and typographic circuits are extracted separately for 13 effective attack words.

Key Experimental Results¶

Main Results¶

Experiments cover two medium-sized models: a supervised ViT-B trained on ImageNet, and OpenCLIP ViT-B/32. Both are represented as directed computation graphs over attention heads and MLP blocks connected by residual-stream edges. Baselines are EAP, EAP-IG (3, 5, and 10 steps), and random edge pruning as a lower bound; EAP and EAP-IG are adapted to ViTs using the ViTPrisma library with the same simplified computation graph enforced, keeping the comparison fair. Circuit quality is measured by faithfulness (classification accuracy when computation is restricted to the circuit, higher is better) and sparsity (fraction of edges retained, lower is better). Cross-class analysis uses Jaccard similarity \(|A \cap B| / |A \cup B|\).

Table 1: Comparison of edge-based circuit extraction methods (ViT-B / OpenCLIP, all ImageNet classes)

Method	Circuit granularity	Source of edge importance	Faithfulness on ViT-B	Edge fraction needed for comparable faithfulness
Random	edge	random	near chance	lowest (lower bound)
EAP	edge	first-order gradient approximation	close to random pruning throughout	very high
EAP-IG-3 / 5 / 10	edge	integrated-gradient approximation	strongest baseline; marginal differences across step counts	high
Vi-CD (Ours)	edge	per-edge activation patching (real intervention)	near-perfect target-class accuracy	< 10%

⚠️ The original reports this result as accuracy-versus-sparsity curves (Figure 4) and provides no directly transcribable numeric table; the qualitative conclusions above and the "< 10%" figure are taken from the paper's prose, and "roughly 10x sparser" is relative to the strongest baseline, EAP-IG. The pattern holds across models: Vi-CD stays highly faithful at extreme sparsity, whereas baselines plateau at low accuracy until a much larger fraction of edges is included.

Table 2: Circuit steering against typographic attacks (OpenCLIP, ImageNet)

Setting	Metric	Big Text Base → Steered	Small Text Base → Steered	Bezel Base → Steered
Clean ImageNet	Top-1 Acc. (%)	57.0 → 55.8	57.1 → 55.7	57.1 → 49.9
Clean ImageNet	Top-5 Acc. (%)	81.9 → 80.9	81.9 → 80.9	81.9 → 75.6
Corrupted ImageNet	Top-1 Acc. (%)	34.7 → 50.0	34.7 → 49.1	34.4 → 45.0
Corrupted ImageNet	Top-5 Acc. (%)	69.9 → 75.9	70.1 → 75.3	69.8 → 71.2
Corrupted ImageNet	ASR Top-1 (%)	39.1 → 2.8	39.4 → 1.6	39.5 → 3.1
Corrupted ImageNet	ASR Top-5 (%)	68.0 → 8.1	68.4 → 7.6	68.4 → 12.7

The three attack regimes are large text overlays, small text overlays, and bezel-style overlays (examples in Appendix C). After steering, Top-1 attack success rate falls from the ~39% range to below 3%, and accuracy on attacked images actually recovers substantially (Top-1 from 34.7% to about 50%) — indicating that steering restores the model's judgment of the image content itself rather than merely suppressing a logit.

Ablation Study¶

Table 3: Analysis and controls for steering behavior

Config / analysis	Key metric	Note
RoCOCO base (no steering)	Rmean 62.74%, RSMS 11.68%	unsteered baseline (\(\alpha = 0.4\), Appendix C.6)
RoCOCO + Assault Rifle circuit	Rmean 63.00 ± 0.37, RSMS 5.15 ± 0.24	danger recall more than halved, retrieval quality retained
RoCOCO + Revolver circuit	Rmean 62.68 ± 0.22, RSMS 5.69 ± 0.41	same as above
RoCOCO + Rifle circuit	Rmean 64.37 ± 0.47, RSMS 4.86 ± 0.21	Rmean is even 1.6 points above the baseline
Random circuits (same size)	no clear ASR reduction on typographic attacks	Appendix G; shows the effect comes from the discovered circuits, not generic ablation
Maximal receiver layer intervened (early vs deep)	early-layer intervention already lowers ASR	scanned jointly with steering strength (Figure 6)
Pruning criterion: target logit diff vs KL divergence	target logit difference is better	Appendix F
Segmentation mask quality	see Appendix H	no numbers in the main text
Cross-run stability	Jaccard 0.6–0.8 for same-class circuits	shared core edge set, varying extra connections

Key Findings¶

Sparsity is the paper's hardest result: faithful circuits found by Vi-CD are roughly 10x sparser than those from the strongest baseline, EAP-IG; on ViT-B, keeping fewer than 10% of the edges already yields near-perfect target-class accuracy. Among gradient approximations EAP-IG consistently beats EAP, while EAP is essentially indistinguishable from random pruning throughout — suggesting that first-order gradient approximation deviates badly from true causal contribution on continuous visual inputs.
Circuits reflect semantic structure: overlap is markedly higher between semantically related classes. Pembroke Corgi and Cardigan Corgi share substantially more edges with each other than with unrelated categories such as Revolver or Tench, while the Revolver circuit is most similar to other weapon classes (Assault Rifle, Rifle). The paper explicitly frames this as a descriptive analysis of how the model organizes computation, not as a quality measure for Vi-CD.
The circuit for a class is not unique: across runs, circuits for the same class show Jaccard similarity of only 0.6–0.8, sharing a core edge set with varying additional connections. The authors read this not as a method failure but as genuine redundancy in the model — multiple partially overlapping subgraphs can implement similar behavior, consistent with the OR-circuit structure observed in language models — so circuit discovery is better understood as producing a distribution over plausible circuits than as recovering a single ground-truth subgraph (Appendix D).
Attack information travels along a localizable early-layer pathway: intervening only on fairly early receiver layers already lowers typographic attack success while clean performance is largely preserved, and random circuits of the same size show no such effect, confirming that the discovered object is a real attack pathway rather than generic damage from ablation.
Safety gains come at no retrieval cost: on RoCOCO all three weapon circuits cut RSMS from 11.68% to around 5%, while R@1/R@5/R@10 and Rmean are all maintained or slightly improved (the Rifle circuit lifts Rmean from 62.74% to 64.37%).
Circuits compose: unions of class-specific circuits support zero-shot binary classification without additional training (Appendix E), indicating that the model organizes computation into separable mechanisms rather than entangled representations.

Highlights & Insights¶

Turning corruption from a noise problem into a semantics problem. Vision has no discrete tokens; the transferable move is to actively take task evidence out of the input by segmenting the foreground and inpainting it back. This both solves the non-localizability of corruption and incidentally suppresses background shortcut edges — a very clean reframing of the problem.
Pruning criterion and fidelity metric are deliberately different quantities. Search with target logit difference, validate with classification accuracy, so faithfulness is never measured with the same ruler used to construct the circuit. This discipline is worth copying in any circuit-extraction work.
Causal validation by steering rather than curves alone. Faithfulness is necessary but not sufficient; a predictable intervention effect is what shows the subgraph carries a mechanism, and the paper turns this into quantitative defense results on two independent attack settings, which is far more convincing than a sparsity comparison by itself.
The attention-input node is a cheap and general trick. It changes only the receiver structure and not the function, yet cuts the candidate edges by a factor of \(H\); any transformer analysis that needs per-edge intervention can reuse it directly.
Transferable directions: the corruption strategy ports to any modality whose inputs admit semantic masking plus inpainting (images, video frames, some document layouts); the "circuit as a switch" idea also transfers to the data side — activation differences on a concept circuit can serve as a detector for anomalous or out-of-distribution inputs.

Limitations & Future Work¶

Computational cost is the biggest bottleneck (admitted by the authors): compared with approximations such as EAP and EAP-IG, Vi-CD requires patching each edge at least once, which is far more expensive; the method currently scales only to medium-sized foundation models such as OpenCLIP, and efficient discovery in the largest transformers remains open.
Polysemanticity is unaddressed (admitted): individual neurons routinely encode several unrelated features at once, and edge-based circuits do not decompose this. Applying Vi-CD over monosemantic feature directions derived from sparse autoencoders is the natural extension the authors name.
Stability certification does not port directly (admitted): recent randomized-smoothing certification for node-based circuits requires resampling clean inputs, whereas the edge-based setting would need resampling paired clean–corrupted inputs — a non-trivial change left explicitly to future work.
Model coverage is narrow (my observation): every experiment lands on ViT-B and OpenCLIP ViT-B/32, with no self-supervised or segmentation foundation models such as DINOv2 or SAM, and no larger ViTs or vision-language generative models. The word "vision transformers" in the title and conclusions therefore rests on two medium-sized models, both evaluated on ImageNet classification behavior.
The faithfulness evaluation setting is narrow (my observation): class circuits target telling class A apart from competing classes — a two-way setting. Whether circuits remain equally faithful under full multi-class accuracy, and how the choice of competing classes affects results, is not explored in the main text.
Corruption quality is relegated to the appendix (my observation): segmentation mask quality and the fidelity of the inpainting model directly determine whether the performance drop is caused purely by missing class information, yet the main text does not quantify this sensitivity; Appendix H is the only evidence.
Improvement ideas: cut the number of per-edge patches with surrogate models or early stopping; extend circuit discovery to non-classification tasks such as grounding and retrieval; combine edge-level analysis with SAE feature directions for a joint edge-and-neuron account; and track how circuit structure evolves during training to date the emergence of capabilities.

vs EAP [34]: EAP estimates edge importance with a first-order gradient approximation, trading faithfulness for scalability; Vi-CD performs real per-edge activation patching, slow but accurate. In the experiments EAP is essentially indistinguishable from random pruning on vision, indicating that on continuous patch-embedding inputs the gradient approximation deviates badly from true causal contribution.
vs EAP-IG [18]: the integrated-gradient variant stabilizes attribution estimates and is the strongest and closest baseline; this paper's faithfulness definition (Def. 3.2) is itself adopted from that work. Even so EAP-IG needs far more edges to reach comparable performance, a sparsity gap of roughly 10x, and differences across step counts are marginal.
vs ACDC / Conmy et al. [9]: Vi-CD directly inherits its edge-based circuit definition and sequential patching-pruning framework, but supplies two vision-specific adaptations — semantic corruption and the attention-input graph simplification — and swaps the pruning criterion for target logit difference, which aligns with the classification decision. All edge-based methods were previously developed and evaluated on language models only; this is the first time the paradigm is applied to vision transformers.
vs neuron-level vision methods [7, 13, 16, 24, 31, 38]: those works answer "which components/neurons/features matter," whereas Vi-CD answers "how information flows between components"; the edge-based formulation additionally buys two things — the ability to disentangle a polysemantic neuron's contribution across several circuits, and fine-grained intervention without ablating whole components. The two are complementary rather than competing, and combining edge-level with neuron-level analysis (finding which neurons inside a circuit edge actually carry the semantics) is named by the authors as a natural extension.
vs Certified Circuits [1]: that work provides stability guarantees for node-based circuits via randomized smoothing, certifying that circuit membership decisions stay invariant under bounded dataset edits. This paper points out that porting the guarantee to the edge level is non-trivial (it requires resampling paired inputs) and lists it as an explicit future direction; its own finding of 0.6–0.8 Jaccard variation across runs is almost the same problem approached from the empirical side.

Rating¶

Novelty: ⭐⭐⭐⭐ First systematic transfer of edge-based circuit discovery to vision transformers, with two non-trivial adaptations (corruption strategy and graph simplification); the skeleton (sequential patching pruning, edge-based circuit definition) is inherited from ACDC, and the formalization is largely carried over.
Experimental Thoroughness: ⭐⭐⭐⭐ Two models, all ImageNet classes, cross-class similarity, and steering validation on two attack settings make a fairly complete story, and the parts without numeric tables are covered by curves; the main comparison, however, is only Vi-CD vs EAP/EAP-IG/random, the faithfulness claims are presented as curves without reproducible numbers, and scale stops at the ViT-B level.
Writing Quality: ⭐⭐⭐⭐ The formalization is clear, the research questions are well organized, and the framework figure explains the three-step flow well; but several key conclusions (mask quality, criterion comparison, random-circuit controls, multi-circuit redundancy) point to the appendix, leaving the main text somewhat declarative.
Value: ⭐⭐⭐⭐ Provides a usable tool for visual mechanistic analysis and demonstrates through two attack-scenario defenses that circuits are actionable rather than merely interpretable; the price is that the cost of per-edge patching limits scaling to larger models.