🔬 Interpretability

🎞️ ECCV2026 · 30 paper notes

📌 Same area in other venues: 📷 CVPR2026 (34) · 🔬 ICLR2026 (195) · 💬 ACL2026 (63) · 🧪 ICML2026 (91) · 🤖 AAAI2026 (37) · 🧠 NeurIPS2025 (80)

🔥 Top topics: Multimodal/VLM ×4 · Adversarial Robustness ×3

CFM: Language-aligned Concept Foundation Model for Vision

CFM decomposes language-aligned visual features into sparse concepts with spatial locations, hierarchical relations, and textual names, making downstream predictions explainable and steerable while largely preserving classification, segmentation, and captioning performance.

On the Reliability of Cue Conflict and Beyond

This paper systematically questions the reliability of the widely used cue-conflict benchmark on both the stimulus-construction and the measurement side, identifying four problems — stylization entangles the cues, cue informativeness is imbalanced, ratio-based bias hides absolute cue sensitivity, and a restricted label space distorts predictions — and offers REFINED-BIAS as a replacement, built from human-perceptual cue definitions, controlled purified cue generation, and a full-label-space MRR sensitivity metric; on 32 training-strategy models, multiple architecture families, OpenCLIP/DINOv2 large models, and multi-level quantized models it resolves contradictory conclusions that the original benchmark could not explain.

Seeing Through Circuits: Faithful Mechanistic Interpretability for Vision Transformers

The paper formalizes a ViT as a directed graph over residual-stream connections, builds clean/corrupted input pairs via semantic segmentation plus inpainting, and prunes edges sequentially with target logit difference as the criterion (Vi-CD) to automatically recover class-specific circuits in ViT-B and OpenCLIP that are faithful yet roughly 10x sparser than the strongest baseline, and it shows these circuits act as causal switches that defend CLIP against typographic attacks and lower the danger retrieval score on RoCOCO.

Caption Bottleneck Models

CaBM converts images into taxonomy-censored captions, recognizes classes with an independent text-only classifier, and discovers concepts from its textual evidence, removing the visual-feature bypass while achieving competitive but not uniformly leading classification results.

DEX-AR: A Dynamic Explainability Method for Autoregressive Vision-Language Models

DEX-AR treats the gradient of each generation step's attention maps with respect to intermediate logits as the attribution signal, then dynamically filters attention heads and generated tokens by their "visual minus textual" maximum-gradient difference, producing both per-token and sequence-level image heatmaps for autoregressive VLMs that consistently beat Grad-CAM, attention-based, and perturbation-based baselines on ImageNet/VQAv2 perturbation and PascalVOC localization.

From Drop-off to Recovery: A Mechanistic Analysis of Segmentation in MLLMs

Without adding any new module to the MLLM, this paper dissects its segmentation competence through three interventional experiments: layerwise linear probing reveals a representation drop-off at the adapter when visual features are projected into the language space, followed by progressive recovery of mIoU across LLM layers; attention knockout shows that this recovery is not a passive by-product of residual connections but is actively mediated by cross-token attention, in which correctly classified tokens act as semantic anchors pulling misclassified neighbours toward the correct label; finally, causal attention is shown to starve the earliest image tokens of global context, a cost that bidirectional attention among image tokens eliminates.

This Looks Distinctly Like That: Grounding Interpretable Recognition in Stiefel Geometry against Neural Collapse

AMP replaces class prototypes with Stiefel-constrained orthonormal bases and uses sparse capacity and spatial constraints to select and localize distinct evidence, achieving 88.4% accuracy with ResNet50 on CUB, although its strict guarantee concerns basis rank rather than necessarily distinct semantic parts.

Towards Robustness against Typographic Attack with Training-free Concept Localization

This paper localizes typographic-attack vulnerability to a small set of "text-reading" attention heads in CLIP ViTs: it samples concept directions in the multi-head subspace, scores each head with a gradient-based text-focus metric (nTAS) to mine those heads, and then applies a training-free test-time intervention (attention reweighting or zero ablation) to them — raising object classification accuracy by more than 6 points on average across five CLIP backbones and four typographic-attack benchmarks while cutting the text confusion rate by 10–36 points.

World Knowledge in the Weights: Reading Concept Circuits of Vision Transformers

The work ports Cross-Layer Transcoders (CLTs) from language models to ViTs, reading a global concept circuit — the model's "world knowledge" — directly from input-invariant encoder/decoder weights and instance concept circuits from sparse activations; co-activation-reweighted global weights are validated by mean ablation to carry genuine functional importance, enabling fully automatic spurious-correlation discovery (graffiti for Freight Car, bird feeder for Hummingbird) and a +23% worst-group intervention gain on Waterbird, while CLT circuits beat SAE baselines on faithfulness and completeness across the board.

Blind to Position, Biased in Language: Probing Mid-Layer Representational Bias in Vision-Language Encoders for Zero-Shot Language-Grounded Spatial Understanding

B2G probes frozen vision-language encoders and reuses spatially sensitive visual features and more language-consistent text features for region ranking, increasing average multilingual referring segmentation mIoU from 42.42 to 49.62 across nine benchmarks, with additional multilingual inference cost.

Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations

This paper proposes an interpretability evaluation framework for SAEs based on human concept annotations. It features the FBMP algorithm supporting many-to-one matching and the TAPAScore causal validation metric based on target attribute perturbation. Furthermore, two synthetic perturbation datasets, synCUB and synCOCO, are constructed. Experiments show that existing automated evaluation metrics (FMS, MS, CKNNA) fail the sanity checks, whereas the proposed matching metrics and TAPAScore reliably distinguish trained from untrained SAEs. Additionally, increasing overcompleteness is found to degrade the quality of perturbation alignment.

Learn to Rank: Visual Attribution by Learning Importance Ranking

To overcome the three-way trade-off among computational efficiency, causal faithfulness, and fine-grained spatial resolution in visual attribution, this paper proposes Amortized Hybrid Attribution (AHA), which relaxes discrete sorting into differentiable permutation learning via Gumbel-Sinkhorn to directly optimize Deletion and Insertion metrics end-to-end, producing dense pixel-level heatmaps in a single forward pass.

Mapping the Concept Landscape: Structural Perception of Global Distributions for Transparent Data Pruning

Addressing the core limitation of opaque embedding-based data pruning that obscures fine-grained semantic coverage and lacks human interpretability, this paper proposes Mapping the Concept Landscape (MCL), which decouples image-text samples into explicit concept graphs of entities, events, and attributes, aggregates them into a dataset-level semantic landscape, and applies a greedy marginal coverage maximization strategy for efficient and transparent pruning.

On the Faithfulness of Post-Hoc Concept Bottleneck Models

This paper systematically analyzes the faithfulness of concept projection in post-hoc CBMs. It demonstrates that classifier accuracy cannot serve as a proxy metric for the quality of the bottleneck layer (since random projections can also achieve competitive performance), identifies two fundamental causes of unfaithfulness—covariate shift in auxiliary datasets and systematic errors in VLM surrogate labels—and proposes corresponding diagnostic metrics (\(\mathcal{H}\Delta\mathcal{H}\)-divergence and error-activation correlation).

REDistill: Robust Estimator Distillation for Balancing Robustness and Efficiency

Grounded in robust M-estimation, REDistill substitutes the conventional KL divergence in knowledge distillation with a theoretically derived power divergence (\(\lambda=2/3\)), adaptively downweighting unreliable teacher predictions and achieving state-of-the-art distillation across diverse architectures without hyper-parameter tuning.

TRINITY: A Multi-Perspective Benchmark for Personal-Style Video Highlight Detection

Addressing the limitation of traditional event-centric video highlight detection, this paper introduces TRINITY, a multi-perspective benchmark that decomposes saliency into Event, Emotion, and Nature, along with a shared-backbone multi-branch architecture featuring view-specific experts that achieves new state-of-the-art results on Mr. HiSum and YouTube Highlights.

Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Intervention

This paper introduces a controlled subspace intervention framework utilizing Singular Value Decomposition (SVD) on converged linear probe weights to decode geometric representations in self-supervised ViTs, proving that DINOv2 neatly aligns explicit geometry into a highly compressible low-rank subspace while MAE disperses geometric signals across spatial and channel dimensions.

Weight-Space Mixture-of-Experts for Implicit Neural Representation Classification

Addressing the challenges of high parameter dimensionality and layer-wise functional heterogeneity in classifying implicit neural representation (INR) weights, this paper proposes a hierarchical Mixture-of-Experts (HMoE) Transformer coupled with meta-learning that achieves new state-of-the-art results across standard benchmarks and introduces Grad-CAG, a first-order weight-space attribution and pruning method to uncover discriminative subcircuits.

Causal Intervention in Concept Bottleneck Models

Proposes Causal Intervention (CI), a training-free intervention approach that leverages concept-correlation structures already encoded inside trained Concept Bottleneck Models by treating the input image as a shared upstream carrier to propagate human feedback through the \(x \to c\) pathway, achieving state-of-the-art performance with minimal intervention effort.

GridVQA-X: A Diagnostic Framework for Evaluating Multimodal Explainability Methods

Addressing the lack of ground-truth causal validation in multimodal explainable AI (MxAI), this paper introduces GridVQA-X, the first closed-world diagnostic framework featuring mathematically guaranteed unique ground-truth explanations and paired models (\(M_{\text{pure}}\) vs. \(M_{\text{spur}}\)), revealing that prominent explainers fail to distinguish genuine spatial reasoning from shallow Bag-of-Words shortcuts.

Learning Physics-based Forward Model Corrections in Unrolled Networks for Diffuser-based Imaging

Addressing the severe forward model mismatch caused by spatially-varying optical aberrations in diffuser-based lensless cameras, this paper proposes EK-DUN, an unrolled architecture incorporating an Eigen-Kernel decomposition of the point spread function alongside an iteration-aware late-activation scheme, achieving state-of-the-art reconstruction on real hardware while preserving linear and lightweight adjoint operators.

Multi-Block-Attention-based Color Constancy

HybridCC introduces a compact 1.1 MB hybrid architecture that guides a differentiable soft-maximum Retinex prior using learned saliency maps and refines block features via multi-head self-attention to eliminate aggregation bias, achieving state-of-the-art cross-camera illuminant estimation without requiring camera metadata or calibration.

Multiple Images Distract Large Multimodal Models via Attention Fragmentation

This paper diagnoses the mechanism behind open-source LMM performance drops and recency bias across multi-image tasks as "attention fragmentation"—caused by recurrent background attention sinks and causal masking—and proposes Attention Remasking (AR), a training-free edit that suppresses visual sinks and dynamically routes attention to task-relevant forward links using an uncorrupted first-layer grounding prior.

OPAL: Orthonormal Prototype Alignment Learning for Interpretable Image Classification

OPAL eliminates complex multi-stage prototype clustering and projection pipelines by anchoring classes onto predefined orthonormal subspaces and enforcing channel-wise spatial competition, achieving auxiliary-loss-free, single-stage interpretable image classification that outperforms black-box CNN baselines.

ProtoMappingNet: Interpretable Hierarchical Prototypes through Relational Prototype Mappings

ProtoMappingNet incorporates learnable bidirectional cross-layer semantic mappings and alignment losses across low (texture), mid (part), and high (object) abstraction levels, achieving structured, spatially nested part-whole prototype reasoning without requiring ground-truth part annotations.

R-ESC: Robustly Erasing Space Concepts via Stochastic Feature Remapping

Addressing the vulnerability of prior concept erasure methods that degrade remain-data utility and leave cohesive latent clusters recoverable by fine-tuning, R-ESC introduces a training-free framework that combines prototype-orthogonal projection with multi-expert stochastic remapping to achieve robust, irreversible feature-level erasure.

SPICE: Simple Polysemantic feature Interpretation via Clustering-based Explanations

Addressing the challenge of polysemanticity where individual neurons activate for multiple unrelated concepts, SPICE introduces an architecture-agnostic framework that adaptively determines cluster count \(K\) via attribution footprints and kernel density estimation, enabling the first unified polysemanticity comparison across CNNs and Vision Transformers.

STVFocus: Query-guided Spatio-Temporal Visual Focusing for Video LLMs

Addressing the intrinsically low task-relevant information density and substantial redundancy in video inputs, STVFocus introduces a training-free spatio-temporal visual focusing framework that combines coarse-to-fine hierarchical temporal frame sampling with query-guided intra-frame spatial saliency pruning, reinvesting the saved spatial budget into supplementary keyframes to significantly enhance Video LLM reasoning accuracy and interpretability.

VNC: A Scale-Space Foundation for Learnable 3D Surface Evolution

Addressing the numerical irregularity and lack of geometric interpretability in conventional 3D surface evolution, this paper introduces a dimensionless, differentiable Variational Neighborhood Curvature (VNC) operator alongside a spacetime-balanced VNC-Flow algorithm, training a time-conditioned Controllable VAE that spontaneously organizes a physically interpretable radial latent structure.

Why Linear Probing Works: Non-Vacuous Generalization Bounds via Effective Dimension

Resolving the theoretical discrepancy where linear probing requires exponentially fewer samples than classical bounds predict, this paper leverages steep covariance spectral decay to construct covariance-aligned data-dependent Gaussian priors, deriving the first non-vacuous PAC-Bayes generalization certificates governed by effective dimension \(d_{\text{eff}} \ll d\) and classification margin \(\gamma\).