Skip to content

DEX-AR: A Dynamic Explainability Method for Autoregressive Vision-Language Models

Conference: ECCV 2026
Paper: ECCV 2026
Code: https://walidbousselham.com/DEX-AR
Area: Interpretability
Keywords: interpretability, vision-language model, attention gradient attribution, autoregressive generation, attention head filtering

TL;DR

DEX-AR treats the gradient of each generation step's attention maps with respect to intermediate logits as the attribution signal, then dynamically filters attention heads and generated tokens by their "visual minus textual" maximum-gradient difference, producing both per-token and sequence-level image heatmaps for autoregressive VLMs that consistently beat Grad-CAM, attention-based, and perturbation-based baselines on ImageNet/VQAv2 perturbation and PascalVOC localization.

Background & Motivation

Autoregressive vision-language models (the LLaVA, BakLLaVA, PaliGemma, Florence-2 family) can already answer questions and write descriptions about images, yet there is almost no tool for asking the obvious follow-up: which part of the image is the model actually looking at when it emits the word dog? Explainability methods built for classification — Grad-CAM, Guided Backprop, Integrated Gradients — all assume the output is a set of logits over fixed classes, so they lose their attribution target the moment the output becomes a token-by-token text sequence. Attention-based methods (Raw Attention, Attention Rollout) are a natural fit for transformers, but prior work has repeatedly shown that attention weights are not feature importance, and this is even less reliable in multimodal layers where visual and textual tokens alternate. Methods developed for contrastive models such as CLIP (Chefer et al.'s attention reweighting, LeGrad by the same first author) explain a static image-text alignment and cannot express the changing state of "which part of the context does the token generated at step t depend on."

The real difficulty has two layers: one temporal, one lexical. Temporally, the same image region plays completely different roles at different generation steps — it is decisive evidence when the model writes dog, and contributes nothing to the and that immediately follows — so attribution must be tied to a specific generation step rather than summarized for the whole sentence as in classification models; causal attention further makes every token's hidden state accumulate the entire preceding context, so attribution errors on early tokens propagate down the sequence. Lexically, in "The young woman is wearing a floral dress" only woman, floral, and dress are governed by visual evidence, whereas the, is, and wearing are almost entirely predicted from linguistic priors; averaging them together with the content words dilutes the explanation into a map that glows faintly everywhere. The closest prior work, TAM, mitigates context interference with static visual features plus post-hoc statistical causal estimation, but it does not observe how the attention dynamics change at each generation step.

This paper's angle is that a transformer already records, layer by layer, whom the current token attends to — so the gradient of the attention maps with respect to the current prediction logit measures how sensitive the prediction is to each head at each layer, and two filters built from the same quantity (the maximum-gradient difference between visual and textual tokens), requiring essentially no hyper-parameters, can suppress heads that mainly process text and tokens driven by linguistic priors. Attribution then lands on the parts that are genuinely supported by visual evidence. Core idea: split the explanation of an autoregressive VLM into a two-level reweighting — per-token attention-gradient attribution, dynamic head filtering, and sequence-level token filtering — so that each generation step's explanation consists only of the attention pathways and tokens that truly depend on visual evidence.

Method

Overall Architecture

Notation first: the visual encoder turns an image into \(N\) visual tokens, which are concatenated with \(T_c\) prompt context tokens and fed to the LLM; the LLM has \(L\) layers with \(h\) heads each and autoregressively generates \(T_a\) answer tokens. At step \(t\) the sequence length is \(N+T_c+t\), and causal attention guarantees that the first \(N+T_c\) positions do not change as generation proceeds. DEX-AR is a fully post-hoc method: it trains nothing, modifies no weights, and requires no threshold, waiting until the VLM has produced the whole answer and then tracing back the explanation for every step. Its inputs are the image, the prompt, and the completed answer; its outputs are \(T_a\) per-token heatmaps (one per generated word, showing the image regions that word depends on) plus a single sequence-level heatmap aggregating the sentence.

The pipeline runs as follows. A single forward pass yields the intermediate logits at the "about to be predicted" position of every layer (logit lens); gradients are taken with respect to the attention maps. Only the query row of the predicted token itself and the columns of the visual tokens are kept. Each head is then weighted by its visual-minus-textual maximum gradient difference (head filtering), producing the per-token heatmaps. The same difference, maximized over layers and heads, weights each generated token (token filtering), and the weighted maps are aggregated into the sequence-level heatmap. Note that the two filters quantify the same question — "does this attention pathway / this token really depend on the image?" — but act on different axes: the first reweights heads (a spatial dimension), the second reweights generation steps (a temporal dimension).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["image + prompt + completed answer"] --> B["per-token attention-gradient attribution"]
    B --> C["dynamic attention head filtering<br/>→ per-token heatmaps"]
    C --> D["sequence-level token visual-relevance filtering"]
    D --> E["sequence-level heatmap"]

Key Designs

1. Per-token attention-gradient attribution: pinning down which image region a step looks at via intermediate-layer attention gradients

DEX-AR does not attribute at the final layer. It uses the logit lens instead: the hidden state of the last position at layer \(l\) is passed directly through the language model head to obtain that layer's "if we had to emit a word right now" intermediate logits \(o^{l,t}=\text{LM\_Head}(Z^{l,t}_{-1})\in\mathbb{R}^{V}\), the logit \(\hat{o}^{l,t}\) of the actually sampled word is selected, and the gradient is taken with respect to that layer's attention maps \(A^{l,t}\in\mathbb{R}^{h\times T_t\times T_t}\). Using intermediate layers is safe because the residual stream keeps them in-distribution for the LM head, and the supplementary ablation (Supp. Tab. 7) confirms that intermediate features improve IoU consistently across all four architectures. Taking the gradient with respect to attention maps — rather than input pixels or hidden states — is what makes one formulation span decoder-only, prefix-decoder, and encoder-decoder VLMs unchanged.

From the resulting gradient tensor only two parts are kept: the row of the predicted token itself (under causal attention the intermediate logit at position \(n\) depends only on row \(n\) of the attention matrix) and the columns of the visual tokens. What remains, \(\nabla A^{l,t}_{-1,v}\in\mathbb{R}^{h\times N}\), is the raw signal for "how much prediction sensitivity this head placed on the image at this step." Because the explanation is post-hoc and the full answer already exists, DEX-AR can also collapse the \(T_a\) forward passes into one: prompt and answer are fed as a single sequence, and causal masking guarantees every position's hidden state matches the one produced during autoregressive generation, so no future-token information leaks. Better still, since the gradients of different target tokens occupy disjoint rows of the attention matrix, summing the logits of all \(T_a\) target positions and running a single backward pass per layer recovers mathematically identical per-token gradients. Cost drops from \(T_a\) forward and \(T_a\times L\) backward passes to 1 forward and \(L\) backward passes — a measured ~18× speedup at \(T_a=100\).

2. Dynamic attention head filtering: only heads that look at the image more than at the text get to speak

Not every head looks at the image — many specialize in syntax, coreference, and context, and adding their gradients only injects noise into the heatmap. The ablation says this bluntly: with no filtering at all SNR is only 1.64 (LLaVA) / 5.27 (BakLLaVA), and scoring heads by the average of all gradients is even worse than no filtering. DEX-AR therefore computes, for every (layer \(l\), head \(i\), step \(t\)), the maximum gradient magnitude with respect to visual tokens and to textual tokens, \(S^{l,t,i}_{\text{img}}\) and \(S^{l,t,i}_{\text{text}}\), and uses their difference through a ReLU as the head's weight:

\[w^{l,t,i}=\left(\max_j\left|\nabla A^{l,t}_{-1,v}[i,j]\right|-\max_j\left|\nabla A^{l,t}_{-1,\text{text}}[i,j]\right|\right)^{+}\]

The ReLU zeroes out any head that looks at text more than at the image, so no threshold has to be chosen and no retention ratio has to be tuned — the strength of the filtering is determined entirely by the model's own gradients. Using a maximum rather than a mean is the key detail: averaged over its spatial extent a small object (a tennis ball) is swamped by a large region (the sky), whereas the maximum captures the strongest visual evidence regardless of the object's size, which makes localization more accurate across objects of different scales (the paper's Sec. 4.5 gives the controlled comparison). The per-token heatmap is the weighted sum over all heads and layers, reshaped back onto the 2D grid and min-max normalized:

\[\bar{E}^{(t)}=\sum_{l=1}^{L}\sum_{i=1}^{h}w^{l,t,i}\cdot\nabla A^{l,t}_{-1,v}[i,:]\in\mathbb{R}^{N},\qquad E^{(t)}=\texttt{Norm}\!\left(\texttt{Reshape}\!\left(\bar{E}^{(t)}\right)\right)\in\mathbb{R}^{W\times H}\]

Because \(w^{l,t,i}\) depends on the generation step \(t\), the same head can carry completely different weight for different tokens — the first sense in which the method is "dynamic."

3. Sequence-level token filtering: dropping purely linguistic tokens out of the aggregation

Averaging all tokens into a sequence-level heatmap implicitly assumes every word depends on the image equally, yet the, is, and wearing are predictable from linguistic priors alone. DEX-AR reads gradients as a measure of prediction sensitivity: the larger the gradient magnitude of a feature, the more the model's confidence in the current token degrades when that feature is perturbed. It therefore defines a token's "visual necessity" with exactly the same visual-minus-textual maximum gradient difference used for heads — except that this time the difference is taken over the maximum across layers and heads rather than summed, because the question here is whether the token has at least one sufficiently strong chain of visual evidence: a single pathway depending on the image is enough to rule out the claim that the token is purely linguistic.

\[\delta^t=\left(\max_{l,i}S^{l,t,i}_{\text{img}}-\max_{l,i}S^{l,t,i}_{\text{text}}\right)^{+},\qquad \bar{E}=\sum_{t=1}^{T_a}\delta^t\,E^{(t)}\in\mathbb{R}^{N}\]

Tokens with \(\delta^t=0\) are removed from the aggregation entirely — words whose textual sensitivity is at least their visual sensitivity contribute nothing to the final heatmap. This is the second sense of "dynamic": the weighting coefficient varies with the generation step and is determined by the model's own gradients. The effect is direct: on PascalVOC-QA, enabling head filtering alone lifts SNR from 9.16 to 16.84, enabling token filtering alone lifts it to 89.29, and both together reach 96.12. Token-level filtering is thus the workhorse of the dual filtering, with head filtering acting as the cleaner that shuts down noisy pathways and hands token filtering a cleaner input.

A Worked Example

Take one image, the prompt "What is the content of the image?", and the generated answer "The image features a dog and a cat sitting together in a grassy field." The original Fig. 1 annotates the per-token weights for this very sentence (⚠️ transcribed from the figure; tokenization boundaries may differ, refer to the original paper): the content words dog (5.9), cat (5.2), grass(y) (4.1), and sitting (3.5) carry almost all the mass, together only 0.4, and The, image, features, a, and, in are all exactly 0. When the sequence-level heatmap is aggregated, the two words dog and cat contribute the overwhelming majority of the weight, articles and prepositions are zeroed out, and the template phrase "The image features…" leaves no response on the map at all.

Those weights come from the head-filtering stage. Take one head at one layer (the numbers here are illustrative, only to show the mechanism): if its maximum gradient magnitude is 0.8 on visual tokens and 0.2 on textual tokens, its weight at this step is 0.6 and its gradient map enters \(\bar{E}^{(t)}\) at strength 0.6. A head that mainly watches the preceding text, with 0.5 on the textual side and only 0.1 on the visual side, is zeroed by the ReLU and its gradient map takes no part in this step's explanation. Fig. 1 of the paper contrasts "w/ head filtering" against "w/o head filtering" and "w/ token filtering" against "w/o token filtering": turning head filtering off lets speckle-like noise appear on the map, and turning token filtering off brings back the background response caused by the template sentence — each filter owns a different kind of noise, and neither is dispensable.

Key Experimental Results

Main Results

The perturbation protocol replaces the top-\(p\)% pixels of the heatmap (\(p\in\{0\%,10\%,\dots,90\%\}\)) with the dataset mean pixel value and scores the area under the normalized-perplexity curve (AUC): a higher positive-perturbation value means the removed pixels really were critical, while a lower negative-perturbation value means irrelevant pixels were not mistaken for important ones. The table below is redrawn from the paper's Table 1, keeping two representative models; Attention Rollout is absent for LLaVA-1.5 because its attention implementation does not expose the required per-layer maps.

Model Method ImageNet Pos↑ ImageNet Neg↓ VQAv2 Pos↑ VQAv2 Neg↓
LLaVA-1.5 Raw Attention 2.17 1.01 0.88 0.89
LLaVA-1.5 GradCAM 1.43 1.10 0.91 0.77
LLaVA-1.5 CheferCAM 2.06 1.03 0.93 0.78
LLaVA-1.5 Attn×Grad 1.94 1.00 0.90 0.80
LLaVA-1.5 Int.Grad 1.63 1.52 0.91 0.80
LLaVA-1.5 RISE 1.24 0.99 0.92 0.78
LLaVA-1.5 IIA 1.66 0.90 0.92 0.77
LLaVA-1.5 DEX-AR 2.31 0.96 0.93 0.77
BakLLaVA-v1 Raw Attention 11.47 4.36 0.98 0.86
BakLLaVA-v1 Rollout 6.13 3.12 0.95 0.90
BakLLaVA-v1 GradCAM 7.49 4.39 1.01 0.86
BakLLaVA-v1 CheferCAM 11.15 4.07 1.05 0.84
BakLLaVA-v1 Attn×Grad 12.60 3.74 1.06 0.86
BakLLaVA-v1 Int.Grad 13.50 12.77 1.03 0.84
BakLLaVA-v1 RISE 6.39 3.87 1.09 0.86
BakLLaVA-v1 IIA 11.08 3.77 1.02 0.86
BakLLaVA-v1 DEX-AR 18.10 2.48 1.13 0.81

Segmentation-based localization is evaluated on PascalVOC with a simple classification prompt ("Classify the image") against the ground-truth masks of all objects present, so the model must localize several objects at once. The three metrics are complementary: soft-IoU operates directly on continuous attribution maps without thresholding, IoU takes the best of 20 equally spaced thresholds, and EPG measures the share of attribution energy falling inside the object mask.

Model Method soft-IoU↑ IoU↑ EPG↑
LLaVA-1.5 Raw Attention 2.00 19.10 16.00
LLaVA-1.5 GradCAM 10.20 28.90 19.30
LLaVA-1.5 CheferCAM 1.60 21.01 17.10
LLaVA-1.5 Attn×Grad 5.10 24.20 26.60
LLaVA-1.5 DEX-AR 17.70 36.34 27.75
PaliGemma Raw Attention 4.11 20.38 16.55
PaliGemma GradCAM 8.44 22.55 26.95
PaliGemma Attn×Grad 6.55 23.32 23.52
PaliGemma DEX-AR 15.26 23.55 20.43

Ablation Study

The dual-filtering ablation runs on PascalVOC-QA with LLaVA-1.5-7B. That dataset is built automatically from PascalVOC images with templated descriptions (e.g. "I see a {object 1} as well as a {object 2}"), which yields token-level ground truth marking filler words versus content-bearing tokens.

Head filtering Token filtering SNR↑ MSE↓ EPG↑
✗ ✗ 9.16 0.13 44.96
✓ ✗ 16.84 0.10 63.38
✗ ✓ 89.29 0.11 92.50
✓ ✓ 96.12 0.12 95.04

Head-filtering strategy ablation (PascalVOC; no filtering / maximum gradient only / top-\(k\)% gradients / average of all gradients):

Head filtering strategy LLaVA SNR↑ LLaVA MSE↓ BakLLaVA SNR↑ BakLLaVA MSE↓
no filtering 1.64 0.33 5.27 0.15
max (ours) 3.64 0.22 6.02 0.14
top 5% 3.35 0.24 4.13 0.17
top 50% 1.45 0.34 1.62 0.27
avg (mean of all gradients) 1.09 0.30 1.05 0.30

Key Findings

  • Token-level filtering is the workhorse; head filtering clears the way. Token filtering alone lifts SNR from 9.16 to 89.29, head filtering alone only to 16.84, and combining them reaches 96.12 — the noise they remove barely overlaps, so they are complementary rather than redundant. Note though that MSE does not improve monotonically (0.13 → 0.10/0.11 → 0.12); "dual filtering is uniformly better" holds only for SNR and EPG.
  • Visual information is highly concentrated, so selectivity beats coverage. Stepping down from max to top 5%, to top 50%, to averaging all gradients degrades SNR monotonically, with the average ending up worse than no filtering at all (1.64 → 1.09). The heads being averaged in are essentially noise sources, which is why the simplest max strategy — keeping only the single strongest difference — works best.
  • Positive perturbation gains are clear; negative-perturbation gains are not stable. On ImageNet, BakLLaVA's AUC exceeds Attn×Grad by 5.5 (18.10 vs 12.60), but its negative score is only 2.48, and on PaliGemma 0.90 trails GradCAM's 0.87; on VQAv2, BakLLaVA's 1.13 is closely followed by RISE's 1.09. This is the same phenomenon as the segmentation picture — a large soft-IoU win but an EPG loss to GradCAM on PaliGemma: DEX-AR's maps cover more of the object and follow its outline better, but they are not the narrow, peaky maps that EPG and negative perturbation reward.
  • Absolute localization accuracy remains limited. The best IoU is only 36.34, so attribution maps are far from segmentation masks; the authors frame segmentation explicitly as a spatial sanity check, with perturbation-based faithfulness as the primary evaluation.
  • Speed is the practical differentiator. One ImageNet image takes DEX-AR 0.71 seconds versus 6.20 / 8.20 / 11.8 / 15.3 seconds for CheferCAM / Int.Grad / RISE / IIA — an 8–21× gap — and the batched variant is another 18× faster at \(T_a=100\). For real use cases such as short-answer VQA (5–10 tokens) and classification (1–3 tokens), that cost is what makes the method usable at all.
  • Consistency across architectures. DEX-AR leads on decoder-only (LLaVA-1.5, BakLLaVA), prefix-decoder (PaliGemma), and encoder-decoder (Florence-2) models alike, indicating the gain comes from the mechanism rather than from one implementation detail.

Highlights & Insights

  • Gradients with respect to attention maps, not to inputs or hidden states. This choice is what makes the method architecture-agnostic: any model exposing attention maps works, with no change to the formulation, whereas Grad-CAM needs convolutional feature maps and RISE needs repeated forward passes.
  • The "visual minus textual maximum gradient difference" filter is zero-hyper-parameter and intuitively grounded. No threshold, no retention ratio, no trainable weights, and the sign of the difference maps naturally onto "does this head/token depend more on the image?" Using max rather than mean has a concrete justification too — size invariance — which encodes a specific pain point (localization degrading with object size) directly into the scoring function.
  • Reinterpreting gradients as "visual necessity" yields a free reusable intermediate. \(\delta^t\) is a costless per-token visual-dependence score: it can serve as a hallucination detector (tokens with near-zero \(\delta^t\) generated with high confidence are exactly where a language prior is inventing content) or as a decoding-time gate (lower the sampling temperature, or force another look at the image, when visual dependence is low).
  • The disjoint-row batching argument generalizes beyond DEX-AR. Any per-token attribution over a causal LM can reuse it: sum the logits of the target positions, run one backward pass per layer, and recover mathematically identical per-token gradients — a cheap, general optimization.
  • The evaluation protocol is itself a contribution. Normalized-perplexity AUC replaces "ask another LLM to judge" or manual grading, avoiding external-model bias and producing a continuous rather than binary score; PascalVOC-QA automates filler-versus-content ground truth through templated sentences, making token-level filtering quantitatively measurable for the first time.

Limitations & Future Work

  • Heavy white-box requirements. The method needs per-layer attention maps and back-propagation. A telling side note is that even Attention Rollout cannot run on LLaVA-1.5 because its attention implementation does not expose per-layer maps; for API-only VLMs or heavily fused inference paths (e.g. FlashAttention), the engineering obstacles for DEX-AR would only be larger.
  • Evaluation is limited to older, smaller models. All four VLMs are 7B-class and relatively early (LLaVA-1.5, BakLLaVA, PaliGemma, Florence-2); there is no Qwen2.5-VL, InternVL, or GPT-4o-class model, and the full comparison against TAM, the closest competitor, is deferred to the supplementary material.
  • Applicability to open-ended generation is questionable. Normalized perplexity needs reference answers (VQAv2 has ground truth), so the metric is unavailable for free-form captions or open dialogue; PascalVOC-QA's filler ground truth comes from a fixed template, whereas in free generation the boundary is far blurrier — hallucinated objects, figurative phrasing, and invented details are all hard to classify as filler or content.
  • The hard ReLU cut can kill the wrong tokens. \(\delta^t\) zeroes any token whose textual sensitivity is at least its visual sensitivity, yet strong textual evidence does not mean the image was unused (a model may first confirm the category name from the image and then organize the sentence linguistically). Both filters rest on maximum gradients, making them sensitive to a single outlying large gradient.
  • The perturbation protocol has artifacts of its own. Replacing top-\(p\) pixels with the dataset mean introduces out-of-distribution regions, and under negative perturbation this can easily push the "important region" onto the background; evaluating both directions mitigates but does not remove the problem.
  • Directions worth pursuing: soften or even learn/amortize \(\delta^t\) and the head weights to avoid hard cutoff; use \(\delta^t\) as a decoding-time hallucination gate, turning explanation into intervention; generalize the batching trick to per-token attribution for any causal LLM; and on video/audio autoregressive VLMs, let token filtering extend naturally into temporal localization along the time axis.
  • vs Grad-CAM / CheferCAM / Attn×Grad: all belong to the "gradient × attention" family, but Grad-CAM targets convolutional feature maps and classification logits, while CheferCAM / Attn×Grad are designed for contrastive models and encoder-decoders and reweight static attention per layer. DEX-AR moves the attribution target to each layer's intermediate logits (logit lens) and binds it to a specific generation step, reaching 1.7–11× their soft-IoU (17.70 vs 10.20 / 1.60 / 5.10); yet it loses EPG to GradCAM on PaliGemma, showing that "broader maps" and "peakier maps" are different objectives and no single metric settles the comparison.
  • vs Attention Rollout: Rollout aggregates attention weights directly, assumes attention is additive, and uses no gradients; DEX-AR uses gradients to weight attention by sensitivity. Rollout trails on nearly every metric here and cannot even run on LLaVA-1.5 because the required per-layer maps are not exposed.
  • vs TAM: TAM mitigates context interference with static visual features and post-hoc statistical causal estimation, making it the most direct competitor; DEX-AR differs by recomputing hierarchical gradients at every generation step, capturing step-varying attention dynamics, while its batched implementation keeps cost at 1/8 to 1/21 of IIA / RISE / Int.Grad.
  • vs RISE / Integrated Gradients: both are model-agnostic perturbation or axiomatic methods that need no attention access; the price is lower accuracy and an order-of-magnitude higher cost (11.8 / 8.2 seconds per image), and they explain input pixels without distinguishing generation steps.
  • vs LeGrad: the same first author's earlier work, attributing for contrastive models such as CLIP via feature-formation sensitivity. DEX-AR carries the same family of ideas into autoregressive generation; the genuine increment is the two dynamic axes — generation step and token — plus the accompanying filtering mechanisms.

Rating

  • Novelty: ⭐⭐⭐⭐ The "gradient × attention" family already exists, but adapting it fully to autoregressive VLMs together with a zero-hyper-parameter dual filter — with concrete arguments such as size invariance behind the design — is a clear new contribution.
  • Experimental Thoroughness: ⭐⭐⭐⭐ Four architectures × two perturbation datasets, plus segmentation localization, two filtering ablations, and a runtime analysis, with PascalVOC-QA and the normalized-perplexity metric contributed on top; the gaps are the absence of newer VLMs, unstable negative-perturbation gains, and the TAM comparison being relegated to the supplement.
  • Writing Quality: ⭐⭐⭐⭐ The causal chain from method to ablation is clear (why max instead of mean, which filtering stage produces the SNR gain), but the formula typesetting is broken, one sentence contradicts itself (calling contrastive-model explainability methods "designed for autoregressive generation"), and Table 1's two-column layout makes row labels easy to misread.
  • Value: ⭐⭐⭐⭐ It supplies an immediately usable VLM attribution tool and a reusable evaluation protocol, and intermediate quantities such as \(\delta^t\) and the head weights leave room for secondary uses like hallucination detection and decoding-time gating.