Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignment¶

Conference: ECCV 2026
Paper: ECCV official page · Paper PDF
Code: https://github.com/snumprlab/moras
Area: LLM Safety
Keywords: Multimodal safety, visual attention, risk assessment, activation steering, inference-time alignment

TL;DR¶

MoRAS improves multimodal risk assessment with a short image description, then uses the estimated risk to steer early response activations for the original query toward refusals, reducing LLaVA-1.5-7B's MM-Safety attack success rate from 40.1% to 2.6% while retaining its reported general-task aggregate scores.

Background & Motivation¶

Multimodal large language models inherit text-based safety alignment, but this does not ensure that safety-relevant visual evidence influences their decisions. When an instruction depends on its accompanying image, the model must connect the textual intent to the relevant visual regions before deciding whether to answer or refuse. The paper investigates insufficient attention to these regions rather than a complete inability to recognize image content. Its attention and representation analyses support this explanation, but do not establish it as the sole cause of every multimodal safety failure.

Existing inference-time defenses introduce different costs: safety prompts can push benign requests toward refusals, while checking and revising a complete response adds latency. Activation-steering approaches may also require substantial calibration data, model-specific intervention-strength searches, or directions that transfer poorly beyond the calibration distribution. The paper links these problems through query-level risk estimation: a reliable risk signal could reserve strong interventions for the inputs that need them. The challenge is to exploit a safety-enhanced input for diagnosis without allowing its refusal bias to degrade the final answer on ordinary tasks.

MoRAS first verbalizes important image content while retaining the image itself, making visual evidence easier to connect with the request. It then examines only a short response prefix from this enhanced input and returns to the original query for final generation. Core idea: separate visually informed risk assessment from answer generation, using an enhanced query for the former and a risk-scaled, localized refusal intervention on the original query for the latter.

Method¶

Overall Architecture¶

Multimodal Risk-Adaptive Steering, or MoRAS, takes an image and a text query and produces a model response. The underlying multimodal model remains frozen: the method neither fine-tunes its weights nor adds reinforcement learning. Before deployment, it prepares refusal-behavior prototypes and calibration statistics; at inference time, it performs vision-aware query reformulation, exponentially weighted risk evaluation, and scaled activation steering.

Vision-aware query reformulation uses the same model to generate a concise visual context and combines it with a safety prompt and the original image–text query. Exponentially weighted risk evaluation examines the probability distributions at a few initial response positions, rather than checking a completed answer. Scaled activation steering then restarts generation from the original query and uses the estimated risk to determine how far early hidden states should move toward refusal prototypes. MoRAS therefore reduces reliance on iterative response revision; it does not eliminate additional model computation.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Original image and query"] --> B["Vision-aware<br/>query reformulation"]
    B --> C["Exponentially weighted<br/>risk evaluation"]
    P["Prepared prototypes<br/>and calibration statistics"] --> C
    C -->|"Continuous risk score"| D["Scaled activation<br/>steering"]
    A -->|"Restart from original query"| D
    P --> D
    D --> E["Helpful answer or safe refusal"]

Key Designs¶

1. Vision-aware query reformulation: make relevant visual evidence accessible to safety reasoning

A safety prompt alone tells the model to be cautious without necessarily helping it identify which image regions matter to the request. MoRAS first generates a short image description with the frozen model, then combines that description, a safety prompt, the original text, and the image. The description supplies language-accessible visual cues rather than directly assigning a safety label. This gives the safety instruction concrete visual evidence to act on instead of merely increasing the model's general tendency to refuse. Importantly, the method retains the image rather than replacing multimodal assessment with text-only classification.

Sections 3.1–3.2 hold a textual request fixed while varying the accompanying images and compare cross-modal attention and representation separability under different formulations. The Fisher Discriminant Ratio, or FDR, measures separation of the two groups' activation means relative to their within-group covariance; larger values indicate better separation of safe and unsafe requests. The analysis uses the final input-token activations that determine the first response token, not a separately trained inference-time classifier. Figures 3–4 show limited improvement from safety prompting alone, whereas adding visual context makes the safety prompt more effective. Removing the image reduces separability again, supporting complementarity between the description and the original visual evidence. The cached FDR equation has corrupted matrix notation, so this note explains its meaning without reconstructing the equation or estimating precise values from the plotted curves.

2. Exponentially weighted risk evaluation: convert refusal-like behavior into a continuous control signal

Although the reformulated query exposes risk more clearly, directly using it to answer can still over-refuse because of the added safety prompt. MoRAS therefore treats this branch as a diagnostic probe, measuring how closely its initial response distributions resemble typical refusal behavior. Before deployment, the authors use a small collection of unsafe textual requests to elicit the model's existing refusals and collect last-layer activations at the first few response positions. A separate mean activation is calculated for each position, yielding position-specific prototypes. The paper calls these “unsafe prototypes,” but they represent responses to unsafe requests—specifically refusal behavior—not harmful answers. The main text reports 50 samples for prototype construction; request-source details and alternative-source comparisons are deferred to the supplement, and no unsafe request content is reproduced here.

For an input under assessment, the model generates the first \(N\) response tokens from the reformulated query and maps their activations through its language-model head and softmax. The prototype activations pass through the same head to obtain reference vocabulary distributions. On each side, distributions are weighted by \(\gamma^{n-1}\) and summed across response positions; their cosine similarity gives the score \(S\). This is neither similarity between completed response texts nor a direct cosine comparison of hidden states. Exponential decay prioritizes the beginning of the response, reflecting the authors' observation that refusal behavior is often concentrated there. The main text's efficiency discussion uses the first 3 response tokens, not a full generated safety assessment.

A held-out calibration set supplies the baseline similarity \(S_b\), defined as the mean similarity across its samples. The authors use a random subset from the same calibration source as AutoSteer and report 100 samples for this estimate. They then map similarity to a continuous risk score with a sigmoid centered on that baseline:

\[ r(S)=\sigma\!\left(\alpha(S-S_b)\right). \]

The positive parameter \(\alpha\) controls the mapping's steepness; the main text explains its normalization purpose but leaves detailed selection to the supplement. Similarity above the baseline indicates stronger refusal-like behavior and a larger intervention signal; values below it yield weaker intervention. This score is a control proxy, not a demonstrated, probability-calibrated estimate of actual harmfulness. Algorithm 1 confirms the subtraction and parentheses in the sigmoid despite corruption in the extracted display equation.

3. Scaled activation steering: intervene on the original query only as strongly as the risk requires

After estimating risk, MoRAS discards the diagnostic branch's output and begins generation again from the original image and text. At each affected response position, the steering direction is the corresponding refusal prototype minus the current activation for that original query. Unlike a fixed difference between safe and unsafe group means, this direction depends on the current request's state while retaining a position-specific refusal target. The risk score determines the amount of movement rather than imposing the same intervention magnitude on every input.

\[ \widetilde h^{n}=h^{n}+r(S)\left(\mu_p^{n}-h^{n}\right),\qquad n\leq N. \]

Here \(h^n\) is the original-query branch's last-layer activation and \(\mu_p^n\) is the prototype at the matching response position. This expression combines Equations 5–6; Algorithm 1, lines 22–23, confirms the operations missing from the extracted display formulas. Near-zero risk approximately preserves the original state, near-one risk moves it toward the prototype, and intermediate scores interpolate continuously. The model computes the next-token distribution from the modified activation and continues autoregressive decoding; it is not replacing text after generation. The main method intervenes only at the last layer and only for the first \(N\) response positions, after which ordinary generation resumes. Limiting the intervention and keeping additional safety prompts out of final-answer generation are central to preserving utility on benign requests.

Loss & Training¶

MoRAS adds no training loss and does not update the backbone, but inference-time alignment still requires deployment preparation. Prototype estimation and risk-baseline calibration serve different purposes and should not be conflated into one training dataset. The former defines target refusal states, while the latter maps observed similarity into intervention strength. Visual-context generation, short-prefix assessment, and final generation from the original query are distinct computational steps, not one ordinary forward pass. Exact prompt templates, the decay coefficient, and other implementation settings are deferred to supplementary material that is absent from the local main-paper cache.

Key Experimental Results¶

Main Results¶

The following selection from Table 1 compares methods on the same LLaVA-1.5-7B backbone. ASR is the proportion of evaluated requests judged successful at eliciting unsafe responses by MD-Judge-v0.2-Internlm2; lower is better, while the official MM-Vet aggregate score is higher-is-better. SPA-VL, FigStep, MM-Safety, and VAA are distinct safety benchmarks, so their numerical levels should not be interpreted as interchangeable measures of input severity. The main text refers exact benchmark splits and sample counts to Supplementary Section 3; these results are not relabeled here as full-dataset evaluations.

Method	SPA-VL ASR ↓ (%)	MM-Safety ASR ↓ (%)	VAA ASR ↓ (%)	MM-Vet aggregate ↑
Vanilla	47.2	40.1	43.1	30.5
ETA	17.0	15.8	12.1	30.4
AutoSteer	8.3	37.6	19.8	29.3
MoRAS	7.6	2.6	14.3	30.5

On MM-Safety, MoRAS improves over vanilla by 37.5 percentage points and over ETA by 13.2 percentage points, not relative percentages. It is not the best method on VAA: its 14.3% ASR exceeds ETA's 12.1%, ruling out a claim of superiority on every benchmark. In the same table, FigStep ASR decreases from 59.3% to 2.8%, while GQA and Sci-QA remain at 61.9 and 69.5. Preserved aggregate scores for this backbone do not imply identical outcomes on every example, sub-capability, or model.

Ablation Study¶

Table 2 uses LLaVA-1.5-7B to compare direct answering with the reformulated query against full MoRAS; MM-Vet Total is the paper's weighted aggregate of task scores. This is a combined-stage ablation, not an independent removal study for risk evaluation and activation steering separately.

Configuration	SPA-VL ASR ↓ (%)	MM-Safety ASR ↓ (%)	MM-Vet Total ↑	MM-Vet math ↑
Vanilla	47.2	40.1	30.5	11.5
Vision-aware query reformulation only	7.1	3.5	28.6	7.7
Full three-stage MoRAS	7.6	2.6	30.5	11.5

Reformulation alone substantially reduces ASR but costs 1.9 MM-Vet points. The full method restores those 1.9 points and improves MM-Safety, although SPA-VL ASR rises slightly from 7.1% to 7.6%. Individual Table 2 sub-capabilities move in different directions, so the supported claim is restoration of the aggregate rather than uniformly lossless behavior.

Key Findings¶

The most informative result is the separation of risk detection from final answering: the first stage already identifies substantial risk, while the later stages reduce indiscriminate application of its refusal bias to benign requests.
Utility preservation is not exact across backbones. Table 1 reports LLaVA-OneVision-7B's MM-Vet score changing from 52.8 to 50.8, which warrants “broadly comparable utility,” not universal equality.
Section 4.2 reports 50 prototype samples, 100 baseline-calibration samples, and intervention on the first 3 response tokens; generating the visual description still adds computation.
Detailed throughput breakdowns, hyperparameter sensitivity, and finer component ablations are in the supplement. This note does not infer throughput coordinates or general speedup factors from the poorly extracted Figure 7.

Highlights & Insights¶

Risk assessment need not directly produce the final answer. An enhanced input can serve as an internal diagnostic view whose compact control signal influences a separate original-input branch, reducing persistent prompt interference.
Verbalizing visual evidence is not the same as discarding vision. The description helps focus on relevant evidence while the retained image supplies information absent from the summary; the with-image versus without-image analysis supports that distinction.
Adaptive magnitude complements a query-dependent direction. The risk score controls distance, and the difference from the current activation to the refusal prototype controls direction, rather than imposing identical movement on every input.

Limitations & Future Work¶

Acknowledged by the authors: the method does not eliminate all vulnerabilities and requires continued safety evaluation and mitigation; it is not a deployment safety guarantee.
Reader assessment: both image descriptions and refusal prototypes depend on the backbone's existing capabilities, so errors in a new domain can propagate into the control signal; the main text establishes no universal distribution-shift guarantee.
Reader assessment: access to internal activations is required, preventing direct application through a closed text-only API. Short-prefix intervention also provides no formal guarantee about the safety of an entire long response.
Evidence boundary: automated judging, exact evaluation splits, supplementary over-refusal tests, and parameter sensitivity affect the strength of the conclusions. The main tables contain no confidence intervals, and the missing supplement prevents verification of statistical significance or complete reproduction settings.

Compared with CoCA: CoCA calibrates outputs using distributions with and without safety prompts. MoRAS adds visual context and uses the enhanced query diagnostically rather than requiring the same calibration-style overhead throughout final generation.
Compared with ECSO and ETA: these approaches check and revise completed responses. MoRAS uses a short diagnostic prefix before generating from the original query, reducing dependence on full-output revision without eliminating all extra generation.
Compared with AutoSteer and ASTRA: all intervene on internal activations, but MoRAS combines visually enhanced assessment, small-scale calibration, and query-dependent interpolation toward refusal prototypes. Initialization cost and inference cost must be compared separately.

Rating¶

Novelty: 4/5. A coherent connection between visual-attention analysis and risk-adaptive intervention, building on established prototype and steering ideas.
Experimental Thoroughness: 3.5/5. Multiple backbones, safety benchmarks, and utility tests are reported, but finer ablations and reproduction settings depend on the unavailable supplement.
Writing Quality: 4/5. Motivation, three-stage design, and combined ablation align clearly; local equation extraction complicates reading.
Value: 4/5. A practically relevant defensive design that still needs independent evaluation of out-of-distribution reliability and unnecessary refusals.