Skip to content

CAVD-Guard: Defending Against Token Fragmentation Jailbreaks via Attention Footprints

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/serendipity098/CAVD-Guard
Area: LLM Safety
Keywords: LLM Safety, Jailbreak Defense, Token Fragmentation, Attention Footprints, Canonical Views

TL;DR

Addresses Token Fragmentation Jailbreaks that split harmful keywords into recoverable fragments to evade safety filters by introducing CAVD, a training-free prefill attention footprint detector, paired with CAVD-Guard, a wrapper that routes flagged prompts through tiered deterministic canonical views for guard evaluation, reducing attack success rates to 1.63% while keeping benign false-positive rates below 0.75%.

Background & Motivation

Large language models deployed across user-facing applications require stringent safety alignment to ensure that malicious or dangerous requests are systematically refused. Existing deployment defenses typically operate at two primary intervention points: front-end guard models (such as LLaMA Guard, ShieldLM, and WildGuard) that inspect incoming prompts before they reach the main system, and internal or decoding-time defenses (such as perplexity filtering or safe decoding). However, these screening defenses fundamentally rely on a shared premise: the instruction inspected during safety screening is assumed to expose complete, recognizable harmful keywords or contiguous semantic cues necessary to trigger safety refusal classifiers.

Recent attacks have exposed a critical vulnerability in this assumption through Token Fragmentation Jailbreaks (TFJ). By replacing sensitive keywords with non-contiguous, recoverable fragmentsโ€”such as 2D ASCII art layouts (ArtPrompt, ArtPerception), interleaved emojis (Emoji Attack), redundant non-semantic syllables (Ubbi Dubbi language games), or subword-altering delimitersโ€”attackers decouple safety screening from target model generation. Front-end guard models only observe fragmented tokens and fail to detect the underlying harm, passing the prompt as benign. In contrast, capable target LLMs leverage their strong contextual binding capacity during autoregressive generation to seamlessly reconstruct the hidden keywords and execute the dangerous request.

To close this gap between screening and generation without costly retraining or modifying target generators, this paper investigates the internal mechanics of LLMs processing fragmented text and identifies an intrinsic attention footprint: during prefill, attention mass concentrates intensely among the dispersed keyword fragments, while non-local attention links between these fragments and normal complete words become drastically depleted. Core idea: exploit the structural anomaly of prefill attention void and concentrated anchor links to construct a training-free, length-calibrated detector (CAVD), using it as a reliable routing signal to direct suspicious instructions into tiered deterministic canonical views before safety guard adjudication.

Method

Overall Architecture

CAVD-Guard is a training-free wrapper for guard models that coordinates three sequential stages: prefill attention extraction and void scoring, length-conditioned adaptive routing, and deterministic canonical-view generation with serial guard adjudication. When an instruction arrives, the system extracts the head-averaged causal attention matrix from the terminal layer of the guard model (or a lightweight front-end proxy). It masks out the local diagonal syntactic band and calculates the Attention-Void Ratio (AVR). This score is evaluated against a dynamic, length-conditioned threshold calibrated on benign data. If unflagged (\(z=0\)), the instruction proceeds through standard guard inspection. If flagged (\(z=1\)), the instruction is routed to generate an ordered list of deterministic canonical viewsโ€”spanning cue-specific strict recovery, lenient structural cleanup, and universal normalizationโ€”evaluated by the guard with early-stopping rejection.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Instruction x"] --> B["Prefill Attention Extraction<br/>Terminal-layer head-averaged ยฏA"]
    B --> C["Attention-Void Ratio Computation (AVR)<br/>Exclude local window and prefix keys"]
    C --> D["Length-Conditioned Calibration & Detection (CAVD)<br/>Evaluate against boundary T(L)"]
    D -->|Unflagged z=0| E["Original View Guard Decision G(x)"]
    D -->|Flagged z=1| F["Tiered Deterministic Canonical Views<br/>t0 Strict Recovery โ†’ t1 Structural Cleanup โ†’ t2 Universal Normalization"]
    F --> G["Serial Guard Adjudication & Early Stopping (CAVD-Guard)<br/>D(x) = max G(v)"]
    E --> H["Safe Acceptance / Blocking Decision"]
    G --> H

Key Designs

1. Attention-Void Ratio Computation (AVR): Isolating local syntactic bias to quantify non-local link depletion

Token fragmentation forces the model to allocate excessive attention mass across separated fragment tokens (overwhelming the top 20% strongest links), leaving the vast majority of non-local query-key pairs with abnormally low attention mass (depleting the bottom 30% weakest links). AVR measures this structural footprint by counting non-local attention entries that fall significantly below an instruction-specific reference mean. Formally, given the head-averaged causal attention matrix \(\bar{A}\) at the terminal layer, a local syntactic window \(W_{\text{local}}=5\) along the diagonal is excluded, yielding the non-local region \(F = \{(i, j) : i > j, i - j > W_{\text{local}}\}\). To prevent sequence-initial attention sinks from skewing the baseline, the first \(W_{\text{prefix}}=30\) key columns are excluded to compute the reference mean \(\mu(x) = |C|^{-1} \sum_{(i,j)\in C} \bar{A}_{ij}\) over \(C = \{(i, j) \in F : j \ge W_{\text{prefix}}\}\). The ratio is then defined as:

\[ \text{AVR}(x) = \frac{1}{|F|} \sum_{(i,j)\in F} \mathbf{1}\left[\bar{A}_{ij} < \alpha \mu(x)\right] \]

With \(\alpha=0.1\) by default, AVR counts entries that fall at least one order of magnitude below the reference mean. As a structural routing metric independent of semantic safety labels, AVR generalizes robustly across diverse obfuscation formats.

2. Length-Conditioned Calibration & Detection (CAVD): Neutralizing sequence-length scaling without attack-sample tuning

As sequence length \(L\) expands, quadratic attention geometry naturally multiplies low-intensity background entries, causing benign long instructions to exhibit elevated baseline AVR scores. Using a fixed scalar threshold would inevitably trigger excessive false alarms on long benign prompts or miss attacks on short prompts. CAVD resolves this by fitting a linear boundary \(T(L) = w L + b + k \sigma\) using ordinary least squares regression over XSTEST-CAL, an augmented pool of benign, length-balanced prompts derived from safe XSTest instances. Here, \(\sigma\) represents residual standard deviation and \(k=1\) provides a conservative safety margin:

\[ T(L) = w L + b + k \sigma \]

At inference time, the detector raises an attention-footprint flag \(z(x) = \mathbf{1}[\text{AVR}(x) > T(|x|)]\) in a single prefill forward pass. The entire calibration process requires zero attack samples, zero benign test samples, and zero gradient updates, achieving parameter-free, data-leak-free detection.

3. Tiered Deterministic Canonical Views & Serial Early Stopping: Structured recovery across multi-level primitives

When CAVD flags an instruction (\(z(x)=1\)), the system avoids blind rejection and instead routes the input to a deterministic reconstruction pipeline that exposes candidate views to the safety guard. Canonicalization generates candidate views across three ordered tiers: - t0 (Cue-specific strict recovery): Reconstructs contiguous text from 2D character grids (ASCII art), decodes rule-based recurring patterns (e.g. stripping Ubbi Dubbi syllable insertions), and isolates high-risk payload islands fenced inside benign wrapper templates; - t1 (Lenient structural cleanup): Removes tokenization-disrupting non-semantic characters including emojis, zero-width joiners, and pictographic separators, compresses repeated punctuation and blank lines, and applies bounded character-edit repair using a fixed confusables dictionary; - t2 (Universal layout normalization): Applies standard Unicode NFKC normalization, whitespace flattening, and trailing whitespace trimming.

The guard evaluates candidate views \(U^+(x) = [C_1(x), \dots, C_K(x)]\) strictly in priority order \(t_0 \to t_1 \to t_2\), stopping immediately upon encountering the first unsafe classification (\(G(v)=1\)). This serial early-stopping design avoids redundant guard invocations while ensuring broad coverage against adversarial variations.

Loss & Training

CAVD and CAVD-Guard are completely training-free and require no fine-tuning or parameter updates on either target LLMs or guard models. The default hyperparameters (\(W_{\text{local}}=5\), \(W_{\text{prefix}}=30\), \(\alpha=0.1\), \(k=1\)) were chosen from a broad stable empirical sensitivity region and remained frozen across all models, guard architectures, and attack datasets. Furthermore, the attention footprint transfers effectively across model families: a compact proxy model (Qwen3-0.6B) can compute the CAVD flag in just 17.18 ms, reducing latency to 12.1% of a single downstream guard inference pass.

Key Experimental Results

Main Results

The main defense evaluation tests six general-purpose LLMs (Open-source: Llama-3-8B, Mistral-7B, Qwen3-8B, Gemma-2-9B; Closed-source: DeepSeek-V3.2, GPT-4o-mini) against four token-fragmentation attack datasets: ArtPrompt (ARTP), ArtPerception (ARTPERC), Emoji Attack (EMOJI), and Ubbi Game (UBBI). Post-defense Attack Success Rate (ASR %) and Harmfulness Severity (HS, 0โ€“5 scale) are judged automatically by Qwen-Plus (temperature 0.0), where ASR counts responses with harmfulness \(\ge 3\).

The table below reports post-defense ASR (%) across target models and the cross-target average:

Dataset Defense Method Llama-3-8B Mistral-7B Qwen3-8B Gemma-2-9B DeepSeek-V3.2 GPT-4o-mini Avg. ASR (%)
ARTP No Defense 18.01 63.99 58.03 31.99 41.97 10.94 37.49
PPL Filter 23.96 68.01 54.99 31.02 โ€“ โ€“ 44.50
ABD Defense 17.04 62.05 57.06 32.96 โ€“ โ€“ 42.28
LLAMA GUARD 12.05 41.97 32.96 21.05 31.99 10.94 25.16
+ CAVD-Guard 0.00 0.00 0.00 0.00 0.00 0.00 0.00
SHIELDLM + CAVD-Guard 0.00 0.00 0.00 0.00 0.00 0.00 0.00
WILDGUARD + CAVD-Guard 0.97 0.97 0.00 0.00 0.00 0.00 0.32
ARTPERC No Defense 78.67 68.00 58.67 46.67 62.67 76.00 65.11
LLAMA GUARD 72.00 76.00 61.33 52.00 62.00 78.67 67.00
+ CAVD-Guard 0.00 0.00 0.00 0.00 0.00 0.00 0.00
SHIELDLM + CAVD-Guard 0.00 0.00 0.00 0.00 0.00 0.00 0.00
WILDGUARD + CAVD-Guard 0.00 0.00 0.00 0.00 0.00 0.00 0.00
EMOJI No Defense 70.96 70.00 86.92 5.96 1.92 5.00 40.13
LLAMA GUARD 19.04 15.00 25.00 4.04 0.96 4.04 11.35
+ CAVD-Guard 1.92 1.92 1.92 0.96 0.00 0.00 1.12
SHIELDLM + CAVD-Guard 0.00 0.00 0.00 0.00 0.00 0.00 0.00
WILDGUARD + CAVD-Guard 0.00 0.00 0.00 0.00 0.00 0.00 0.00
UBBI No Defense 66.03 89.02 59.93 56.97 86.93 71.95 71.81
LLAMA GUARD 62.02 86.06 48.95 64.98 81.01 73.00 69.34
+ CAVD-Guard 18.12 19.16 13.07 16.03 17.94 18.99 17.22
SHIELDLM + CAVD-Guard 0.00 0.00 0.00 0.00 0.00 0.00 0.00
WILDGUARD + CAVD-Guard 0.87 1.05 0.52 0.87 1.05 1.05 0.90

Across all evaluated target models and guard combinations, CAVD-Guard reduces the average post-defense ASR to 1.63%. ShieldLM and WildGuard wrappers achieve complete 0.00% ASR across ARTP and ARTPERC, and WildGuard + CAVD-Guard curbs the severe 71.81% unmitigated attack success rate on UBBI down to 0.90%.

Ablation Study

A component ablation on a balanced 800-instruction workload (100 each from AdvBench and HarmBench, 50 each from four fragmentation attack sets, and 200 each from Dolly and Code-Ins) decouples the detection and recovery mechanisms. It compares a heuristic surface-feature filter (Surface-only), standalone guard classification (Guard-only), standalone attention-footprint blocking (CAVD-only), exhaustive canonicalization (Always-CV Guard), and selective routing (CAVD-Guard).

The table below summarizes Block Rates (BR %) on non-fragmented and fragmented attacks, False Positive Rates (FPR %) on benign inputs, and downstream guard invocation counts:

Guard Architecture Defense Method Non-Frag. BR (%) โ†‘ Frag. Attack BR (%) โ†‘ Benign FPR (%) โ†“ Net Gain (Avg. BR - FPR) โ†‘ Downstream Guard Calls โ†“
Model-free Surface-only (Heuristic) 0.00 100.00 34.50 32.17 โ€“
LLAMA GUARD Guard-only (Standard) 82.50 28.00 0.00 46.17 800
CAVD-only (Direct Block) 0.00 100.00 6.50 60.17 โ€“
Always-CV Guard (All Inputs) 83.50 93.00 0.50 89.33 2,903
CAVD-Guard (Route+Reclassify) 82.50 93.00 0.25 89.25 1,067
SHIELDLM Guard-only (Standard) 98.00 49.00 0.50 64.83 800
CAVD-only (Direct Block) 0.00 100.00 10.25 56.42 โ€“
Always-CV Guard (All Inputs) 100.00 99.00 2.00 97.33 2,622
CAVD-Guard (Route+Reclassify) 98.00 99.00 0.75 97.92 1,021
WILDGUARD Guard-only (Standard) 99.50 58.50 0.25 71.92 800
CAVD-only (Direct Block) 1.50 100.00 31.50 35.67 โ€“
Always-CV Guard (All Inputs) 99.50 97.50 0.25 97.92 2,656
CAVD-Guard (Route+Reclassify) 99.50 97.00 0.25 97.58 1,395

Key Findings

  • Conventional guard models fail under token fragmentation: Standalone guard models exhibit high vulnerability to fragmentation, blocking only 28.00% (LLaMA Guard), 49.00% (ShieldLM), and 58.50% (WildGuard) of fragmented attacks, despite achieving 82.5%โ€“99.5% block rates on standard non-fragmented prompts.
  • Direct threshold blocking creates unacceptable false alarms: Standalone surface-feature rules and direct CAVD threshold-blocking trigger on 100% of attacks, but produce prohibitive false positive rates of 6.50%โ€“34.50% on technical benign datasets like Code-Ins. Routing flagged instructions through canonical views for guard verification suppresses benign FPR to 0.25%โ€“0.75%.
  • Selective routing preserves defense while cutting calls by 57.4%: Compared to applying canonical views unconditionally to all incoming prompts (Always-CV Guard), CAVD-Guard maintains virtually identical defense capability (96.33% vs 96.50% average fragmented BR) while slashing downstream guard invocations from 2,727 to 1,161 on average.
  • Proxy model acceleration maintains full detection accuracy: Computing the attention-footprint flag with an ultra-lightweight Qwen3-0.6B proxy achieves 100% attack trigger coverage at an inference latency of only 17.18 ms, cutting computation cost to 12.1% of a standard guard pass.

Highlights & Insights

  • From surface matching to internal attention dynamics: Highlights that the fundamental process of subword/character reconstruction leaves an inescapable mechanistic footprint in prefill attentionโ€”namely, hyper-concentrated fragment clusters paired with non-local voidโ€”providing an internal signal immune to superficial string mutations.
  • Decoupling anomaly detection from safety adjudication: Treating the detector as a confidence router rather than an executioner decouples "determining when an input is too obfuscated for direct inspection" from "determining whether the restored semantic intent violates safety policies," reconciling sensitivity with low false positives.
  • Deterministic and auditable recovery primitives: Employs rule-based transformations rather than generative LLM rewrites, eliminating hallucination risks, secondary prompt injection vectors, and uncontrollable latency overheads.

Limitations & Future Work

  • Requirement for accessible prefill attention weights: Calculating AVR requires internal attention weights from the terminal layer, constraining direct native deployment to open-weight or instrumented guard models; black-box API setups necessitate front-end proxy models.
  • Specialized scope bounded to fragmentation attacks: The defense specifically targets attacks that dismantle keywords into recoverable spatial or character pieces; it is not designed to counter role-playing, persuasion-based jailbreaks, or gradient-based suffix optimization that do not fragment keywords.
  • Over-routing on symbol-dense benign domains: On naturally symbol-heavy formats such as terminal console logs (43.0% trigger) or configuration files (32.0% trigger), CAVD exhibits a 25.17% false routing rate, which induces minor redundant canonical-view inspections despite remaining safe from false blocks.
  • vs Perplexity Filtering (PPL, Alon and Kamfonas, 2023): PPL flags unusual token likelihoods, but overreacts severely to benign structured code and fails against attacks that preserve local token fluency; CAVD inspects non-local relational structure directly.
  • vs Attention-Based Defenses (ABD, Pu et al., 2025): ABD monitors broad attention dispersion across the whole target model, whereas CAVD specifically captures non-local void and fragment-to-fragment anchor concentration with calibrated length boundaries.
  • vs Randomized Perturbation / Smoothing Defenses (SmoothLLM, Robey et al., 2023; Jain et al., 2023): Stochastic character noise degrades prompt fidelity and carries heavy multi-sample inference costs; CAVD-Guard uses targeted, tiered deterministic canonical views with serial early stopping to achieve high efficiency.

Rating

  • Novelty: โญโญโญโญโญ Discovers the characteristic attention-void footprint inherent to token fragmentation and builds a calibrated, parameter-free detector.
  • Experimental Thoroughness: โญโญโญโญโญ Comprehensive cross-model validation across 6 target LLMs, 3 guard models, 4 attack families, and extensive ablation benchmarks.
  • Writing Quality: โญโญโญโญโญ Clear, structured exposition supported by rigorous empirical characterization and mechanistic interpretations.
  • Value: โญโญโญโญโญ Delivers an efficient, plug-and-play defense wrapping existing guardrails against an otherwise evasive family of jailbreak attacks.