Skip to content

CAVD-Guard: Defending Against Token Fragmentation Jailbreaks via Attention Verification

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/serendipity098/CAVD-Guard
Area: LLM Safety
Keywords: Jailbreak Defense, Token Fragmentation, Attention Footprint, Attention-Void Ratio, Guard Models

TL;DR

Addressing Token Fragmentation Jailbreaks that bypass safety filters by decomposing harmful keywords into recoverable subword pieces, this paper discovers a distinctive "attention-void" footprint during prefill and proposes CAVD-Guard—a training-free, length-calibrated attention detector paired with deterministic canonical-view routing that slashes attack success rates from 53.61% to 1.63% with near-zero benign false positives.

Background & Motivation

As large language models (LLMs) are widely deployed across real-world interactive systems, safety alignment and robust jailbreak defense have emerged as indispensable guardrails. Current industry deployments and research defenses predominantly rely on front-end guard models (such as Llama Guard, ShieldLM, and WildGuard) to screen inputs and filter out toxic queries before inference, or leverage target-model internal interventions like perplexity thresholding and representation steering. However, nearly all front-end safety classifiers operate on a foundational assumption: the inspected instruction preserves explicit, contiguous harmful keywords or recognizable semantic indicators that allow a safety classifier to trigger a refusal. When attackers split harmful keywords into dispersed pieces using 2D ASCII art (e.g., ArtPrompt, ArtPerception), interspersed emojis (e.g., Emoji Attack), redundant syllable language games (e.g., Ubbi Game), or tokenization-altering characters, standard guard models suffer from "fragmentation blindness" and fail to detect the underlying harm.

This defense failure originates from an asymmetric representation capability between safety screening and generative decoding. During the prefill screening stage, the guard classifier only observes an obfuscated, seemingly disjoint sequence of harmless symbols and characters, and thus passes the instruction as benign. In contrast, during the target model's auto-regressive generation stage, the model's powerful contextual reconstruction capabilities easily bridge non-contiguous fragments, assemble the scattered symbols back into complete harmful concepts, and faithfully fulfill the malicious intent. Because these fragmentation transformations leave high-level harmful intent intact while evading generic perplexity filters and surface-pattern heuristics, existing defense wrappers struggle to balance attack blocking rates against the false-positive penalty on symbol-dense benign user queries.

To bridge this operational vulnerability, this paper investigates whether instructions containing recoverable keyword fragments leave an internal, measurable footprint during LLM prefill. Through comprehensive analyses of prefill attention matrices across open-weight backbones and guard models, the authors discover that fragmented inputs induce an unmistakable attention anomaly: attention mass concentrates tightly within the fragmented token cluster, while non-local attention connections connecting those fragments to regular complete-word context become extraordinarily sparse and weak. Core idea: measure this prefill terminal-layer "attention-void" footprint via a training-free, length-calibrated detector (CAVD), and wrap guard models with deterministic canonical-view routing so that flagged inputs are re-inspected under normalized and reconstructed views without retraining or modifying downstream generators.

Method

Overall Architecture

CAVD-Guard is a training-free external defense wrapper designed for existing guard models, operating without fine-tuning downstream target models or collecting attack samples. The pipeline proceeds in two coordinated stages: First, Calibrated Attention-Void Detection (CAVD) inspects the terminal-layer prefill causal attention matrix of the guard model. After filtering out the local syntactic diagonal band and prefix key sink columns, CAVD computes the Attention-Void Ratio (AVR)—the fraction of non-local connections falling an order of magnitude below the instruction's reference mean—and compares it against a dynamic, length-conditioned threshold \(T(L)\) calibrated on benign samples. Second, Deterministic Canonical Views Routing handles decision dispatching: unflagged inputs (\(z(x)=0\)) proceed through the standard guard classification \(G(x)\), while flagged inputs (\(z(x)=1\)) generate a prioritized list of canonicalized recovery views for early-stopping guard evaluation, blocking the request if any reconstructed view violates safety guidelines.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    IN["Input User Instruction x"] --> D1["Attention-Void Ratio Measurement<br/>Strip local window and prefix keys to quantify weak-link ratio AVR"]
    D1 --> D2["Length-Conditioned Benign Boundary Calibration<br/>Evaluate against dynamic threshold T(L) to produce flag z(x)"]
    D2 -->|Normal z(x)=0| GUARD_ORIG["Original View Safety Screening<br/>Guard directly evaluates original surface G(x)"]
    D2 -->|Flagged z(x)=1| D3["Tiered Deterministic Canonical-View Routing<br/>Sequentially generate t0/t1/t2 canonical views with early stopping"]
    GUARD_ORIG --> OUT["Accept / Block Instruction"]
    D3 --> OUT

Key Designs

1. Attention-Void Ratio Measurement: Non-contiguous keyword fragments cause anomalous sparsity -> Strip local syntactic and sink keys to isolate non-local weak links -> Training-free extraction of token-fragmentation footprints To quantify the internal representation anomaly caused by fragmented tokens, CAVD extracts the terminal-layer head-averaged causal attention matrix \(\bar{A}\) during the prefill phase. Because adjacent-token syntactic binding clusters near the main diagonal and sequence-initial tokens act as "attention sinks" absorbing high weights irrespective of semantics, CAVD explicitly removes these confounding connections. It isolates a measured non-local region \(F = \{(i, j) : i > j, i - j > W_{\text{local}}\}\) (default window \(W_{\text{local}}=5\)) and calculates the instruction-level reference mean \(\mu(x)\) over an inner reference set \(C = \{(i, j) \in F : j \ge W_{\text{prefix}}\}\) (default prefix exclusion \(W_{\text{prefix}}=30\)): $\(\mu(x) = \frac{1}{|C|} \sum_{(i,j) \in C} \bar{A}_{ij}\)$ The Attention-Void Ratio (AVR) is formally defined as the proportion of non-local pairs whose attention weight falls at least one order of magnitude below \(\mu(x)\) (\(\alpha=0.1\)): $\(\text{AVR}(x) = \frac{1}{|F|} \sum_{(i,j) \in F} \mathbb{I}\left[\bar{A}_{ij} < \alpha \mu(x)\right]\)$ When an instruction contains fragmented ASCII art, repeated syllables, or interspersed emojis, attention links concentrate heavily among the mutually reconstructive fragment tokens, leaving the vast majority of non-local contextual token pairs exceptionally depleted and driving AVR to abnormally high levels.

2. Length-Conditioned Benign Boundary Calibration: Natural attention dilution across long sequences causes false positives -> Regress benign AVR on sequence length with a safety margin -> Zero-attack unsupervised adaptive decision boundary Empirical profiling reveals that AVR naturally scales with instruction length \(L\) even on benign prompts, as longer sequences inherently contain more weakly related non-local token pairs. A static scalar threshold would inevitably trigger massive false alarms on long benign requests. To address this, CAVD calibrates a model-specific dynamic threshold using XSTEST-CAL, an augmented pool of 770 safe-labeled benign instructions spanning diverse length brackets (from under 20 to over 1,600 tokens). Fitting a linear regression on benign AVR versus sequence length yields the dynamic decision boundary: $\(T(L) = w L + b + k \sigma\)$ where \(w\) and \(b\) represent the fitted regression slope and intercept, \(\sigma\) is the residual standard deviation, and \(k\) is a safety margin multiplier (fixed at \(k=1\)). At runtime, CAVD requires only a single prefill pass; an attention-footprint flag \(z(x) = \mathbb{I}[\text{AVR}(x) > T(|x|)]\) is raised whenever an instruction crosses this boundary, signaling that the surface text is untrustworthy for direct safety screening.

3. Tiered Deterministic Canonical-View Routing: Direct blocking causes benign formatting over-killing -> Reconstruct views via structured decoding, cleanup, and normalization with early stopping -> Minimal guard inference overhead and negligible false-positive rate Treating a CAVD flag as an immediate block would severely penalize benign symbol-dense prompts like Markdown tables or source code. CAVD-Guard therefore decouples detection from blocking, using \(z(x)\) strictly as a routing trigger to construct an ordered canonical-view list \(U^+(x) = [C_1(x), \dots, C_K(x)]\). The canonical views are empirical, deterministic recovery primitives organized into three fixed tiers: - Cue-specific strict recovery (\(t_0\)): Parses regular spatial grids, fills masked-slot letters (e.g., ASCII grids), decodes recurring fragmentation patterns (e.g., Ubbi Dubbi rule-based syllable removal), and isolates fenced or quoted high-risk payload islands; - Lenient structural cleanup (\(t_1\)): Strips inline emojis, zero-width spaces, and pictographic separators, compresses repeated punctuation and abnormal spacing, and repairs bounded single-character edit/swap perturbations; - Universal normalization (\(t_2\)): Standardizes Unicode (NFKC compatibility), flattens whitespace, and trims irregular lines. The downstream guard model inspects views in the fixed priority order \(t_0 \to t_1 \to t_2\), evaluating \(G(v) \in \{0, 1\}\) and halting at the first detected violation. The overall decision is \(D(x) = \max_{v \in U^+(x)} G(v)\) when flagged, and \(D(x) = G(x)\) otherwise. This cascading early-stopping mechanism exposes reconstructed keywords to the guard while avoiding unnecessary forward passes and preserving high benign utility.

Benchmark & Evaluation Protocol

The evaluation covers 4 primary Token Fragmentation Jailbreak benchmarks: 2D ASCII-art datasets ARTPROMPT (722 prompts) and ARTPERCEPTION (150 prompts), as well as interspersed symbol benchmarks EMOJI ATTACK (520 prompts) and UBBI GAME (574 prompts). Non-fragmented attacks AdvBench (520 prompts) and HarmBench (200 prompts) serve as standard adversarial baselines, while DOLLY (15,011 prompts) and CODE-INS (30,000 prompts) evaluate benign utility. Evaluated target generators include 4 open-source LLMs (Llama-3-8B, Mistral-7B, Qwen3-8B, Gemma-2-9B) and 2 proprietary models (DeepSeek-V3.2, GPT-4o-mini). Safety evaluation is judged by Qwen-Plus on a 0–5 scale, where Harmfulness Severity (HS) is the average score and Attack Success Rate (ASR) measures the percentage of responses scoring \(\ge 3\). Human validation on 100 sample pairs confirmed 93.0% agreement with the automatic judge (Cohen's \(\kappa = 0.84\)).

Key Experimental Results

Main Results

Table 1 demonstrates that in the absence of defense, token fragmentation attacks achieve alarming attack success rates across all target models (averaging 37.49% to 71.81%). Non-guard baselines (PPL, ABD) struggle because they do not reconstruct the concealed keywords. Standard guard models evaluate fragmented surfaces directly and suffer catastrophic bypasses (e.g., Llama Guard achieves 67.00% ASR on ARTPERCEPTION). Wrapping these guards with CAVD-Guard drives downstream attack success rates down to near zero across almost all settings, lowering overall average ASR from 53.61% to 1.63%.

Dataset Defense Method Llama-3 ASR(%) / HS Mistral ASR(%) / HS Qwen3 ASR(%) / HS Gemma-2 ASR(%) / HS DeepSeek-V3.2 ASR(%) / HS GPT-4o-mini ASR(%) / HS Avg. ASR(%) / HS
ARTP No Defense 18.01 / 1.59 63.99 / 3.32 58.03 / 2.77 31.99 / 2.14 41.97 / 2.37 10.94 / 1.38 37.49 / 2.26
(ASCII Art) Llama Guard 12.05 / 1.18 41.97 / 2.20 32.96 / 1.79 21.05 / 1.49 31.99 / 1.63 10.94 / 1.11 25.16 / 1.57
+ CAVD-Guard 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00
ShieldLM 1.94 / 0.10 5.96 / 0.26 4.02 / 0.16 1.94 / 0.16 4.99 / 0.17 0.97 / 0.10 3.30 / 0.16
+ CAVD-Guard 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00
WildGuard 0.97 / 0.05 1.94 / 0.06 0.00 / 0.04 0.00 / 0.04 0.00 / 0.03 0.00 / 0.02 0.49 / 0.04
+ CAVD-Guard 0.97 / 0.05 0.97 / 0.05 0.00 / 0.04 0.00 / 0.04 0.00 / 0.03 0.00 / 0.02 0.32 / 0.04
ARTPERC No Defense 78.67 / 3.57 68.00 / 3.39 58.67 / 2.80 46.67 / 2.61 62.67 / 3.09 76.00 / 3.57 65.11 / 3.17
(Perceptual ASCII) Llama Guard 72.00 / 3.53 76.00 / 3.54 61.33 / 2.86 52.00 / 2.78 62.00 / 2.94 78.67 / 3.59 67.00 / 3.21
+ CAVD-Guard 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00
ShieldLM 73.33 / 3.26 68.00 / 3.22 50.67 / 2.48 50.00 / 2.48 54.00 / 2.66 68.00 / 3.05 60.67 / 2.86
+ CAVD-Guard 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00
WildGuard 18.67 / 0.90 18.67 / 0.89 16.00 / 0.78 12.00 / 0.66 18.67 / 0.90 21.33 / 0.90 17.56 / 0.84
+ CAVD-Guard 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00
EMOJI No Defense 70.96 / 3.63 70.00 / 3.67 86.92 / 3.87 5.96 / 1.24 1.92 / 1.09 5.00 / 1.20 40.13 / 2.45
(Emoji Insertion) Llama Guard 19.04 / 0.94 15.00 / 0.86 25.00 / 1.10 4.04 / 0.42 0.96 / 0.30 4.04 / 0.40 11.35 / 0.67
+ CAVD-Guard 1.92 / 0.08 1.92 / 0.10 1.92 / 0.08 0.96 / 0.06 0.00 / 0.02 0.00 / 0.03 1.12 / 0.06
ShieldLM 69.04 / 3.50 68.08 / 3.62 85.00 / 3.89 5.96 / 1.20 3.08 / 1.09 4.04 / 1.14 39.20 / 2.41
+ CAVD-Guard 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00
WildGuard 29.04 / 1.29 25.96 / 1.27 28.08 / 1.33 0.00 / 0.32 0.00 / 0.32 3.08 / 0.42 14.36 / 0.83
+ CAVD-Guard 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00
UBBI No Defense 66.03 / 3.29 89.02 / 4.30 59.93 / 2.93 56.97 / 3.05 86.93 / 4.41 71.95 / 3.68 71.81 / 3.61
(Language Game) Llama Guard 62.02 / 3.31 86.06 / 4.13 48.95 / 2.91 64.98 / 3.16 81.01 / 4.20 73.00 / 3.78 69.34 / 3.58
+ CAVD-Guard 18.12 / 0.84 19.16 / 0.90 13.07 / 0.62 16.03 / 0.76 17.94 / 0.88 18.99 / 0.94 17.22 / 0.82
ShieldLM 55.05 / 2.86 85.02 / 4.11 39.02 / 2.48 50.00 / 2.81 74.04 / 3.88 63.07 / 3.45 61.03 / 3.27
+ CAVD-Guard 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00 0.00 / 0.00
WildGuard 67.07 / 3.34 86.06 / 4.23 44.08 / 2.82 55.05 / 3.11 86.93 / 4.42 71.08 / 3.69 68.38 / 3.60
+ CAVD-Guard 0.87 / 0.08 1.05 / 0.09 0.52 / 0.04 0.87 / 0.08 1.05 / 0.09 1.05 / 0.08 0.90 / 0.08

Ablation Study

Table 2 reports the mechanism ablation over an 800-instruction balanced mixture containing non-fragmented malicious requests, fragmented attacks, and benign user queries. CAVD-Guard matches the high blocking capacity of brute-force canonicalization (Always-CV Guard) while cutting downstream guard invocations by over 57%.

Guard Model Defense Variant Non-Fragmented Attack BR (%) ↑
[AdvBench / HarmBench / Avg.]
Token Fragmentation BR (%) ↑
[ARTP / PERC / EMOJI / UBBI / Avg.]
Benign FPR (%) ↓
[DOLLY / CODE-INS / Avg.]
Net Margin (Avg. BR − FPR) ↑ Downstream Guard Calls ↓
Model-free Surface-only 0.00 / 0.00 / 0.00 100.00 / 100.00 / 100.00 / 100.00 / 100.00 10.00 / 59.00 / 34.50 32.17 –
Llama Guard Guard-only 94.00 / 71.00 / 82.50 40.00 / 2.00 / 70.00 / 0.00 / 28.00 0.00 / 0.00 / 0.00 46.17 800
CAVD-only 0.00 / 0.00 / 0.00 100.00 / 100.00 / 100.00 / 100.00 / 100.00 0.00 / 13.00 / 6.50 60.17 –
CAVD-Guard 94.00 / 71.00 / 82.50 100.00 / 98.00 / 94.00 / 80.00 / 93.00 0.00 / 0.50 / 0.25 89.25 1,067
Always-CV Guard 94.00 / 73.00 / 83.50 100.00 / 98.00 / 94.00 / 80.00 / 93.00 0.50 / 0.50 / 0.50 89.33 2,903
ShieldLM Guard-only 100.00 / 96.00 / 98.00 96.00 / 34.00 / 14.00 / 52.00 / 49.00 1.00 / 0.00 / 0.50 64.83 800
CAVD-only 0.00 / 0.00 / 0.00 100.00 / 100.00 / 100.00 / 100.00 / 100.00 1.50 / 19.00 / 10.25 56.42 –
CAVD-Guard 100.00 / 96.00 / 98.00 100.00 / 100.00 / 98.00 / 98.00 / 99.00 1.00 / 0.50 / 0.75 97.92 1,021
Always-CV Guard 100.00 / 100.00 / 100.00 100.00 / 100.00 / 98.00 / 98.00 / 99.00 1.50 / 2.50 / 2.00 97.33 2,622
WildGuard Guard-only 100.00 / 99.00 / 99.50 96.00 / 66.00 / 72.00 / 0.00 / 58.50 0.50 / 0.00 / 0.25 71.92 800
CAVD-only 0.00 / 0.00 / 0.00 100.00 / 100.00 / 100.00 / 100.00 / 100.00 13.50 / 49.50 / 31.50 35.67 –
CAVD-Guard 100.00 / 99.00 / 99.50 100.00 / 100.00 / 98.00 / 90.00 / 97.00 0.50 / 0.00 / 0.25 97.58 1,395
Always-CV Guard 100.00 / 99.00 / 99.50 100.00 / 100.00 / 100.00 / 90.00 / 97.50 0.50 / 0.00 / 0.25 97.92 2,656

Key Findings

  • Stand-alone Detection Direct Blocking Fails on Benign Code: Surface-only rules block all token-fragmentation attacks but produce a catastrophic 59.00% FPR on CODE-INS; similarly, direct blocking based on CAVD flags (CAVD-only) inflicts up to 31.50% FPR on WildGuard. This underscores that an attention footprint signals surface unreliability rather than inherent toxicity, making post-detection canonical routing indispensable.
  • Component Contributions in CAVD: Ablating detector components (Table 9) confirms that replacing the length-conditioned threshold \(T(L)\) with a static ceiling drops attack coverage to 79.91%, while omitting local diagonal masking or prefix-window sinks elevates benign trigger rates from 22.51% to over 25.0%.
  • Ultra-lightweight Front-end Proxy: Using a tiny Qwen3-0.6B proxy model to compute the CAVD flag (Table 3) incurs only 17.18 ms latency per prompt (just 12.1% of a full guard forward pass), maintains 100% attack trigger coverage, and lowers total downstream guard calls to 921, delivering substantial compute savings.

Highlights & Insights

  • Reconstruction Intent as the Root Cause: Rigorous paired experiments with identical ASCII characters (Table 14) show that explicit reconstruction instructions boost attention to the ASCII token zone by up to \(+0.414\), verifying that the footprint reflects active subword reconstruction rather than raw symbolic noise.
  • Training-Free, Zero-Adversarial Calibration: CAVD does not require fine-tuning guards or collecting adversarial samples; its threshold is calibrated exclusively on harmless benign text (XSTEST-CAL), making it inherently resistant to adversarial overfitting.
  • Modular and Extensible View Design: Detection (when to intervene) and normalization (how to transform) are cleanly decoupled; new attack variants (such as reversed text in FlipAttack) can be defended by appending a deterministic reversal primitive to \(t_0\) without retraining the detector.

Limitations & Future Work

  • White-Box Attention Weight Dependency: CAVD requires access to terminal-layer prefill attention weights; deploying it alongside black-box API guardrails requires introducing a local open-weight proxy model (such as Qwen3-0.6B).
  • Scope Limited to Structural Token Fragmentation: The defense is tailor-made for attacks that break keyword contiguity; it does not protect against non-fragmented semantic exploits such as complex role-play, multi-turn deception, or continuous adversarial suffixes.
  • False Routing on Naturally Symbol-Dense Inputs: Stress testing on 600 benign symbol-dense prompts (Table 18) indicates an average 25.17% false routing rate on logs, LaTeX, and terminal outputs, pointing to the need for more context-aware routing filters in future work.
  • vs. ArtPrompt / ArtPerception / Ubbi Game: These attack frameworks established that 2D ASCII art and language games easily penetrate alignment; CAVD-Guard provides the first training-free, dedicated defense by inverting their tokenization-evasion mechanisms.
  • vs. ABD (Attention-based Defense, Pu et al., 2025): While ABD relies on holistic target-model attention shifts and requires white-box target model access, CAVD operates entirely on the guard model and pairs structural void detection with deterministic keyword reconstruction.
  • vs. SmoothLLM / Character Perturbation Filters: Generic character perturbators degrade legibility without restoring semantic structure in 2D ASCII art; CAVD-Guard's tiered canonical views systematically reconstruct masked and formatted tokens back into readable words.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Discovers the distinctive attention-void prefill footprint and presents an elegant decouple-and-route defense.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Evaluates 6 target models, 3 guard models, 4 fragmentation attack families, and extensive ablations with latency profiling.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally structured narrative, crystal-clear motivation, and meticulous mathematical formulation.
  • Value: ⭐⭐⭐⭐⭐ Offers a practical, drop-in, highly cost-effective blueprint for securing production LLM guardrails against token fragmentation attacks.