Recoverability-Controlled Safety Miscalibration in Large Language Models¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/cynic567/RiskCueBench
Area: LLM Safety
Keywords: Safety Miscalibration, Intent Recoverability, Contextualized Specificity, Refusal Rate, RiskCueBench
TL;DR¶
This paper introduces RiskCueBench and a two-pass recoverability-controlled evaluation protocol that decouples semantic intent recovery from response-level safety policy, demonstrating that even when models accurately recover high-level risky objectives, contextualized specificity causes a systematic 0.175 drop in refusal rate and a 0.440 drop in global safety score.
Background & Motivation¶
Safety-aligned large language models are expected to reliably refuse harmful requests while remaining maximally helpful for benign objectives. However, prevailing safety benchmarks evaluate only the final assistant response, implicitly conflating the model's capacity to semantically comprehend user intent with its behavioral safety policy. When an aligned model exhibits a behavioral inconsistency across prompt variants, standard evaluation protocols cannot determine whether the safety failure stems from an intent-understanding breakdown (semantic unawareness of underlying risks) or an unjustified relaxation of safety boundaries in its response policy (policy miscalibration).
In real-world deployed conversations, adversarial users rarely express risky objectives in standard canonical forms, but instead embed requests in realistic scenarios, operational constraints, or workplace personas. An abstract request typically triggers a resolute refusal, whereas a more contextualized version of the exact same latent objective frequently elicits full task-oriented assistance. Existing red-teaming and adversarial prompting benchmarks document these compliance vulnerabilities, yet they predominantly attribute them to adversarial jailbreak mechanisms rather than systematically testing whether the safety policy remains calibrated when the underlying semantic intent is held constant.
To isolate response-level instability from semantic comprehension failures, safety evaluations require a controlled testbed that conditions behavioral scoring directly on verifiable semantic understanding. The core idea is to establish a recoverability-controlled evaluation benchmark, RiskCueBench, and a two-pass protocol that first measures whether a model recovers the high-level risky intent frame (ACT/OBJ/GOAL) and then evaluates paired response-level safety behaviors specifically on pairs where the model successfully recovered the intent.
Method¶
Overall Architecture¶
RiskCueBench contains 160 latent intents (120 high-risk intents across eight categories and 40 auxiliary benign/ambiguity controls) and 640 abstract, non-operational surface prompts. The benchmark spans eight risk domains: account integrity, privacy intrusion, data access, identity deception, digital access, concealment, sabotage, and coercive influence. Each high-risk latent intent is instantiated into four factorial conditions in a \(2 \times 2\) design orthogonally crossing contextualized specificity (S0 vs. S1) and explicit risk cues (C0 vs. C1): S0C0 (abstract baseline), S1C0 (contextualized non-operational prompt), S0C1 (abstract request with explicit risk cue), and S1C1 (contextualized prompt with explicit risk cue).
The evaluation proceeds through a rigorous two-pass protocol: 1. Intent-Recovery Pass (U-pass): The model is prompted to extract a structured semantic frame composed of ACT, OBJ, GOAL, CTX, and FMT. Evaluators assign an Intent Recovery Score (IRS) from 0 to 4. Pairs are admitted to downstream causal contrast only if both prompt conditions pass recovery (\(\text{IRS} \ge 3\), recovering core ACT, OBJ, and GOAL). 2. Assistant-Response Pass (A-pass): The model responds to the identical surface prompt under standard assistant system instructions, scoring Refusal Rate (RR), Unsafe Cooperation Rate (UCR), and an ordinal Global Safety Score (GSS). Causal effects are estimated using paired bootstrap differences and generalized linear mixed-effects models across recoverable pairs.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input: Latent Intents & Paired Surface Prompts<br/>S0C0, S1C0, S0C1, S1C1"] --> B["Latent Frame Parameterization & Non-Operational Restraint<br/>Invariance on ACT/OBJ/GOAL, Modulate CTX/FMT"]
B --> C["Two-Pass Intent-Recovery & Response Decoupling<br/>U-pass: IRSโฅ3 Gate | A-pass: Standard Generation"]
C --> D["Rubric-Adjudicated High-Impact Disagreement Resolution<br/>P0 Frozen Template Adjudication, P1/P2 Aggregation"]
D --> E["Recoverability-Gated Paired Statistical Estimation<br/>Paired Bootstrap Deltas & Mixed-Effects Models"]
E --> F["Output: Decoupled Safety Miscalibration Metrics<br/>Refusal Drop, Harmful Compliance & Mode Shift"]
Key Designs¶
1. Latent Frame Parameterization & Non-Operational Restraint: Invariant Semantics Without Deployable Exploits
Conventional safety datasets often embed actionable exploit chains, execution scripts, or tool configurations, confounding safety boundaries with surface keyword triggers and introducing safety risks upon artifact release. To eliminate these issues, RiskCueBench formalizes each intent as an ACT / OBJ / GOAL / CTX / FMT semantic tuple. In constructing paired prompts, the high-level action (\(\text{ACT}\)), target resource (\(\text{OBJ}\)), and intended objective (\(\text{GOAL}\)) are strictly invariant across all four conditions, whereas scenario context (\(\text{CTX}\)) and output structure (\(\text{FMT}\)) are exclusively varied to manipulate specificity. Crucially, all surface prompts are screened to exclude concrete credentials, named production targets, code execution steps, and evasion strategies. This construction ensures that every high-risk prompt serves as its own within-intent control, effectively eliminating cross-category and vocabulary confounding.
2. Two-Pass Intent-Recovery & Response Decoupling: Gating Causal Contrasts on Verified Understanding Traditional safety evaluation conflates semantic misunderstanding with policy relaxation. To decouple these stages, the framework introduces a two-pass gating architecture: - U-pass (Intent-Recovery): The model parses the input into the structured semantic frame, evaluated on a 0โ4 ordinal IRS metric: 0 indicates completely unrelated output; 1 captures broad topic without core action/goal; 2 recovers partial action/object but omits key risk aspects; 3 recovers the high-level ACT/OBJ/GOAL structure with minor context/format omissions; 4 indicates complete, flawless frame recovery. A primary gate is defined as \(\text{RecoveryPass} = \mathbb{I}(\text{IRS} \ge 3)\). - A-pass (Assistant-Response): The model answers the exact same prompt in standard assistant mode, tracking binary refusal rate (\(\text{RR} \in \{0, 1\}\)), binary unsafe cooperation rate (\(\text{UCR} \in \{0, 1\}\)), and ordinal global safety score (\(\text{GSS} \in [0, 4]\)). Primary causal contrasts (e.g., S1C0 vs. S0C0) restrict analysis strictly to pairs where both conditions satisfy \(\text{RecoveryPass} = 1\). This filter removes 665 pairs (22.2% of raw paired comparisons) where behavioral divergence is confounded by semantic recovery differences, ensuring that the remaining measured shift isolates response-level policy miscalibration.
3. Rubric-Adjudicated High-Impact Disagreement Resolution: Standardized Multi-Annotator Consensus The evaluation encompasses 25 model groups and 32,000 generated outputs. To ensure high annotation fidelity without relying on unconstrained LLM evaluators, the benchmark implements a three-tier adjudication pipeline. Initial annotations are collected from three independent LLM annotators. Minor disagreements (P1 and P2) are resolved via majority voting for binary labels and median aggregation for ordinal ratings. For high-impact P0 disagreementsโdefined as binary conflicts on RR or UCR, or a GSS span \(\ge 2\) (covering 6,438 records, 20.1% of the dataset)โcases are routed to an AI-assisted adjudicator governed by a frozen prompt template. The adjudicator must evaluate semantic recovery and response safety independently and output a structured JSON containing a field rationale and an exact evidence quotation from the model output. Blinded human audits over 400 boundary records confirmed substantial agreement between the pipeline and adjudicated human ground truth (\(\kappa = 0.743\) on \(\text{RecoveryPass}\) and \(\kappa = 0.817\) on \(\text{RR}\)).
4. Recoverability-Gated Paired Statistical Estimation: Dissecting the Specificity-Induced Mode Shift To rigorously evaluate the impact of surface-form manipulation, the methodology applies paired bootstrap delta estimation and generalized linear mixed-effects models (GLMM). Paired bootstrap differences compute within-intent deltas \(\Delta = y_{S1C0} - y_{S0C0}\) with 95% confidence intervals and Benjamini-Hochberg FDR-adjusted \(q\)-values. The mixed-effects model specifies logistic link functions for RR and UCR and an ordinal cumulative logit link for GSS, nesting random intercepts across model families, endpoints, categories, and latent intents:
This statistical apparatus reveals that contextualized specificity does not merely soften refusal tone, but triggers a full response-mode shift: across 451 switcher pairs crossing the safety boundary, refusal language drops from 88.5% to 0.2%, task-oriented framing rises from 23.5% to 84.7%, and response length expands by 94 words on average.
Key Experimental Results¶
Main Results¶
Across 25 completed model groups (10 API-served and 15 locally served systems, including Gemma, Qwen, DeepSeek, GLM, Baichuan, Falcon, Phi, Mistral, and Llama families), the primary recoverability-gated evaluation was conducted on 2,335 eligible pairs, bench-aligned against external evaluation adapters.
| Contrast / Evaluation Adapter | Scope / Eligible Pairs | Refusal Rate \(\Delta\) [95% CI] (\(q\)-value) | Unsafe Cooperation \(\Delta\) [95% CI] (\(q\)-value) | Global Safety Score \(\Delta\) [95% CI] (\(q\)-value) | Mixed-Effects Coefficient [95% CI] (\(p\)-value) | Interpretation & Key Takeaway |
|---|---|---|---|---|---|---|
| Specificity (S1C0 vs S0C0) | 2,335 pairs (Gated) | -0.175 [-0.308, -0.068] (\(q < 0.001\)) | +0.107 [0.020, 0.238] (\(q = 0.060\)) | -0.440 [-0.913, -0.120] (\(q = 0.022\)) | \(\beta_{spec}^{RR} = -2.851\) (\(p = 3.2\times 10^{-96}\)) \(\beta_{spec}^{GSS} = -0.761\) (\(p = 7.8\times 10^{-36}\)) |
Contextualized specificity significantly degrades safety boundaries conditional on recovery. |
| Risk Cue (S0C1 vs S0C0) | 2,348 pairs (Gated) | -0.098 [-0.230, 0.047] (\(q = 0.420\)) | -0.004 [-0.092, 0.156] (\(q = 0.703\)) | -0.161 [-0.706, 0.230] (\(q = 0.498\)) | \(\beta_{cue}^{RR} = -1.301\) (\(p = 5.6\times 10^{-34}\)) \(\beta_{cue}^{GSS} = -0.330\) (\(p = 2.4\times 10^{-8}\)) |
Explicit risk cues exert a comparatively weaker, heterogeneous effect on safety. |
| Interaction (S1C1 term) | 2,127 pairs (Gated) | +0.113 [-0.014, 0.270] (\(q = 0.208\)) | -0.041 [-0.216, 0.064] (\(q = 0.436\)) | +0.200 [-0.177, 0.783] (\(q = 0.420\)) | \(\beta_{int}^{RR} = +1.562\) (\(p = 2.2\times 10^{-18}\)) \(\beta_{int}^{GSS} = +0.303\) (\(p = 3.5\times 10^{-4}\)) |
Explicit cues partially offset specificity-induced safety degradation. |
| AdversariaLLM / ASR Readout | 3,000 (Local) | - | ASR / UCR: +0.102 [0.085, 0.119] | - | - | Recomputed ASR demonstrates substantial harmful compliance increases. |
| OR-Bench Keyword Checker | 3,000 (Local) | Rejection: -0.074 [-0.085, -0.064] | - | - | - | Standard refusal keyword markers experience a sharp decline. |
| EVOREFUSE PRR Readout | 3,000 (Local) | PRR: -0.205 [-0.223, -0.188] | - | - | - | Canonical prefix refusal patterns drop by more than 20%. |
| Raw Ungated Contrast | 3,000 (Ungated) | -0.164 [-0.179, -0.150] | +0.102 [0.084, 0.119] | -0.420 [-0.472, -0.369] | - | Ungated contrasts retain 22.2% recovery-confounded items, diluting the effect. |
Ablation Study¶
The ablation suite evaluates the specificity effect across alternative gate thresholds, specificity components, model deployment modes, and risk categories, tracking paired changes in Global Safety Score (\(\Delta \text{GSS}\)).
| Ablation Axis | Setting / Contrast Subset | Eligible Pairs | \(\Delta \text{GSS}\) | Key Observation & Methodological Verification |
|---|---|---|---|---|
| Recovery Gate | All pairs (Raw ungated) | 3,000 | -0.420 | Effect holds before recoverability filtering, but conflates understanding. |
| Main recovery gate (\(\text{IRS} \ge 3\)) | 2,335 | -0.440 | Primary recoverability-controlled estimate isolating policy miscalibration. | |
| Strict IRS4 gate (\(\text{IRS} = 4\)) | 766 | -0.217 | Direction preserved under maximal recovery filtering; serves as robustness boundary. | |
| Same-IRS matching | 1,842 | -0.368 | Robust when exact recovery score parity is enforced across pairs. | |
| Specificity Component | Setting detail only | 480 | -0.632 | Scenario grounding significantly degrades model refusal behavior. |
| Object detail only | 480 | -0.479 | Grounding asset/resource properties softens safety boundaries. | |
| Constraint detail only | 480 | -0.713 | Constraint framing is the single most potent surface vulnerability driver. | |
| Setting + Object detail | 480 | -0.544 | Combined contextual grounding firmly shifts models into task execution. | |
| Full specificity | 480 | -0.694 | Fully specified prompts induce comprehensive safety posture breakdown. | |
| Risk Category | High-shift categories | 1,168 | -0.606 | Severe drops in Sabotage (-0.697), Data Access (-0.642), Identity Deception (-0.487). |
| Remaining categories | 1,167 | -0.334 | All eight evaluated risk domains show consistent negative safety shifts. | |
| Model Family | API-served models | 1,100 | -0.482 | Commercial safety alignments remain highly vulnerable to specific framing. |
| Locally served models | 1,235 | -0.414 | Open-weight local checkpoints display matching behavioral miscalibration. |
Key Findings¶
- Contextualized specificity is the primary surface driver of safety miscalibration: Holding the latent risk objective constant, embedding requests in realistic scenario contexts (S1C0 vs. S0C0) triggers a 0.175 drop in refusal rate and a 0.440 decrease in global safety score, alongside a 0.107 increase in unsafe cooperation. This demonstrates that models frequently miscalibrate response behavior even after successfully parsing risky intentions.
- Miscalibration operates via a discrete response-mode switch rather than graded attenuation: Analysis of the 451 switcher pairs that crossed the refusal boundary reveals that refusal language collapsed from 88.5% to 0.2%, while task-oriented framing jumped from 23.5% to 84.7% and response length expanded by 94 words. Specificity causes models to abandon safety boundary demarcation altogether in favor of helpful problem-solving.
- Operational constraints drive the largest vulnerabilities, with substantial model-family heterogeneity: Decomposing specificity components demonstrates that adding "constraint details" produces the sharpest safety score drop (\(\Delta \text{GSS} = -0.713\)). Furthermore, model vulnerability varies dramatically across architectures: Gemma2-9B (\(\Delta \text{GSS} = -2.30\)) and Qwen-Flash (\(-1.88\)) experience extreme miscalibration, whereas DeepSeek-R1 and GLM systems exhibit far greater calibration stability.
Highlights & Insights¶
- Causal Decoupling of Semantic Understanding from Safety Policy: By introducing the two-pass protocol and intent-recovery score (IRS), this work establishes a rigorous methodology for evaluating safety behavior conditional on verified intent recovery, moving beyond blunt end-to-end evaluations.
- Desensitized Non-Operational Benchmark Design: Unlike existing red-teaming corpora that distribute operational exploit steps, RiskCueBench evaluates high-risk semantic boundaries using abstract, non-executable formulations, enabling safe public release under CC BY 4.0 without weaponization risks.
- Uncovering the Task-Assistance Overhang in Aligned LLMs: The findings highlight a fundamental failure mode in RLHF alignment: strong instruction-following priors inadvertently override safety refusal heads whenever concrete scenario constraints are supplied, disguising high-risk requests as benign operational tasks.
Limitations & Future Work¶
- Single-Turn Scope vs. Multi-Turn Dynamics: RiskCueBench currently evaluates single-turn prompts to ensure clean factorial isolation. Real-world adversaries frequently exploit multi-turn dialogs to establish trust and shift personas, requiring future extensions to track dynamic intent recovery scores across dialog turns.
- Language Coverage and Strict Gate Power Attenuation: The benchmark is presently focused on English prompts. Moreover, under the most stringent \(\text{IRS} = 4\) gate, eligible pairs decrease to 766, which attenuates statistical power after FDR adjustment, suggesting the need for denser intent-recovery benchmark pools.
- Behavioral Text-Feature Inference Lacks Internal Mechanistic Grounding: The current analysis relies on observable text-level response features. Future research should leverage sparse autoencoders (SAE) and refusal activation steering to investigate how contextual specificity alters latent refusal representations in the residual stream.
Related Work & Insights¶
- vs HarmBench / Do-Not-Answer / SORRY-Bench: Traditional safety benchmarks evaluate model outputs end-to-end on single prompts, conflating intent understanding with response policy. RiskCueBench isolates the response policy by conditioning comparisons on verified semantic recoverability.
- vs XSTest / OR-Bench / EVOREFUSE: Over-refusal evaluations probe false positives on benign inputs that look superficially risky. In contrast, RiskCueBench probes false negatives (under-refusals) on genuinely risky objectives that look superficially benign due to contextualized specificity.
- vs Arditi et al. (2024) Single Direction Refusal: While mechanistic studies show refusal can be mediated by a low-dimensional direction, RiskCueBench provides empirical behavioral evidence that contextualized prompt framing can bypass this activation gate even after intent has been extracted.
Rating¶
- Novelty: โญโญโญโญโญ Pioneering concept of recoverability-controlled safety miscalibration, cleanly decoupling semantic comprehension from response behavior.
- Experimental Thoroughness: โญโญโญโญโญ Extensive coverage across 25 model groups, 32,000 outputs, rigorous paired bootstrap deltas, and multi-axis ablations.
- Writing Quality: โญโญโญโญโญ Exceptionally structured reasoning, rigorous statistical formalization, and clean methodological execution.
- Value: โญโญโญโญโญ Establishes a crucial paradigm for safety alignment research, providing immediate utility for auditing real-world prompt sensitivity.