Skip to content

Probing Counterfactual Reasoning in Large Language Models

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Large Language Models, Counterfactual Reasoning, Pseudo-Harmful Queries, Over-Refusal, Multimodal Safety

TL;DR

Investigating why large models excessively reject requests whose isolated harmful speech is rendered benign by surrounding acoustic contexts, this work introduces AOR-Bench with 3,000 pseudo-harmful audio queries across six scenarios, benchmarks 12 leading LALMs, and validates inference-time Chain-of-Thought and representation steering mitigations.

Background & Motivation

As Large Audio Language Models (LALMs) rapidly transition from research prototypes to practical deployment in interactive speech assistants, embodied perception, and spoken dialogue systems, safeguarding them against malicious misuse and policy violations has become an urgent prerequisite. Frontline developers and frontier labs commonly employ refusal mechanisms trained via safety alignment to reject requests that breach ethical boundaries. However, these safety measures frequently induce a detrimental side effect known as over-refusal, where models develop hypersensitivity and erroneously reject completely benign, legitimate user prompts.

In the pure text modality, over-refusal typically stems from surface-level keyword matching or lexical ambiguity, such as incorrectly refusing "how to kill a python process" due to the token "kill." In audio language modeling, however, counterfactual reasoning faces a significantly more complex multimodal challenge: spoken utterances that appear dangerous or illicit when evaluated in isolation can become completely benign when contextualized by surrounding acoustic cues. For example, a spoken query asking "how to pass through a red light intersection at high speed" appears blatantly illegal on its own, but when interpreted alongside the blaring sirens and dispatch chatter of a fire engine background, the actual speaker intent clearly reflects an emergency rescue response. When models lack the counterfactual reasoning ability to integrate non-textual acoustic cues and infer the true benign intent, they default to premature refusal.

Existing safety evaluation benchmarks either focus strictly on textual inputs or directly synthesize speech from text-only safety datasets, overlooking the acoustic scene dynamics, background sounds, and paralinguistic cues that fundamentally alter intent. To resolve this critical blind spot, this paper systematically explores how foundation models perform counterfactual reasoning over complex acoustic environments. Core idea: by pairing isolated harmful-sounding speech with grounded background audio that counterfactually inverts the semantic intent into a benign scenario, the authors construct AOR-Bench to probe audio-specific over-refusal, exposing the vulnerability of current models in contextual causal reasoning.

Method

Overall Architecture

AOR-Bench is constructed through a grounded multimodal synthesis and moderation pipeline covering six real-world scenario categories: agriculture, cooking, emergency, gaming, entertainment shows, and sports. The benchmark construction architecture decouples scene generation, text candidate prompting, multi-model consensus verification, and acoustic speech synthesis, followed by systematic probing across 12 foundation LALMs and lightweight intervention strategies.

The full construction and counterfactual evaluation pipeline flows as follows:

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Scenario Script Authoring"] --> B["Video-to-Audio Grounded Track Extraction<br/>Sora-2 video generation and sound separation"]
    B --> C["Cooperative Text Candidate Generation<br/>Acoustic scene captioning and speech drafting"]
    C --> D["Multi-LLM Ensemble Moderation<br/>Validating harmful-in-isolation yet benign-in-context"]
    D --> E["High-Fidelity TTS Paralinguistic Synthesis<br/>Varying gender, emotion, and speech speed"]
    E --> F["Audio Mixing & Benchmark Assembly<br/>Yielding 3000 AOR-Bench test samples"]
    F --> G["LALM Contextual Inference & Over-Refusal Probing<br/>Evaluating ORR, TRR, and MB-Score"]
    G --> H["Chain-of-Thought & Activation Steering Analysis<br/>Exploring causal reasoning and alignment trade-offs"]

Key Designs

1. Video-Grounded Background Audio Extraction: Grounding complex scene acoustics without temporal decoherence
Direct text-to-audio synthesis models frequently produce disjointed, synthetic sound effects that lack physical coherence and temporal narrative structure. To ensure that background acoustics provide rich and authentic contextual evidence strong enough to invert perceived intent, the authors implement a video-to-audio extraction strategy. Detailed scripts describing realistic activities (e.g., firefighters navigating emergency intersections, chefs performing culinary techniques, or stage actors rehearsing action sequences) are provided to the state-of-the-art text-to-video model Sora-2. The audio tracks extracted from these generated video clips inherently capture coherent multi-source environmental sounds, spatial acoustics, and realistic temporal dynamics, establishing a dependable factual context for subsequent counterfactual inference.

2. Multi-LLM Ensemble Moderation: Eliminating human subjectivity and single-model alignment bias
A core hurdle in constructing pseudo-harmful benchmarks is guaranteeing that every candidate query is genuinely harmful in isolation but unambiguously benign and legal when interpreted alongside the acoustic context. Human annotation is notoriously prone to subjective variance across cultural and ethical norms, while any single LLM judge carries its own systematic safety prejudices. The authors introduce a multi-LLM voting panel comprising MiniMax-M2.5, GLM-5, Gemini-3, and Claude-Haiku-4.5. A candidate pair is retained only when all ensemble judges unanimously confirm that the combined scenario is safe and benign, while the isolated speech text exhibits sufficient surface harm. If a category lacks sufficient valid pairs, iterative prompting is triggered until balanced quotas are satisfied.

3. Systematic Paralinguistic Audio Attribute Variation: Disentangling acoustic feature perturbations from contextual intent
Human speech conveys rich non-verbal signals—including speaker gender, affective tone, and cadence—that foundation models may conflate with safety threat levels. Utilizing the neural text-to-speech engine CosyVoice-v3-flash, the authors systematically perturb the 500 validated speech scripts along three orthogonal paralinguistic dimensions: speaker gender (male, female), emotion (neutral, happy, angry/fearful), and speech rate (slow, normal, fast). By crossing these acoustic variations against the 50 unique background acoustic scenarios, the benchmark scales to 3,000 diverse evaluation samples, enabling precise auditing of whether safety classifiers overreact to speaker agitation or pitch rather than contextual intent.

4. Balanced Trade-off Metric & Lightweight Inference Mitigation: Reconciling over-refusal reduction with genuine safety alignment
Optimizing solely for a lower Over-Refusal Rate (\(ORR\)) risks degrading a model's vigilance against truly malicious inputs, measured by the True-Refusal Rate (\(TRR\)). To capture this dual objective, the authors formulate the harmonic balance metric \(MB\text{-Score}\): $\(ORR = \frac{|D_{\text{over-refused}}|}{|D_{\text{benign}}|}, \quad TRR = \frac{|D_{\text{correctly-refused}}|}{|D_{\text{harmful}}|}\)$ $\(MB\text{-Score} = \frac{2 \cdot TRR \cdot (1 - ORR)}{TRR + (1 - ORR)}\)$ To address the root cause of contextual neglect without costly retraining, the authors formulate a step-by-step Chain-of-Thought (CoT) prompt that instructs the LALM to explicitly analyze the background sounds and infer user intent prior to rendering a safety verdict. In parallel, they investigate activation steering via the difference-in-means representation vector \(\Delta h^{(\ell)}\) computed over contrastive hidden states: $\(\tilde{h}^{(\ell)} = h^{(\ell)} - \alpha \cdot \Delta h^{(\ell)}\)$ probing the representational limits of linear concept steering in multimodal safety alignment.

Benchmark & Evaluation Protocol

The benchmark evaluates 12 representative LALMs across 6 prominent model families, spanning closed-source flagships (Gemini-3-Flash-Preview, Gemini-2.5-Flash-Lite, Gemini-2.0-Flash-Lite, GPT-Audio, GPT-Audio-mini) and open-weight architectures (Step-Audio-2, Step-Audio-2-mini, MiMo-V2-Omni, MiMo-Audio-7B-Instruct, Qwen3-Omni, Qwen2-Audio-Instruct, Kimi-Audio-Instruct). Deterministic decoding (temperature 0.0, or 0.6 where default prevents degenerate loops) is enforced. Refusal semantics are judged via DeepSeek-V4 and cross-validated across heterogeneous high-performance LLM evaluators.

Key Experimental Results

Main Results

The table below presents the Over-Refusal Rate (ORR, lower is better) and the harmonic safety-utility trade-off (MB-Score, higher is better) across the six scenario categories evaluated on AOR-Bench:

Model Agriculture (%) Cook (%) Emergency (%) Game (%) Show (%) Sports (%) Average ORR (↓) Average MB-Score (↑)
Gemini-3 51.82 22.50 33.64 65.00 68.33 30.00 43.80 55.90
Gemini-2.5 69.09 42.50 48.18 76.67 85.00 73.33 66.20 47.97
Gemini-2 55.45 17.50 34.55 28.33 46.67 33.33 38.20 54.16
GPT-Audio 44.55 52.50 59.09 30.00 63.33 19.17 42.80 48.17
GPT-Audio-Mini 23.64 12.50 30.91 13.33 45.00 0.83 20.20 39.22
Step-Audio-2 50.00 32.50 23.64 43.33 51.67 58.33 44.20 55.90
Step-Audio-2-Mini 40.00 0.00 10.00 30.00 18.33 41.67 26.80 68.06
MiMo-Audio 25.45 0.00 15.45 20.00 28.33 13.33 18.00 54.84
MiMo-V2-Omni 56.36 47.50 33.64 46.67 73.33 30.83 45.40 55.68
Qwen2-Audio 22.73 5.00 5.45 26.67 23.33 21.67 17.80 82.80
Qwen3-Omni 60.00 40.00 27.27 50.00 73.33 48.33 48.80 58.71
Kimi-Audio 63.64 2.50 17.27 3.33 13.33 41.67 30.00 78.90

Ablation Study & Mitigation Analysis

Evaluating inference-time interventions reveals that guiding models through explicit contextual reasoning substantially restores counterfactual understanding:

Intervention / Setting Evaluated Model Baseline ORR (%) Mitigated ORR (%) Absolute Δ Empirical Behavior & Insights
Chain-of-Thought (CoT) GPT-Audio 43.00 18.00 -25.00% Forces step-by-step acoustic context parsing, curbing knee-jerk refusal
Chain-of-Thought (CoT) Gemini-3 31.17 5.20 -25.97% Over 80% relative drop in false refusals, activating counterfactual reasoning
Chain-of-Thought (CoT) MiMo-V2-Omni 43.20 20.60 -22.60% Halves over-refusal by clarifying legitimate intent from background sound
Textual Background Oracle All 12 LALMs Raw Audio Input Text Caption Added Consistent Drop Proves failure stems from poor audio context encoding rather than safety logic
Activation Steering (\(\alpha=1\)) Open-Source LALMs Unsteered Base Hidden Shift \(\tilde{h}\) ORR Reduced Diminishes false refusals but induces collateral drop in TRR and overall MB-Score

Key Findings

  • Audio over-refusal is widespread and severe across model families: All tested models exhibit non-zero over-refusal rates on benign-in-context queries. Gemini-2.5 reaches an alarming average ORR of 66.20%, and half of the evaluated models incorrectly refuse more than 40% of legitimate requests.
  • Acoustic scene context is fundamentally underutilized: When models are supplied with gold textual transcriptions of the background audio, over-refusal rates collapse uniformly across all models. This provides definitive empirical proof that LALMs possess the cognitive reasoning capacity to discern benign intent, but their audio encoders fail to ground and propagate acoustic background semantics into the safety deliberation layer.
  • Fragile balance between safety compliance and helpfulness: Qwen2-Audio (82.80) and Kimi-Audio (78.90) achieve the most balanced trade-offs, whereas models like GPT-Audio-mini obtain low ORR simply due to overall under-refusal tendencies, collapsing their MB-Scores.
  • High vulnerability to system prompt framing and paralinguistics: Priming models with safety-centric system instructions drives over-refusal rates toward 100%. Concurrently, speech speed and emotion provoke erratic, model-dependent shifts (e.g., faster speech increases refusals in Gemini-2.5 but reduces them in MiMo-V2-Omni), indicating that paralinguistic cues introduce unpredictable noise into current safety boundaries.

Highlights & Insights

  • Multimodal counterfactual intent formulation: Shifts safety benchmarking from surface lexical matching to deep causal reasoning, formalizing how non-verbal context counterfactually flips the semantic validity of spoken requests.
  • Sora-powered video-to-audio synthesis paradigm: Overcomes the lack of realistic environmental dynamics in audio generation by tapping into physically grounded text-to-video simulations to curate authentic soundscapes.
  • Inference-time Chain-of-Thought as an immediate remedy: Proves that structured causal decomposition (background identification \(\rightarrow\) spoken request interpretation \(\rightarrow\) holistic intent inference) slashes over-refusal by up to 25.97% without fine-tuning weights.

Limitations & Future Work

  • Scenario breadth and real-world acoustic coverage: While AOR-Bench introduces 6 representative domains across 50 base scenes, the physical world contains vastly more diverse acoustic ambiguities and polyphonic overlapping speech events.
  • Coarse perception of subtle paralinguistic nuances: Current LALM audio frontends remain relatively coarse in processing fine-grained acoustic prosody and emotional cadence, leading to inconsistent paralinguistic sensitivity across architectures.
  • Collateral vulnerability of representation steering: Vector subtraction interventions reduce over-refusal at the expense of lowering defense barriers against genuinely adversarial prompts, highlighting the need for context-conditional steering mechanisms.
  • vs XSTest / OR-Bench (Textual Over-Refusal): Whereas text benchmarks rely on polysemous puns, medical metaphors, or homonyms within isolated sentences, AOR-Bench establishes a multimodal paradigm where visual/acoustic context provides the necessary factual ground truth to overturn apparent danger.
  • vs Jailbreak-AudioBench / JALMBench (Audio Red-Teaming): Prior audio safety works focus predominantly on jailbreak vulnerabilities and under-refusal attacks; AOR-Bench targets the opposing failure mode of exaggerated safety, completing the safety-utility evaluation spectrum.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Formulates the first dedicated multimodal counterfactual reasoning benchmark for audio over-refusal with a grounded video-audio extraction framework.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorously benchmarks 12 frontier models across 6 families, providing comprehensive analyses across system prompts, paralinguistic perturbations, and internal activations.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear narrative progression with precise formal metrics, intuitive motivating examples, and meticulous ablation reporting.
  • Value: ⭐⭐⭐⭐⭐ Offers an indispensable diagnostic asset for the speech and audio AI community, highlighting the necessity of deep multimodal contextual understanding in safety alignment.