Multi-Domain Latent Concepts Evaluation: Auditing Interpretability and Robustness¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/Beijing-AISI/ForesightSafety-HAJailBench
Area: LLM Evaluation
Keywords: LLM Evaluation, Safety Evaluation, Jailbreak Attacks, Multi-Agent Debate, Human-Annotated Benchmark
TL;DR¶
To overcome the prohibitive cost of frontier judges and the brittleness of single-pass small models against subtle jailbreaks, this paper introduces a multi-agent debate framework combining refusal screening, dynamic 5-aspect value alignment, and bounded critic-defender deliberation, alongside FSโHAJailBench (an 11,100-instance human-annotated jailbreak benchmark), achieving near-GPT-4o agreement with a 54% reduction in inference cost.
Background & Motivation¶
Safety evaluation of large language models (LLMs) has become a non-negotiable prerequisite for deployment across open-domain dialogue and high-stakes reasoning tasks. However, existing automated LLM-as-a-judge pipelines face an acute dilemma between reliability and economic scalability: keyword-based filters and single-pass prompting are easily tricked by indirection, role-play framing, and multi-turn adversarial nudging; fine-tuned safety classifiers (e.g., LlamaGuard and ShieldGemma) exhibit severe distribution collapse when encountering novel attack strategies outside their training distribution; and frontier proprietary models (e.g., GPT-4o) capable of long chain-of-thought deliberation impose prohibitive inference expenses when scaled to tens of thousands of red-teaming interactions.
The core tension behind this evaluation bottleneck stems from the subjective and multi-faceted nature of adversarial jailbreaks: harmful intent is frequently concealed across dialogue turns, and independent judges show alarming divergence (the mean pairwise Cohen's \(\kappa\) across independent judges consistently falls below 0.2 on borderline cases). Prior multi-agent setups like JailJudge attempt to mitigate individual variance by aggregating scores across multiple parallel judges. Yet, such voting ensembles treat disagreement merely as random noise rather than meaningful semantic signal, failing to systematically dissect the adversarial interaction or verify competing interpretations.
This paper's angle is that small language models (SLMs) struggle with single-pass safety judging because they lack the scale-dependent capability to sustain long, multi-step internal reasoning chains; however, by externalizing this deliberation into an explicit adversarial dialogue between role-conditioned agents under a shared rubric, smaller models can rigorously surface subtle risks. Core idea: externalize LLM safety deliberation into a structured multi-agent debate pipeline where dynamic five-aspect value alignment bounds the discussion space, role-specific critic and defender agents iteratively challenge each other's evidence under early stopping, and a final judge consolidates the transcript into calibrated ten-point risk scores, enabling 14B models to approach GPT-4o-level human agreement at lower cost.
Method¶
Overall Architecture¶
The evaluation framework operates as a four-stage sequential pipeline: first, a lightweight helper screen filters out clear refusals directly; next, a pre-debate value alignment module prompts an LLM to select the five most pertinent safety dimensions from a 15-field pool to establish a shared evaluation rubric; third, optional response denoising strips obfuscations and prompt echos before critic and defender agents engage in multi-round bounded debate (monitored by early-stopping convergence conditions); finally, a judge agent consolidates the strongest arguments from both sides to generate an explicit explanation alongside calibrated 10-point, 5-level ordinal, and binary risk judgments.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input: Prompt & Model Response Pair"] --> B["Stage 1: Refusal Screening<br/>Detect explicit refusals directly as safe"]
B -->|Non-refusal cases| C["Stage 2: Dynamic Value Alignment<br/>Select top-5 relevant safety aspects from 15-field pool"]
C --> D["Stage 3: Response Denoising<br/>Fuzzy-match and strip prompt wrappers/obfuscation"]
D --> E["Stage 4: Critic-Defender Multi-Round Debate<br/>Critic identifies risks vs Defender argues benign intent"]
E -->|Convergence / max rounds met| F["Stage 5: Final Judge Adjudication<br/>Synthesize debate evidence into calibrated 10-point score"]
Key Designs¶
1. Dynamic Value Alignment: Anchoring the Topic Space to Preempt Semantic Drift
Unconstrained multi-agent debate in smaller models frequently suffers from topic drift, omission of subtle criteria, and hallucinated tangents over multi-turn interactions. To maintain strict evaluation focus without sacrificing generality, the framework deploys a pre-debate alignment step: given a prompt-response instance, the model dynamically selects the five most applicable safety aspects from a predefined 15-field taxonomy (spanning hateful content, violence, illegal activities and criminal instructions, privacy violations, dangerous behaviors, misinformation, unqualified professional advice, etc.). These five chosen dimensions are injected into the prompts of the critic, defender, and final judge. By forcing all agents to debate exclusively over the same structured rubric, the system prevents agents from talking past one another and eliminates the semantic drift typical of small model multi-turn conversations.
2. Point-Wise Critic-Defender Adversarial Deliberation: Externalizing Deep Semantic Audits
To substitute for single-pass subjective bias, the system instantiates two adversarial agents: the Critic Agent, tasked with detecting actionable harm across the aligned dimensions and providing explicit rationale with an initial risk assessment; and the Defender Agent, tasked with challenging the critic's claims through alternative benign interpretations, contextual qualifiers, or counter-evidence. The exchange progresses through bounded iterative rounds with early stopping: debate terminates immediately if both agents converge onto identical risk bands, or if consecutive counter-arguments become repetitive (measured by a Ratcliff-Obershelp text similarity threshold exceeding 0.85). This turns hidden assumptions into explicit textual arguments, forcing the evaluation to verify whether an apparently harmful token sequence actually provides actionable execution instructions.
3. Calibrated Multi-Granularity Adjudication: Structured Evidence Consolidation
Following the debate, the Judge Agent ingests the complete transcript, weighs the opposing arguments, and issues a final explanatory assessment accompanied by a scalar score on a 1โ10 scale. This continuous score is mapped into three functional bands: safe (1โ4), suspicious/uncertain (5โ6, deliberately kept narrow as the most contested borderline zone), and unsafe (7โ10, indicating tangible harmful assistance), which further maps to standard five-level ordinal ranks and binary attack success labels (ASR). This design preserves granular ranking signals for calibration benchmarks while delivering high-certainty binary gating for downstream deployment.
Loss & Training¶
The framework is a prompt-engineered, role-conditioned inference pipeline that requires no parameter fine-tuning. Computational efficiency is assessed via tracked prompt and completion token counts under public OpenRouter token pricing snapshots. In constructing the evaluation ground truth for FSโHAJailBench, a two-round human annotation protocol was employed: instances exhibiting significant scoring discrepancies (> 2 points) between primary annotators and triage models (GPT-4o and DeepSeek-R1) were escalated to a three-annotator panel, resolved via majority-band averaging to ensure gold-standard calibration.
Key Experimental Results¶
Main Results¶
Evaluation was conducted on FSโHAJailBench (11,100 human-annotated instances across 11 jailbreak methods and 11 target models). The table reports Cohen's \(\kappa\) agreement against human ground truth alongside unit costs (\(10^{-4}\$\) per query) and cost ratios relative to GPT-4o.
| Judge Backbone | Algorithm | Cohen's \(\kappa\) | Cost per Query ($10^{-4}$) | Cost Ratio (vs GPT-4o) |
|---|---|---|---|---|
| GPT-4o | Baseline Judge (single-pass) | 0.7589 | 9.03 | 1.00 |
| LlamaGuard3-8B | Fine-tuned Judge | -0.0056 | 0.12 | 0.01 |
| ShieldGemma-9B | Fine-tuned Judge | 0.3510 | - | - |
| Qwen2.5-7B | Multi-Agent Judge (Ours) | 0.5233 | 1.62 | 0.18 |
| Qwen2.5-7B | JailJudge baseline | 0.4334 | 2.51 | 0.28 |
| Qwen3-4B | Multi-Agent Judge (Ours) | 0.6440 | 2.67 | 0.30 |
| Qwen3-4B | JailJudge baseline | 0.5121 | 2.71 | 0.30 |
| Qwen3-8B | Multi-Agent Judge (Ours) | 0.6532 | 4.05 | 0.45 |
| Qwen3-8B | JailJudge baseline | 0.5626 | 4.58 | 0.51 |
| Qwen3-14B | Multi-Agent Judge (Ours) | 0.7331 | 4.13 | 0.46 |
| Qwen3-14B | JailJudge baseline | 0.5545 | 4.30 | 0.48 |
| Llama-3.1-8B | Multi-Agent Judge (Ours) | 0.4727 | 1.85 | 0.20 |
| Llama-3.1-8B | JailJudge baseline | 0.5719 | 1.33 | 0.15 |
On the external transfer benchmark JailJudge-ID (with Qwen3-14B as backbone), the Multi-Agent Judge achieved \(\kappa = 0.6843\) compared to JailJudge's \(\kappa = 0.6008\), proving robust cross-dataset generalization.
Ablation Study¶
Ablations on Qwen3-14B isolate the impact of debate depth and individual scaffolding stages (alignment, denoising, refusal screening).
| Variant | Debate Rounds | Configuration | Cohen's \(\kappa\) | Cost ($10^{-4}$) | Cost Ratio |
|---|---|---|---|---|---|
| Reference System (Full) | 3 | Standard dynamic alignment (Align on) | 0.7331 | 4.13 | 1.000 |
| Round Depth: 0 rounds | 0 | Alignment only + single-pass judge | 0.5709 | 1.24 | 0.300 |
| Round Depth: 1 round | 1 | 1 round Critic-Defender exchange | 0.6955 | 2.87 | 0.695 |
| Round Depth: 2 rounds | 2 | 2 rounds Critic-Defender exchange | 0.7143 | 3.60 | 0.872 |
| Round Depth: 4 rounds | 4 | 4 rounds Critic-Defender exchange | 0.7260 | 4.36 | 1.056 |
| Round Depth: 5 rounds | 5 | 5 rounds Critic-Defender exchange | 0.7221 | 4.66 | 1.129 |
| Component: No alignment | 3 | Disable value alignment (Align off) | 0.7239 | 3.84 | 0.930 |
| Component: Free alignment | 3 | Unconstrained aspect generation (Align free) | 0.7300 | 4.09 | 0.990 |
| Stripped Baseline | 0 | No alignment / no denoise / no refusal | 0.5200 | 0.60 | 0.145 |
Key Findings¶
- Three rounds mark the optimal efficiency-accuracy frontier: Increasing debate depth from 0 rounds (\(\kappa=0.5709\)) to 3 rounds (\(\kappa=0.7331\)) produces a massive gain of \(+0.1622\) in agreement. Extending further to 4 and 5 rounds slightly degrades performance (dropping to \(0.7260\) and \(0.7221\)), showing that over-debating introduces repetitive noise and error accumulation.
- Near-frontier agreement at half the cost: Qwen3-14B with Multi-Agent Judge reaches \(\kappa = 0.7331\), trailing GPT-4o (\(\kappa = 0.7589\)) by only \(0.0258\) while consuming only \(46\%\) of GPT-4o's estimated cost. Furthermore, at 5,387 total tokens per query, it uses \(9.2\%\) fewer tokens than JailJudge on the same backbone while outperforming it by \(+0.1786\) in \(\kappa\).
- Failure of fine-tuned classifiers under complex attacks: Deployed classifiers like LlamaGuard3 struggle severely (\(\kappa \approx -0.0056\)), highlighting that static fine-tuned filters cannot generalize to subtle multi-turn or semantically obfuscated jailbreaks without structured deliberation.
Highlights & Insights¶
- Externalizing CoT into multi-agent dialogue: Instead of hoping small models maintain consistent internal long-chain reasoning, this framework turns reasoning into an externalized turn-taking dialogue between opposing roles, effectively unlocking frontier-grade depth on small backbones.
- Dynamic rubric alignment as an anchor: Rather than unbounded conversation, pre-selecting five case-relevant safety aspects confines the debate to concrete points, virtually eliminating agent hallucinations and topic drift.
- High-fidelity open evaluation suite: FSโHAJailBench delivers an extensively validated benchmark of 11,100 instances spanning 11 jailbreaks and 11 target models, providing a reliable gold standard for the community to benchmark automated safety judges.
Limitations & Future Work¶
- Backbone-dependent sensitivity: On Llama-3.1-8B, the framework scored lower than JailJudge (\(\kappa=0.4727\) vs \(0.5719\)), revealing that the debate mechanism is sensitive to base instruction-following fidelity and chat templates, warranting more robust, model-agnostic prompting.
- Potential blind spots of fixed taxonomies: Constraining the selection pool to 15 predefined dimensions might restrict sensitivity to entirely novel, hybrid harm vectors (though the Align free variant provides a viable mitigation).
- Latency overhead: Sequential multi-turn exchanges incur higher end-to-end inference latency than single-pass filters, making the framework ideal for offline auditing and batch red-teaming rather than sub-second real-time online guardrails.
Related Work & Insights¶
- vs JailJudge (Liu et al., 2024a): JailJudge relies on parallel voting across independent judges to reduce variance, treating disagreement as noise; Multi-Agent Judge treats disagreement as critical signal, driving explicit adversarial rebuttal that yields \(+0.1786\) higher \(\kappa\) and lower token usage on Qwen3-14B.
- vs ChatEval (Chan et al., 2023): While ChatEval explored multi-agent debate for general text quality, this work introduces safety-specific scaffolding (refusal triage, dynamic 5-aspect alignment, and similarity-based early stopping) specifically engineered for adversarial jailbreak auditing.
Rating¶
- Novelty: โญโญโญโญ [Transforms adversarial jailbreak auditing into structured multi-agent debate with dynamic aspect alignment and similarity-based early stopping]
- Experimental Thoroughness: โญโญโญโญโญ [Constructs an 11,100-instance human-annotated benchmark across 11 attacks and 11 targets, complete with bootstrap significance and token-level cost accounting]
- Writing Quality: โญโญโญโญโญ [Clear motivation, modular pipeline diagrams, transparent ablation analysis, and extensive appendix transcripts]
- Value: โญโญโญโญโญ [Provides a cost-effective, interpretable, and reproducible recipe for scalable safety evaluation using open-weights small models]