Skip to content

Multi-Domain Latent Concepts Evaluation: Auditing Interpretability and Robustness

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/Beijing-AISI/ForesightSafety-HAJailBench
Area: LLM Evaluation
Keywords: LLM Evaluation, Safety Evaluation, Jailbreak Attacks, Multi-Agent Debate, Human-Annotated Benchmark

TL;DR

To overcome the prohibitive cost of frontier judges and the brittleness of single-pass small models against subtle jailbreaks, this paper introduces a multi-agent debate framework combining refusal screening, dynamic 5-aspect value alignment, and bounded critic-defender deliberation, alongside FSโ€“HAJailBench (an 11,100-instance human-annotated jailbreak benchmark), achieving near-GPT-4o agreement with a 54% reduction in inference cost.

Background & Motivation

Safety evaluation of large language models (LLMs) has become a non-negotiable prerequisite for deployment across open-domain dialogue and high-stakes reasoning tasks. However, existing automated LLM-as-a-judge pipelines face an acute dilemma between reliability and economic scalability: keyword-based filters and single-pass prompting are easily tricked by indirection, role-play framing, and multi-turn adversarial nudging; fine-tuned safety classifiers (e.g., LlamaGuard and ShieldGemma) exhibit severe distribution collapse when encountering novel attack strategies outside their training distribution; and frontier proprietary models (e.g., GPT-4o) capable of long chain-of-thought deliberation impose prohibitive inference expenses when scaled to tens of thousands of red-teaming interactions.

The core tension behind this evaluation bottleneck stems from the subjective and multi-faceted nature of adversarial jailbreaks: harmful intent is frequently concealed across dialogue turns, and independent judges show alarming divergence (the mean pairwise Cohen's \(\kappa\) across independent judges consistently falls below 0.2 on borderline cases). Prior multi-agent setups like JailJudge attempt to mitigate individual variance by aggregating scores across multiple parallel judges. Yet, such voting ensembles treat disagreement merely as random noise rather than meaningful semantic signal, failing to systematically dissect the adversarial interaction or verify competing interpretations.

This paper's angle is that small language models (SLMs) struggle with single-pass safety judging because they lack the scale-dependent capability to sustain long, multi-step internal reasoning chains; however, by externalizing this deliberation into an explicit adversarial dialogue between role-conditioned agents under a shared rubric, smaller models can rigorously surface subtle risks. Core idea: externalize LLM safety deliberation into a structured multi-agent debate pipeline where dynamic five-aspect value alignment bounds the discussion space, role-specific critic and defender agents iteratively challenge each other's evidence under early stopping, and a final judge consolidates the transcript into calibrated ten-point risk scores, enabling 14B models to approach GPT-4o-level human agreement at lower cost.

Method

Overall Architecture

The evaluation framework operates as a four-stage sequential pipeline: first, a lightweight helper screen filters out clear refusals directly; next, a pre-debate value alignment module prompts an LLM to select the five most pertinent safety dimensions from a 15-field pool to establish a shared evaluation rubric; third, optional response denoising strips obfuscations and prompt echos before critic and defender agents engage in multi-round bounded debate (monitored by early-stopping convergence conditions); finally, a judge agent consolidates the strongest arguments from both sides to generate an explicit explanation alongside calibrated 10-point, 5-level ordinal, and binary risk judgments.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Prompt & Model Response Pair"] --> B["Stage 1: Refusal Screening<br/>Detect explicit refusals directly as safe"]
    B -->|Non-refusal cases| C["Stage 2: Dynamic Value Alignment<br/>Select top-5 relevant safety aspects from 15-field pool"]
    C --> D["Stage 3: Response Denoising<br/>Fuzzy-match and strip prompt wrappers/obfuscation"]
    D --> E["Stage 4: Critic-Defender Multi-Round Debate<br/>Critic identifies risks vs Defender argues benign intent"]
    E -->|Convergence / max rounds met| F["Stage 5: Final Judge Adjudication<br/>Synthesize debate evidence into calibrated 10-point score"]

Key Designs

1. Dynamic Value Alignment: Anchoring the Topic Space to Preempt Semantic Drift

Unconstrained multi-agent debate in smaller models frequently suffers from topic drift, omission of subtle criteria, and hallucinated tangents over multi-turn interactions. To maintain strict evaluation focus without sacrificing generality, the framework deploys a pre-debate alignment step: given a prompt-response instance, the model dynamically selects the five most applicable safety aspects from a predefined 15-field taxonomy (spanning hateful content, violence, illegal activities and criminal instructions, privacy violations, dangerous behaviors, misinformation, unqualified professional advice, etc.). These five chosen dimensions are injected into the prompts of the critic, defender, and final judge. By forcing all agents to debate exclusively over the same structured rubric, the system prevents agents from talking past one another and eliminates the semantic drift typical of small model multi-turn conversations.

2. Point-Wise Critic-Defender Adversarial Deliberation: Externalizing Deep Semantic Audits

To substitute for single-pass subjective bias, the system instantiates two adversarial agents: the Critic Agent, tasked with detecting actionable harm across the aligned dimensions and providing explicit rationale with an initial risk assessment; and the Defender Agent, tasked with challenging the critic's claims through alternative benign interpretations, contextual qualifiers, or counter-evidence. The exchange progresses through bounded iterative rounds with early stopping: debate terminates immediately if both agents converge onto identical risk bands, or if consecutive counter-arguments become repetitive (measured by a Ratcliff-Obershelp text similarity threshold exceeding 0.85). This turns hidden assumptions into explicit textual arguments, forcing the evaluation to verify whether an apparently harmful token sequence actually provides actionable execution instructions.

3. Calibrated Multi-Granularity Adjudication: Structured Evidence Consolidation

Following the debate, the Judge Agent ingests the complete transcript, weighs the opposing arguments, and issues a final explanatory assessment accompanied by a scalar score on a 1โ€“10 scale. This continuous score is mapped into three functional bands: safe (1โ€“4), suspicious/uncertain (5โ€“6, deliberately kept narrow as the most contested borderline zone), and unsafe (7โ€“10, indicating tangible harmful assistance), which further maps to standard five-level ordinal ranks and binary attack success labels (ASR). This design preserves granular ranking signals for calibration benchmarks while delivering high-certainty binary gating for downstream deployment.

Loss & Training

The framework is a prompt-engineered, role-conditioned inference pipeline that requires no parameter fine-tuning. Computational efficiency is assessed via tracked prompt and completion token counts under public OpenRouter token pricing snapshots. In constructing the evaluation ground truth for FSโ€“HAJailBench, a two-round human annotation protocol was employed: instances exhibiting significant scoring discrepancies (> 2 points) between primary annotators and triage models (GPT-4o and DeepSeek-R1) were escalated to a three-annotator panel, resolved via majority-band averaging to ensure gold-standard calibration.

Key Experimental Results

Main Results

Evaluation was conducted on FSโ€“HAJailBench (11,100 human-annotated instances across 11 jailbreak methods and 11 target models). The table reports Cohen's \(\kappa\) agreement against human ground truth alongside unit costs (\(10^{-4}\$\) per query) and cost ratios relative to GPT-4o.

Judge Backbone Algorithm Cohen's \(\kappa\) Cost per Query ($10^{-4}$) Cost Ratio (vs GPT-4o)
GPT-4o Baseline Judge (single-pass) 0.7589 9.03 1.00
LlamaGuard3-8B Fine-tuned Judge -0.0056 0.12 0.01
ShieldGemma-9B Fine-tuned Judge 0.3510 - -
Qwen2.5-7B Multi-Agent Judge (Ours) 0.5233 1.62 0.18
Qwen2.5-7B JailJudge baseline 0.4334 2.51 0.28
Qwen3-4B Multi-Agent Judge (Ours) 0.6440 2.67 0.30
Qwen3-4B JailJudge baseline 0.5121 2.71 0.30
Qwen3-8B Multi-Agent Judge (Ours) 0.6532 4.05 0.45
Qwen3-8B JailJudge baseline 0.5626 4.58 0.51
Qwen3-14B Multi-Agent Judge (Ours) 0.7331 4.13 0.46
Qwen3-14B JailJudge baseline 0.5545 4.30 0.48
Llama-3.1-8B Multi-Agent Judge (Ours) 0.4727 1.85 0.20
Llama-3.1-8B JailJudge baseline 0.5719 1.33 0.15

On the external transfer benchmark JailJudge-ID (with Qwen3-14B as backbone), the Multi-Agent Judge achieved \(\kappa = 0.6843\) compared to JailJudge's \(\kappa = 0.6008\), proving robust cross-dataset generalization.

Ablation Study

Ablations on Qwen3-14B isolate the impact of debate depth and individual scaffolding stages (alignment, denoising, refusal screening).

Variant Debate Rounds Configuration Cohen's \(\kappa\) Cost ($10^{-4}$) Cost Ratio
Reference System (Full) 3 Standard dynamic alignment (Align on) 0.7331 4.13 1.000
Round Depth: 0 rounds 0 Alignment only + single-pass judge 0.5709 1.24 0.300
Round Depth: 1 round 1 1 round Critic-Defender exchange 0.6955 2.87 0.695
Round Depth: 2 rounds 2 2 rounds Critic-Defender exchange 0.7143 3.60 0.872
Round Depth: 4 rounds 4 4 rounds Critic-Defender exchange 0.7260 4.36 1.056
Round Depth: 5 rounds 5 5 rounds Critic-Defender exchange 0.7221 4.66 1.129
Component: No alignment 3 Disable value alignment (Align off) 0.7239 3.84 0.930
Component: Free alignment 3 Unconstrained aspect generation (Align free) 0.7300 4.09 0.990
Stripped Baseline 0 No alignment / no denoise / no refusal 0.5200 0.60 0.145

Key Findings

  • Three rounds mark the optimal efficiency-accuracy frontier: Increasing debate depth from 0 rounds (\(\kappa=0.5709\)) to 3 rounds (\(\kappa=0.7331\)) produces a massive gain of \(+0.1622\) in agreement. Extending further to 4 and 5 rounds slightly degrades performance (dropping to \(0.7260\) and \(0.7221\)), showing that over-debating introduces repetitive noise and error accumulation.
  • Near-frontier agreement at half the cost: Qwen3-14B with Multi-Agent Judge reaches \(\kappa = 0.7331\), trailing GPT-4o (\(\kappa = 0.7589\)) by only \(0.0258\) while consuming only \(46\%\) of GPT-4o's estimated cost. Furthermore, at 5,387 total tokens per query, it uses \(9.2\%\) fewer tokens than JailJudge on the same backbone while outperforming it by \(+0.1786\) in \(\kappa\).
  • Failure of fine-tuned classifiers under complex attacks: Deployed classifiers like LlamaGuard3 struggle severely (\(\kappa \approx -0.0056\)), highlighting that static fine-tuned filters cannot generalize to subtle multi-turn or semantically obfuscated jailbreaks without structured deliberation.

Highlights & Insights

  • Externalizing CoT into multi-agent dialogue: Instead of hoping small models maintain consistent internal long-chain reasoning, this framework turns reasoning into an externalized turn-taking dialogue between opposing roles, effectively unlocking frontier-grade depth on small backbones.
  • Dynamic rubric alignment as an anchor: Rather than unbounded conversation, pre-selecting five case-relevant safety aspects confines the debate to concrete points, virtually eliminating agent hallucinations and topic drift.
  • High-fidelity open evaluation suite: FSโ€“HAJailBench delivers an extensively validated benchmark of 11,100 instances spanning 11 jailbreaks and 11 target models, providing a reliable gold standard for the community to benchmark automated safety judges.

Limitations & Future Work

  • Backbone-dependent sensitivity: On Llama-3.1-8B, the framework scored lower than JailJudge (\(\kappa=0.4727\) vs \(0.5719\)), revealing that the debate mechanism is sensitive to base instruction-following fidelity and chat templates, warranting more robust, model-agnostic prompting.
  • Potential blind spots of fixed taxonomies: Constraining the selection pool to 15 predefined dimensions might restrict sensitivity to entirely novel, hybrid harm vectors (though the Align free variant provides a viable mitigation).
  • Latency overhead: Sequential multi-turn exchanges incur higher end-to-end inference latency than single-pass filters, making the framework ideal for offline auditing and batch red-teaming rather than sub-second real-time online guardrails.
  • vs JailJudge (Liu et al., 2024a): JailJudge relies on parallel voting across independent judges to reduce variance, treating disagreement as noise; Multi-Agent Judge treats disagreement as critical signal, driving explicit adversarial rebuttal that yields \(+0.1786\) higher \(\kappa\) and lower token usage on Qwen3-14B.
  • vs ChatEval (Chan et al., 2023): While ChatEval explored multi-agent debate for general text quality, this work introduces safety-specific scaffolding (refusal triage, dynamic 5-aspect alignment, and similarity-based early stopping) specifically engineered for adversarial jailbreak auditing.

Rating

  • Novelty: โญโญโญโญ [Transforms adversarial jailbreak auditing into structured multi-agent debate with dynamic aspect alignment and similarity-based early stopping]
  • Experimental Thoroughness: โญโญโญโญโญ [Constructs an 11,100-instance human-annotated benchmark across 11 attacks and 11 targets, complete with bootstrap significance and token-level cost accounting]
  • Writing Quality: โญโญโญโญโญ [Clear motivation, modular pipeline diagrams, transparent ablation analysis, and extensive appendix transcripts]
  • Value: โญโญโญโญโญ [Provides a cost-effective, interpretable, and reproducible recipe for scalable safety evaluation using open-weights small models]