AAAI 2026 Audio & Speech Policy Debate Multi-Agent Collaboration LLM Evidence Retrieval Autonomous Debating TTS

DeepDebater: A Superpersuasive Autonomous Policy Debating System¶

Conference: AAAI 2026
arXiv: 2511.17854
Code: GitHub
Area: Audio and Speech
Keywords: Policy Debate, Multi-Agent Collaboration, LLM, Evidence Retrieval, Autonomous Debating, TTS

TL;DR¶

This paper proposes DeepDebater, the first autonomous multi-agent system capable of participating in and winning a full policy debate tournament (eight speech rounds + cross-examinations). It utilizes a hierarchical agent workflow division of labor to construct Affirmative (Advantage) and Negative (DA + CP + Kritik) arguments. Supported by retrieval augmentation on over 3 million evidence cards from OpenDebateEvidence, alongside GPT-4o TTS speech synthesis and EchoMimic digital human animation, the system significantly outperforms human-authored cases across all metrics in expert evaluations (Quality 4.32 vs 3.65) and achieves an 85% win rate in simulated matchups.

Background & Motivation¶

Highly complex, evidence-based, and strategically adaptive persuasion remains a fundamental challenge for AI. IBM Project Debater was previously the most renowned AI debating system, but it suffers from three fundamental limitations:

First, the format is overly simplified—it adopts a non-standard, extremely short debating format tailored for lay audiences, a configuration for which no real-world tournaments exist. Authentic American policy debates consist of eight speech rounds and four cross-examinations, with a strictly defined and highly complex structure.

Second, the use of evidence is superficial—Project Debater cites scant evidence for short speeches, whereas the foundation of real policy debate lies in "cards" (evidence units comprising pages of direct quotations, highlighting, and taglines). Debaters must cite evidence intensively and almost verbatim.

Third, it lacks iterative game-playing—Project Debater does not engage in full, multi-turn debates, failing to handle refutations and strategic gameplay. Real debates require addressing every point made in the preceding round speech by speech.

High school and collegiate Policy Debate is an ideal testing ground for AI argumentation research: it features strict time limits, relies heavily on massive quantities of high-quality evidence, is highly structured, and demands both long-term strategic planning and real-time tactical decision-making. Core Idea: Decompose the complex debating task into a pipeline of hierarchical and specialized agent workflows, where each agent team is responsible for a discrete argumentative task.

Method¶

Overall Architecture¶

A modular pipeline framework comprising two core components: the OpenDebateEvidence library indexed by DuckDB (over 3 million debate cards, queried via BM25 retrieval) and a hierarchical multi-agent conversation architecture based on the AG2/Autogen framework (powered by gpt-4.1-mini), complemented by an end-to-end presentation utilizing GPT-4o TTS for speech synthesis and EchoMimic V1 for digital human animation.

Key Designs¶

Reusable Multi-Agent Workflow Pattern
- Function: Defines the fundamental building blocks of argument generation.
- Mechanism: Each workflow is comprised of specialized cooperating agents: the Generator produces draft arguments, the Retriever retrieves and ranks evidence (often filtering the top cards from hundreds of retrieved candidates for each point), and the Critic reviews the quality and proposes revisions. The process iterates until the Reviewer Agent is satisfied or the iteration limit is reached. Pydantic models are used to enforce structured output, ensuring agent messages are machine-readable.
- Design Motivation: Each argumentative component of a policy debate (such as Link, Impact, and Uniqueness in an Advantage, or the various Stock Issues in a Disadvantage) requires independent evidence support and logical formulation. Separating agent roles allows each task to be specialized, while the Critic mechanism prevents low-quality outputs.
Debate Generation Pipeline
- Function: Generates the complete debate sequentially according to the eight-round structure of policy debate.
- Mechanism: 1AC (1st Affirmative Constructive): Divided into three stages: Plan text generation (formulating the policy proposal) $\rightarrow$ Stock Issue workflow (individual agent teams for Harms, Inherency, and Solvency) $\rightarrow$ Advantage generation (each containing a chain of Uniqueness, Link, Internal Link, and Impact evidence). 1NC (1st Negative Constructive): Strategic combination generation $\rightarrow$ Off-Case workflow (Topicality/Theory, Disadvantages with complete evidence chains, Counterplans with alternative advocacy, and Kritiks comprising philosophical/ethical challenges + Alternatives) $\rightarrow$ On-Case refutation (directly targeting the evidence of the 1AC). Subsequent Speeches: 2AC $\rightarrow$ 2NC $\rightarrow$ 1NR $\rightarrow$ 1AR $\rightarrow$ 2NR $\rightarrow$ 2AR, with each round taking the complete preceding transcript as context. Cross-Examination: A two-agent dialogue simulating strategic question-and-answer exchanges. Adjudication: A standalone Judge Agent (Claude, Gemini, or GPT-4.1) reviews the entire transcript and renders a Reason for Decision (RFD).
- Design Motivation: Strictly adheres to the authentic format of policy debates—format compliance is a core factor distinguishing novice debaters from champions.
End-to-End Presentation + Human-AI Collaboration
- Function: Translates textual debates into an audible and visual interactive experience.
- Mechanism: GPT-4o-mini TTS synthesizes speech from the written scripts $\rightarrow$ EchoMimic V1 drives static portraits with the audio to generate lip-synced digital human videos. Text transcriptions on the screen are preserved to emulate the traditional "flowing" practice in debate. It supports three modes: fully automated (AI vs AI), mixed hybrid teams (AI + Human), and competitive matches (AI vs Human).
- Design Motivation: The essence of policy debate is live interaction—judges need to hear the speech and observe the delivery. Digital human rendering grants the AI debaters a sense of "presence." The human-AI collaborative design allows human intervention at any stage, serving both as a research tool and increasing academic engagement.

Loss & Training¶

This work does not involve model training. It relies entirely on zero-shot prompt reasoning of LLMs (gpt-4.1-mini) + BM25 retrieval augmentation. The cost is approximately $1–3 per round (text only), $3–5 with speech synthesis, and $20–50 with digital human video generation.

Key Experimental Results¶

Main Results¶

Experiment 1: Expert Evaluation (5 debate coaches with 10+ years of experience, scored on a 1-5 scale)

Metric	DeepDebater	Human-Authored	Gain
Quality (Strategic Coherence + Persuasiveness)	4.32 ± 0.31	3.65 ± 0.52	+0.67
Factuality (Factual Accuracy)	4.45 ± 0.25	3.98 ± 0.23	+0.47
Faithfulness (Tagline Fidelity)	4.81 ± 0.19	4.05 ± 0.48	+0.76

Experiment 2: Simulated Matches (20 rounds, evaluated by AI judges)

Scenario	Matches	DeepDebater Win Rate
System as Neg vs Human Aff Case	10	90%
System as Aff vs Human Neg Strategy	10	80%
Total	20	85%

Ablation Study¶

Experiment 3: Cross-Judge Robustness (using the same 20 debate transcripts)

Judge Model	Win Rate (%)	Δ vs Gemini (pp)	Cohen's $\kappa$ vs Gemini
Gemini	85	0	—
Claude	80	-5	0.75
GPT-4.1	83	-2	0.89

Key Findings¶

Faithfulness shows the largest gain (+0.76): The AI demonstrates its most pronounced advantage in "accurately summarizing evidence with taglines," which happens to be one of the core skills in policy debate.
The judges' RFDs frequently point out that the system exhibits higher quality and density of evidence, alongside more comprehensive line-by-line refutations.
Good consistency is observed across the three judge models ($\kappa = 0.75 - 0.89$), though the risk of AI judge bias remains.
The system achieves a higher win rate when acting as the Negative (90% vs. 80%), likely because the Negative can deploy more targeted strategies against a known Affirmative case.

Highlights & Insights¶

First Complete Policy Debate AI System: Covers eight rounds of speeches, cross-examination, refutation, and adjudication, with a level of complexity far exceeding IBM Project Debater.
Exquisite Hierarchical Multi-Agent Division of Labor: Every debating component (Advantage, Disadvantage, Counterplan, Kritik, Topicality) is handled by a specialized workflow. Quality is ensured through an iterative generation-retrieval-review loop.
Heavy Evidence Support: Grounded in over 3 million real-world debate evidence cards, where every claim can be traced back to specific citations—representing a robust step toward realizable and trustworthy AI argumentation.
Human-AI Collaborative Design holds practical value: It serves not only as a fully automated tool but also as a training and preparatory assistant for human debaters.
Candid Discussion on AI Persuasion Risks: The authors responsibly list the potential misuse risks, such as micro-targeted manipulation, information warfare, and social engineering attacks.

Limitations & Future Work¶

Weakness in BM25 Retrieval: The authors acknowledge that dense retrieval would significantly enhance quality, but it was not implemented due to portability and computational costs. Relying on sparse keyword matching may overlook high-quality evidence that is semantically relevant but uses different vocabulary.
Small Evaluation Scale and AI Judge Bias: The evaluation relies on only 5 experts $\times$ 3 replicates for human evaluation and 20 simulated matchups evaluated by AI judges. This limited scale fails to meet rigorous standards for statistical significance, and AI judges might suffer from stylistic inheritance or familial biases.
Outdated Evidence Base (ending in 2022): The system is prompted to simulate being in 2022. Empowering the system to automatically "cut cards" (generating new evidence units directly from open literature) would drastically boost capabilities.
Limited to English and US-style Policy Debate: The system has not been tested on other debate formats (e.g., British Parliamentary, Lincoln-Douglas) or in non-English languages.
Unverified Adversarial Robustness: No stress testing has been performed against adversarial opponents, such as those employing highly unconventional strategies, prompt injection attacks, or poisoned evidence.
Computational Costs and API Dependencies: Generating a full debate entails a vast number of API calls (gpt-4.1-mini + TTS + EchoMimic). Reproducibility is tightly coupled with API availability and model drift.
Winning Strategy != Finding Truth: The system's optimization objective is to secure a ballot from the judge rather than pursuing objective truth or calibrating uncertainty.

Compared with IBM Project Debater, DeepDebater achieves a qualitative leap in task complexity—moving from short speeches in a simplified format to a full eight-round policy debate. Compared to standard applications of multi-agent frameworks like AutoGen, this work demonstrates the capability of hierarchical agent workflows on extremely complex and creative tasks.

The paper's title and capitalization format serve as a tribute and response to the final publication of IBM Project Debater ("An autonomous debating system")—advancing from "an autonomous" to "a superpersuasive autonomous."

The system's hierarchical workflow architecture can be generalized to other scenarios requiring multi-stage, multi-role, and evidence-intensive content generation (e.g., legal drafting, academic writing, and think-tank reporting). Discussions on the "dual-use" risks of AI persuasion offer valuable references for AI safety research—particularly the idea of decomposing persuasive capability into independently evaluable components (evidence retrieval, argument construction, and strategic planning).

Rating¶

Novelty: ⭐⭐⭐⭐ First complete policy debate system with a novel hierarchical agent architecture, though the core techniques (RAG + multi-agent) are not intrinsically new.
Experimental Thoroughness: ⭐⭐⭐ Includes expert assessments and simulated matchups, but the scale is relatively small (5 experts $\times$ 3 replicates, 20 matchups) and lacks statistical power.
Writing Quality: ⭐⭐⭐⭐ Contains detailed background information, a clear system architecture description, and a comprehensive, responsible discussion on dual-use risks.
Value: ⭐⭐⭐⭐ Demonstrates the potential of LLMs + multi-agent systems in extremely complex argumentative tasks, providing instructions for AI safety and persuasion research.