Skip to content

Structure-Aware Data Augmentation for Argument Mining

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/YuqiHuang2003/QOS_DOS
Area: NLP Understanding
Keywords: Argument Mining, Data Augmentation, Probabilistic Context-Free Grammar, Structure-Aware Generation, TANL Framework

TL;DR

This paper proposes a structure-aware data augmentation framework for argument mining that samples diverse pseudo-argumentative tree topologies using a PCFG learned from training data, linearizes them into TANL templates, and guides an LLM to generate text under structure-aware demonstration selection and automated diagnostic feedback, significantly boosting component and relation extraction performance.

Background & Motivation

Argument mining (AM) aims to automatically extract argumentative units and their logical dependencies (such as support and attack relations) from unstructured texts, organizing discourse into tree or directed acyclic graph (DAG) topologies. Because argumentative structures span entire documents and exhibit intricate multi-level hierarchies, constructing high-quality annotated corporaโ€”such as the persuasive essay corpus AAEC or the biomedical abstract dataset AbstRCTโ€”requires immense human effort. Bound by specific essay prompts and individual annotator styles, existing benchmark datasets capture only a small fraction of potential argumentative structures. In the AAEC test set, for instance, all 80 documents have distinct argumentative topologies, yet only 5.00% of these configurations ever appear in the training corpus. Such severe structural sparsity causes neural argument mining models to overfit to narrow argumentative formulas and struggle when generalizing to varied real-world discourse.

To mitigate data scarcity, data augmentation has been widely applied across NLP tasks. However, conventional techniques primarily introduce lexical variations (e.g., EDA, back-translation), which leave underlying inter-sentential topologies untouched; meanwhile, syntax-guided subtree perturbations operate strictly within local sentence boundaries and cannot synthesize document-level argumentative topologies. Recent AM-oriented synthesis strategies have explored quality-oriented synthesis (QOS) and diversity-oriented synthesis (DOS): QOS strictly preserves existing structures to generate paraphrases, failing to introduce structural diversity; conversely, DOS prompts LLMs without explicit structural constraints to produce novel essays, but relies on post-hoc auto-annotation to reconstruct argument structures, inherently propagating severe annotation noise into the augmented training pool.

To resolve these challenges, this paper reverses the standard generative order: rather than generating raw text and attempting error-prone post-hoc parsing, the system first deterministically creates valid, novel argumentative tree skeletons and then guides an LLM to synthesize natural text within these structural boundaries. Core idea: learn a probabilistic context-free grammar (PCFG) from training argument topologies and sample diverse pseudo-argumentative trees via beam search, linearize these trees into TANL templates with explicit slot dependencies, and employ structure-aware demonstration selection (SDS) alongside an automated diagnostic revision loop to guide an LLM in generating text with exact label alignment.

Method

Overall Architecture

The method casts argument mining as a structured text-to-text generation task within the TANL (Translation between Augmented Natural Languages) framework. The end-to-end data augmentation pipeline comprises three collaborative modules: the Topic Generator, the Structure Generator, and the Augmenter. First, the Topic Generator produces novel essay topics or paper titles, while the Structure Generator learns a PCFG from gold argument trees and samples a rich pool of non-isomorphic candidate tree skeletons via beam search. The top ranked skeletons are linearized into bracketed TANL templates with unique identifiers and explicit dependency bindings. Finally, the Augmenter uses structure-aware demonstration selection based on tree edit distance to retrieve in-context examples, prompting an LLM to instantiate fluent text across all template slots, followed by an automated validator that triggers up to two revision rounds to guarantee perfect syntactic and relational conformance.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Original Training Corpus<br/>Extract text-free argument tree topologies"] --> B["Train Document-Level PCFG<br/>Estimate rule production probabilities via MLE"]
    B --> C["PCFG Beam Search & Deduplication<br/>Filter depth/node constraints to select Top-K trees"]
    C --> D["Linearize into TANL Templates<br/>Assign unique slot variables and explicit relation targets"]
    E["Topic Generator<br/>Zero-shot prompting for diverse essay/paper titles"] --> F["Structure-Aware Demonstration Selection (SDS)<br/>Retrieve nearest examples via Tree Edit Distance"]
    D --> F
    F --> G["Text Augmenter & Few-Shot Synthesis<br/>LLM instantiates coherent discourse along template slots"]
    G --> H{"Rule Validation & Automatic Revision<br/>Validate bracket matching/label order/exact references"}
    H -->|Invalid output| I["Diagnostic Feedback Re-prompting<br/>Targeted structural repair with error feedback"]
    I --> H
    H -->|Valid or budget met| J["Decouple into Raw Text & Gold TANL Labels<br/>Feed into two-stage model fine-tuning"]

Key Designs

1. Document-Level Argument Topology Generation via PCFG and Beam Search: Breaking the Structural Scarcity Bottleneck

To overcome the severe topological sparsity of manual annotations, this design abstracts training documents into text-free tree structures while preserving hierarchical dependencies among components. Each production rule models how a parent component expands into child components along with their connective relations: \(\alpha \to \beta\), where \(\alpha \in \mathcal{C}\) is a parent component label and \(\beta = \{(r_1, c_1), \dots, (r_k, c_k)\}\) represents child labels and relation types (e.g., claim -> {(support, premise), (attack, premise)}). Production probabilities \(P(\alpha \to \beta)\) are estimated via maximum likelihood based on observed expansion frequencies in the training data, incorporating a special terminal symbol None to capture leaf node distributions.

During topological inference, beam search recursively expands non-terminal nodes starting from ROOT. Each candidate tree is scored by the log product of its applied production rules:

\[\log P(T_j^{\text{aug}}) = \sum_{(\alpha \to \beta) \in T_j^{\text{aug}}} \log P(\alpha \to \beta)\]

To exclude degenerate skeletons (overly trivial, excessively deep, or repetitive), the search enforces structural filters: AAEC structures must have at least 6 nodes with a maximum depth of 6, while AbstRCT structures require at least 1 node with a maximum depth of 4. Completed valid trees are collected, topologically deduplicated, and ranked by probability to retain the top-\(K\) unique configurations (2,000 for AAEC and 1,000 for AbstRCT). This guarantees that synthesized skeletons reflect plausible human reasoning conventions while exploring unobserved topological combinations.

2. TANL Template Linearization and Structure-Aware Demonstration Selection (SDS): Eliminating Post-Hoc Labeling Noise

To enable language models to interpret abstract tree graphs and fill them reliably, each generated tree is traversed and flattened into a TANL template sequence where each component is assigned a unique variable identifier. For example, an argument branch is linearized into: [ <MC0> | major claim ] [ <C1> | claim for ] [ <P2> | premise | support = <C1> ]. By binding relations explicitly to placeholder tokens rather than fuzzy span indices, target boundaries and dependencies are structurally fixed before generation begins.

To provide optimal in-context context for a given structural skeleton, the authors introduce Structure-aware Demonstration Selection (SDS). Drawing from the original training corpus, SDS evaluates the normalized Tree Edit Distance (TED) between the generated target tree \(T^{\text{aug}}\) and each training tree \(T_m^{\text{org}}\):

\[\text{sim}(T^{\text{aug}}, T_m^{\text{org}}) = 1 - \text{TED}(T^{\text{aug}}, T_m^{\text{org}})\]

The system retrieves the top-\(k\) most topologically similar instances (\(k=1\) in experiments), formatting their topics, structural skeletons, and gold TANL texts into an in-context demonstration. This topological proximity dramatically lowers the LLM's uncertainty when instantiating complex nested hierarchies.

3. Grammatically Constrained Synthesis with an Automated Diagnostic Revision Loop: Guaranteeing High-Fidelity Label Alignment

In the generation phase, the Topic Generator's output \(S_i^{\text{aug}}\) and structural template \(T_j^{\text{aug}}\) are combined across pairs (e.g., 100 topics \(\times\) 2,000 structures yield 200,000 candidate instances for AAEC). The prompt instructs the LLM to strictly maintain bracket spans, label categories, and sequence order while replacing placeholders with natural discourse, mandating that the text following the relation operator (=) must match the generated target claim verbatim.

Following generation, a deterministic validator inspects the output. A generation is valid only if it parses correctly, contains no unexpanded placeholders, matches the exact span count and label sequence, and maintains verbatim claim references in all premise relations. If any structural flaws are detected (e.g., malformed brackets, dropped spans, mismatched references), diagnostic feedback is formulated (specifying expected vs. observed span counts and violation locations) and fed back into an iterative revision loop capped at two rounds. Empirically, initial generation yields only 49.3% structural validity, but two revision rounds elevate validity to 90.7% (a 41.4 percentage point gain), delivering high-precision training data without excessive inference cost.

Loss & Training

Downstream argument mining models are based on FLAN-T5 (Base, Large, XL, and XXL), fine-tuned efficiently via QLoRA (\(r=16, \alpha=32\), dropout 0.05). Training employs a two-stage regime: 1. Stage 1 (Silver Data Pre-tuning): Fine-tuning on the augmented synthetic corpus for 10,000 steps with a batch size of 8 (consuming up to 80,000 synthetic samples), optimized via adamw_bnb_8bit with a constant learning rate (2e-4 for Base/Large, 1e-4 for XL/XXL). 2. Stage 2 (Gold Data Adaptation): Fine-tuning on the original gold dataset for an additional 10,000 steps with linear learning rate decay (initial learning rate 4e-4 for Base/Large, 2e-4 for XL/XXL). Model checkpoints are selected according to the highest average of Component-F1 and Relation-F1 on the development set.

Key Experimental Results

Main Results

Experiments are evaluated on the student essay benchmark AAEC (80 test documents) and the biomedical abstract benchmark AbstRCT (100 test documents) under exact-match Component-F1 and Relation-F1 metrics. Baselines include training without augmentation, Quality-Oriented Synthesis (QOS), Diversity-Oriented Synthesis (DOS), and their combination (QOS + DOS at 25:75).

Model Size Method AAEC Comp. AAEC Rel. AbstRCT Comp. AbstRCT Rel.
FLAN-T5 Base w/o Augmentation 76.02 53.59 70.35 40.08
QOS (Bao et al., 2025) 77.10 55.64 71.59 40.22
DOS (Bao et al., 2025) 76.63 54.80 71.12 40.08
QOS + DOS (Bao et al., 2025) 77.02 55.94 71.62 40.19
Ours 77.48 56.36 72.00 40.30
FLAN-T5 Large w/o Augmentation 77.09 56.91 72.45 44.11
QOS (Bao et al., 2025) 77.47 57.02 73.62 44.85
DOS (Bao et al., 2025) 77.84 57.11 74.55 45.24
QOS + DOS (Bao et al., 2025) 77.33 56.99 74.43 45.88
Ours 78.97 57.19 75.73 45.78
FLAN-T5 XL w/o Augmentation 78.20 57.77 73.00 44.32
QOS (Bao et al., 2025) 79.36 58.72 75.02 46.32
DOS (Bao et al., 2025) 78.99 57.15 75.74 45.73
QOS + DOS (Bao et al., 2025) 79.28 58.73 75.26 45.99
Ours 79.92 58.88 75.20 46.11
FLAN-T5 XXL w/o Augmentation 80.96 61.73 75.10 47.37
QOS (Bao et al., 2025) 81.22 62.15 75.70 47.84
DOS (Bao et al., 2025) 81.10 61.98 75.42 47.63
QOS + DOS (Bao et al., 2025) 81.71 61.86 75.55 47.82
Ours 82.35 62.49 76.05 48.09

Ablation Study

An ablation on AAEC with FLAN-T5 Large isolates the impact of each core component:

Configuration Component-F1 Relation-F1 Note
Ours (Full Method) 78.97 57.19 Complete pipeline with PCFG, Topic Generator, and SDS
- w/o Structure Generator (PCFG) 77.80 56.85 Reusing gold structures without PCFG sampling (largest drop in Comp-F1)
- w/o Topic Generator 77.85 56.12 Reusing gold topics without novel generation (largest drop in Rel-F1)
- w/o SDS 78.45 57.07 Selecting in-context demonstrations randomly instead of by TED
Baseline (w/o Augmentation) 77.09 56.91 Fine-tuning exclusively on gold data without synthetic augmentation

Key Findings

  • Structural Coverage Directly Correlates with Performance Gains: On the AAEC test set (where all 80 documents exhibit distinct structures), the original training data covers only 5.00% of test tree configurations. While DOS only reaches 10.00%, the proposed method alone covers 21.25% (and 23.75% when combined with the training data), quadrupling the structural exposure and explaining the pronounced F1 gains on AAEC.
  • Complementary Roles of Structure and Topic: Disabling the PCFG Structure Generator causes the sharpest drop in Component-F1 (from 78.97 to 77.80), demonstrating that structural diversity is vital to prevent models from overfitting to rigid span layouts. Disabling the Topic Generator causes the steepest drop in Relation-F1 (from 57.19 to 56.12), indicating that lexical and topical diversity are essential for robust relational inference across varied contexts.
  • Data Scaling Dynamics: Scaling the synthetic volume from 10k to 200k samples (and sampled structures from 100 to 2,000) produces monotonic gains in Component-F1, plateauing at 78.97. In contrast, Relation-F1 initially dips below the no-augmentation baseline at 10k (56.83 vs. 56.91), requires at least 50k samples to show net benefits, and peaks at 200k (57.19) before degrading at 250k (57.10). This indicates that relation classification requires substantial exposure to diverse structures to learn generalizable patterns, whereas over-scaling forces beam search into low-probability long-tail trees that deviate from coherent human argumentation.

Highlights & Insights

  • Structure-First Generation Paradigm: Rather than relying on unconstrained generation followed by noisy post-hoc tagging, establishing a deterministic grammar skeleton before lexical instantiation guarantees zero label noise and strict syntactic fidelity.
  • Discourse-Level PCFG Adaptation: Elegantly adapts classical probabilistic context-free grammars from sentence-level syntactic parsing to document-level rhetorical structures, effectively capturing hierarchical argumentative transitions at minimal computational cost.
  • Dataset Diversity Determines Augmentation Utility: The findings empirically demonstrate that structural augmentation yields substantial returns on structurally dispersed tasks (such as AAEC, with 5% training-test structural overlap), whereas its marginal benefits diminish on formulaic domains (such as AbstRCT, with 57% overlap).

Limitations & Future Work

  • Bounded by Training Production Distributions: Because PCFG transition probabilities are derived from MLE on training data, beam search predominantly recombines observed high-frequency rules, leaving fundamentally rare structural archetypes underexplored.
  • Restricted to Tree Structures (Excluding DAGs): The PCFG formulation assumes each node has exactly one incoming parent edge. Consequently, it cannot directly accommodate directed acyclic graphs (DAGs) with multi-parent dependencies or cyclical reasoning paths, such as those in the CDCP legal corpus.
  • Hallucination in Domain-Specific Discourse: Synthetic texts generated by LLMs prioritize rhetorical coherence over factual veracity. In high-stakes domains such as clinical trials, synthetic abstracts must strictly serve as structural training signals rather than factual evidence.
  • vs. Quality-Oriented Synthesis (QOS / Back-Translation) (Bao et al., 2025; Sennrich et al., 2016): QOS strictly preserves existing document structures, providing zero additional structural coverage (remaining at 5.00% on AAEC); the proposed method expands test structure coverage to 21.25%, outperforming QOS by 1.13 Component-F1 points on FLAN-T5 XXL.
  • vs. Diversity-Oriented Synthesis (DOS) (Bao et al., 2025): DOS prompts LLMs freely to alter components, requiring post-hoc auto-annotation that injects label noise; this paper's structure-first TANL templates guarantee perfect label alignment, surpassing DOS across all model scales.
  • vs. Sentence-Level Syntactic Augmentation (Zhang et al., 2025): While prior syntactic augmentations mutate intra-sentential parse trees, this framework scales grammatical modeling to entire discourse-level argumentative structures.

Rating

  • Novelty: โญโญโญโญโ˜† (Pioneering structure-first generation with document-level PCFG and TANL templates to eliminate post-hoc annotation noise)
  • Experimental Thoroughness: โญโญโญโญโญ (Thorough evaluations spanning four model scales, structural coverage audits, scaling curves, and external LLM judge verification)
  • Writing Quality: โญโญโญโญโญ (Rigorous formalization, insightful analysis of structural coverage, and clear architectural diagrams)
  • Value: โญโญโญโญโ˜† (Provides an actionable, high-precision paradigm for document-level structured information extraction and synthetic data generation)