Skip to content

Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors

Conference: ACL2025
arXiv: 2505.24523
Code: gpucce/control_mgt
Area: LLM/NLP
Keywords: Machine-Generated Text Detection, Adversarial Attacks, DPO, Linguistic Style Transfer, Robustness Evaluation, Linguistic Feature Analysis

TL;DR

Fine-tuning LLMs via DPO aligns their writing style with the linguistic feature distribution of human text, generating machine-generated text (MGT) that is significantly harder to detect, exposing the over-reliance of existing MGT detectors on shallow linguistic cues.

Background & Motivation

Background: The generation quality of LLMs (such as GPT-4, Llama 3, DeepSeek V3, etc.) has reached a level that is difficult for humans to distinguish, prompting various Machine-Generated Text (MGT) detection methods like MAGE, RADAR, and Binoculars. In shared tasks, top-performing systems achieve over 96% accuracy.

Limitations of Prior Work: Existing benchmarks saturate rapidly—detectors perform exceptionally well in controlled environments, but suffer severe performance degradation when facing out-of-distribution (OOD) samples. Doughman et al. (2025) pointed out that detectors rely heavily on shallow linguistic cues, such as punctuation patterns and average word length.

Key Challenge: The high accuracy of detectors creates an illusion that the problem is solved; in reality, they only learn surface-level stylistic differences between MGT and HWT, rather than deep semantic distinctions. This "linguistic shortcut learning" makes them highly fragile in real-world scenarios.

Goal: How to systematically expose the vulnerability of MGT detectors? Can a more challenging test benchmark be generated by aligning the writing style of LLMs?

Key Insight: Since detectors rely on the differences in linguistic feature distributions between MGT and HWT, aligning the generation style of LLMs with human writing using DPO can eliminate these shortcuts.

Core Idea: Align the linguistic feature distributions of LLMs (TTR, POS distribution, sentence length, etc.) with human text using DPO, generating stylistically human-like MGT to stress-test detectors.

Method

Overall Architecture

An iterative adversarial evaluation pipeline is proposed (Algorithm 1): 1. Select a human text dataset D (e.g., XSUM news, arXiv abstracts). 2. Generate MGT using LLM M prompted with titles to construct (HWT, MGT) parallel corpora. 3. Evaluate the performance of SOTA detectors on this corpus. 4. Fine-tune M to M' using DPO to make the generation style closer to HWT. 5. Iterate: Let M \(\leftarrow\) M' and repeat steps 3-4.

Key Designs 1: Two DPO Data Selection Strategies

Function: Construct the preference dataset, where HWT is preferred and MGT is dispreferred. Why: DPO directly adjusts model weights via preference pairs without training a reward model, serving as an efficient method for style alignment. Implementation: - dpo (random selection): Directly take HWT-MGT pairs as preference data, labeling HWT as preferred. - dpo-ling (linguistic-guided selection): Train an SVM classifier to extract the 10 most discriminative linguistic features, then select the top-k pairs with the maximum absolute distance between HWT and MGT for each feature.

Key Designs 2: Linguistic Feature Profiling System

Function: Extract over 130 linguistic features using the ProfilingUD tool. Why: Previous research indicates systematic differences in the distribution of linguistic phenomena between MGT and HWT. Implementation: Features cover three levels—lexical (TTR, lexical density, character/token ratio), morphosyntactic (UPOS distribution, verb morphology), and syntactic (clause length, proportion of postverbal subjects). An SVM based on these features achieves over 0.94 F1.

Key Designs 3: Iterative Alignment

Function: DPO can be executed iteratively (dpo-1 \(\rightarrow\) dpo-2). Why: A single alignment step might be insufficient, particularly for smaller models like Gemma. Implementation: Different sample pairs are used in each iteration to avoid repetition, and dpo-ling selects a different subset of linguistic features each time.

Loss & Training

  • Models: Llama 3.1-8B-Instruct, Gemma 2-2B-IT
  • LoRA: rank=32, applied to all attention layers
  • DPO hyperparameter search: Grid search on \(\beta\) and learning rate
  • Training set size: XSUM ~7k pairs, arXiv ~6k pairs

Key Experimental Results

Main Results: Detector Performance Degradation (Table 1 — Macro F1)

Detector Llama Original dpo-1 dpo-1-ling Drop
Mage (XSUM) 0.76 0.40 0.47 -36pp / -29pp
Radar (XSUM) 0.94 0.79 0.58 -15pp / -36pp
Binoculars (XSUM) 0.99 0.33 0.38 -66pp / -61pp
LLM-DetectAIve (XSUM) 0.72 0.53 0.54 -19pp / -18pp
Detector Gemma Original dpo-2 dpo-2-ling Drop
Mage (arXiv) 0.76 0.63 0.70 -13pp / -6pp
Binoculars (arXiv) 0.58 0.40 0.47 -18pp / -11pp
SVM† (arXiv) 0.99 0.83 0.87 -16pp / -12pp

Ablation Study: TPR @ Low FPR (Table 2)

Detector Llama dpo-1 dpo-1-ling
Mage TPR@5%FPR 0.997 0.057 0.176
Radar TPR@1%FPR 0.932 0.620 0.324
DetectAIve TPR@1%FPR 0.312 0.001 0.017

Linguistic Alignment Analysis (Table 3 — MANOVA Pillai's Trace)

Comparison Pillai's Trace Explanation
HWT vs Original MGT 0.7628 76.28% variance is distinguishable
HWT vs dpo-1 0.7635 Random DPO did not significantly reduce the difference
HWT vs dpo-1-ling 0.7137 Linguistically guided DPO effectively reduced the difference

Key Findings

  1. A single DPO iteration drastically degrades detector performance: average absolute drop of 5-35 percentage points (pp), with Binoculars dropping by up to 66pp.
  2. Mechanisms and distinctions between dpo and dpo-ling: Random dpo affects a broader feature distribution (thus better at fooling detectors), while dpo-ling accurately aligns specifically selected features (confirmed by MANOVA to be closer to HWT).
  3. RADAR is the most robust: Its adversarial training (simulating paraphrasing attacks) provides some resilience against distribution shift.
  4. Gemma is naturally harder to detect even originally, but benefits further after DPO (especially after the second iteration); Llama's alignment effect nearly saturates after the first iteration.
  5. Human Evaluation: Fleiss' Kappa is only 0.06-0.10, with most annotators' accuracies ranging between 0.40 and 0.60 (near random guess), indicating that MGT and HWT are already highly indistinguishable to humans.

Highlights & Insights

  • "Using your spear to strike your shield" paradigm: Instead of improving the detection end, this approach exposes the systematic weaknesses of the detector from the generation end—a highly effective path toward prompting the development of robust detectors.
  • Interpretable analysis of linguistic features: Conducting feature-by-feature style shift analysis via Jensen-Shannon divergence yields deeper insights than simply comparing accuracy—for example, Llama's TTR features are easiest to align, while Gemma's POS distribution is easiest to align.
  • Significant stylistic shifts with only ~7k samples: This demonstrates that LLM writing styles are not deeply embedded, and can be adjusted relatively easily through lightweight alignment.
  • Detector saturation \(\neq\) problem solved: A wake-up call for the MGT detection field, emphasizing the critical transition from "pursuing higher accuracy" to "aiming for robust generalization".

Limitations & Future Work

  1. Only 2 models were tested (8B and 2B): It remains unverified whether larger models (70B+) exhibit similar effects, and closed-source models were not tested.
  2. Only news and scientific writing domains were covered: High-risk scenarios such as social media, code, and dialogue were not investigated.
  3. Limited scale of human evaluation: Only 100 pairs per condition with 5 annotators, which may restrict statistical power.
  4. Defense strategies were not explored: The study only demonstrates the effectiveness of the attack without proposing methods to defend detectors against it.
  5. DPO alignment may impact generation quality: Although human evaluation did not observe obvious degradation, automated fluency and coherence evaluations are lacking.

vs RADAR (Hu et al., 2023)

RADAR enhances robustness through adversarial training (simulating paraphrasing attacks), making it the most resilient detector evaluated. Insight: Introducing distribution shift simulation during detector training is an effective strategy to improve robustness. However, RADAR only simulates paraphrasing attacks and still exhibits blind spots toward systematic, style-level distribution shifts.

vs Doughman et al. (2025)

This work diagnosed that detectors rely on shallow cues (punctuation patterns, average word length) but remained purely analytical. Ours translates this diagnosis into action—actively eliminating these cue differences using DPO—and quantifies the exact performance degradation.

vs MAGE (Li et al., 2024)

MAGE improves generalization via a massive training set constructed with 27 LLMs across 7 tasks, yet remains fragile against in-domain adversarial samples (F1 drops from 0.76 to 0.40). This indicates that data diversity cannot substitute for robustness against writing style alignment attacks.

Rating

  • Novelty: ⭐⭐⭐⭐ Combining linguistic feature profiling with DPO alignment to attack MGT detectors represents a highly novel and practical approach.
  • Experimental Thoroughness: ⭐⭐⭐⭐ Very comprehensive evaluation, featuring 6 detectors × 2 models × 2 domains × multiple DPO configurations + human evaluation + in-depth linguistic feature analysis.
  • Writing Quality: ⭐⭐⭐⭐ Highly structured and step-by-step presentation from methodology to experiments and analysis, featuring informative figures and tables.
  • Value: ⭐⭐⭐⭐ Provides practical tools and a solid methodology for robustness evaluation in the MGT detection field, delivering critical guidance for future work on the exposed "linguistic shortcut" issue.