Skip to content

Can AI-Generated Persuasion Be Detected? Persuaficial Benchmark and AI vs. Human Linguistic Differences

Conference: ACL 2026 arXiv: 2601.04925 Code: https://github.com/ArkadiusDS/Persuaficial Area: Robotics & Embodied AI Keywords: Persuasion Detection, AI-Generated Text, Multilingual Benchmark, Linguistic Difference Analysis, Controllable Generation

TL;DR

Persuaficial is a high-quality multilingual benchmark covering six languages for AI-generated persuasive text. Systematic evaluation reveals that subtle AI persuasion is harder to detect than human persuasion (F1 drops ~20%), while intensified persuasion is paradoxically easier to detect.

Method

Key Designs

  1. Four Controllable Generation Strategies: Paraphrasing (semantic equivalence), subtle rewriting (more covert), intensified rewriting (enhanced persuasion), and open-ended generation. Each simulates different real-world AI persuasion abuse scenarios.

  2. Multilingual Multi-Source Construction: Three human persuasion datasets × four LLMs × six languages.

  3. 196-Dimensional Linguistic Feature Analysis: Using StyloMetrix for interpretable fine-grained analysis.

Key Experimental Results

Strategy F1 Change vs Human
Human 0.740 —
Subtle Rewriting 0.403 ↓46%
Intensified Rewriting 0.815 ↑10%
Open-Ended 0.896 ↑21%

Highlights & Insights

  • "More subtle = harder to detect, more intense = easier to detect" is intuitive but first quantitatively verified here
  • First systematic study of AI persuasion vs human persuasion detectability differences

Rating

  • Novelty: ⭐⭐⭐⭐
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐
  • Writing Quality: ⭐⭐⭐⭐
  • Value: ⭐⭐⭐⭐