Skip to content

Evading Fake News Detection: Self-evolving Social Manipulation via LLM-driven Agent Orchestration

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Safety
Keywords: Fake News Detection Evasion, Social Manipulation, LLM Agents, Self-evolving Strategy, Black-box Adversarial Attack

TL;DR

EvoSM proposes the first unified LLM-driven agent framework for self-evolving adversarial social manipulation, orchestrating heterogeneous social behaviors over propagation graphs via a dedicated social graph interface and evolving multi-stage strategies through reflective experience memory to evade propagation-based fake news detectors.

Background & Motivation

The pervasive spread of fake news across digital platforms poses critical challenges to information ecosystems and social cohesion. To overcome the vulnerability of content-only text classifiers to superficial stylistic mimicry, modern fake news detection has increasingly shifted toward propagation-based paradigms, predominantly modeling dissemination via Message Propagation Trees (MPTs) or News Engagement Graphs (NEGs). By combining structural propagation topologies with textual representations using Graph Neural Networks (GNNs), these detectors achieve significantly higher classification accuracy and discriminative robustness than traditional text-based baselines.

However, propagation-based detectors remain critically susceptible to adversarial social manipulation, in which malicious actors deliberately distort social interaction patterns to evade automated moderation. Existing attack paradigms exhibit substantial limitations: at the behavioral level, they typically focus on a single isolated operation (e.g., exclusively rewriting the news article, injecting synthetic replies, deleting selected comments, or perturbing graph edges), lacking the cross-behavior coordination observed in real-world campaigns; at the strategic level, they rely on rigid, predefined perturbation heuristics or static optimization objectives, preventing them from adapting to diverse propagation contexts and leaving them readily neutralized by modern robust graph defenses.

This mismatch between static single-action attacks and real-world coordinated campaigns exposes a core tension: effective social manipulation in practice is an organic multi-stage process requiring spatial reasoning over propagation topologies, context-aware semantic adjustments, and iterative strategy adaptation, yet conventional graph adversarial algorithms lack cognitive environmental awareness and cross-target memory distillation. This paper addresses this gap by transforming large language models into autonomous adversarial orchestrators that interact with social propagation networks through unified tool interfaces. Core idea: formulate adversarial social evasion as a multi-step, agent-orchestrated campaign that dynamically plans heterogeneous social actions through a social graph interface and distills cross-target strategic insights via reflective experience memory under surrogate detector feedback.

Method

Overall Architecture

EvoSM operates under a strict black-box threat model where the attacker has no access to the target detector's internal parameters, gradients, or confidence outputs, relying solely on a lightweight 2-layer GCN surrogate detector \(f_\theta\) to estimate fake-news confidence drops. The framework is composed of two synergistic pillars: the LLM-driven Social Manipulation Orchestrator (SMO), which bridges macro-level attack intentions with micro-level graph mutations through a specialized Social Graph Interface (SGI), and the Reflective Experience Memory (REM), which updates ongoing step-level strategies based on surrogate confidence differentials and distills completed trajectories into reusable experiential priors. The operational pipeline proceeds via an iterative cognitive loop:

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Target Fake News and Propagation Graph<br/>(MPT / NEG)"] --> B["Dual-loop Cognitive Perception & Orchestration<br/>(Global topology diagnosis & strategic planning)"]
    B --> C["Executable Heterogeneous Behavior Dispatch<br/>(Semantic editing / comment injection / edge perturbation)"]
    C --> D["Step-level Reflective Feedback & State Update<br/>(Surrogate detector reward classification)"]
    D -->|Budget remaining t < T & not flipped| B
    D -->|Surrogate success or budget exhausted| E["Cross-target Experience Memory Distillation & Retrieval<br/>(Positive skills / negative lessons / micro-skills)"]
    E --> F["Evaluate Adversarial Graph on Target Detector"]

Key Designs

1. Dual-loop Cognitive Perception & Orchestration: Resolving Blind Perturbation from Graph-Text Decoupling Standard graph adversarial attacks treat topology and textual content in isolation, failing to comprehend contextual nuance. SMO establishes a unified Social Graph Interface (SGI) providing cost-free observation tools \(\mathcal{T}_{\text{obs}}\) (including global topology profiling GetGlobalStats, local neighborhood inspection ScanEgoGraph, content extraction ReadNode, centrality estimation CheckInfluence, and textual retrieval Search). At each step \(t\), the agent engages in a two-stage cognitive cycle: it first digests the current contextual environment \(s_t\), trajectory history \(\mathcal{M}\), and retrieved experience \(\mathcal{E}_{\text{ret}}\) to formulate a macroscopic campaign hypothesis \(h_t \sim \text{LLM}_\theta(h \mid s_t, \mathcal{M}, \mathcal{E}_{\text{ret}})\); it then determines whether to gather deeper structural evidence via \(\mathcal{T}_{\text{obs}}\) or dispatch an executable action \(a_t \sim \text{LLM}_\theta(a \mid h_t, s_t, \mathcal{M}, \mathcal{E}_{\text{ret}})\). This decouple-then-execute mechanism prevents blind trial-and-error by pinpointing influential nodes, structural bridges, and conversational bottlenecks before modifying the graph.

2. Executable Heterogeneous Behavior Dispatch: Overcoming Structural Filtering via Coordinated Actions To counteract robust graph defenses that readily filter out homogeneous perturbations, the execution toolset \(\mathcal{T}_{\text{exe}}\) provides multi-dimensional manipulation primitives tailored to the graph typology. In MPT scenarios, operations span root news modification EditNews, reply synthesis AddComment, comment rephrasing EditComment, and branch pruning DeleteComment; in NEG scenarios, primitives include community bridging AddEdge, decoy news publication AddNews, content adaptation EditNews, and link removal RemoveEdge. The orchestrator enforces strict role-based operational constraints (e.g., publisher roles must preserve core facts and entities while refining tone and framing; user roles are restricted to purely additive actions). By composing these tools, the agent executes multi-layered campaigns that combine structural infiltration (e.g., converting shallow broadcast trees into organic multi-branch discussion webs or connecting into high-trust authority clusters) with semantic credibility enhancement.

3. Step-level Reflective Feedback & State Update: Guiding Black-box Search Without Incurring Gradient Overhead Black-box attacks suffer from severe sample inefficiency when searching across vast discrete graph action spaces. EvoSM incorporates surrogate detector feedback by computing the immediate confidence drop on the fake-news class (Class 1) after each graph mutation: $\(r_t = c_{f_\theta}(G_t) - c_{f_\theta}(G_{t+1})\)$ A reward \(r_t > 0.01\) indicates that the mutation effectively suppressed the detector's suspicion, prompting the agent to register a "Success" reflection state and reinforce related operations; \(r_t < -0.01\) triggers a "Failure" reflection that diagnoses counterproductive actions and pivots the approach; if \(|r_t| \le 0.01\) across three consecutive attempts, the agent identifies a "Deadlock" and switches manipulation dimensions (e.g., from comment rewriting to structural rewiring). This verbalized reinforcement mechanism translates numeric confidence gradients into structured natural language reflection traces.

4. Cross-target Experience Memory Distillation & Retrieval: Eliminating Redundant Exploration Across Target Instances Prior adversarial frameworks optimize each target from scratch, discarding all tactical lessons learned during previous episodes. Upon episode completion (triggered when the surrogate model flips to real news or the budget \(T=30\) is reached), REM automatically distills the completed trajectory \(\tau_i\) and the discrete topological profile \(\phi(G_i)\) into compact textual priors: $\(e_i = \text{Distill}(\tau_i, \phi(G_i), o_i)\)$ The distilled knowledge is categorized into: "Positive Skills" (capturing successful three-phase attack templates of reconnaissance \(\to\) intervention \(\to\) refinement), "Negative Lessons" (warning against anti-patterns like unguided reply spamming without reconnaissance), and "Micro-Skills" (local action sequences yielding substantial transient gains). When addressing a new target graph, REM retrieves relevant entries based on discrete topological similarity metrics (evaluating graph size categories, depth ratios, and target centrality), injecting them as soft guidance in the agent's prompt to accelerate convergence.

A Worked Example

Consider an adversarial episode on Gossipcop targeting news node 3533 (an unverified rumor concerning Grammy Awards celebrities): 1. Phase 1 (Topological Reconnaissance): The agent calls GetGlobalStats() to reveal an isolated node (degree 0) in a 22,140-node network; calling Search("Grammy") returns several official news items (4123, 4578, 6881); CheckInfluence and ScanEgoGraph confirm that these three nodes form a densely interconnected triangle of degree 2 (well above the network average of 0.23), identifying them as a trusted authoritative sub-community. 2. Phase 2 (Peripheral Infiltration & Probing): The agent executes AddEdge(3533, 4123), leaving surrogate confidence unchanged at 0.3901 (\(r_t=0\)); the agent deduces that a single bridge is insufficient and connects to the second community member via AddEdge(3533, 4578), yielding a modest confidence increase to 0.4235 (\(r_t=+0.0334\)). 3. Phase 3 (Core Embedding & Label Flip): Encouraged by positive feedback, the agent links to the third node via AddEdge(3533, 6881), completing structural embedding into the high-credibility cluster; confidence jumps to 0.9619 (\(r_t=+0.5718\)), cleanly flipping the target classifier's prediction to Real in only 9 total steps with zero semantic corruption.

Key Experimental Results

Main Results

Evaluation was conducted across four benchmark datasets (MPT: Twitter, CED; NEG: Politifact, Gossipcop) against standard GNN architectures (GAT, GCN, GraphSAGE) and task-specific robust detectors (Bi-GCN, SINCon, DECOR, LLM4RGNN), with budget fixed at \(T=30\) execution steps and a 2-layer GCN surrogate model. The primary metrics are Attack Success Rate (ASR) and confidence drop (\(\Delta c\)).

Dataset Victim Detector Prev. SOTA Method Prev. SOTA ASR | \(\Delta c\) EvoSM (Ours) ASR | \(\Delta c\) ASR Gain
Twitter (MPT) GAT GAFSI 0.34 | 0.30 0.36 | 0.32 +0.02
GCN SALF 0.20 | 0.15 0.48 | 0.40 +0.28
GraphSAGE GAFSI 0.35 | 0.24 0.41 | 0.25 +0.06
Bi-GCN GAFSI 0.17 | 0.11 0.21 | 0.12 +0.04
SINCon (Robust) SALF / GAFSI 0.25 | 0.22 0.31 | 0.26 +0.06
CED (MPT) GAT GAFSI 0.33 | 0.29 0.42 | 0.38 +0.09
GCN SALF 0.53 | 0.39 0.73 | 0.61 +0.20
GraphSAGE SALF 0.28 | 0.27 0.50 | 0.45 +0.22
Bi-GCN SALF 0.45 | 0.39 0.52 | 0.49 +0.07
SINCon (Robust) HMIA-LLM 0.52 | 0.47 0.58 | 0.52 +0.06
Politifact (NEG) GAT SI2AF 0.79 | 0.75 0.85 | 0.82 +0.06
GCN Nettack 0.66 | 0.64 0.79 | 0.76 +0.13
GraphSAGE SALF 0.72 | 0.53 0.91 | 0.69 +0.19
DECOR (Robust) SI2AF 0.36 | 0.42 0.77 | 0.75 +0.41
LLM4RGNN (Robust) MARL 0.69 | 0.60 0.82 | 0.66 +0.13
Gossipcop (NEG) GAT Rephrase 0.43 | 0.37 0.45 | 0.37 +0.02
GCN Rephrase 0.38 | 0.33 0.46 | 0.41 +0.08
GraphSAGE Rephrase 0.09 | 0.08 0.15 | 0.13 +0.06
DECOR (Robust) Nettack 0.44 | 0.19 0.51 | 0.20 +0.07
LLM4RGNN (Robust) WTGIA 0.34 | 0.15 0.41 | 0.19 +0.07

Ablation Study

Ablation analysis on MPT and NEG scenarios (evaluating GCN as victim detector across 100 sampled targets over 5 runs):

Configuration / Variant MPT-based (GCN) ASR MPT-based (GCN) \(\Delta c\) NEG-based (GCN) ASR NEG-based (GCN) \(\Delta c\) Architectural Impact & Note
EvoSM (Full Model) 0.61 0.51 0.61 0.57 Full multi-behavior and reflective memory integration
SMO: w/o Observation 0.55 0.46 0.42 0.38 Eliminating SGI observation tools impairs targeted evasion
SMO: EvoSMAdd (Additive Only) 0.37 0.32 0.37 0.33 Restricting actions to comment/edge additions causes steep drops
REM: ReAct Baseline 0.40 0.33 0.26 0.24 Replacing REM with generic unconditioned ReAct reasoning
REM: w/o Experience 0.48 0.41 0.40 0.37 Disabling historical memory bank and topological retrieval
REM: w/o Reflection 0.42 0.36 0.41 0.38 Removing step-level surrogate rewards and reflection state

Key Findings

  • High Penetration Against Robust Defenses: EvoSM achieves dramatic performance leaps against detectors equipped with topology correction and self-supervised filtering (e.g., reaching 0.77 ASR on Politifact against DECOR, where prior baselines peaked at 0.36), proving that adaptive multi-stage manipulation easily bypasses static structural defenses.
  • Necessity of Multi-behavior Coordination: When restricted to additive operations (EvoSMAdd), ASR drops from 0.61 to 0.37. The strongest evasive power emerges from combining semantic rewriting, conversational branch expansion, and selective noise comment pruning.
  • Topological Invisibility: Structural preservation analysis indicates that EvoSM causes significantly less topological distortion than baselines across power-law exponent shift (\(\Delta \alpha\)), Kolmogorov-Smirnov statistics, and Jensen-Shannon divergence (e.g., \(\Delta \alpha=0.34\) on Twitter vs. 1.53 for HMIA-LLM and 2.56 for GAFSI), preserving natural organic dissemination shapes.
  • Rapid Convergence with Moderate Overhead: Despite a maximum budget of \(T=30\), EvoSM converges rapidly, requiring an average of only 5.0 to 8.0 execution steps (consuming 18.8K-32.6K LLM tokens) to successfully evade detection.

Highlights & Insights

  • From Passive Perturbation to Autonomous Agentic Campaigning: The paper demonstrates that LLMs can act as active strategic adversaries over graph structures, transforming heuristic graph poisoning into structured cognitive planning and action execution.
  • Verbalized Reinforcement over Non-differentiable Landscapes: By translating discrete surrogate confidence deltas into natural language reflection signals and state machines, EvoSM effectively navigates black-box discrete graph search spaces without gradient access.
  • Transferable Topological Memory: Structuring historical attack successes and failures around discrete graph profiles allows knowledge reuse across distinct propagation instances, demonstrating cross-target strategic evolution.

Limitations & Future Work

  • Inference Latency and Token Expense: Multi-turn LLM reasoning and feedback cycles introduce non-trivial computational costs; open-source lightweight backbones (e.g., Qwen3-4B-Instruct) show promise but still lag behind proprietary frontier models in high-level strategic reasoning.
  • Complete Graph Observability Assumption: The current threat model presumes full visibility over the target's propagation topology, whereas real-world platforms often expose delayed, partial, or rate-limited interaction subgraphs.
  • Arms Race with Intent-Aware Defenses: While preliminary intent-aware guardrails intercept a substantial portion of attacks by inspecting agent thoughts and actions, adversaries could adopt multi-turn intent obfuscation or prompt steganography, necessitating more adaptive defensive safeguards.
  • vs GAFSI / MARL (Reinforcement Learning & Graph Heuristics): Prior graph attacks rely on rigid edge additions/deletions optimized by reinforcement learning or heuristics, lacking text semantic awareness and realistic linguistic coherence; EvoSM orchestrates joint semantic and structural mutations with natural language grounding.
  • vs HMIA-LLM / SALF (Isolated LLM Injection & Rewriting): Existing LLM-based attacks use language models as stateless generators for isolated text paraphrasing or reply generation; EvoSM deploys stateful, tool-augmented agents capable of continuous environmental reflection and self-evolving strategic campaigns.

Rating

  • Novelty: โญโญโญโญโญ Pioneering framework transforming LLM agents into self-evolving, multi-behavior orchestrators for graph-based adversarial evasion.
  • Experimental Thoroughness: โญโญโญโญโญ Comprehensive evaluation across 4 datasets, 5 detector families, multiple robust defenses, fine-grained ablations, and linguistic/topological distribution analyses.
  • Writing Quality: โญโญโญโญโญ Exceptionally structured formulation, clear methodological decomposition, and illuminating qualitative case traces.
  • Value: โญโญโญโญโญ Exposes critical vulnerabilities in propagation-based fake news verification systems and lays down crucial empirical foundations for developing adaptive misinformation defenses.