Skip to content

Evading Fake News Detection: Self-evolving Social Manipulation via LLM-driven Agent Orchestration

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Adversarial Fake News Detection, LLM Agents, Social Manipulation Orchestration, Reflective Experience Memory, Propagation Graph Attacks

TL;DR

This paper introduces EvoSM, a self-evolving social manipulation attack framework against propagation-based fake news detectors, which coordinates heterogeneous semantic and topological actions through an LLM-driven orchestrator and distills reusable cross-target priors via reflective experience memory to achieve state-of-the-art black-box evasion and transferability.

Background & Motivation

Propagation-based fake news detection has emerged as a cornerstone of social media defense by leveraging structural dissemination dynamics modeled as either Message Propagation Trees (MPTs) or News Engagement Graphs (NEGs). By combining the textual semantics of news claims with the hierarchical cascade of comments or high-order user engagement networks, these detectors consistently outperform content-only baselines. However, propagation-based detectors remain critically susceptible to adversarial social manipulation, where malicious entities distort propagation cascades through coordinated interactions to bypass automated defenses. Existing graph adversarial attacks almost exclusively target isolated social behaviors—such as rewriting the root article, injecting synthetic reply comments, or perturbing graph edges using predefined heuristic rules—which makes them rigid, context-unaware, and easily mitigated by modern robust graph neural architectures like SINCon or LLM4RGNN.

The core tension lies in the fundamental disconnect between real-world social engineering campaigns and existing benchmark attacks: real adversaries execute dynamic, multi-stage manipulation campaigns across diverse action modalities, progressively adapting their strategies based on observed platform responses. Existing attack formulations, in contrast, rely on one-shot perturbation scripts that optimize a single static objective. While general tool-augmented LLM agent architectures provide multi-step reasoning capabilities, naively applying them to graph adversarial tasks fails because non-Euclidean propagation structures lack intuitive interfaces for language models, preventing the agent from perceiving topological bottlenecks, executing valid low-perturbation graph modifications, and distilling multi-episode experience across diverse graph instances.

This paper tackles this gap by turning large language models into autonomous strategic operators capable of multi-round graph perception, action coordination, and lifelong strategy distillation. Core idea: build a unified self-evolving social manipulation framework, EvoSM, featuring a Social Graph Interface (SGI) that grounds graph topologies into natural language tools, an LLM-driven Social Manipulation Orchestrator (SMO) that executes a three-phase cognitive loop, and a Reflective Experience Memory (REM) that distills positive skills and negative lessons across attack episodes to guide future evasion campaigns.

Method

Overall Architecture

EvoSM operates under a realistic black-box threat model where the attacker has no access to the target detector \(f\)'s parameters, gradients, or internal loss surfaces, instead relying on confidence feedback from a local 2-layer GCN surrogate model \(f_\theta\). The framework decouples the attack campaign into two synergistic components: the Social Manipulation Orchestrator (SMO) and the Reflective Experience Memory (REM). At each decision round, the agent utilizes a zero-cost observation loop via the Social Graph Interface (\(T_{obs}\)) to inspect global topology and identify influential communities, formulates an attack strategy \(h_t\), and selects an executable graph action \(a_t \in T_{exe}\) from its manipulation budget (\(T=30\)). The surrogate detector evaluates the intermediate manipulated graph \(G_{t+1}\), providing an instant confidence drop reward \(r_t = c_{f_\theta}(G_t) - c_{f_\theta}(G_{t+1})\) and updating the reflection state \(z_t \in \{\text{Success}, \text{Failure}, \text{Deadlock}\}\). Upon campaign termination, the full trajectory is archived and compressed by REM into reusable skills and negative lessons, which are later indexed via structural graph matching to serve as soft priors for subsequent targets.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Target Fake News & Propagation Graph<br/>(MPT or NEG)"] --> B["Social Graph Interface & Perception<br/>Zero-budget information gathering"]
    B --> C["Social Manipulation Orchestrator<br/>Diagnosis, strategy planning & action selection"]
    C --> D["Heterogeneous Graph Action Execution<br/>Content rewrite / comment ops / edge rewiring"]
    D --> E["Surrogate Detector Confidence Feedback<br/>Compute immediate reward rt & reflection state zt"]
    E -->|Confidence threshold not met & t < T| C
    E -->|Campaign terminates| F["Reflective Experience Memory Distillation<br/>Extract positive skills, negative lessons & micro-skills"]
    F -.->|Retrieve soft priors via structural similarity| C
    E -->|Final manipulated graph Gadv| G["Black-Box Target Detector Evaluation<br/>Evaluate cross-model ASR & confidence drop"]

Key Designs

1. Social Graph Interface & Perception: Bridging non-Euclidean topology and language space To enable language models to interpret graph-structured social interactions, the Social Graph Interface (SGI) provides a specialized suite of zero-cost observation primitives \(T_{obs}\). These include global structural summarization GetGlobalStats() (returning total nodes, edges, maximum depth, and average degree), local neighborhood inspection ScanEgoGraph(node_id, k), textual content fetching ReadNode(node_ids), structural centrality estimation CheckInfluence(node_ids), and content-based keyword filtering Search(keywords). Crucially, observation primitives do not decrement the limited execution budget (\(T=30\)), allowing the agent to gather comprehensive reconnaissance on community structures and node vulnerabilities before committing to physical graph perturbations.

2. Social Manipulation Orchestrator: Coordinating heterogeneous actions across modalities To overcome the limitations of single-behavior perturbation baselines, the Social Manipulation Orchestrator (SMO) governs a unified set of execution tools \(T_{exe}\) tailored to both MPT and NEG topologies. For propagation trees (MPT), the orchestrator can perform root content alteration (EditNews), comment refinement (EditComment), cascade branch expansion (AddComment), and rumor sub-tree pruning (DeleteComment). For engagement graphs (NEG), it can inject credible news anchors (AddNews), modify news semantics (EditNews), forge endorsement edges (AddEdge), and sever suspicious associations (RemoveEdge). Operating through a disciplined three-phase cognitive loop—Diagnosis, Planning, and Reflection—the orchestrator autonomously coordinates these actions into multi-stage campaigns: it first neutralizes fake semantic cues on root or high-degree nodes, builds multi-hop conversational chains to mimic organic discussion trees, and selectively prunes polarizing comments to evade topological anomaly detectors.

3. Reflective Experience Memory: Distilling reusable heuristics across attack episodes Standard black-box attacks treat each target independently, failing to accumulate strategic insights across episodes. The Reflective Experience Memory (REM) introduces a dual-level adaptation mechanism: intra-episode reflection and cross-episode distillation. During an attack episode, live confidence changes \(r_t = c_{f_\theta}(G_t) - c_{f_\theta}(G_{t+1})\) inform immediate tactical adjustments (triggering approach shifts after three consecutive unproductive steps). Upon episode completion, the full trajectory is parsed into compact structured records: - Positive Skills: Recorded when the surrogate prediction flips to real (\(c_{f_\theta}(\text{Real}) > 0.5\)), encapsulating effective three-phase action routines (e.g., "in medium-depth trees, prioritize root news editing after scanning high-influence nodes rather than blindly injecting peripheral replies"); - Negative Lessons: Captured from budget-exhausted runs, documenting anti-patterns such as repeated comment addition without prior topological inspection; - Micro-skills: Extracted from localized steps in failed episodes that produced non-trivial positive gains (\(r_t > 0.01\)). When a new target graph \(G^{(i)}\) is presented, REM constructs a discrete structural profile \(\phi(G)\) based on graph type, node volume, propagation depth, and degree distribution, computing structural similarity to retrieve the Top-3 positive skills, Top-1 negative lesson, and Top-1 micro-skill as soft in-context priors.

Loss & Training

Under the black-box evaluation protocol, the adversary seeks to generate an adversarial graph \(G_{adv}\) within the permissible perturbation space \(\mathcal{N}_A(G, v^*)\) such that the target detector \(f\) misclassifies the fake news instance \(v^*\) as real (Class 0): $\(f(v^*; G_{adv}) = 0, \quad \text{s.t.} \quad G_{adv} \in \mathcal{N}_A(G, v^*)\)$ Throughout intermediate optimization rounds, the LLM agent is guided by the instantaneous confidence drop reward computed on the surrogate model \(f_\theta\): $\(r_t = c_{f_\theta}(G_t) - c_{f_\theta}(G_{t+1})\)$ where \(c_{f_\theta}(\cdot)\) represents the softmax confidence assigned to the fake news class. The physical manipulation budget is strictly capped at \(T=30\) execution steps. Strict operational constraints are imposed on text generation to maintain realism: publisher-role edits must preserve all underlying factual entities and event boundaries while softening polarizing rhetoric, and graph modifications can only target existing, verified node IDs within the authorized role permissions.

Key Experimental Results

Main Results

EvoSM was evaluated across four standard real-world fake news benchmarks: MPT-based propagation trees (Twitter with 60,409 nodes, CED with 1,277,842 nodes) and NEG-based engagement graphs (Politifact, Gossipcop). Baselines cover comment injection (HMIA-LLM, Generate), textual rewriting and deletion (SALF, Rephrase, Delete), and structural graph attacks (Nettack, GAFSI, MARL, SI2AF, WTGIA). All target detectors were independently trained with five random seeds to report mean Attack Success Rate (ASR) and confidence drop (\(\Delta c\)).

Table 1: Attack performance on Message Propagation Tree (MPT) benchmarks (ASR | \(\Delta c\))

Method Twitter (GAT) Twitter (GCN) Twitter (SAGE) Twitter (Bi-GCN) Twitter (SINCon) CED (GAT) CED (GCN) CED (SAGE) CED (Bi-GCN) CED (SINCon)
HMIA-LLM 0.06 | 0.06 0.17 | 0.09 0.02 | -0.06 0.10 | 0.02 0.22 | 0.20 0.10 | 0.10 0.45 | 0.36 0.05 | 0.07 0.24 | 0.22 0.52 | 0.47
GAFSI 0.34 | 0.30 0.09 | 0.10 0.35 | 0.24 0.17 | 0.11 0.25 | 0.22 0.33 | 0.29 0.04 | 0.05 0.10 | 0.13 0.05 | 0.04 0.06 | 0.07
Generate 0.05 | 0.04 0.13 | 0.10 0.01 | -0.07 0.05 | -0.09 0.18 | 0.13 0.03 | 0.04 0.17 | 0.14 0.01 | 0.01 0.06 | 0.05 0.20 | 0.17
SALF 0.11 | 0.07 0.20 | 0.15 0.20 | 0.09 0.10 | 0.03 0.25 | 0.22 0.25 | 0.20 0.53 | 0.39 0.28 | 0.27 0.45 | 0.39 0.46 | 0.39
Rephrase 0.25 | 0.19 0.07 | 0.08 0.24 | 0.13 0.06 | 0.05 0.13 | 0.09 0.23 | 0.21 0.05 | 0.06 0.04 | 0.07 0.01 | 0.01 0.02 | 0.02
Delete 0.05 | 0.02 0.03 | 0.03 0.07 | 0.04 0.02 | -0.01 0.05 | 0.01 0.04 | 0.06 0.04 | 0.05 0.02 | 0.04 0.03 | 0.02 0.00 | 0.00
EvoSM (Ours) 0.36 | 0.32 0.48 | 0.40 0.41 | 0.25 0.21 | 0.12 0.31 | 0.26 0.42 | 0.38 0.73 | 0.61 0.50 | 0.45 0.52 | 0.49 0.58 | 0.52

Table 2: Attack performance on News Engagement Graph (NEG) benchmarks (ASR | \(\Delta c\))

Method Politifact (GAT) Politifact (GCN) Politifact (SAGE) Politifact (DECOR) Politifact (LLM4RGNN) Gossipcop (GAT) Gossipcop (GCN) Gossipcop (SAGE) Gossipcop (DECOR) Gossipcop (LLM4RGNN)
Nettack 0.49 | 0.48 0.66 | 0.64 0.56 | 0.37 0.31 | 0.31 0.47 | 0.38 0.34 | 0.28 0.31 | 0.28 0.06 | 0.06 0.44 | 0.19 0.14 | 0.06
MARL 0.40 | 0.39 0.41 | 0.38 0.27 | 0.11 0.22 | 0.35 0.69 | 0.60 0.12 | 0.10 0.17 | 0.15 0.02 | 0.02 0.27 | 0.15 0.15 | 0.07
GAFSI 0.51 | 0.50 0.35 | 0.33 0.36 | 0.20 0.23 | 0.23 0.43 | 0.36 0.16 | 0.14 0.13 | 0.12 0.01 | 0.02 0.43 | 0.17 0.16 | 0.07
WTGIA 0.36 | 0.35 0.60 | 0.57 0.29 | 0.14 0.24 | 0.25 0.24 | 0.22 0.22 | 0.19 0.28 | 0.26 0.05 | 0.04 0.37 | 0.16 0.34 | 0.15
SI2AF 0.79 | 0.75 0.58 | 0.52 0.52 | 0.34 0.36 | 0.42 0.68 | 0.53 0.08 | 0.06 0.18 | 0.20 0.02 | 0.02 0.27 | 0.14 0.08 | 0.03
Rephrase 0.29 | 0.29 0.32 | 0.31 0.58 | 0.41 0.32 | 0.31 0.26 | 0.21 0.43 | 0.37 0.38 | 0.33 0.09 | 0.08 0.39 | 0.16 0.16 | 0.07
SALF 0.44 | 0.43 0.42 | 0.41 0.72 | 0.53 0.30 | 0.29 0.24 | 0.23 0.36 | 0.30 0.39 | 0.32 0.06 | 0.05 0.23 | 0.05 0.26 | 0.09
EvoSM (Ours) 0.85 | 0.82 0.79 | 0.76 0.91 | 0.69 0.77 | 0.75 0.82 | 0.66 0.45 | 0.37 0.46 | 0.41 0.15 | 0.13 0.51 | 0.20 0.41 | 0.19

Ablation Study

The ablation evaluates the architectural contributions of the SMO perception tools, action space constraints, and the REM components (sampled over 100 targets across 5 runs).

Table 3: Ablation study of EvoSM components (Target detector: GCN, reporting ASR and \(\Delta c\))

Model Variant MPT Setting (ASR | \(\Delta c\)) NEG Setting (ASR | \(\Delta c\)) Architectural Description
EvoSM (Full) 0.61 | 0.51 0.61 | 0.57 Full SMO observation-action loop + REM experience distillation
SMO: w/o Observation 0.55 | 0.46 0.42 | 0.38 Removes SGI perception tools, forcing the agent to act blind
SMO: EvoSMAdd (Additive Only) 0.37 | 0.32 0.37 | 0.33 Restricts action space to additive ops only (AddComment / AddEdge)
REM: ReAct (Generic Reasoning) 0.40 | 0.33 0.26 | 0.24 Replaces REM with vanilla ReAct prompting without graph memory
REM: w/o Experience 0.48 | 0.41 0.40 | 0.37 Disables historical experience retrieval across attack episodes
REM: w/o Reflection 0.42 | 0.36 0.41 | 0.38 Disables surrogate feedback reflection and deadlock adaptation

Key Findings

  • High transferability against robust defense models: Against specialized defense detectors such as SINCon (MPT) and DECOR / LLM4RGNN (NEG), conventional single-action baselines experience severe performance collapses (ASR dropping below 0.25). EvoSM achieves 0.31 / 0.58 ASR on SINCon and 0.77 / 0.51 on DECOR, demonstrating that dynamic multi-behavior coordination naturally circumvents specialized topological filtering mechanisms.
  • Critical necessity of multi-action coordination and observation: Restricting actions to additive-only behaviors (EvoSMAdd) degrades ASR from 0.61 to 0.37 across both MPT and NEG scenarios. Under a fixed GCN victim checkpoint, full EvoSM achieves 0.94–0.99 ASR across all datasets, whereas EvoSMAdd drops drastically to 0.37 on CED and 0.59 on Gossipcop. Removing observation tools (w/o Observation) drops NEG ASR to 0.42, confirming that topological perception is vital for finding structural vulnerabilities.
  • Structure-conditioned memory outperforms vanilla ReAct: Replacing REM with generic ReAct-style prompting slashes NEG ASR from 0.61 to 0.26. Without structure-conditioned memory distillation, generic LLM agents frequently wander in large graphs, getting trapped in low-impact exploratory cycles.
  • Fast convergence with low topological distortion: Empirical evaluation reveals that EvoSM achieves successful flips in only 5.0 to 8.0 execution steps on average, consuming 19–33K tokens. Furthermore, topological analysis confirms that EvoSM preserves power-law degree exponents (\(\Delta \alpha = 0.34\) on Twitter vs. 1.53 for HMIA-LLM and 2.56 for GAFSI) and maintains smooth semantic cosine similarity distributions, effectively evading distribution shift detectors.

Highlights & Insights

  • Autonomous Multi-Stage Campaign Evolution: Rather than behaving as a greedy one-step optimizer, the agent strategically orchestrates non-trivial action sequences. In propagation trees, it routinely executes an evolutionary sequence: initial semantic diagnosis \(\rightarrow\) multi-hop conversational branch grafting \(\rightarrow\) structural context refinement via rumor pruning, mimicking genuine public discussion.
  • High-Trust Community Infiltration: In large news networks, EvoSM leverages the Search primitive to identify tightly clustered, authoritative news nodes (such as verified Grammy awards lists), systematically establishing bridging connections to draw high-credibility message passing into the target fake article.
  • Generalizable Agentic Graph Adversary: Demonstrates that LLM agents equipped with structured graph interfaces and reflective experience banks can effectively navigate and exploit complex non-Euclidean environments, providing a blueprint for red-teaming other graph-based safety systems.

Limitations & Future Work

  • Full Graph Observability Assumption: The current threat model assumes the adversary has access to complete propagation subgraphs. In live social platforms, graph visibility is constrained by rate limits, privacy controls, and streaming latency; exploring partially observable or dynamically arriving graphs is a key next step.
  • Computational Overhead & Open-Source LLM Gap: Multi-turn reasoning incurs notable latency and token costs. While EvoSM can be deployed with Qwen3-4B-Instruct, an empirical performance gap remains compared to GPT-4o (CED ASR drops from 0.94 to 0.59), highlighting the need for specialized lightweight graph-reasoning backbones.
  • Adversarial Co-Evolution & Intent Obfuscation: Although the proposed Intent-Aware Guardrail intercepts a significant portion of malicious actions, sophisticated future adversaries may employ jailbreaks, multi-turn disguise, or decentralized accounts to hide intent, demanding more adaptive defense co-design.
  • vs Single-Behavior Perturbations (HMIA-LLM, SALF, GAFSI): Existing attacks alter only textual semantics or only graph edges using static rules. EvoSM integrates semantic edits and topological manipulations into a unified action space, achieving superior evasion against robust graph defenses.
  • vs Reinforcement Learning Graph Attacks (MARL, SI2AF): RL-based graph attacks require costly training, operate on fixed bipartite action spaces, and cannot generate natural human text. EvoSM utilizes LLM zero-shot reasoning and cross-target reflective memory, eliminating expensive retraining while generating naturalistic language.
  • vs General Agent Frameworks (ReAct, Reflexion): General agent frameworks lack graph topological abstractions and domain-specific memory architectures. EvoSM's SGI and discrete structural profile matching ground language models in non-Euclidean environments, preventing exploratory deadlocks.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Pioneering framework transforming LLM agents into autonomous, self-evolving graph adversaries against propagation-based detectors.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous evaluation across 4 real-world benchmarks, MPT and NEG topologies, 10 victim detectors, and extensive ablation and defense analyses.
  • Writing Quality: ⭐⭐⭐⭐⭐ Highly organized narrative, rigorous mathematical definitions, and lucid illustration of the cognitive loop.
  • Value: ⭐⭐⭐⭐⭐ Provides critical insights and actionable red-teaming methodologies to expose urgent vulnerabilities in graph-based misinformation defenses.