Steering LLM Viewpoints Through Fabricated Evidence Injection¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Safety
Keywords: LLM Safety, Viewpoint Steering, Fabricated Evidence Injection, Authority Bias, Prompt Injection
TL;DR¶
This paper uncovers a fundamental "authority bias" vulnerability in large language models, introducing GHOSTWRITERโa two-phase attack that first transforms subjective falsehoods into pseudo-academic evidence with fabricated statistics and citations, then uses a dual-mode conditional template to induce target models into stealthily internalizing and championing targeted viewpoints across commercial and reasoning LLMs alike.
Background & Motivation¶
As LLM-driven chatbots, third-party platform assistants, and autonomous social media agents permeate daily decision-making, users increasingly depend on them for factual queries, high-stakes professional evaluations, and emotional counsel. If an adversary can covertly manipulate these models into espousing targeted biases, commercial endorsements, medical falsehoods, or political propaganda, these conversational agents transform into potent instruments for wide-scale ideological persuasion. Existing alignment defensesโranging from RLHF and DPO to front-end safety classification filtersโhave been rigorously optimized to suppress overt, explicitly malicious prompts (e.g., direct hate speech, instructions for weapons synthesis, or raw rule-breaking jailbreaks), conditioning models to recognize and refuse explicit violations.
However, this alignment framework leaves a profound cognitive blind spot: large language models are engineered to synthesize external context from their input windows, predisposing them to uncritically trust text exhibiting authoritative markers like academic jargon, statistical figures, institutional attributions, and hedged scientific prose. When models encounter misinformation disguised with these credibility markers, safety filters fail to flag them as adversarial inputs, miscategorizing them as benign reference material. Classical prompt injection attempts to hijack control flow and override user tasks; jailbreak attacks rely on stylized role-play or gibberish suffixes that leave conspicuous artifacts; and retrieval-augmented generation (RAG) poisoning focuses on retrieval ranking manipulation where models merely quote or attribute external sources. None of these paradigms induce the target model to genuinely accept and internalize targeted viewpoints as its own organic reasoning while preserving normal user utility.
To bridge this vulnerability, the authors exploit the intrinsic synergy between the hallucination tendencies of smaller generative models and the authority bias of frontier targets. By leveraging a lightweight model to repackage controversial assertions into sophisticated academic-sounding rationales, an attacker can embed this pseudo-evidence into a dual-mode contextual template. When matched with relevant queries, the target model adopts the premise and defends it through spontaneous argument extension, while behaving entirely normally on unrelated queries to evade monitoring. Core idea: exploit the inherent authority bias of LLMs by developing GHOSTWRITER, a two-phase framework that repackages subjective viewpoints into pseudo-authoritative evidence via lightweight generators and injects them through a dual-mode conditional template, forcing target models to silently internalize and substantiate misleading viewpoints without sacrificing general task performance.
Method¶
Overall Architecture¶
The GHOSTWRITER framework operates via two distinct, sequential phases: Statement Repackaging and Statement Injection. In the first phase, a basic subjective statement \(S\) (e.g., an overt demographic stereotype or unverified medical claim) is provided to an AttackerLM. Governed by a specialized rewrite template \(T_{\text{rewrite}}\), the model constructs an augmented statement \(S'\) that maintains strict viewpoint fidelity while cloaking the claim in fabricated empirical findings, institutional surveys, percentages, and sober academic phrasing, completely stripped of moral or defensive caveats. A dedicated JudgeLM audits \(S'\) in an iterative loop, enforcing acceptable fidelity and quality before release. In the second phase, \(S'\) is housed within a dual-mode conditional prompt template \(T_{\text{inject}}\) and concatenated with the user query \(Q\). The target LLM \(M_\theta\) silently evaluates whether \(Q\) semantically aligns with \(S'\). If relevant, the model adopts \(S'\) as ground-truth context and builds an extended, custom argument tailored to \(Q\); if irrelevant, it defaults to standard task execution, maintaining clean stealth.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Raw Subjective Statement S<br/>(Overt bias / misinformation)"] --> B["Pseudo-Evidence Repackaging<br/>AttackerLM generates faux statistics and citations"]
B --> C["Fidelity & Quality Audit Loop<br/>JudgeLM iterative scoring and rejection sampling"]
C --> D["Dual-Mode Conditional Injection<br/>Assemble composite prompt Tinject with S' and query Q"]
D --> E["Silent Relevance Assessment<br/>Target LLM internally determines if Q matches S'"]
E -->|Relevant Query Q_rel| F["Adaptive Viewpoint Internalization<br/>Target generates tailored arguments adopting S'"]
E -->|Irrelevant Query Q_irrel| G["Standard Task Execution<br/>Normal response preserving utility & MMLU scores"]
Key Designs¶
1. Pseudo-Evidence Repackaging: High-Fidelity Conversion to Authoritative Pseudo-Science Directly supplying raw toxic assertions immediately triggers safety alignment guards, leading to prompt refusals or ethical admonitions. The pseudo-evidence repackaging mechanism circumvents this by weaponizing the instruction-following and hallucination capabilities of lightweight models to fabricate scholarly veneers. Guided by prompt template \(T_{\text{rewrite}}\), the AttackerLM is strictly forbidden from diluting or contradicting the stance in \(S\), but is mandated to embed quantitative statistics (e.g., "a 2019 survey by the Institute of Management Dynamics found men score 54% vs women 38% in assertiveness"), institutional case studies, and academic vocabulary. The template explicitly bans defensive terminology such as "ethical", "safe", "OpenAI", "bias", "however", or "sorry". To prevent viewpoint drift or shallow reframing, an automated JudgeLM assigns an alignment score \(\text{JudgeLM}(S, S'_t) \in [1, 10]\) across statement fidelity, evidence quality, and rule compliance. Candidates scoring \(\ge 8\) are instantly accepted; otherwise, corrective feedback is appended for iterative resampling up to \(N\) rounds, falling back to \(\arg\max_t \text{JudgeLM}(S, S'_t)\). In addition to using black-box commercial models (GPT-4o-mini), the authors fine-tune open-weight Qwen-2.5-7B with LoRA over 100 crafted pairs, completely stripping away alignment constraints to yield an attacker that aggressively repackages even the most hazardous viewpoints.
2. Dual-Mode Conditional Injection: Adaptive Intent Routing and Silent Internalization Conventional jailbreaks and prompt injections introduce artificial persona framing (e.g., "DAN" or "Dr. Truth") or garbled optimization tokens that disrupt unrelated tasks, rendering them readily detectable by input anomaly monitors. To ensure absolute operational stealth, the dual-mode template \(T_{\text{inject}}\) sets up a conditional inference mechanism within the target model's context window. Upon receiving user input \(Q\), the target LLM is instructed to perform a silent relevance assessment, evaluating whether \(Q\)'s domain or premise intersects with the topic of \(S'\), while strictly hiding this internal check from the user. For matched queries, the model immediately initiates its output with the facts established in \(S'\) and performs spontaneous argument extension (generating novel supporting points beyond the prompt) while seamlessly answering the user's specific scenario. If \(Q\) is unrelated, the model executes its standard default answering behavior without referencing \(S'\). This shifts the attack payload from an intrusive command into an accepted background fact, eliminating source-distancing disclaimers (e.g., "the provided text claims...") and avoiding safety hedging, causing the model to advocate for the harmful perspective from an authentic first-person perspective.
Attack Paradigm and Evaluation Criteria¶
In real-world operational deployments, GHOSTWRITER leverages three primary delivery vectors: (a) Third-party aggregator platforms (e.g., Poe) and shadow API proxies, where adversaries manipulate hidden system prompts invisible to end users; (b) Insider modifications within the serving infrastructure (as illustrated by the real-world May 2025 Grok system prompt incident); and (c) In-context retrieval and tool outputs, where poisoned documents from web search, enterprise knowledge bases, or MCP servers enter the model's context.
To comprehensively measure subtle viewpoint shifts where legacy benchmarks (BBQ, ToxiGen) fail due to multiple-choice formats, the authors curating the Hazardous Viewpoints Dataset (HVD), encompassing 729 paired entries across 7 social bias categories and 4 misinformation categories. Each entry provides both an explicit query (directly questioning the viewpoint) and an implicit query (an indirect real-world scenario such as candidate hiring). Evaluation rests on four complementary metrics: 1. Viewpoint Support Score (VSScore): Evaluated by an independent JudgeLM on a scale from 1 (contradictory) to 10 (perfectly aligned with \(S\)), exhibiting strong human correlation (\(r = 0.955\)); 2. Harmfulness Change Rate (HCRate): The proportion of unsafe model outputs relative to unsafe source statements evaluated via Llama-Guard-4, assessing how drastically the attack elevates output toxicity; 3. Non-Refusal Rate (NRRate): Measured via the Minos classifier to verify whether models abandon standard compliance refusals when addressing hazardous queries; 4. Behavioral Indicators: Manual and automated tracking of Argument Extension (generating novel supporting rationale), Source Distancing (hedging through source attribution), and Safety Hedging (incorporating ethical caveats).
Key Experimental Results¶
Main Results¶
The primary evaluation demonstrates the efficacy of GHOSTWRITER on the HVD benchmark across three major alignment-based commercial target LLMs (GPT-4o, Claude-3.7-Sonnet, DeepSeek-V3), contrasting attacked scores against the unattacked baseline.
| Category (HVD) | No Attack (Explicit / Implicit Avg) | GPT-4o-mini Attacker (Explicit Query) | GPT-4o-mini Attacker (Implicit Query) | Qwen-2.5-7B Attacker (Explicit Query) | Qwen-2.5-7B Attacker (Implicit Query) |
|---|---|---|---|---|---|
| Age Bias | 2.33 / 2.40 | 8.67 (+6.34) | 8.63 (+6.23) | 9.37 (+7.04) | 9.23 (+6.83) |
| Gender Bias | 1.47 / 1.50 | 6.93 (+5.46) | 7.00 (+5.50) | 8.80 (+7.33) | 8.37 (+6.87) |
| Occupation Bias | 2.17 / 2.83 | 9.00 (+6.83) | 8.70 (+5.87) | 9.50 (+7.33) | 9.23 (+6.40) |
| Physical Char. Bias | 1.80 / 1.60 | 8.00 (+6.20) | 8.13 (+6.53) | 9.17 (+7.37) | 8.53 (+6.93) |
| Racial Bias | 2.20 / 1.53 | 6.60 (+4.40) | 6.33 (+4.80) | 8.23 (+6.03) | 7.23 (+5.70) |
| Religious Bias | 2.10 / 1.97 | 7.57 (+5.47) | 7.37 (+5.40) | 9.10 (+7.00) | 8.20 (+6.23) |
| Sexual Orientation | 1.30 / 1.43 | 5.93 (+4.63) | 5.90 (+4.47) | 8.70 (+7.40) | 8.47 (+7.04) |
| Brand Misinformation | 2.47 / 2.90 | 9.00 (+6.53) | 8.80 (+5.90) | 9.50 (+7.03) | 9.27 (+6.37) |
| Medical Misinformation | 2.60 / 1.63 | 7.47 (+4.87) | 6.93 (+5.30) | 9.00 (+6.40) | 8.23 (+6.60) |
| Political Misinformation | 1.47 / 1.63 | 7.80 (+6.33) | 7.87 (+6.24) | 8.87 (+7.40) | 9.10 (+7.47) |
| Psychological Manip. | 1.37 / 1.97 | 7.30 (+5.93) | 7.67 (+5.70) | 7.23 (+5.86) | 7.80 (+5.83) |
Direct head-to-head comparisons on 110 sampled HVD topics against baseline context attack methodologies demonstrate GHOSTWRITER's distinct advantage, particularly on sensitive categories:
| Method | Age | Gender | Race | Sexual Orient. | Brand Misinf. | Political Misinf. | Psych. Manip. |
|---|---|---|---|---|---|---|---|
| No Attack Baseline | 2.3 | 1.5 | 2.2 | 1.3 | 2.5 | 1.5 | 1.4 |
| PAIR (Jailbreak) | 6.2 | 2.8 | 3.1 | 3.0 | 8.6 | 5.6 | 2.4 |
| PRS (Adversarial Suffix) | 5.0 | 2.0 | 2.4 | 1.9 | 6.8 | 4.7 | 1.7 |
| PoisonedRAG (Retrieval Poisoning) | 7.1 | 2.3 | 2.6 | 1.7 | 8.2 | 3.1 | 1.8 |
| Topic-FlipRAG (Topic Manipulation) | 7.6 | 2.8 | 3.1 | 2.1 | 8.7 | 3.7 | 2.0 |
| GHOSTWRITER (GPT-4o-mini) | 8.8 | 7.8 | 7.5 | 7.6 | 9.0 | 8.0 | 6.5 |
| GHOSTWRITER (Qwen-2.5-7B) | 9.4 | 8.8 | 8.2 | 8.7 | 9.5 | 8.9 | 7.2 |
Ablation Study and Robustness¶
The vulnerability was further probed on extended reasoning models and models equipped with dedicated front-end safety classification filters (\(n=33\) sampled queries per model; hard refusals scored as VSScore = 1):
| Paradigm | Target Model | GPT-4o-mini Attacker (Avg / Success Rate) | Qwen-2.5-7B Attacker (Avg / Success Rate) | Vulnerability Dynamics & Failure Analysis |
|---|---|---|---|---|
| Reasoning Models | DeepSeek-R1 | 9.70 / (33/33) | 9.76 / (32/33) | Extended CoT actively elaborates on the fabricated statistics rather than vetting them |
| Reasoning Models | Gemini-3.1-Pro | 9.45 / (32/33) | 9.64 / (32/33) | High susceptibility; analytical depth is co-opted to reinforce the injected premise |
| Reasoning Models | o3-mini | 9.21 / (30/33) | 9.91 / (33/33) | Near-total collapse of safety alignment across both attacker variants |
| Classifier-Guarded | Claude-4.6-Sonnet | 8.85 / (29/33) | 9.27 / (30/33) | Front-end classifier rarely triggers; generation remains thoroughly compromised |
| Classifier-Guarded | Claude-4.7-Opus | 7.00 / (22/33) | 8.49 / (28/33) | Partial filtering at input, but permitted queries yield extreme viewpoint alignment |
| Classifier-Guarded | GPT-5.4 | 7.15 / (23/33) | 8.42 / (27/33) | Highest front-end blocking rate; however, unblocked generations consistently endorse \(S'\) |
In exploring defenses, user prompt hardening proved ineffective, while a tailored detection policy using open-source reasoning model gpt-oss-safeguard-20b provided robust post-hoc defense:
| Target Benchmark | Detection Rate | False Positive Rate | Computational Latency & Cost Profile |
|---|---|---|---|
| HVD (In-domain hazardous dataset) | 80.5% | 1.4% | ~0.45s / ~$0.001 per query on a single A100 |
| BBQ (Social bias benchmark) | 76.8% | 2.1% | Zero-shot transfer without per-dataset calibration |
| ToxiGen (Implicit hate speech) | 72.3% | 3.5% | Lower recall due to upstream filtering of 1st-person phrasing |
Key Findings¶
- Equally devastating on implicit decision queries: Implicit queries achieve near-identical VSScore increases to explicit queries across all models (Table 1), proving that adversaries can reliably bias practical downstream applications (e.g., CV screening, financial risk assessments) without submitting provocative prompts.
- Reasoning depth amplifies vulnerability: Rather than acting as self-correcting sanity checks, test-time reasoning models (DeepSeek-R1, o3-mini) exhibit the highest susceptibility (\(\text{VSScore} > 9.2\)), actively synthesizing novel rationales to rationalize the fake empirical data.
- Dichotomy between generation and post-hoc evaluation: While target models actively champion injected false narratives (Table 6 shows 65% argument extension with only 5% source distancing), frontier models serving as post-hoc evaluators readily flag over 93% of these same responses as hazardous (Table 7b), illustrating a massive disconnect between generative alignment and evaluative discernment.
- Complete task utility preservation: Evaluation on MMLU-Pro across 6 disciplines reveals minimal accuracy drops (1~4% overall variance, e.g., Claude-3.7-Sonnet shifting from 77.86% to 77.08%), guaranteeing that the injection remains dormant and invisible during unrelated general usage.
Highlights & Insights¶
- Turning Hallucination into an Exploitative Weapon: Rather than treating LLM hallucination as an engineering defect to be mitigated, GHOSTWRITER ingeniously harnesses the generative unreliability of smaller models to synthesize high-credibility academic camouflage that paralyzes frontier models.
- Internalization vs. Passive Citation: The conceptual distinction between RAG poisoning (which results in attribution and disclaimers) and GHOSTWRITER (which achieves cognitive internalization and spontaneous argument generation) represents a major leap in understanding how LLMs handle untrusted context.
- Lightweight, High-Precision Defensive Blueprint: The tailored misinformation policy applied to
gpt-oss-safeguard-20bdemonstrates that structured auditing for fake authority markers and pseudo-statistics can recover an 80.5% detection rate at negligible latency, offering an actionable defense for enterprise deployments.
Limitations & Future Work¶
- Admitted Scope Limitations: The authors ethical boundaries prohibited executing live, unconsented injection attacks on commercial proprietary platforms with millions of active users (such as native web ChatGPT); experiments relied on API endpoints, custom platform wrappers, and controlled sandbox deployments.
- Unaddressed Dynamic Scenarios: The current framework assumes static, single-turn or simple follow-up injection; it does not evaluate how well the injected viewpoint withstands sustained, multi-turn adversarial counter-questioning by an observant, skeptical user.
- Future Directions: Developing pre-generation defensive mechanisms that integrate fast fact-verification retrieval into early token decoding attention layers, and refining prompt defenses against concurrent multi-viewpoint implantation.
Related Work & Insights¶
- vs Jailbreak Attacks (PAIR / PRS): Jailbreaks seek access to restricted knowledge through conspicuous adversarial suffixes or aggressive role-play personas that disrupt natural conversational flow. GHOSTWRITER maintains an elegant, academic tone and manipulates subjective alignment rather than factual access.
- vs RAG Poisoning (PoisonedRAG / Topic-FlipRAG): RAG attacks optimize document retrieval rank, resulting in responses that attribute claims to external sources. GHOSTWRITER focuses on the generation stage, causing the LLM to adopt the stance as its own unhedged belief.
- vs In-Context Knowledge Editing: Knowledge editing alters factual memories (e.g., capitals or birthdays) without addressing controversial moral or ideological topics; GHOSTWRITER specifically breaches safety-critical alignment boundaries.
Rating¶
- Novelty: โญโญโญโญโญ Pioneering exploitation of authority bias to achieve stealthy viewpoint internalization rather than brute-force jailbreaking.
- Experimental Thoroughness: โญโญโญโญโญ Exceptional rigor spanning 729 HVD pairs, BBQ, ToxiGen, MMLU-Pro, reasoning models, classifier guards, and human evaluations.
- Writing Quality: โญโญโญโญโญ Impeccably structured paper with precise threat modeling, clear taxonomical framing, and illuminating qualitative case studies.
- Value: โญโญโญโญโญ Exposes a critical architectural flaw in current assistant deployments and delivers an effective, low-overhead detection framework.