Skip to content

MDHB: A Multi-Dimensional Online Harm Detection Benchmark for Proactive Psychosocial Risk Forecasting

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/yutingh1018-maker/MDHB
Area: LLM Safety
Keywords: Online Harm Detection, Multi-Dimensional Benchmark, Self-Evolving Agent, Non-Offensive Sarcasm, Logical Fallacies

TL;DR

To overcome the blind spots of single-task detection paradigms and the superficiality of generative jailbreak alignment benchmarks, MDHB provides a 7,500-sample multi-dimensional benchmark covering misinformation, toxicity, sarcasm (uniquely including non-offensive sarcasm), and logical fallacies, alongside Self-Refining Sentinel (SRS)โ€”a parameter-free dual-agent architecture that distills failure trajectories into an external experience repository, substantially outperforming static baselines and outscaling larger models.

Background & Motivation

The ubiquitous spread of social media has dramatically accelerated information dissemination while simultaneously converting digital spaces into complex ecosystems harboring multifaceted online harms. Over recent years, digital risks on the open web have evolved significantly beyond explicit, overt violations (such as terrorism or direct slurs) into subtle, covert malicious behaviors. Modern harmful content frequently manifests through intricate rhetorical camouflage, veiled interpersonal harassment, and disguised emotional venting that signals latent individual or collective crises. However, existing content moderation research remains constrained by two fundamental bottlenecks. First, conventional studies overwhelmingly focus on single-task discriminative classificationโ€”such as isolated fake news verification or explicit hate speech filteringโ€”which lacks a comprehensive governance perspective and creates systemic blind spots when confronting multifaceted real-world threats. Second, emerging multi-dimensional safety benchmarks (e.g., Do-Not-Answer, SafetyBench, DecodingTrust) almost exclusively emphasize generative safety alignment, measuring refusal behavior against adversarial jailbreak prompts that feature flat, simplified logical premises and overt adversarial signatures, thereby failing to reflect the deep semantic reasoning demands of organic social media discourse.

Viewed through the lens of Kahneman's dual-process theory, current safety guardrails predominantly cater to intuitive, fast "System 1" processingโ€”such as filtering toxic obscenities that provoke immediate emotional disgust or identifying factual falsehoods via superficial heuristics. Conversely, real-world implicit threats, notably informal logical fallacies (such as slippery slopes or false dilemmas) and pragmatic intent inversion (sarcasm), necessitate deliberate, analytic "System 2" reasoning to unpack discursive structures and deduce communicative intentions. Furthermore, existing governance approaches adhere to a reactive paradigm that merely suppresses explicit violations, remaining oblivious to the predictive utility of passive-negative, non-offensive sarcasm (such as self-deprecation and veiled complaints). Grounded in psycholinguistics and emotional contagion theory, such non-confrontational expressions often serve as critical linguistic precursors to escalating individual psychological crises and widespread societal anxiety. Transitioning online safety from reactive blocking to proactive psychosocial risk warning demands a unified evaluation suite capable of assessing subtle semantic anomalies.

Empirical investigations reveal that state-of-the-art language models, including GPT-5 and scaled open-source models, suffer from severe cross-dimensional cognitive interference when evaluated on multi-dimensional harm detection. Even when equipped with search-augmented ReAct frameworks, external retrieval often injects factual noise into context-dependent pragmatic tasks like sarcasm, causing precision degradation; furthermore, static LLMs suffer from context collapse and catastrophic forgetting across independent inference sessions. Core idea: Establish MDHB, a hierarchical multi-dimensional online harm benchmark incorporating non-offensive sarcasm and logical fallacies, and develop the Self-Refining Sentinel (SRS), a weights-free self-evolving dual-agent framework that dynamically distills failed reasoning trajectories into an external experience repository to resolve cross-dimensional interference without parameter updates.

Method

Overall Architecture

The Self-Refining Sentinel (SRS) departs from conventional static model fine-tuning by establishing an error-driven, test-time self-evolving dual-agent architecture. SRS comprises three integral components: a forward-reasoning Proposer Agent, a backward-diagnostic Miner Agent, and a persistent Dynamic Experience Repository (ER). When evaluating an input social media post, the framework first performs a dense-sparse hybrid retrieval over ER to extract relevant historical cognitive lessons, which are injected into the Proposer's prompt as prior guardrails. The Proposer executes ReAct-style iterative reasoning with access to an external web search tool to yield a prediction. If the prediction diverges from ground truth, the Miner Agent is triggered to conduct backward flaw attribution, abstracting the failure trajectory into a generalized logical guideline. This guideline is evaluated via Jaccard-based deduplication and appended to the persistent experience repository, achieving continual, non-parametric capability expansion.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Social Media Text z"] --> B["Hybrid Experience Retrieval & Dynamic Context Injection<br/>Dense Cosine + Sparse BM25 retrieval of top-k lessons"]
    B --> C["Dual-Agent Inference & Backward Diagnostic Attribution<br/>Proposer generates prediction ลท / Miner performs flaw diagnosis"]
    C -->|Correct Decision ลท = y| D["Output Harm Classification Label"]
    C -->|Incorrect Decision ลท โ‰  y| E["Deduplication Assessment & Experience Persistence<br/>Jaccard similarity thresholding and persistent storage"]
    E --> F["Dynamic Experience Repository ER Expansion<br/>Single-pass sequential zero-shot self-evolution"]
    F -.->|Real-time prior retrieval| B

Key Designs

1. Hybrid Experience Retrieval & Dynamic Context Injection: Input representation sparsity and cross-dimensional confusion -> Joint dense semantic and sparse lexical retrieval -> Provisioning targeted cognitive heuristics for forward inference Social media texts exhibit subtle stylistic variations and informal syntax; relying solely on dense semantic embeddings often overlooks specific lexical triggers, whereas pure sparse keyword search fails to match abstract argumentative structures. To ensure that retrieved historical experiences capture both broad semantic context and specific reasoning pitfalls, SRS generates a concise natural-language query summary \(q\) characterizing the input text \(z\). The system then executes a hybrid search over the experience repository \(\text{ER}\), calculating a joint similarity score that balances dense embedding cosine similarity and sparse BM25 lexical matching: $\(\text{Score}(q, e) = \alpha \cdot \cos(E(q), E(e)) + (1 - \alpha) \cdot \text{BM25}(q, e)\)$ where \(E(\cdot)\) is the dense text encoder and \(\alpha\) is a balancing coefficient. The top-\(k\) most relevant experiences form the context set \(C = \arg\max_{e \in \text{ER}}^k \text{Score}(q, e)\), which is dynamically prepended to the Proposer's system prompt to anchor its decision boundaries against known failure modes.

2. Dual-Agent Inference & Backward Diagnostic Attribution: Static LLMs lack self-correction and accumulate unidirectional errors -> Decoupling task execution from retrospective diagnostic analysis -> Abstracting failure trajectories into cross-scenario transferable guidelines Standard reasoning agents lack internal mechanisms to learn from unforced misclassifications across independent queries. SRS decouples forward task execution from backward cognitive attribution through a two-agent workflow. The Proposer employs a ReAct formulation over input \(z\), prior context \(C\), and a DuckDuckGo search tool, generating multi-hop thought-action sequences \(T\) and terminating with predicted label \(\hat{y}\). If the prediction is erroneous (\(\hat{y} \ne y\)), the Miner Agent is activated. Receiving \(z\), the full reasoning chain \(T\), and ground truth \(y\), the Miner operates under a restricted decoding temperature (\(T=0.1\)) to conduct reverse diagnosis. It interrogates the root cognitive flawโ€”such as being misled by literal praise, conflating sarcasm with misinformation, or missing implicit premise-conclusion gapsโ€”and distills an abstract, entity-agnostic natural-language lesson \(e_{\text{new}} = \text{Miner}(z, T, y)\).

3. Deduplication Assessment & Experience Persistence: Unbounded memory expansion causing retrieval redundancy and prompt bloat -> Jaccard-based novelty filtering and controlled persistence -> Establishing a high-purity lifelong evolving memory store Unchecked accumulation of ad-hoc feedback quickly floods an external memory store with repetitive or conflicting directives, impairing retrieval precision and consuming inference context. To maintain knowledge diversity and compact representation, SRS enforces a strict deduplication gate. When the Miner generates candidate lesson \(e_{\text{new}}\), the system calculates its maximum Jaccard similarity against existing rules in \(\text{ER}\): $\(\text{Sim}_{\max} = \max_{e \in \text{ER}} \text{Jaccard}(e_{\text{new}}, e)\)$ Only if \(\text{Sim}_{\max} < \tau\) (with threshold \(\tau = 0.6\)) is \(e_{\text{new}}\) appended to persistent storage. Across sequential evaluation on 1,250 validation and 6,250 test samples, GPT-5 accumulated 352 and 1,639 rules respectively, exhibiting a smooth marginal convergence that confirms repository stability without redundant rule proliferation.

Benchmark & Evaluation Protocol

To benchmark multi-dimensional harm detection rigorously, the authors created MDHB comprising 7,500 curated textual instances. The dataset is partitioned into Validation (1,250 instances) and Test (6,250 instances) sets at a strict 1:5 ratio, maintaining an exact 20% balanced representation across five mutually exclusive categories (1,500 instances each): 1. Misinformation (1,500 samples): Encompasses political manipulation, health/pseudoscience, social emergencies, and urban legends. Sourced from the multi-hop fact-checking benchmark AVeriTeC and secondary cross-verified using Bing web search. 2. Toxicity (1,500 samples): Derived from the implicit hate speech corpus Latent Hatred. Instances labeled as irony were systematically purged to prevent inter-dimensional leakage. Spans racial discrimination, threats/intimidation, stereotypes, incitement to violence, and inferiority language. 3. Sarcasm (1,500 samples): Constructed from IAC-V2 and iSarcasm, partitioned into offensive sarcasm (interpersonal ridicule) and non-offensive sarcasm (self-deprecation, benign emotional venting). To alleviate real-world scarcity of non-offensive negative sarcasm, targeted LLM augmentation grounded in psycholinguistic crisis markers was performed. 4. Logical Fallacy (1,500 samples): Sourced from Cocolofa and LOGIC benchmarks, covering eight prevalent informal fallacies: Appeal to Authority, Appeal to Tradition, Appeal to Majority, Appeal to Nature, Slippery Slope, False Dilemma, Hasty Generalization, and Appeal to Worse Problems. Implements contextual fusion and rewriting to integrate comments and news context, eliminating length-based shortcut learning. 5. Normal Data (1,500 samples): Benign control samples extracted from negative subsets of source corpora, screened through automated violation detectors and human cross-validation.

All texts underwent strict length-distribution normalization. Five independent annotators verified 1,250 validation instances, achieving substantial inter-rater reliability with Fleiss' \(\kappa = 0.73\) (Normal: 0.81, Toxicity: 0.79, Misinformation: 0.75, Sarcasm: 0.68, Logical Fallacy: 0.60).

Key Experimental Results

Main Results

The benchmark was evaluated across human annotators, open-source models (Qwen3.5 series, Ministral3 series, DeepSeek-V4-Pro), and proprietary GPT-5 under Direct, ReAct, and SRS paradigms.

Base Model Method / Paradigm Test Acc (%) Test Macro-P (%) Test Macro-R (%) Test Macro-F1 (%) Val Acc (%) Val Macro-F1 (%)
Human Annotators - - - - - 66.87 69.03
Qwen3.5-4B Direct 46.78 54.85 46.78 47.26 45.48 46.05
Qwen3.5-4B ReAct 49.31 57.94 49.31 48.80 49.68 49.19
Qwen3.5-4B SRS (Ours) 62.47 67.86 62.47 63.67 58.30 59.34
Qwen3.5-9B Direct 53.02 61.89 53.02 53.44 54.72 55.66
Qwen3.5-9B ReAct 54.66 61.34 54.66 55.12 57.52 58.14
Qwen3.5-9B SRS (Ours) 67.93 70.89 67.93 68.23 65.49 65.69
Ministral3-8B Direct 60.51 63.80 60.51 61.02 60.47 60.38
Ministral3-8B ReAct 64.72 64.29 64.72 63.95 62.85 62.10
Ministral3-8B SRS (Ours) 68.66 70.39 68.66 69.30 64.16 64.97
Ministral3-14B Direct 62.19 66.16 62.19 62.73 61.68 62.45
Ministral3-14B ReAct 65.98 65.40 65.98 65.29 65.04 64.06
Ministral3-14B SRS (Ours) 69.46 71.24 69.46 70.13 66.72 67.55
Qwen3.5-27B Direct 63.65 71.87 63.65 64.47 62.72 63.26
Qwen3.5-27B ReAct 63.78 70.16 63.78 64.37 63.92 64.51
Qwen3.5-27B SRS (Ours) 73.55 77.04 73.55 74.48 71.23 71.99
Qwen3.5-397B-A17B Direct 64.02 73.52 64.02 64.55 64.07 64.67
Qwen3.5-397B-A17B ReAct 66.56 70.35 66.56 66.83 65.44 65.91
Qwen3.5-397B-A17B SRS (Ours) 72.80 76.65 72.80 73.70 70.16 71.15
DeepSeek-V4-Pro Direct 68.16 71.13 68.16 68.55 68.96 69.20
DeepSeek-V4-Pro ReAct 68.29 70.31 68.29 68.57 70.56 71.09
DeepSeek-V4-Pro SRS (Ours) 74.61 76.86 74.61 75.34 72.88 73.62
GPT-5 Direct 67.46 75.70 67.46 67.97 65.92 66.42
GPT-5 ReAct 71.66 74.34 71.66 72.03 70.48 71.05
GPT-5 SRS (Ours) 73.79 77.71 73.79 74.37 72.00 72.82

Ablation & In-Depth Diagnostic Analysis

The following table presents ablation results for Qwen3.5-27B on the validation set, alongside performance of the specialized moderation model Llama-Guard-3-8B and breakdown of sarcasm sub-dimensions.

Target Setting / Variant Dimension / Subtask Acc (%) Macro-P (%) Macro-R (%) Macro-F1 (%)
SRS Full Pipeline (All) 5-way Validation Ablation 71.23 74.85 71.23 71.99
w/o Experience Repository (ReAct) 5-way Validation Ablation 63.92 68.80 63.92 64.51
w/o Miner (Fixed 100 human rules) 5-way Validation Ablation 68.80 73.00 68.80 69.40
Random Retrieval (Random 5 rules) 5-way Validation Ablation 65.60 70.10 65.60 66.60
Llama-Guard-3-8B (Validation) Binary Harm Aggregation 33.68 92.54 18.60 30.97
Llama-Guard-3-8B (Test) Binary Harm Aggregation 33.95 91.38 19.29 31.86
Human (Validation breakdown) Logical Fallacy - 89.00 37.00 52.27
Human (Validation breakdown) Normal Data - 51.50 89.50 65.38
Qwen3.5-27B (Direct) Offensive / Non-Offensive Acc 58.57 (Offensive) - - 40.78 (Non-Offensive)
Qwen3.5-27B (ReAct) Offensive / Non-Offensive Acc 59.29 (Offensive) - - 47.57 (Non-Offensive)
Qwen3.5-27B (SRS) Offensive / Non-Offensive Acc 60.29 (Offensive) - - 57.28 (Non-Offensive)
Human Baseline Offensive / Non-Offensive Acc 61.11 (Offensive) - - 68.18 (Non-Offensive)

Key Findings

  • Human and Moderation Guard Blind Spots on Implicit Harms: Human evaluators exhibited severe cognitive divergence: on logical fallacies, humans achieved high precision (89.00%) but dismal recall (37.00%), indicating frequent distraction by inflammatory rhetoric while missing broken inferences; on normal data, humans reached 89.50% recall but only 51.50% precision, misclassifying non-explicit sarcasm as benign. Similarly, specialized guard model Llama-Guard-3-8B demonstrated 91.38% precision but an abysmal 19.29% recall on the test set, demonstrating that standard guardrails fail catastrophically on covert harms.
  • SRS Outperforms Massive Parameter Scaling: Equipping Qwen3.5-27B with SRS yields 74.48% Macro-F1, decisively outperforming the massive Qwen3.5-397B-A17B under ReAct (66.83% F1) and proprietary GPT-5 under ReAct (72.03% F1). Dynamically accumulating cognitive lessons at test time proves more effective and economical than scaling raw parameter counts.
  • Mitigating ReAct's Pragmatic Noise: Direct web retrieval in ReAct introduces factual bias that degrades precision in context-dependent sarcasm tasks (e.g., test precision drops by 1.71% in Qwen3.5-27B and 1.36% in GPT-5). SRS mitigates this limitation, boosting non-offensive sarcasm accuracy from 40.78% (Direct) and 47.57% (ReAct) to 57.28%, confirming that distilled abstract rules counteract superficial literal matching.
  • Essential Interplay of Pipeline Components: Ablations prove that eliminating experiential memory incurs a 7.48% drop in F1 (71.99% to 64.51%), showing that web facts alone cannot solve multi-dimensional implicit harms. Substituting the dynamic Miner with 100 static human rules drops F1 by 2.59%, while injecting random experiences drops F1 by 5.39%, demonstrating that static heuristics fail on long-tail cases and irrelevant experiences introduce damaging context noise.

Highlights & Insights

  • Decoupled Sarcasm for Psychosocial Risk Forecasting: Decouples sarcasm into offensive versus non-offensive variants, repositioning non-hostile self-deprecation and venting as vital early-warning signals for collective psychological distress rather than mere linguistic noise.
  • Weights-Free Test-Time Evolution: Implements a self-sustaining evolutionary loop between Proposer and Miner agents without gradient updates, achieving continual test-time learning that scales with dataset size.
  • Contextual Rewriting to Eliminate Shortcut Biases: Employs an LLM-assisted context fusion protocol to integrate comments with background news and enforce uniform length normalization, preventing models from exploiting text length as a spurious classification shortcut.
  • Structured Error Attribution Schema: The Miner's three-part prompt schema ("flaw analysis -> generalizable rule -> corrected reasoning") provides a reusable template for agentic reflection across general NLP reasoning domains.

Limitations & Future Work

  • Corpus Sourcing and Demographic Biases: Primary data sources derive from public English social platforms, risking demographic skew. Without extended conversational history, moderation models may disproportionately misidentify colloquial dialects (e.g., African American English) or in-group reclaimed slurs as toxic.
  • Text-Centric Scope Excluding Multimodal Memes: MDHB currently addresses pure text, whereas contemporary harms proliferate via internet memes where malice emerges from image-text incongruity. Extending SRS to multimodal conflict grounding is an essential frontier.
  • Socio-Cultural and Multilingual Boundaries: Implicit harms and pragmatic conventions are deeply intertwined with local cultural contexts and regional idioms. Expanding beyond English discourse to diverse multicultural settings remains unaddressed.
  • vs AVeriTeC / ToxiGen: Prior benchmarks isolate single harm dimensions under simplified classification settings. MDHB unifies four core threats into a cohesive hierarchical taxonomy, exposing severe cross-dimensional confusion.
  • vs Do-Not-Answer / SafetyBench / DecodingTrust: These benchmarks assess generative safety alignment against explicit adversarial prompts, whereas MDHB evaluates discriminative moderation over organic, implicit UGC text.
  • vs Reflexion / ACE: Reflexion relies on within-session trial-and-error without cross-instance persistence; ACE adapts contexts via external environment feedback. SRS implements test-time cross-query knowledge accumulation via autonomous dual-agent reflection and Jaccard-gated experience storage.

Rating

  • Novelty: โญโญโญโญโญ Pioneering incorporation of non-offensive sarcasm for proactive psychosocial risk warning alongside an autonomous dual-agent evolutionary framework.
  • Experimental Thoroughness: โญโญโญโญโญ Comprehensive benchmarking across 8 diverse LLMs and human raters over 7,500 samples, complete with fine-grained subtask analyses and rigorous ablations.
  • Writing Quality: โญโญโญโญโญ Rigorous conceptual framing grounded in psycholinguistics and dual-process cognitive theory, supported by structured mathematical formulations and clear empirical illustrations.
  • Value: โญโญโญโญโญ Provides an indispensable evaluation benchmark and a parameter-free architectural blueprint for real-world automated content moderation.