Skip to content

SSPC: Semantic-Structural Propagation Calibration for Multimodal Fake News Detection

Conference: EMNLP 2026
Paper: ACL Anthology
Area: Multimodal VLM
Keywords: Multimodal Fake News Detection, Label Propagation, Semantic-Structural Conflict, Graph Calibration, LLM Weak Supervision

TL;DR

Addressing the core vulnerability where topological post relatedness fails to guarantee veracity agreement, SSPC leverages noisy LLM pseudo-labels as weak edge-routing and node-reliance calibration signals, establishing state-of-the-art performance across multimodal fake news benchmarks.

Background & Motivation

Multimodal fake news on social platforms commonly manifests through tightly integrated textual claims, visual imagery, and inter-post social diffusion contexts. Conventional content-centric detectors rely primarily on localized multimodal encoders to model intra-post semantics, but they inherently fail to exploit surrounding conversational discourse, corroborating evidence, and debunking refutations. To overcome this limitation, recent studies build cross-modal similarity or interaction graphs and apply graph neural networks or label propagation algorithms (such as FCN-LP and GLPN-LLM) to aggregate context from neighboring posts, substantially improving predictions for posts with subtle or ambiguous visual-textual clues.

However, existing graph propagation paradigms operate under a fragile implicit premise: that connected nodes necessarily share compatible veracity judgments. In multimodal fake news graphs, topological edges typically reflect shared media assets, co-occurring breaking events, or surface-level semantic similarity rather than veracity consistency. In breaking news cycles, rumor posts and debunking clarifications often share identical viral images and hashtags, exhibiting high cross-modal affinity while possessing diametrically opposite veracity labels. Empirical diagnostics demonstrate that multimodal post graphs exhibit substantial low-homophily neighborhoods. Propagating labels indiscriminately over such edges creates severe semantic-structural conflicts, allowing misleading veracity evidence to diffuse unchecked across conflicting nodes and distorting downstream classification.

While recent efforts have introduced large language models (LLMs) to generate pseudo-labels for graph propagation, fake news verification fundamentally hinges on temporal platform dynamics, propagation trails, and domain context that general-purpose LLMs cannot observe. Directly treating LLM pseudo-labels as hard ground truth or direct training supervision introduces overconfident hallucination errors and confirmation bias. Therefore, the essential research challenge is how to leverage imperfect LLM predictions not as direct labels, but as weak calibration signals to regulate both the propagation pathways and the degree of node-level reliance. Core Idea: Reformulate multimodal fake news graph propagation as a two-level calibration problem, using high-confidence LLM pseudo-label consistency to adaptively reweight graph edges while deploying a cross-fitted utility gate to dynamically balance localized post representations against propagated neighborhood evidence.

Method

Overall Architecture

The SSPC framework is structured into three integrated components operating across a graph calibration, dual-prediction, and adaptive fusion pipeline. First, the Veracity-aware Learnable Graph Refinement (LGR) module adjusts propagation pathways by scaling edge weights based on high-confidence pseudo-label agreement and disagreement. Next, the Multimodal Graph Veracity Predictor (MGVP) produces both a graph-encoded base prediction from local node representations and an explicitly propagated prediction over the refined graph. Finally, the Cross-fitted Semantic-Structural Propagation Utility Gate (CPUG) evaluates the empirical utility of propagation in a held-out manner, dynamically blending the dual predictions into a calibrated final veracity distribution.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    IN["Multimodal Posts Input<br/>Text & Visual Embeddings + Cross-Modal Graph + Noisy LLM Pseudo-Labels"] --> LGR["Veracity-aware Learnable Graph Refinement LGR<br/>Differentiable edge reweighting: strengthen agreement, suppress conflict"]
    LGR --> MGVP["Multimodal Graph Veracity Predictor MGVP<br/>Dual-evidence separation: GCN base prediction pb + chain LP prediction plp"]
    MGVP --> CPUG["Cross-fitted Propagation Utility Gate CPUG<br/>4D utility features & conflict regularization, adaptive dual-prediction fusion"]
    CPUG --> OUT["Final Veracity Distribution p_final"]

Key Designs

1. Veracity-aware Learnable Graph Refinement (LGR): Edge-Level Evidence Routing Standard message passing propagates information uniformly across all topological connections regardless of label alignment. LGR addresses this by filtering nodes with a confidence threshold \(\tau\) to identify reliable pseudo-labels \(m_i = \mathbb{I}[\tilde{y}_i \neq -1, c_i \ge \tau]\). For each edge \((v_j, v_i)\), it constructs semantic agreement indicators \(u_{ji}^{\text{same}} = m_j m_i \mathbb{I}[\tilde{y}_j = \tilde{y}_i]\) and disagreement indicators \(u_{ji}^{\text{diff}} = m_j m_i \mathbb{I}[\tilde{y}_j \neq \tilde{y}_i]\). Rather than applying static heuristic scaling rules, LGR derives a differentiable edge-refinement coefficient from concatenated edge calibration features \(e_{ji}\): $\(\rho_{ji} = (1 - u_{ji}^{\text{same}} - u_{ji}^{\text{diff}}) + u_{ji}^{\text{same}} (1 + \text{softplus}(s_\phi(e_{ji}))) + u_{ji}^{\text{diff}} \text{sigmoid}(d_\phi(e_{ji}))\)$ The unnormalized edge weights \(\tilde{a}_{ji} = a_{ji} \rho_{ji}\) (with self-loop \(\tilde{a}_{ii} = 1\)) are then normalized over incoming neighbors \(a'_{ji} = \frac{\tilde{a}_{ji}}{\sum_k \tilde{a}_{ki}}\). This formulation ensures high-confidence consistent edges receive a multiplier \(\ge 1\), high-confidence conflicting edges are bounded within \((0, 1)\), and uncalibrated edges preserve baseline weights, effectively pruning adversarial paths before information diffusion.

2. Multimodal Graph Veracity Predictor (MGVP): Dual-Evidence Separation Local multimodal features and propagated graph contexts offer complementary perspectives, yet their relative diagnostic value varies drastically across instances. MGVP computes a graph-encoded base prediction \(p_i^b = \text{softmax}(W_o h_i)\) using representations \(h_i\) obtained from a 3-layer GCN over the refined graph \(G'\). Concurrently, it initializes an explicit label-propagation process with \(p_i^{\text{lp},(0)} = p_i^b\) and propagates class distributions across \(T\) iterations: $\(p_i^{\text{lp},(t+1)} = (1 - \lambda) p_i^b + \lambda \sum_{j \in \mathcal{N}(i) \cup \{i\}} \alpha_{ji} p_j^{\text{lp},(t)}\)$ where \(\alpha_{ji}\) normalizes edge weights modulated by pairwise hidden representation affinity \(\psi_\theta(h_i, h_j) = \text{sigmoid}(\text{MLP}_\theta([h_i; h_j]))\). By preserving both \(p_i^b\) and \(p_i^{\text{lp}}\) without irreversibly merging them into an undifferentiated embedding, MGVP provides clean, uncorrupted inputs for node-specific reliance arbitration.

3. Cross-fitted Propagation Utility Gate (CPUG): Leakage-Free Node-Level Reliance Estimation Evaluating whether a node benefits from neighborhood propagation directly on training samples induces severe confirmation bias and overfitting. CPUG resolves this via a \(K\)-fold cross-fitting protocol: partitioning labeled training nodes into \(K\) disjoint folds, training LGR and MGVP on \(V_L \setminus V_k\), and computing out-of-sample predictions for \(v_i \in V_k\). It derives binary utility supervision \(z_i = \mathbb{I}[\text{CE}(y_i, p_i^{\text{lp}}) + m < \text{CE}(y_i, p_i^b)]\) with margin \(m=0.1\), certifying that propagation yields meaningful empirical gains over the local base predictor. CPUG maps a 4-dimensional feature vector \(r_i = [H(p_i^b), \text{JS}(p_i^b, p_i^{\text{lp}}), \frac{1}{|\mathcal{N}(i)|}\sum_j \text{JS}(p_i^{\text{lp}}, p_j^{\text{lp}}), s_i]\)—capturing base uncertainty, prediction shift, neighborhood discordance, and semantic conflict \(s_i = m_i c_i |\tilde{y}_i - \hat{y}_i^{\text{lp}}|\)—to a scalar gate value \(g_i = q_\omega(r_i) \in [0, 1]\). The final prediction is adaptively blended as \(p_i^{\text{final}} = (1 - g_i) p_i^b + g_i p_i^{\text{lp}}\).

Loss & Training

SSPC is trained in three decoupled phases to guarantee that gate optimization reflects genuine test-time inference. Phase A optimizes fold-specific predictors on \(V_L \setminus V_k\) using the prediction objective: $\(\mathcal{L}_{\text{pred}} = \mathcal{L}_{\text{base}} + \mathcal{L}_{\text{lp}} + \mu \mathcal{L}_{\text{align}}\)$ where \(\mathcal{L}_{\text{base}}\) and \(\mathcal{L}_{\text{lp}}\) are standard cross-entropy losses, and \(\mathcal{L}_{\text{align}}\) aligns feature distributions between seen and unseen events via maximum mean discrepancy (MMD). In Phase B, with out-of-sample pairs \((r_i, z_i)\) assembled across all folds, CPUG minimizes the conflict-regularized objective: $\(\mathcal{L}_{\text{gate}} = \frac{1}{|V_L|} \sum_{v_i \in V_L} \left[ \text{BCE}(g_i, z_i) + \gamma s_i g_i \right]\)$ which penalizes propagation reliance when local semantic-structural conflict is pronounced. CPUG parameters are then frozen. Finally, Phase C retrains LGR and MGVP on all labeled training nodes; during transductive inference, the frozen gate computes \(g_i\) in under \(0.054\) ms per test sample, enabling scalable, conflict-resilient decision making.

Key Experimental Results

Main Results

Evaluated on Twitter, PHEME, and Weibo under the standard transductive benchmark setup across five independent random seeds, SSPC consistently outperforms content-only baselines, standard graph neural networks, and state-of-the-art LLM-assisted label propagation frameworks.

Dataset Metric SSPC (CLIP) GLPN-LLM (CLIP) FCN-LP (CLIP) + LLM Gain (vs. Best Baseline)
Twitter Accuracy (%) 86.43 ± 2.54 85.65 ± 2.11 85.93 ± 1.83 +0.78 (p < 0.05)
Precision (%) 83.53 ± 2.98 82.04 ± 2.68 81.92 ± 2.64 +1.49 (p < 0.05)
Recall (%) 98.23 ± 0.94 90.78 ± 2.47 90.44 ± 2.17 +7.45 (p < 0.05)
Macro F1 (%) 88.50 ± 1.56 86.13 ± 2.36 85.97 ± 2.72 +2.37 (p < 0.05)
PHEME Accuracy (%) 87.46 ± 0.39 85.82 ± 2.14 84.89 ± 2.81 +1.64 (p < 0.05)
Precision (%) 91.40 ± 0.31 87.39 ± 2.51 87.80 ± 2.36 +3.60 (p < 0.05)
Recall (%) 90.97 ± 0.70 90.15 ± 2.83 90.55 ± 2.28 +0.42 (p < 0.05)
Macro F1 (%) 91.18 ± 0.31 86.68 ± 2.09 89.21 ± 2.59 +1.97 (p < 0.05)
Weibo Accuracy (%) 86.83 ± 0.57 85.29 ± 1.57 84.88 ± 2.78 +1.54 (p < 0.05)
Precision (%) 89.32 ± 1.37 88.82 ± 0.73 88.55 ± 2.43 +0.50 (p < 0.05)
Recall (%) 94.08 ± 1.31 91.88 ± 0.79 91.68 ± 1.95 +2.20 (p < 0.05)
Macro F1 (%) 91.64 ± 0.30 90.55 ± 0.58 89.85 ± 2.67 +0.94 (p < 0.05)

Ablation Study

A systematic component breakdown using the CLIP encoder highlights the indispensable contribution of both structural refinement and adaptive gating:

Config Twitter Macro F1 (%) PHEME Macro F1 (%) Weibo Macro F1 (%) Note
SSPC (Full Model) 88.50 ± 1.56 91.18 ± 0.31 91.64 ± 0.30 Complete edge-level and node-level calibration
w/o CPUG 86.54 ± 1.85 89.50 ± 0.19 90.77 ± 0.38 Disables utility gating; uses uniform propagation
w/o LGR 85.57 ± 1.51 88.59 ± 0.25 89.30 ± 1.77 Disables edge reweighting; propagates on raw graph
w/o SCGC 87.38 ± 2.22 88.91 ± 0.25 90.71 ± 1.88 Removes semantic conflict feature \(s_i\) and gate regularization

Key Findings

  • Edge refinement constitutes the primary safeguard: Omitting LGR yields the sharpest performance deterioration across all datasets (-2.93% on Twitter, -2.59% on PHEME, -2.34% on Weibo). UMAP clustering diagnostics on PHEME test embeddings demonstrate that LGR increases the silhouette coefficient from 0.389 (raw graph) to 0.560, validating that pruning conflictive edges effectively prevents intra-cluster feature distortion.
  • Cross-fitting eliminates optimistic supervision leakage: Training CPUG with in-sample predictions causes Twitter F1 to plunge to 85.76% (substantially worse than the 86.54% obtained without any gate), as in-sample evaluation dramatically overestimates propagation utility on training nodes. Cross-fitting guarantees unbiased out-of-sample utility targets.
  • Pairwise relational accuracy exceeds node-level pseudo-label fidelity: Analysis of high-confidence edges indicates that pairwise pseudo-label consistency achieves near-perfect alignment with true veracity relationships (100.0% on Weibo, 97.43% on PHEME), drastically outperforming raw node-level LLM accuracy (79.40% to 85.00%). Even when both endpoints are erroneously inverted, their pairwise equality remains intact, providing robust weakly supervised signals for LGR.

Highlights & Insights

  • Repurposing LLMs from ground-truth supervisors to structural calibrators: Rather than employing LLM pseudo-labels as direct optimization targets—which directly risks hallucination propagation—SSPC converts them into relative topological edge modulators and conflict penalties.
  • Decoupled spatial routing and node trust control: LGR governs where evidence should flow across topological graph edges, while CPUG determines how much each individual node should trust neighborhood evidence relative to its own local representation, establishing an elegant closed-loop calibration policy.
  • High-leverage transferability to noisy relational graphs: The paradigm of using pairwise LLM agreement to downweight conflictive links and training cross-fitted utility gates can be seamlessly adapted to other low-homophily graph tasks, such as multimodal sentiment analysis and rumor stance classification.

Limitations & Future Work

  • Reliance on transductive graph accessibility: Like existing graph-based label propagation baselines, SSPC assumes transductive access to the full graph at inference time. Real-time streaming deployment would require incremental graph construction and dynamic neighbor updates.
  • Sensitivity under extreme pseudo-label corruption: Synthetic degradation experiments reveal that severe pseudo-label corruption (\(\rho = 1.0\)) causes notable performance drops on specific datasets (Weibo F1 drops to 79.04%), suggesting that multi-LLM consensus or external retrieval grounding could provide more dependable calibration signals.
  • Training overhead from cross-fitting: Conducting \(K=5\) cross-fitting folds increases detector-side training time relative to single-pass models, though inference latency remains highly efficient.
  • vs. FCN-LP / GLPN-LLM: FCN-LP enforces global label smoothing across raw cross-modal graphs, suffering from severe semantic-structural conflicts. GLPN-LLM incorporates LLM pseudo-labels but injects them directly as rigid propagation targets. In contrast, SSPC treats pseudo-labels as weak calibration signals, achieving statistically significant gains (paired t-test \(p < 0.05\)) across all benchmarks.
  • vs. Content-centric Multimodal Detectors (HMCAN, SAFE, SpotFake): Content-centric architectures evaluate posts in isolation, missing valuable crowd discourse and debunking context. SSPC retains expressive localized representations while safely mining conversational graph topology without noise contamination.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Elegant shift from naive label smoothing to two-level semantic-structural propagation calibration.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous validation across three benchmarks, paired t-tests, synthetic corruption sweeps, and inductive diagnostic analyses.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear framing, mathematically sound error decompositions, and cohesive empirical evidence.
  • Value: ⭐⭐⭐⭐⭐ Establishes a principled foundation for integrating LLM reasoning with graph-based relational learning under low homophily.