SSPC: Semantic-Structural Propagation Calibration for Multimodal Fake News Detection¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Multimodal Fake News Detection, Semantic-Structural Conflict, Label Propagation Calibration, LLM Pseudo-labels, Utility Gating
TL;DR¶
Addressing the vulnerability where graph edges connect related posts with opposing veracity, SSPC shifts graph propagation from unconditional label smoothing to calibrated evidence transfer by repurposing noisy LLM pseudo-labels into learnable edge routing and cross-fitted node reliance gating.
Background & Motivation¶
Multimodal fake news on social platforms commonly couples textual claims, visual assets, and intricate inter-post interactions. Traditional content-centric detectors encode multimodal features to classify posts in isolation, yet they inevitably overlook vital conversational and inter-post context—such as debunking refutations, supporting corroborations, or crowd-sourced corrections. To overcome this limitation, graph-based methods construct social interaction, diffusion, or multimodal similarity graphs, aggregating neighborhood representations via Graph Neural Networks (GNNs) and propagating veracity beliefs across connected posts using label propagation algorithms. However, conventional message passing and label propagation implicitly assume homophily: connected posts are assumed to share compatible veracity states.
In realistic multimodal fake news graphs, graph edges typically reflect shared media artifacts, event co-occurrence, or high cross-modal embedding similarity rather than true veracity agreement. Consequently, rumor posts and genuine debunking reports frequently share identical incident photographs or hashtags, creating severe semantic-structural conflict where edges link posts with opposing ground-truth labels. Under this heterophilous regime, unconditional graph propagation forces misleading veracity evidence across conflicting boundaries, inadvertently corrupting initially correct predictions. While recent works incorporate Large Language Model (LLM) pseudo-labels directly into label propagation frameworks, LLMs lack real-time platform propagation context, historical background, and source credibility verifications, causing direct pseudo-label supervision or unconstrained propagation to inject hallucinated, overconfident noise into the network.
This fundamental tension shifts the core question from how to propagate more neighborhood signals to deciding which specific graph edges should transmit veracity evidence and how much each individual post should trust propagated beliefs. Rather than treating LLM pseudo-labels as ground-truth supervision or initial propagation priors, this paper redefines their utility as weak calibration signals. Core idea: formulate multimodal fake news detection as a two-level propagation calibration problem, leveraging high-confidence LLM pseudo-label consistency to dynamically route edge evidence (LGR) and training a cross-fitted utility gate (CPUG) to adaptively balance local post representations against propagated graph beliefs.
Method¶
Overall Architecture¶
SSPC establishes an integrated pipeline consisting of three interconnected modules for graph calibration, dual-view veracity estimation, and adaptive fusion: given multimodal post representations \(X\), an initial post graph \(G=(V, E, A)\), labeled training instances \(Y_L\), and noisy LLM pseudo-labels with confidence scores, the pipeline executes sequentially. First, the Veracity-aware Learnable Graph Refinement (LGR) module adjusts edge transmission capacities based on pairwise pseudo-label agreements, producing a calibrated topology \(G'\). Second, the Multimodal Graph Veracity Predictor (MGVP) executes GCN encoding over \(G'\) to generate a post-level base prediction \(p_i^b\), followed by multi-step chain-style label propagation to produce a context-level propagated prediction \(p_i^{lp}\). Third, the Cross-fitted Semantic-Structural Propagation Utility Gate (CPUG) assesses base uncertainty, prediction discrepancies, and semantic-structural conflict to compute an adaptive reliance score, fusing both predictions into the final post veracity distribution.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Multimodal Post Inputs & Initial Graph<br/>(Text/Image Features + Topology G)"] --> B["Veracity-aware Learnable Graph Refinement (LGR)<br/>Edge Routing via Pairwise Pseudo-Label Alignment"]
B --> C["Multimodal Graph Veracity Predictor (MGVP)<br/>Dual-evidence: GCN Base & Chain Label Propagation"]
C --> D["Cross-fitted Propagation Utility Gate (CPUG)<br/>Node Reliance Gating via Out-of-Sample Utility"]
D --> E["Adaptive Evidence Fusion & Final Veracity Output"]
Key Designs¶
1. Veracity-aware Learnable Graph Refinement (LGR): Preventing misleading evidence flow across conflicting edges
Standard graph propagation operating on fixed adjacency matrices diffuses labels indiscriminately across heterophilous edges. Rather than applying rigid heuristic edge-dropping thresholds, LGR formulates a differentiable reweighting policy conditioned on confidence-filtered LLM pseudo-labels. For node \(v_i\), let \(\tilde{y}_i \in \{0, 1\}\) and \(c_i \in [0, 1]\) denote its LLM pseudo-label and confidence score; a binary mask \(m_i = \mathbb{I}[\tilde{y}_i \ne -1, c_i \ge \tau]\) filters out unreliable predictions. For any edge \((v_j, v_i) \in E\), LGR defines indicator features for high-confidence agreement \(u_{ji}^{\text{same}} = m_j m_i \mathbb{I}[\tilde{y}_j = \tilde{y}_i]\) and disagreement \(u_{ji}^{\text{diff}} = m_j m_i \mathbb{I}[\tilde{y}_j \ne \tilde{y}_i]\). A pair of small networks \(s_\phi\) and \(d_\phi\) operates on the edge-calibration vector \(e_{ji} = [a_{ji}, c_j, c_i, u_{ji}^{\text{same}}, u_{ji}^{\text{diff}}]\) to compute a continuous scaling multiplier \(\rho_{ji}\): $\(\rho_{ji} = (1 - u_{ji}^{\text{same}} - u_{ji}^{\text{diff}}) + u_{ji}^{\text{same}} (1 + \text{softplus}(s_\phi(e_{ji}))) + u_{ji}^{\text{diff}} \text{sigmoid}(d_\phi(e_{ji}))\)$ Unannotated or low-confidence edges preserve their original weights (\(\rho_{ji}=1\)), while agreement edges are boosted (\(\rho_{ji} > 1\)) and conflicting edges are strictly suppressed into \((0, 1)\). Adding self-loops \(\tilde{a}_{ii}=1\) and applying incoming normalization \(a'_{ji} = \tilde{a}_{ji} / \sum_{k \in \mathcal{N}(i) \cup \{i\}} \tilde{a}_{ki}\) yields the calibrated graph \(G' = (V, E, A')\), ensuring that propagation paths align with veracity consistency.
2. Dual-evidence Multimodal Graph Veracity Predictor (MGVP): Decoupling local content representation from explicit graph propagation
Individual posts vary substantially in their informational self-sufficiency: some exhibit obvious textual or visual fabrications, whereas ambiguous cases require supportive or debunking evidence from adjacent posts. To allow downstream adaptive balancing, MGVP explicitly separates the graph-encoded base prediction from the graph-propagated prediction. First, a 3-layer Graph Convolutional Network (GCN) encodes node features over the refined graph \(G'\) to yield hidden representations \(h_i = f_\theta(X, G')_i\), from which the base prediction is obtained as \(p_i^b = \text{softmax}(W_o h_i)\). Because \(p_i^b\) reflects post-level features enriched by immediate neighborhood message passing, it serves as the local content baseline. Next, MGVP initializes chain-style label propagation with \(p_i^{lp, (0)} = p_i^b\) and propagates class distributions over \(G'\) for \(T\) iterations: $\(p_i^{lp, (t+1)} = (1 - \lambda) p_i^b + \lambda \sum_{j \in \mathcal{N}(i) \cup \{i\}} \alpha_{ji} p_j^{lp, (t)}\)$ where propagation weight \(\alpha_{ji}\) normalizes the product of topological weight \(a'_{ji}\) and an adaptive pairwise relation term \(\psi_\theta(h_i, h_j) = \text{sigmoid}(\text{MLP}_\theta([h_i; h_j]))\). The converged output \(p_i^{lp} = p_i^{lp, (T)}\) encapsulates multi-hop contextual consensus, providing an explicit contrast against \(p_i^b\).
3. Cross-fitted Semantic-Structural Propagation Utility Gate (CPUG): Learning node-specific reliance without label leakage
Even on a refined graph, certain nodes reside in noisy or sparse neighborhoods where propagation degrades accuracy. CPUG estimates a node-level scalar gate \(g_i \in [0, 1]\) to control propagation reliance. Crucially, optimizing gate parameters directly on standard training predictions induces severe in-sample overfitting and label leakage, as the model cannot discern true generalization improvements. To resolve this, CPUG employs a \(K\)-fold cross-fitting protocol: the labeled set \(V_L\) is partitioned into disjoint folds \(V_k\), training LGR and MGVP on \(V_L \setminus V_k\) to evaluate held-out instances in \(V_k\). A positive utility label \(z_i = \mathbb{I}[\text{CE}(y_i, p_i^{lp}) + m < \text{CE}(y_i, p_i^b)]\) is assigned only when propagation improves cross-entropy loss by a strict margin \(m\). The gate MLP \(q_\omega\) takes a four-dimensional semantic-structural descriptor \(r_i = [H_i, J_i, D_i, s_i]\): $\(H_i = \mathcal{H}(p_i^b), \quad J_i = \text{JS}(p_i^b, p_i^{lp}), \quad D_i = \frac{1}{|\mathcal{N}(i)|}\sum_{j \in \mathcal{N}(i)} \text{JS}(p_i^{lp}, p_j^{lp}), \quad s_i = \mathbb{I}[\tilde{y}_i \ne -1, c_i \ge \tau] c_i |\tilde{y}_i - \hat{y}_i^{lp}|\)$ capturing base uncertainty, prediction divergence, neighborhood discordance, and direct semantic-structural conflict against high-confidence LLM pseudo-labels. The final prediction fuses both outputs via \(p_i^{\text{final}} = (1 - g_i) p_i^b + g_i p_i^{lp}\).
Loss & Training¶
The framework is optimized through a decoupled three-phase training procedure: 1. Phase A (Cross-fitted Utility Construction): Across \(K=5\) disjoint folds, train LGR and MGVP on \(V_L \setminus V_k\) using the multi-task loss \(L_{\text{pred}} = L_{\text{base}} + L_{\text{lp}} + \mu L_{\text{align}}\), combining cross-entropy losses for \(p^b\) and \(p^{lp}\) with Maximum Mean Discrepancy (MMD) event alignment. This phase extracts out-of-sample feature-utility pairs \(\{(r_i, z_i)\}\) without gold-label contamination. 2. Phase B (Conflict-coupled Gate Optimization): Train the 2-layer gate MLP \(q_\omega\) (hidden dimension 16) on the collected pairs using a conflict-regularized objective: $\(L_{\text{gate}} = \frac{1}{|V_L|} \sum_{v_i \in V_L} [\text{BCE}(g_i, z_i) + \gamma s_i g_i]\)$ The regularizer \(\gamma s_i g_i\) penalizes high propagation reliance whenever semantic-structural conflict \(s_i\) is severe, after which gate parameters \(\omega\) are frozen. 3. Phase C (Full Retraining and Inference): Retrain LGR and MGVP jointly on the entire labeled dataset \(V_L\). At test time, test instances are evaluated on the calibrated graph \(G'\), with the frozen CPUG dynamically outputting \(g_i\) for adaptive fusion.
Key Experimental Results¶
Main Results¶
Experiments were conducted on three benchmark datasets (Twitter, PHEME, and Weibo) using official splits and 5 random seeds, comparing SSPC against unimodal/multimodal baselines, graph neural networks, and LLM-assisted label propagation models.
| Dataset | Metric | SSPC (CLIP) | Strongest CLIP Baseline (GLPN-LLM / FCN-LP+LLM) | Improvement |
|---|---|---|---|---|
| Accuracy (%) | 86.43 ± 2.54 | 85.65 ± 2.11 (GLPN-LLM) | +0.78 (\(p=0.041\)) | |
| Precision (%) | 83.53 ± 2.98 | 82.04 ± 2.68 (GLPN-LLM) | +1.49 (\(p=0.018\)) | |
| Recall (%) | 98.23 ± 0.94 | 90.78 ± 2.47 (GLPN-LLM) | +7.45 (\(p=0.038\)) | |
| Macro F1 (%) | 88.50 ± 1.56 | 86.13 ± 2.36 (GLPN-LLM) | +2.37 (\(p=0.038\)) | |
| PHEME | Accuracy (%) | 87.46 ± 0.39 | 84.89 ± 2.81 (FCN-LP+LLM) | +2.57 (\(p=0.030\)) |
| PHEME | Precision (%) | 91.40 ± 0.31 | 87.80 ± 2.36 (FCN-LP+LLM) | +3.60 (\(p=0.029\)) |
| PHEME | Recall (%) | 90.97 ± 0.70 | 90.55 ± 2.28 (FCN-LP+LLM) | +0.42 (\(p=0.047\)) |
| PHEME | Macro F1 (%) | 91.18 ± 0.31 | 89.21 ± 2.59 (FCN-LP+LLM) | +1.97 (\(p=0.026\)) |
| Accuracy (%) | 86.83 ± 0.57 | 85.29 ± 1.57 (GLPN-LLM) | +1.54 (\(p=0.036\)) | |
| Precision (%) | 89.32 ± 1.37 | 88.82 ± 0.73 (GLPN-LLM) | +0.50 (\(p=0.046\)) | |
| Recall (%) | 94.08 ± 1.31 | 91.88 ± 0.79 (GLPN-LLM) | +2.20 (\(p=0.029\)) | |
| Macro F1 (%) | 91.64 ± 0.30 | 90.55 ± 0.58 (GLPN-LLM) | +1.09 (\(p=0.027\)) |
Under the HMCAN backbone, SSPC similarly establishes SOTA results, achieving 89.68% Acc / 87.74% F1 on PHEME and 90.54% Acc / 91.59% F1 on Weibo.
Ablation Study¶
Ablations on core architectural components with the CLIP encoder (Acc / Prec / Rec / F1 in %):
| Configuration | Twitter (F1) | PHEME (F1) | Weibo (F1) | Mechanism Description & Impact |
|---|---|---|---|---|
| SSPC Full Model | 88.50 ± 1.56 | 91.18 ± 0.31 | 91.64 ± 0.30 | Complete two-level calibration with cross-fitted gate |
| w/o CPUG | 86.54 ± 1.85 | 89.50 ± 0.19 | 90.77 ± 0.38 | Disables utility gate; uniform propagation across all nodes (drops 0.87–1.96% F1) |
| w/o LGR | 85.57 ± 1.51 | 88.59 ± 0.25 | 89.30 ± 1.77 | Reverts to uncalibrated original graph topology (drops 2.34–2.93% F1, largest drop) |
| w/o SCGC | 87.38 ± 2.22 | 88.91 ± 0.25 | 90.71 ± 1.88 | Removes conflict feature \(s_i\) and regularizer \(\gamma s_i g_i\) from gate (drops 0.93–2.27% F1) |
| In-sample Gate | 85.76 ± 2.34 | 88.83 ± 0.11 | 91.26 ± 0.19 | Trains gate on in-sample training predictions; causes severe label leakage & drop |
Key Findings¶
- Edge refinement is the primary performance driver: Removing LGR incurs the sharpest F1 drop across all benchmarks (-2.93% on Twitter, -2.59% on PHEME, and -2.34% on Weibo). UMAP visualizations on PHEME confirm that while uncalibrated propagation yields substantial inter-class blending (silhouette score 0.389), LGR refinement tightens intra-class clusters and expands boundary margins (silhouette score 0.560).
- Pairwise pseudo-label agreement exhibits superior noise robustness: Although node-level LLM pseudo-label accuracy is noisy (75.39% on Twitter, 74.38% on PHEME, and 80.86% on Weibo), edge-level agreement accuracy across high-confidence connected pairs reaches 99.97%, 99.47%, and 100.00%, respectively. Filtering edges pairwise is vastly more reliable than supervising nodes with individual pseudo-labels.
- Cross-fitting eliminates deceptive gate supervision: Replacing the cross-fitted protocol with in-sample training degrades F1 by 2.74% on Twitter and 2.35% on PHEME. In-sample base predictions overfit training labels, misleading the gate into believing propagation is redundant.
Highlights & Insights¶
- Repurposing LLMs from direct supervisors into weak topological calibrators: Rather than forcing LLMs to act as imperfect pseudo-labelers via knowledge distillation, SSPC demonstrates that LLMs excel when used as weak relational filters to diagnose edge homophily and modulate propagation trust.
- Theoretical invariance under symmetric label corruptions: The authors formally prove that even if an LLM systematically flips the binary veracity of two connected nodes (i.e., both node pseudo-labels are simultaneously wrong), their pairwise agreement indicator \(\mathbb{I}[\tilde{y}_i = \tilde{y}_j]\) remains mathematically invariant. This explains the exceptional robustness of LGR under label corruption sweeps.
- High inference efficiency and memory savings: By freezing the lightweight gate after training, SSPC achieves per-sample test latencies below 0.054 ms, while cutting peak training GPU memory by 40.4% to 63.1% compared to global LLM propagation baselines like GLPN-LLM.
Limitations & Future Work¶
- Transductive graph dependency: Like existing graph-based label propagation baselines, SSPC is primarily evaluated in transductive settings where test nodes are present in the adjacency matrix. Adapting the method to streaming social feeds requires engineering real-time feature extraction and dynamic incremental graph updating.
- Single-model calibration source: The current implementation derives pseudo-labels exclusively from GPT-4o. Integrating multi-LLM consensus ensembles or grounding verification via external web retrieval could further enhance calibration reliability under domain shifts.
Related Work & Insights¶
- vs FCN-LP (Zhao et al., ACM MM 2023): FCN-LP pioneered cross-modal tweet graphs for label propagation but treated all links as homophilous, suffering severe degradation when rumor and debunking tweets share identical media; SSPC introduces LGR to prune conflicting edges and CPUG to guard against noisy neighborhoods.
- vs GLPN-LLM (Hu et al., ACL 2025): GLPN-LLM incorporates LLM pseudo-labels as soft priors in global label propagation, risking error propagation when pseudo-labels hallucinate; SSPC confines LLMs to edge routing and gate regularizers, providing far superior stability under severe label noise.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Formulates a rigorous two-level propagation calibration framework addressing semantic-structural conflict with weak LLM supervision.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive cross-lingual benchmarks, statistical significance testing, ablation of training protocols, and formal error bounds.
- Writing Quality: ⭐⭐⭐⭐⭐ Clear motivation, clean mathematical formulation, intuitive visualizations, and cohesive narrative flow.
- Value: ⭐⭐⭐⭐⭐ Delivers actionable insights for heterophilous graph learning, noisy-label calibration, and LLM-assisted graph intelligence.