CoFGraph: Interleaving Reciprocal Calibration with Confidence Feedback for LLM-Augmented Graph Learning¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: LLM-Augmented Graph Learning, Dual-View Reciprocal Calibration, Confidence Feedback, Semantic Proxy, Node Classification
TL;DR¶
CoFGraph introduces a confidence-feedback graph learning framework that eliminates the noise propagation and severe inference latency of static, unidirectional LLM-to-GNN pipelines. By distilling a frozen LLM into a lightweight isomorphic Interpreter offline and establishing an instance-level confidence-driven reciprocal calibration between semantic and structural views, CoFGraph achieves consistent state-of-the-art accuracy across four benchmarks, cuts test-time LLM inference bottlenecks to zero, and exhibits remarkable robustness against semantic noise.
Background & Motivation¶
Graph learning seeks to encode relational data and node attributes into expressive representations to power downstream tasks such as node classification and link prediction. When nodes are enriched with free-form natural language descriptions, traditional graph neural networks (GNNs) typically rely on shallow or fixed text embeddings that lack relational and structural context, inevitably precipitating semantic-structural mismatches. In response, large language models (LLMs) have recently been introduced to text-attributed graph learning due to their unprecedented semantic reasoning and generative prowess. The prevailing paradigm, epitomized by methods like TAPE, LLM4NG, and LLM-GNN, adopts an "LLM-as-enhancer" strategy where an LLM is queried to generate pseudo-labels, soft distributions, and natural language rationales, which are subsequently injected into a downstream GNN as static features or supervision.
Despite preliminary gains, this standard pipeline suffers from two fundamental limitations. First, existing augmentation is fundamentally unidirectional and structure-agnostic: LLMs generate text-driven rationales without topological grounding, making them prone to hallucinations and contextual errors. For instance, a paper titled "Analyzing Historical Documents using Neural Networks" may be erroneously classified by an LLM as History due to superficial keyword bias, completely overlooking its homogeneous Computer Science citation neighborhood. When such erroneous semantics are injected into a GNN, the message-passing mechanism amplifies and cascades the noise across the graph, while the pipeline lacks any feedback channel for structural signals to recalibrate semantic claims. Second, existing methods incur crippling computational costs, requiring either parameter-heavy LLM fine-tuning or repeated, real-time LLM querying during test-time inference, creating prohibitive latency and cost barriers in deployment.
To resolve the dual bottlenecks of semantic noise accumulation and inference overhead, graph learning requires an efficient, bidirectional paradigm where semantic evidence and structural topology can dynamically audit each other. Core idea: distill reasoning knowledge from a frozen LLM into a lightweight isomorphic Interpreter to eradicate test-time querying overhead, while employing instance-wise absolute and relative certainty metrics to dynamically partition nodes and interleave reciprocal calibration, allowing the locally superior view to correct the other under uncertainty-aware decision fusion.
Method¶
Overall Architecture¶
CoFGraph comprises two coordinated modules: Semantic Proxy Training and Confidence-Based Dual-View Learning. In the Semantic Proxy Training stage, the LLM is kept completely frozen. A unified prompting scheme instructs the LLM to extract hard pseudo-labels, soft label distributions, and explanatory rationales for central nodes, alongside importance scores and rationales for local neighbors; these rich supervisory signals are distilled into a lightweight local Interpreter (comprising a DeBERTa encoder and GNN layers), fully circumventing runtime LLM invocations. In the Confidence-Based Dual-View Learning stage, an isomorphic Student model is paired with the Interpreter to represent graph topology. A Reliability-Aware Sample Partitioning mechanism evaluates instance-wise absolute certainty (prototype distance) and relative certainty (entropy) for each view, partitioning nodes into text-dominated and structure-dominated subsets. The more confident view temporarily acts as a teacher to regularize the weaker student through directionally weighted KL-divergence. At inference, predictions from both views are synthesized via uncertainty-aware decision fusion.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}}%%
flowchart TD
subgraph SP["Semantic Proxy Training"]
direction TB
A["Frozen LLM + Unified Prompts<br/>Extract labels, soft logits, rationales, neighbor weights"] --> B["Multi-Task Distillation into Interpreter<br/>Eliminate test-time LLM inference calls"]
end
subgraph CDL["Confidence-Based Dual-View Learning"]
direction TB
B --> C["Interpreter (Semantic View) & Student (Structural View)"]
C --> D["Reliability-Aware Sample Partitioning<br/>Evaluate prototype certainty a & entropy r to form Q_t and Q_s"]
D --> E["Adaptive Dual-View Learning<br/>Dynamic role switching: reliable view guides weaker view via KL"]
end
E --> F["Uncertainty-Aware Decision Fusion<br/>Modulate view weights by absolute certainty a_i"]
F --> G["Final Node Classification Prediction"]
Key Designs¶
1. Semantic Proxy Training: Offloading Reasoning to a Lightweight Local Proxy
Direct LLM inference during deployment imposes an \(\mathcal{O}(\text{LLM})\) latency bottleneck, while standard fine-tuning is computationally prohibitive. CoFGraph solves this by distilling predictive supervision and topological explanations into a compact Interpreter model. For each node \(v_n\) and its neighborhood \(\mathcal{N}(v_n)\), a unified prompt guides the frozen LLM to generate a pseudo hard label \(y_n\), soft probability distribution \(l_n\), and textual rationale \(r_n\). Furthermore, to instill structural discernment, the LLM performs generative neighborhood evaluation, scoring the relational importance \(s_{n,i}\) and supporting textual rationale \(r_{n,i}\) of each neighbor \(v_i \in \mathcal{N}(v_n)\). The Interpreter replaces raw text with generated rationales and uses \(s_{n,i}\) as graph convolutional aggregation weights. The model is trained via multi-task alignment against hard pseudo-labels and soft logits:
This compresses full-scale LLM reasoning into an isomorphic network while insulating downstream training from LLM querying overhead.
2. Reliability-Aware Sample Partitioning: Dual-Metric Local Teacher Selection
Because neither semantic reasoning nor graph topology is universally reliable across all graph nodes, indiscriminate cross-view distillation can propagate misleading supervisory signals. CoFGraph determines local reliability from the posterior distribution derived from class prototype distances in feature space: \(P(\hat{y}_i = k \mid x_i^v) \propto \exp(-\psi(q_i^v, t_k^v))\). To establish a robust selection criterion, the framework formulates two complementary metrics: - Absolute Certainty (\(a_i^v\)): defined as the peak probability over the prototype posterior, \(a_i^v = \max_k P(\hat{y}_i = k \mid x_i^v)\), measuring instance-level proximity to the most representative class center; - Relative Certainty (\(r_i^v\)): defined as the negative prediction entropy, \(r_i^v = \sum_{k=1}^N P(\hat{y}_i = k \mid x_i^v) \log P(\hat{y}_i = k \mid x_i^v)\), quantifying categorical margin sharpness across all classes.
A sample is assigned to the Text-Dominated set \(Q_t\) or Structure-Dominated set \(Q_s\) only when one view strictly surpasses the other on both criteria simultaneously:
This conservative dual-threshold boundary ensures that mutual calibration is governed exclusively by verified, high-confidence domain experts.
3. Adaptive Dual-View Learning and Decision Fusion: Reciprocal Role Switching and Certainty-Gated Output
Rather than enforcing static teacher-student roles, CoFGraph institutes bidirectional role switching. For samples in \(Q_t\), the semantic view (Interpreter) serves as teacher to align the structural view (Student); conversely, for samples in \(Q_s\), the structural view guides the Interpreter. The directional distillation losses are explicitly weighted by the absolute certainty of the expert view to accentuate clean samples:
The global objective balances standard cross-entropy losses with bidirectional distillation: \(L = L_s^{ce} + L_t^{ce} + \lambda_2 (L_{t \to s}(\theta_s) + L_{s \to t}(\theta_t))\). During inference, rather than using uniform averaging, Uncertainty-Aware Decision Fusion dynamically combines posterior predictions using absolute certainty scores \(a_i^t\) and \(a_i^s\) as normalized weights, ensuring that final classifications are predominantly driven by whichever modality possesses higher localized certainty.
A Worked Example¶
The efficacy of reciprocal calibration is highlighted by two complementary failure corrections on the Cora benchmark: - Case 1 (Semantic Corrects Structure, Node 1827): The paper "Efficient Algorithms for Inverting Evolution" has ground truth Theory. Because its citations connect ambiguously to both Neural Networks and Theory nodes, the structural Student is confounded, predicting Neural Networks with high entropy (\(H_s = 1.3801, a_s = 0.4270\)). In contrast, the semantic Interpreter extracts decisive algorithmic evidence from the abstract, predicting Theory with extreme confidence (\(H_t = 0.0744, a_t = 0.9869\)). CoFGraph assigns the sample to \(Q_t\), calibrates the Student during training, and allocates a high inference weight (\(w = 0.7868\)) to the semantic view, successfully overriding the noisy topology. - Case 2 (Structure Corrects Semantic Hallucination, Node 1693): The paper "Incremental Self-Improvement for Life-Time Multi-Agent Reinforcement Learning" belongs to Reinforcement Learning (RL). The semantic view falls victim to textual ambiguity and hallucination, erroneously predicting Case Based with high entropy (\(H_t = 1.4598, a_t = 0.3797\)). Fortunately, the structural citation neighborhood exhibits strong homophily for RL, leading the Student to predict RL with near certainty (\(H_s = 0.0008, a_s = 0.9999\)). CoFGraph partitions this node into \(Q_s\), suppresses the semantic hallucination, and assigns a minor semantic fusion weight (\(w = 0.1886\), yielding \(81.14\%\) structural dominance) to secure the correct RL label.
Loss & Training¶
CoFGraph follows a two-stage training scheme: 1. Stage 1 (Interpreter Pre-training): Train the Interpreter on the training set using the multi-task loss in Equation (3) balanced by hyperparameter \(\lambda_1\), aligning its representation space with LLM rationales and neighborhood importance distributions. 2. Stage 2 (Joint Dual-View Co-training): Jointly optimize Interpreter and Student backbones using Equation (8) balanced by hyperparameter \(\lambda_2\). Evaluated across 4 runs, models utilize DeBERTa text backbones and 4-layer GNN architectures on two NVIDIA A100 GPUs.
Key Experimental Results¶
Main Results¶
CoFGraph was benchmarked against text-centric models (fine-tuned DeBERTa, LLaMA), classical GNNs with shallow or PLM embeddings (MLP, GCN, GraphSAGE, RevGAT), and state-of-the-art frozen-LLM augmentation frameworks (LLM4NG, LLM-GNN, TAPE). The arxiv_2023 benchmark contains strictly post-2023 literature to guard against pre-training data contamination.
| Dataset | Backbone / Method | Prev. SOTA (TAPE) | Ours (CoFGraph) | Absolute Gain |
|---|---|---|---|---|
| Cora | MLP | 0.8160 ยฑ 0.0103 | 0.8410 ยฑ 0.0090 | +2.50% |
| GCN | 0.8727 ยฑ 0.0092 | 0.8995 ยฑ 0.0076 | +2.68% | |
| GraphSAGE | 0.8759 ยฑ 0.0055 | 0.9041 ยฑ 0.0048 | +2.82% | |
| RevGAT | 0.8782 ยฑ 0.0121 | 0.9065 ยฑ 0.0088 | +2.83% | |
| PubMed | MLP | 0.9495 ยฑ 0.0041 | 0.9630 ยฑ 0.0031 | +1.35% |
| GCN | 0.9370 ยฑ 0.0026 | 0.9520 ยฑ 0.0018 | +1.50% | |
| GraphSAGE | 0.9510 ยฑ 0.0038 | 0.9660 ยฑ 0.0028 | +1.50% | |
| RevGAT | 0.9505 ยฑ 0.0032 | 0.9645 ยฑ 0.0027 | +1.40% | |
| ogbn-products | MLP | 0.7850 ยฑ 0.0088 | 0.8120 ยฑ 0.0068 | +2.70% |
| GCN | 0.7980 ยฑ 0.0055 | 0.8270 ยฑ 0.0036 | +2.90% | |
| GraphSAGE | 0.8120 ยฑ 0.0048 | 0.8435 ยฑ 0.0038 | +3.15% | |
| RevGAT | 0.8215 ยฑ 0.0038 | 0.8525 ยฑ 0.0031 | +3.10% | |
| arxiv_2023 | MLP | 0.7980 ยฑ 0.0013 | 0.8265 ยฑ 0.0031 | +2.85% |
| GCN | 0.7810 ยฑ 0.0059 | 0.8105 ยฑ 0.0032 | +2.95% | |
| GraphSAGE | 0.8005 ยฑ 0.0028 | 0.8320 ยฑ 0.0021 | +3.15% | |
| RevGAT | 0.8060 ยฑ 0.0026 | 0.8370 ยฑ 0.0019 | +3.10% |
Ablation Study¶
Two complementary ablation investigations on the Cora benchmark dissect the contributions of individual components within the Interpreter and the Dual-View learning loop.
Table 1: Ablation of Interpreter Semantic Features
| Config | MLP | GCN | SAGE | RevGAT | Note |
|---|---|---|---|---|---|
| w/o labels | 0.7568 | 0.8382 | 0.8435 | 0.8455 | Omits LLM pseudo-hard and soft label supervision |
| w/o evidence | 0.7609 | 0.8421 | 0.8476 | 0.8494 | Omits generated textual node rationales \(r_n\) |
| w/o evaluation | 0.7585 | 0.8403 | 0.8455 | 0.8477 | Omits neighbor importance scores \(s_{n,i}\) and rationales \(r_{n,i}\) |
| Full Interpreter | 0.7725 | 0.8545 | 0.8602 | 0.8621 | Integrates all distilled multi-task signals |
Table 2: Ablation of Dual-View Learning Mechanisms
| Config | MLP | GCN | SAGE | RevGAT | Note |
|---|---|---|---|---|---|
| w/o AC (Absolute Certainty) | 0.7554 | 0.8347 | 0.8390 | 0.8412 | Partitions solely via entropy; drops ~6.5% accuracy |
| w/o RC (Relative Certainty) | 0.7953 | 0.8788 | 0.8833 | 0.8856 | Omits entropy constraint, inducing boundary confusion |
| w/o ADL (Adaptive Dual-View Loop) | 0.7603 | 0.8401 | 0.8444 | 0.8467 | Disables cross-view reciprocal KL distillation |
| w/o UDF (Uncertainty Decision Fusion) | 0.7676 | 0.8482 | 0.8526 | 0.8548 | Replaces dynamic gating with simple equal averaging |
| Full Model (Proposed) | 0.8140 | 0.8995 | 0.9041 | 0.9065 | Full reciprocal synergy yields peak performance |
Key Findings¶
- Robustness Under Injected Semantic Noise: In a severe stress test on Cora where 30% of the training LLM pseudo-labels and rationales were randomly corrupted, the unidirectional baseline TAPE suffered an 8.4% performance drop (0.8727 \(\to\) 0.7996). In stark contrast, CoFGraph exhibited only a 3.7% decline (0.8995 \(\to\) 0.8664). The confidence partition correctly diagnosed high posterior entropy on corrupted nodes and utilized structural neighborhood homophily to suppress semantic noise.
- Data Efficiency Across Regimes: Across varying label proportions from 0.1% to 60%, CoFGraph maintains a substantial advantage in extreme few-shot regimes (0.1%), demonstrating that dual-view reciprocal constraints provide vital inductive bias when supervised signal is virtually absent.
- Inference Efficiency: While TAPE incurs a \(31\text{s} + \mathcal{O}(\text{LLM})\) latency bottleneck during evaluation due to real-time LLM prompting, CoFGraph restricts its 1.7M token consumption to offline pre-processing, locking test latency at a fixed 21 seconds.
Highlights & Insights¶
- From Unidirectional Injection to Reciprocal Collaboration: The paper overturns the prevailing assumption that LLMs act as flawless global teachers for GNNs, demonstrating that topological evidence is indispensable for auditing semantic reasoning.
- Isomorphic Semantic Proxy: By training a lightweight local DeBERTa-GNN Interpreter to mirror LLM reasoning and neighborhood aggregation, the framework reaps the benefits of LLM knowledge distillation without incurring inference-time latency or cost.
- Dual-Threshold Reliability Criterion: Combining prototype-distance absolute certainty with information-theoretic relative certainty creates a principled filter that prevents overconfident guesses from corrupting cross-view distillation.
Limitations & Future Work¶
- Bounded by LLM Reasoning Quality: The fidelity of the semantic proxy fundamentally depends on the reasoning capability of the frozen LLM. In highly specialized domains (e.g., dense biochemical networks), generic LLMs may produce uninformative rationales that bottleneck downstream performance.
- Preprocessing Overhead: Although test-time LLM calls are eliminated, generating rationales and neighbor evaluations across all training nodes requires substantial offline token expenditure (1.7M tokens on tested graphs), presenting a computational bottleneck when scaling to web-scale dynamic graphs.
- Failure Under Correlated Systematic Bias: As disclosed in Appendix A.2 (Node 153), when both the text description and citation neighborhood share identical interdisciplinary ambiguities (e.g., robotics papers heavily citing RL literature while categorizing as Neural Networks), both views become confidently wrong in the same direction, causing reciprocal calibration to fail.
Related Work & Insights¶
- vs TAPE (He et al., 2024): TAPE pioneered LLM rationale generation for text-attributed graphs but is strictly unidirectional and suffers from test-time LLM latency. CoFGraph introduces an offline isomorphic Interpreter and dynamic bidirectional calibration that actively suppresses TAPE's characteristic semantic noise.
- vs LLM4NG (Yu et al., 2025) & LLM-GNN (Chen et al., 2024b): These methods rely on static node generation or one-way feature alignment. CoFGraph proves that sample-specific, confidence-guided role switching is vastly superior to uniform one-way guidance.
- vs GraphGPT (Tang et al., 2024) & LLaGA (Chen et al., 2024a): Graph instruction tuning aligns latent spaces via full or LoRA-based LLM fine-tuning, incurring immense computational costs. CoFGraph keeps the LLM frozen and distills into a lightweight proxy, achieving superior efficiency for deployment.
Rating¶
- Novelty: โญโญโญโญโ (Creative bidirectional calibration mechanism addressing fundamental unidirectional flaws in LLM-GNN pipelines)
- Experimental Thoroughness: โญโญโญโญโญ (Exemplary experimental coverage across four benchmarks, 30% noise injection, low-resource scaling, and in-depth qualitative failure modes)
- Writing Quality: โญโญโญโญโญ (Rigorous mathematical formulation, clear narrative structure, and insightful case studies)
- Value: โญโญโญโญโญ (Combines significant performance improvements with zero inference-time LLM calling overhead, offering immediate practical utility)