SHAPE: Structural Hypergraph Augmentation with Personalized Event Memory for Temporal Knowledge Graph Reasoning¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Temporal Knowledge Graphs, Hypergraph Neural Networks, Event Memory Augmentation, Plug-and-Play Framework, Extrapolation Reasoning
TL;DR¶
SHAPE is a plug-and-play enhancement framework for temporal knowledge graph reasoning that injects higher-order prototype hypergraph priors before temporal encoding and adds query-denoised event memory bias after encoding, delivering consistent state-of-the-art gains across diverse backbones.
Background & Motivation¶
Temporal knowledge graphs (TKGs) organize evolving real-world facts as timestamped quadruples \((s, r, o, t)\), and the fundamental extrapolation task aims to predict future facts based on historical observations. Modern graph-based TKG models, such as RE-GCN, TiRGN, HGLS, and LogCL, typically marry snapshot graph neural networks with temporal sequence encoders to track relational dynamics across time. However, conventional message-passing mechanisms in snapshot GNNs operate almost exclusively along pairwise edges. Such pairwise propagation fails to capture higher-order semantic correlations and latent functional roles shared across multiple entities and relations simultaneously. For instance, in geopolitical event datasets such as ICEWS, multiple sovereign nations often play structurally identical diplomatic or economic roles, but their shared semantic identity is scattered across disconnected pairwise interactions, leaving the temporal backbone devoid of explicit higher-order structural priors at the representation level.
Simultaneously, the history aggregation mechanisms employed at the decoder stage suffer from query-agnostic noise contamination. To alleviate catastrophic forgetting and capture long-range recurring patterns, existing frameworks often aggregate facts across recent historical windows or incorporate global historical entity vocabularies without query-aware filtering. When reasoning over a specific query (such as predicting a diplomatic counterpart), contemporaneous yet completely irrelevant economic, social, or humanitarian events occurring within the same recent temporal window are indiscriminately pooled into the historical representation. This query-agnostic mixing dilutes the critical predictive signal and introduces misleading supervisory shortcuts, severely undermining the confidence and accuracy of candidate entity ranking.
To systematically overcome these two decoupled bottlenecks without disrupting existing architectures, this work advocates an enhancement philosophy that targets both ends of the standard TKG pipeline. Core idea: Introduce Dual Hypergraph Pre-Enhancement (DHPE) before temporal encoding to inject learnable entity/relation prototype hypergraph priors and Query-Denoised Event Memory (QDEM) after temporal encoding to provide structure-aware, query-adaptive candidate scoring bias, acting as a modular, plug-and-play booster for heterogeneous graph backbones.
Method¶
Overall Architecture¶
SHAPE is intentionally designed around the standardized "embed \(\to\) encode \(\to\) decode" paradigm of graph-based TKG reasoning. The backbone encoder is preserved strictly as an unmodified black box. SHAPE operates at both ends of this pipeline: prior to temporal encoding, the Dual Hypergraph Pre-Enhancement (DHPE) module processes raw entity and relation embedding tables through learnable prototype hypergraphs and a cross-view information bottleneck, yielding structure-aware embeddings \(\tilde{X}\) and \(\tilde{R}\) for the backbone. After temporal encoding produces the dynamic entity matrix \(Z\), the Query-Denoised Event Memory (QDEM) module performs biased top-\(k\) retrieval over a sliding event bank and applies Structure-Aware Expert Gating (SAEG) to blend historical recurrence, dense memory matching, and hypergraph topological proximity into a candidate-level scoring bias vector \(b\) alongside an enhanced query subject representation \(\hat{s}\).
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
In["Input: Entity/Relation Embeddings X, R & Query (s, r, ?, t)"] --> DHPE["Dual Hypergraph Pre-Enhancement (DHPE)<br/>Prototype Modulation + Gather-Scatter + Cross-View Bridge"]
DHPE --> Enc["Backbone Temporal Encoder (Unmodified)<br/>Snapshot Evolution producing Timed Representations Z, R̃"]
Enc --> QDEM["Query-Denoised Event Memory (QDEM)<br/>Biased Event Retrieval + Subject Enhancement"]
QDEM --> SAEG["Structure-Aware Expert Gating (SAEG)<br/>Adaptive Gating of HCP + MES + SPS into Candidate Bias b"]
SAEG --> Out["Final Prediction: Logits = Decoder(ŝ, Z) + b"]
Key Designs¶
1. Dual Hypergraph Pre-Enhancement: Higher-Order Prototype Convolution & Cross-View Bridge To overcome the limitation of pairwise edge propagation, DHPE introduces \(K_e\) learnable entity prototype vectors \(P_e \in \mathbb{R}^{K_e \times d}\) and \(K_r\) relation prototype vectors \(P_r \in \mathbb{R}^{K_r \times d}\). On the entity side, \(P_e\) is first modulated via cross-attention with global relation embeddings to render prototypes relation-aware. It then computes a soft, fully differentiable incidence matrix \(H_e \in \mathbb{R}^{N \times K_e}\) using multi-head cross-attention scaled by a learnable temperature \(\tau\): $\(H_e = \mathrm{softmax}\left(\frac{\mathrm{mean\_head}(X W_q (P'_e W_k)^\top)}{\tau \sqrt{d_h}}\right)\)$ Hypergraph convolution is conducted via a gather-scatter procedure: entity features are gathered into hyperedge prototype representations, refined through a residual feed-forward block, and then scattered back to entity space through degree-normalized projections. The resulting hypergraph-enhanced representation is fused with the original embedding via a gated residual connection. An analogous process generates \(R^{(r)}\) on the relation side. Crucially, to bridge the two separate hypergraph views without quadratic cost, DHPE introduces \(M\) learnable anchor vectors \(B \in \mathbb{R}^{M \times d}\) as a compact information bottleneck. Attention flows from entity features to anchors, and then from anchors to relation features (and vice versa) at a linear complexity of \(\mathcal{O}(M(N+R))\), outputting enriched representations \(\tilde{X}\) and \(\tilde{R}\).
2. Query-Denoised Event Memory: Temporal Decayed Retrieval & Dynamic Subject Enhancement To prevent contemporaneous irrelevant events from corrupting the decoding stage, QDEM maintains a sliding memory bank of recent events over the past \(L\) snapshots. Each quadruple \((s_i, r_i, o_i, t_i)\) and its inverse fact are represented as dense event vectors \(e_i\) incorporating post-encoder node states, temporal distance embeddings for recency offset \(\Delta_i = t - t_i\), and an explicit role-direction vector \(z_{o_i} - z_{s_i}\). For a given query \(q = (s, r, ?, t)\), QDEM assigns a query-conditioned relevance score to each event: $\(a_i = \frac{q^\top W_k e_i}{\sqrt{d}} - \lambda \frac{\Delta_i}{L} + b_s \mathbb{I}[s_i = s] + b_r \mathbb{I}[r_i = r]\)$ where \(\lambda \ge 0\) is a learnable recency decay factor, and \(b_s, b_r\) reward exact subject or relation matches. Facts whose forward or inverse forms match neither the query subject nor relation are masked out. A top-\(k\) selection yields a denoised event subset \(T_k(q)\), across which softmax weights \(\alpha_j\) are computed to form the memory context \(m = \sum_{j \in T_k(q)} \alpha_j W_v e_j\). This context vector is integrated with the backbone subject representation \(z_s\) via a gated residual layer to produce the personalized subject vector \(\hat{s}\) for decoding.
3. Structure-Aware Expert Gating: Tripartite Candidate Scoring Bias Fusion Rather than relying solely on the raw decoder output, QDEM constructs an additive candidate scoring bias vector \(b \in \mathbb{R}^N\) by synthesizing three distinct sources of evidence. The Historical Candidate Prior (HCP) explicitly tallies the recency-decayed historical frequency of each candidate answering \((s, r)\) in the memory bank; the Memory-Entity Scoring (MES) computes a dense dot-product similarity between the projected memory context \(W_m m\) and all entity representations in \(Z\); and the Structural Proximity Score (SPS) transfers topological structural affinity from DHPE by measuring prototype-membership cosine similarity between the query subject and all candidate entities: $\(\mathrm{SPS} = \alpha_{sps} \cdot H_e[s] H_e^\top\)$ These three signals are combined via structure-aware expert gating: $\(b = g_{hcp} \cdot \mathrm{HCP} + \mathrm{MES} + g_{sps} \cdot \mathrm{SPS}\)$ where scalar gates \(g_{hcp} = \sigma(w_{hcp}^\top [q; m])\) and \(g_{sps} = \sigma(w_{sps}^\top [q; m])\) are initialized with negative biases to default to near-zero. This architectural choice ensures that the model falls back safely on dense continuous matching (MES) by default, activating the discrete historical frequency or structural proximity experts only when the query context provides high-confidence cues.
Loss & Training¶
SHAPE is trained end-to-end jointly with the underlying backbone without multi-stage schedules or warm-up phases. The total objective optimizes the backbone's cross-entropy loss \(\mathcal{L}_{task}\) augmented with two prototype regularization terms across both entity and relation hypergraph views (\(v \in \{e, r\}\)): $\(\mathcal{L} = \mathcal{L}_{task} + \sum_{v \in \{e, r\}} \left( \lambda_{div} \|\hat{P}_v \hat{P}_v^\top - I\|_F^2 + \lambda_{bal} \mathrm{Var}(H_v^\top \mathbf{1}) \right)\)$ The diversity term \(\lambda_{div}\) enforces mutual orthogonality among row-normalized prototypes \(\hat{P}_v\) to span diverse semantic subspaces, while the balance term \(\lambda_{bal}\) penalizes variance in prototype assignment sums to avert cluster mode collapse. Hyperparameters are fixed at \(\lambda_{div} = \lambda_{bal} = 0.01\). Optimization uses Adam with learning rate 0.001, weight decay \(10^{-5}\), gradient clipping at 1.0, and validation-MRR early stopping (patience = 10 epochs).
Key Experimental Results¶
Main Results¶
Evaluation was conducted on three widely adopted TKG benchmarks (ICEWS14, ICEWS18, and ICEWS05-15) under the standard time-aware filtered protocol. SHAPE was plugged into two state-of-the-art backbones, HGLS and LogCL, and benchmarked against 18 competitive baselines (all metrics reported in %):
| Backbone & Model Variant | ICEWS14 MRR | ICEWS14 H@1 | ICEWS14 H@10 | ICEWS18 MRR | ICEWS18 H@1 | ICEWS18 H@10 | ICEWS05-15 MRR | ICEWS05-15 H@1 | ICEWS05-15 H@10 |
|---|---|---|---|---|---|---|---|---|---|
| HGLS (Baseline) | 47.00 | 35.06 | 70.41 | 29.32 | 19.21 | 49.83 | 46.21 | 35.32 | 67.12 |
| HGLS-SHAPE (Ours) | 51.08 | 40.10 | 72.41 | 37.15 | 26.32 | 58.64 | 49.96 | 40.37 | 68.04 |
| Relative Gain on HGLS | +8.68% | +14.38% | +2.84% | +26.71% | +37.01% | +17.68% | +8.12% | +14.30% | +1.37% |
| LogCL (Baseline) | 48.87 | 37.76 | 70.26 | 35.67 | 24.53 | 57.74 | 57.04 | 46.07 | 77.87 |
| LogCL-SHAPE (Ours) | 51.23 | 40.63 | 72.02 | 37.89 | 26.83 | 59.90 | 58.95 | 48.43 | 78.72 |
| Relative Gain on LogCL | +4.83% | +7.60% | +2.51% | +6.22% | +9.38% | +3.74% | +3.35% | +5.12% | +1.09% |
(Note: When extended to the recurrent backbone RE-GCN, RE-GCN-SHAPE similarly improved MRR by +3.31 on ICEWS14, +2.41 on ICEWS18, and +2.49 on ICEWS05-15.)
Ablation Study¶
Comprehensive ablations on HGLS and LogCL quantify the distinct impact of each submodule across all three datasets (\(\Delta\text{MRR}\) denotes average absolute drop from Full SHAPE):
| Variant Configuration | ICEWS14 MRR | ICEWS18 MRR | ICEWS05-15 MRR | HGLS \(\Delta\text{MRR}\) | LogCL \(\Delta\text{MRR}\) | Description |
|---|---|---|---|---|---|---|
| Full SHAPE (A0) | 51.08 / 51.23 | 37.15 / 37.89 | 49.96 / 58.95 | — | — | Complete dual-enhancement pipeline |
| w/o DHPE (A1) | 47.44 / 46.55 | 35.04 / 37.16 | 45.59 / 56.81 | -3.37 | -2.52 | Removes pre-encoder hypergraph structural priors |
| w/o QDEM (A2) | 47.77 / 48.55 | 33.73 / 36.27 | 46.88 / 56.73 | -3.27 | -2.17 | Removes post-encoder query-denoised memory |
| w/o Cand. Bias (A3) | 50.01 / 48.83 | 36.59 / 36.16 | 47.05 / 57.11 | -1.52 | -1.99 | Disables candidate bias pathway \(b\) |
| w/o Subj. Enh. (A4) | 50.60 / 48.76 | 36.12 / 37.59 | 47.48 / 58.56 | -1.32 | -1.05 | Disables subject representation update \(\hat{s}\) |
| w/o Cross-View (A5) | 50.31 / 48.68 | 36.62 / 37.61 | 47.02 / 57.56 | -1.40 | -1.41 | Removes anchor-based cross-view bridge |
| w/o Ent. Hyper. (A6) | 50.71 / 48.73 | 36.13 / 37.36 | 46.88 / 57.02 | -1.46 | -1.67 | Retains relation hypergraph only |
| Backbone Only (A7) | 47.00 / 48.87 | 29.32 / 35.67 | 46.21 / 57.04 | -5.55 | -2.16 | Original unmodified backbone |
A finer breakdown of SAEG signals on LogCL-SHAPE reveals that removing HCP, MES, or SPS on ICEWS14 drops MRR from 51.23 to 50.01, 49.84, and 49.88 respectively. Retaining historical frequency alone (HistFreq-only) achieves only 49.50 MRR, proving that simple historical copying is insufficient without structural proximity and dense memory scoring.
Key Findings¶
- Disproportionate Boost on Hits@1: Across all benchmarks, SHAPE delivers much higher relative gains on Hits@1 than on Hits@10 (e.g., +37.01% on ICEWS18 for HGLS). Systematic case analysis shows that out of 4,448 queries misranked by LogCL, 54.72% receive rank promotions and 15.47% move directly to rank 1, demonstrating that SHAPE actively resolves top-rank ambiguities rather than merely shuffling lower-ranked candidates.
- Amplified Efficacy under Structural Deficits and Sparsity: Backbones with weaker structural reasoning (HGLS) benefit significantly more from DHPE. In extreme data sparsity tests (10% to 90% random training triple deletion), the performance advantage of SHAPE over bare backbones widens (e.g., MRR gap expands from +4.08 to +6.60 at 30% deletion on HGLS), confirming that prototype sharing provides robust topological scaffolds when observed facts are sparse.
- Negligible Computational Overhead: SHAPE introduces only ~2.72M parameters, increases GPU memory by merely 0.19–0.41 GB, and adds less than 2% per-epoch training time on an NVIDIA RTX 4090 GPU, validating its practicality as an efficient plug-and-play enhancer.
Highlights & Insights¶
- Dual-Ended Decoupled Architecture: By placing structural enhancement at the input embedding layer and denoised memory enhancement at the decoding scoring layer, SHAPE cleanly bypasses the internal complexity of diverse graph sequence encoders, enabling broad cross-model applicability.
- Anchor-Mediated Information Bottleneck: The use of \(M\) learnable anchor tokens to mediate attention between entity and relation hypergraph views compresses cross-view complexity from quadratic \(\mathcal{O}(N \times R)\) to linear \(\mathcal{O}(M(N+R))\), delivering rich inter-view semantic flow at minimal computational expense.
- Defensive Default in Expert Gating: Initializing the gates for explicit historical counting (HCP) and structural prototype similarity (SPS) with negative biases ensures that the model defaults safely to smooth continuous matching (MES), activating discrete structural and frequency biases only when strong evidence is present.
Limitations & Future Work¶
- Performance Bound by Backbone Expressiveness: As an external plug-in, SHAPE's marginal returns naturally diminish when evaluated on an exceptionally strong backbone that already captures high-capacity temporal dynamics.
- Implicit Prototype Latents Lacking Axiomatic Semantics: The prototype hyperedges are learned purely end-to-end via attention without aligning to explicit real-world ontologies or type taxonomies; incorporating external schema constraints could further enhance interpretability.
- Heuristic Memory Window Sizing: The optimal event memory length \(L\) varies across dataset timescales (\(L=3\) for ICEWS14 vs. \(L=10\) for ICEWS05-15); future work could explore dynamic, query-adaptive history span truncation.
Related Work & Insights¶
- vs DHE-TKG / MGESL: Prior hypergraph methods rely heavily on heuristic event co-occurrence rules to construct static hyperedges and restrict hypergraph operations strictly to the encoder; SHAPE learns soft prototype incidence end-to-end and uniquely propagates hypergraph prototype distributions to decoder candidate scoring (SPS).
- vs TiRGN / CyGNet / LogCL: Traditional historical copy and contrastive methods aggregate history indiscriminately across the temporal window, susceptible to noise; SHAPE enforces hard masking against non-matching entities/relations and introduces structured expert gating over multi-source evidence.
Rating¶
- Novelty: ⭐⭐⭐⭐☆ Elegantly decomposes TKG enhancement into pre-encoder hypergraph priors and post-encoder query-denoised memory scoring without touching the backbone.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluations across three benchmarks, two primary and one auxiliary backbone, extensive fine-grained ablations, data sparsity curves, and case migration analyses.
- Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical formulations, rigorous empirical grounding, and well-structured architectural motivations.
- Value: ⭐⭐⭐⭐⭐ Offers a highly practical, low-overhead design pattern for upgrading temporal graph neural networks in dynamic relational reasoning tasks.