LAMPEAE: LLM-Augmented Manifold Probing for Adaptive Event Argument Extraction¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: NLP Understanding
Keywords: Event Argument Extraction, Low-Rank Manifold Projection, Hypernetwork, Riemannian Tangent Space, Neuro-Symbolic Reasoning
TL;DR¶
Addressing the geometric rigidity of static role anchors in prompt-based event argument extraction, LAMPEAE leverages a frozen LLM to distill high-order reasoning priors and dynamically warps static anchors along Riemannian manifold trajectories via a low-rank hypernetwork, achieving a new state-of-the-art Arg-C F1 of 54.5% on the RAMS benchmark.
Background & Motivation¶
Event Argument Extraction (EAE) aims to identify entity spans that fulfill predefined roles for specific event triggers within unstructured texts. Over recent years, prompt-based span selection paradigms such as PAIE and TabEAE have become prominent, offering an attractive trade-off between parallel extraction throughput and parameter efficiency compared to heavy question-answering or classification pipelines. Nevertheless, existing prompt-based architectures strictly adhere to the "Static Parameterization Hypothesis," where each argument role is initialized as a learnable static embedding matrix functioning as a fixed geometric centroid. In complex document-level contexts characterized by severe semantic drift, multi-sentence dependencies, and long-tail distributions, a single static anchor can only capture the "average" semantics of a role, inherently suffering from geometric rigidity.
The core tension is that event argument semantics in real-world contexts do not exist as isolated discrete points, but rather evolve along continuous semantic trajectories on an underlying manifold. While multi-prototype frameworks such as HMPEAE introduce multiple discrete cluster centers to capture intra-class variance, their discrete nature inevitably creates "semantic coverage vacuums"โuncovered intermediate spaces between finite prototypes that fail to accommodate subtle contextual nuances. Furthermore, empirical estimates reveal that while the ambient embedding space spans 768 dimensions, the average intrinsic dimensionality of event roles is merely 4.30. When models rely on unconstrained static projection weights, they struggle to disambiguate roles in dense contexts, frequently succumbing to frequentist bias or completely missing distant arguments.
This paper's angle of attack is to reformulate EAE as a dynamic manifold matching problem on a Riemannian manifold, using high-order commonsense and reasoning priors distilled from a frozen Large Language Model (LLM) to guide anchor adaptation. Core idea: distill reasoning priors via a non-autoregressive forward pass of a frozen LLM, and dynamically project static role anchors onto Riemannian tangent trajectories through a lightweight low-rank hypernetwork, seamlessly bridging the semantic coverage vacuums across continuous parameter space.
Method¶
Overall Architecture¶
LAMPEAE builds upon the PAIE span extraction backbone and establishes a Dynamic Manifold Probing mechanism to supersede static role anchors. The framework operates across three key stages: First, a Neuro-Symbolic Reasoning Engine extracts high-order implicit reasoning context from the input document and schema-aware instructions using a single forward pass through a frozen LLM, bypassing expensive autoregressive generation. Second, a Riemannian Manifold Alignment module utilizes a low-rank hypernetwork to compute tangent transformation matrices, moving the static anchor along a geodesic trajectory via first-order manifold retraction. Third, the resulting instance-specific dynamic probe interacts with document encodings to predict start and end boundary logits, regularized by manifold orthogonality and displacement constraints.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Document D + Trigger t + Event Type e"] --> B["Neuro-Symbolic Reasoning Engine<br/>Single forward pass extracts reasoning prior z"]
B --> C["Low-Rank Tangent Space Projection<br/>Hypernetwork predicts singular bases A(z) and B(z)"]
C --> D["Manifold Retraction & Trajectory Alignment<br/>Gated retraction produces dynamic probe p'k(z)"]
D --> E["Dynamic Span Extraction & Regularization<br/>Bipartite matching with joint loss optimization"]
Key Designs¶
1. Neuro-Symbolic Reasoning Engine: Distilling Non-Autoregressive High-Order Priors Directly deploying LLMs via autoregressive generation incurs intolerable computational latency and generation hallucinations, whereas small discriminative backbones lack the broad world knowledge needed to resolve distant dependencies. This design constructs a meta-cognitive reasoning instruction \(T_{reason}(t, e)\) that prompts the LLM to analyze the latent semantic properties and directional relations of argument roles. Given the concatenated input sequence \(X_{sym} = [\text{CLS}] \oplus D \oplus [\text{SEP}] \oplus T_{reason}(t, e)\), a frozen LLM produces last-layer hidden states in a single forward step. An attention-mask mean pooling operation aggregates these states into an implicit reasoning vector \(h_{pool}\), which is subsequently projected through a task-aware Semantic Bottleneck Layer to filter out general-purpose redundancy and align with the backbone dimensionality:
2. Low-Rank Tangent Space Projection: Subspace-Regularized Parameter Warping Learning an unconstrained parameter transformation matrix \(W(z) \in \mathbb{R}^{d \times d}\) is ill-posed and highly susceptible to severe overfitting and stochastic noise. Grounded in the low intrinsic dimensionality of role semantics and the Eckart-Young-Mirsky Theorem, effective semantic drift is strictly concentrated in a low-dimensional subspace of the tangent bundle \(T_{p_k}\mathcal{M}\). This design employs a lightweight hypernetwork to decompose the transformation kernel into two low-rank singular matrices \(A(z), B(z) \in \mathbb{R}^{d \times r}\) with bottleneck rank \(r = 32 \ll d = 768\). The contextual semantic drift is computed on the tangent space as \(\xi_{k,z} = p_k \cdot (A_k(z) B_k(z)^T)\), compelling parameter updates to flow exclusively along \(r\) principal directions and filtering out high-dimensional semantic noise.
3. Manifold Retraction and Trajectory Alignment: Topologically Stable Probe Generation To transport the tangent update back to the semantic manifold surface \(\mathcal{M}\) without causing representation divergence, the framework approximates the Riemannian exponential map via a first-order retraction. A learnable scalar gate \(\gamma\) dynamically scales the displacement magnitude according to the instance difficulty, and Layer Normalization is applied to ensure that the adapted probe remains topologically valid:
This residual connection guarantees that the static anchor functions as a reliable global prior, preventing semantic collapse while empowering the probe to smoothly traverse intermediate semantic regions in long-tail cases.
4. Dynamic Span Extraction and Manifold Regularization: Dual-Constraint Multi-Task Objective The instance-specific probe \(p'_k(z)\) computes token-level start and end boundary probabilities against BART document contextual representations \(H_D\), solved via bipartite matching with the Hungarian algorithm. To prevent dynamic probes from drifting too close to one another in dense embedding regions, the model introduces a Manifold Orthogonality Regularization loss \(\mathcal{L}_{orth}\) that penalizes squared cosine similarities between distinct role probes:
Additionally, a magnitude constraint \(\mathcal{L}_{norm} = \frac{1}{K} \sum_{k=1}^K \max(0, \|\Delta p_k\|_2 - \delta)\) restricts excessive displacement from the base anchor. The total objective combines span negative log-likelihood with geometric regularizers: \(\mathcal{J} = \mathcal{L}_{span} + \lambda_1 \mathcal{L}_{orth} + \lambda_2 \mathcal{L}_{norm}\).
A Worked Example¶
Consider a conflict-and-attack incident from RAMS: "The ISIL leader is indicated in the documents as the person behind the transportation of bombs from Syria to Turkey." Under static prompt baselines, because "Turkey" frequently co-occurs as a Target in violence-related training corpora, static anchors erroneously assign "Turkey" to the Target role and fail to trace directional movement. In LAMPEAE, the frozen LLM first distills the directional trajectory ("from Syria to Turkey") and flags the absence of an explicit victim in the sentence. Driven by reasoning vector \(z\), the hypernetwork warps the Target probe into an intermediate region where no document token satisfies the energy threshold, correctly yielding "No Answer". Concurrently, the Place probe is translated along its spatial manifold path, correctly associating "Turkey" with the destination Place role.
Loss & Training¶
The framework is trained end-to-end with the AdamW optimizer (weight decay 0.01) on an NVIDIA VGPU with 48GB memory. The base model (BART-large) is trained with learning rate \(\eta_{base} = 2 \times 10^{-5}\), while the hypernetwork uses \(\eta_{hyper} = 1 \times 10^{-4}\) under a linear decay schedule with 10% warmup. Optimal regularizer hyperparameters are determined via grid search to be \(\lambda_1 = 0.1\) and \(\lambda_2 = 0.05\).
Key Experimental Results¶
Main Results¶
Performance comparison across RAMS 1.0, WikiEvents, and ACE05 benchmarks on Argument Identification (Arg-I) and Argument Classification (Arg-C) F1 scores (%):
| Model | Backbone PLM | RAMS Arg-I | RAMS Arg-C | WikiEvents Arg-I | WikiEvents Arg-C | ACE05 Arg-I | ACE05 Arg-C |
|---|---|---|---|---|---|---|---|
| OneIE (2020) | BERT-large | - | - | - | - | 73.2 | 69.3 |
| EEQA (2020) | BART-large | 48.7 | 46.7 | 56.9 | 54.5 | 70.5 | 68.9 |
| BART-Gen (2021) | BART-large | 51.2 | 47.1 | 66.8 | 62.4 | 69.9 | 66.7 |
| TSAR (2022) | RoBERTa-large | 57.0 | 52.1 | 71.1 | 65.8 | - | - |
| PAIE (2022) | BART-large | 56.9 | 52.8 | 69.7 | 65.2 | 75.9 | 72.5 |
| TabEAE (2023) | RoBERTa-large | 57.2 | 53.2 | 69.8 | 63.7 | 75.9 | 73.4 |
| HMPEAE (2024) | RoBERTa-large | 58.6 | 53.7 | 72.1 | 66.6 | - | - |
| DEGAP (2025) | RoBERTa-large | 58.5 | 54.2 | 72.2 | 67.1 | 76.6 | 74.4 |
| LAMPEAE (Ours) | BART-large | 59.3 | 54.5 | 71.8 | 66.9 | 76.2 | 74.1 |
Ablation Study¶
Ablation analysis on the document-level RAMS benchmark isolating individual components:
| Configuration | Arg-I F1 (%) | Arg-C F1 (%) | Description |
|---|---|---|---|
| LAMPEAE (Full Model) | 59.3 | 54.5 | Complete low-rank manifold probing framework |
| w/o LLM Features | 56.2 | 53.1 | Replaced LLM embeddings with BERT representations |
| w/o Manifold Projection | 57.1 | 52.6 | Replaced low-rank manifold projection with cross-attention |
| w/o LLM Prompt | 57.9 | 53.4 | Used generic prompt instructions rather than meta-cognitive instructions |
| Baseline | 55.8 | 50.7 | Conventional static role prompt architecture |
Key Findings¶
- Superior Long-Distance Disambiguation: On RAMS, performance gains are heavily concentrated in long-range intervals (\(d \ge 15\)), where LAMPEAE outperforms PAIE by +3.2% F1. The gate activation \(\gamma\) increases monotonically from 0.03 in short-range contexts to 0.22 in long-range contexts, directly demonstrating that dynamic manifold probing is autonomously invoked when local contextual cues are scarce.
- Low-Rank Constraint Prevents Noise Injection: Varying the projection rank \(r\) reveals a distinct performance peak at \(r = 32\). Expanding to higher ranks (\(r = 64\)) triggers overfitting and introduces stochastic LLM noise, empirically verifying that task-relevant semantic shifts are confined to a compact subspace.
- High Data Efficiency: Trained on only 20% of the training data, LAMPEAE matches or exceeds the full-data performance of EEQA on WikiEvents, confirming that structural manifold regularizers substantially constrain the hypothesis search space.
Highlights & Insights¶
- Geometric Manifold Formulation of EAE: Challenging the long-standing static parameterization dogma, the paper mathematically formulates instance adaptation as a geodesic displacement on a Riemannian manifold, offering a principled solution to semantic drift.
- Non-Autoregressive Neuro-Symbolic Integration: By extracting pooled hidden states from a frozen 7B model in a single forward pass, the model gains rich reasoning priors with only 2M additional parameters and modest latency overhead (+32ms).
- Cross-Task Generalizability: Beyond event extraction, the dynamic manifold probe delivers robust gains across Named Entity Recognition (CoNLL-2003: 92.5% F1) and Relation Extraction (TACRED: 71.8% F1), validating manifold alignment as a foundational technique for structured information extraction.
Limitations & Future Work¶
- Reliance on 7B LLM Inference: Even without token-by-token autoregression, feeding long documents into a 7B LLM imposes non-negligible GPU memory requirements and limits deployment on resource-constrained edge hardware; future work should explore distilling manifold trajectories into compact models.
- Scalability to Open-World Ontologies: As the role schema scales to hundreds or thousands of open-world relation types, computing hypernetwork transformations per role may scale up parameter overhead; investigating compositional manifold sharing remains an important next step.
Related Work & Insights¶
- vs PAIE & TabEAE: These models pioneered efficient prompt-based parallel extraction, but enforce static centroid vectors; LAMPEAE preserves their non-autoregressive efficiency while introducing dynamic context-conditioned probe displacement.
- vs HMPEAE: HMPEAE uses hyperspherical multi-prototypes to address intra-class variance, but remains bounded by discrete clustering centroids that leave semantic coverage vacuums; LAMPEAE provides continuous manifold traversal.
- vs GEMS & DEGAP: These recent baselines use multi-perspective generation or dual prefix gating; LAMPEAE achieves superior long-document results through lower-level geometric regularizers that prevent representation collapse without procedural overhead.
Rating¶
- Novelty: โญโญโญโญโญ First framework to formalize event argument probing through Riemannian tangent spaces and low-rank hypernetworks.
- Experimental Thoroughness: โญโญโญโญโญ Comprehensive coverage across three benchmarks, intrinsic dimensionality measurement, distance-stratified analysis, and cross-task verification.
- Writing Quality: โญโญโญโญโญ Rigorous mathematical modeling, clean flowcharts, and clear conceptual alignment between theoretical motivation and empirical results.
- Value: โญโญโญโญโญ Provides an extensible, lightweight bridge uniting frozen LLM capabilities with efficient discriminative information extraction.