Skip to content

Ontology-Driven Structural Regularization for Document-Level Relation Extraction

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/mntlra/DocRE-StructuralConsistency
Area: NLP Understanding
Keywords: Document-Level Relation Extraction, Distant Supervision, Ontology Constraints, Structural Regularization, Knowledge Graph Construction

TL;DR

Targeting severe structural logical contradictions in distantly supervised datasets for document-level relation extraction, this paper proposes an ontology-driven structural regularization framework grounded in Web Ontology Language (OWL) axioms that filters ill-formed entities and completes missing inverse relations prior to training, significantly suppressing prediction contradictions and substantially boosting extraction F1 without architectural modifications.

Background & Motivation

Document-level relation extraction (DocRE) seeks to identify all semantic relations expressed between entity pairs across an entire document, providing an indispensable foundation for turning unstructured text into structured, machine-readable knowledge graphs (KGs). Because capturing complex inter-sentence interactions and multi-hop reasoning across long narratives requires extensive human curation, manually annotated benchmark datasets—such as the human-revised splits of DocRED—remain severely limited in scale. To train robust relation extractors, practitioners rely heavily on distant supervision (DS) to construct massive training sets comprising tens of thousands of documents by heuristically aligning Wikidata knowledge base triples with Wikipedia text. However, entities co-occurring within a document frequently lack genuine semantic relationships, introducing substantial supervisory label noise into distant datasets. Existing denoising techniques operate primarily through a model-centric lens, employing Monte Carlo dropout to quantify pseudo-label uncertainty, filtering low-confidence predictions, or utilizing active learning to bolster long-tail relations, all within the isolated probability space of individual triple classification.

Despite their utility, these model-centric paradigms overlook a foundational vulnerability: relational triples are not isolated categorical labels, but rather constituents of an interconnected semantic topology governed by rigid real-world ontological axioms. An in-depth audit of the reference DocRED distant supervision corpus reveals rampant structural invalidities—ranging from non-temporal entities being assigned to death-date relations, to capital-city relations asserted in one direction while omitting the reciprocal inverse, to asymmetric properties asserted simultaneously in opposing directions. Deep neural models possess high representational capacity and readily memorize these structural contradictions, subsequently propagating and amplifying them into downstream predictions, producing graph extractions that are internally incoherent. Existing semantic denoising frameworks lack the formal structural awareness necessary to diagnose or penalize such topological invalidities.

This work addresses this deficiency through a principled data-centric perspective, establishing structural consistency under ontological constraints as a missing yet critical axis of supervision in DocRE. Grounded in the formal semantics of the Web Ontology Language (OWL), the proposed framework defines explicit domain, range, inverse, and cardinality axioms over entity types and relations to audit and rectify supervision signals. Core idea: formulate document-level relation extraction as the construction of an RDF graph compliant with OWL algebraic axioms, diagnose structural pathologies via domain-range, inverse, asymmetric, and cardinality rules, and apply a lightweight, model-agnostic pre-processing pipeline that prunes ill-formed entities and synthesizes missing inverse relations to enforce structural well-formedness before model training.

Method

Overall Architecture

The framework establishes an ontology-driven structural consistency analysis and data-centric regularization pipeline for DocRE corpora. Given an input document \(d\) and an associated set of named entities \(E=\{e_i\}_{i=1}^n\), where each entity comprises mention spans \(M_i=\{m_{i,j}\}_{j=1}^{k_i}\), the DocRE objective is to predict semantic relations from \(R \cup \{\text{na}\}\) for all distinct entity pairs \((e_s, e_o)\). The pipeline maps each predicted relation to an RDF triple \(\langle s, p, o \rangle\) and interprets each relation \(r \in R\) as an owl:ObjectProperty subject to formal semantic axioms. The pipeline integrates three primary phases: formalizing four classes of OWL ontological constraints, conducting statistically grounded error attribution via Bayesian estimation, and executing model-agnostic dataset sanitization and completion to train standard DocRE architectures.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Raw Document and Entity Mentions<br/>DocRED Distant Supervision Dataset"] --> B["1. Formalizing OWL-Driven Ontological Constraints<br/>Domain/range, inverseOf, asymmetry, and bounded cardinality"]
    B --> C["2. Statistically Grounded Root-Cause Attribution<br/>Binomial sampling with 95% HPD credible intervals"]
    C --> D["3. Model-Agnostic Structural Regularization Pipeline<br/>Entity-level cascade pruning and inverse relation closure"]
    D --> E["4. Training Downstream DocRE Models on Well-Formed Data<br/>Evaluate ATLOP and DREEAM architectures"]
    E --> F["Coherent, Structurally Consistent Document Relation Graph"]

Key Designs

1. Formalizing OWL-Driven Ontological Constraints: Imposing Algebraic KG Axioms on Extracted Triples

To operationalize structural well-formedness, the authors formulate four explicit categories of ontological rules grounded in OWL formal semantics: - Invalid Triples: Each entity \(e_i\) is mapped to a discrete entity type \(t(e_i) \in \mathcal{T}\) (comprising PER, ORG, LOC, TIME, NUM, and MISC). Each relation \(r \in R\) possesses permitted subject types \(\text{dom}(r) \subseteq \mathcal{T}\) and object types \(\text{ran}(r) \subseteq \mathcal{T}\), extracted from Wikidata property constraints and smoothed over empirical ReDocRED frequencies (including MISC universally to prevent over-restriction). A triple \(\langle e_s, r, e_o \rangle\) is defined as invalid if: $\(t(e_s) \notin \text{dom}(r) \lor t(e_o) \notin \text{ran}(r)\)$ For example, asserting \(\langle \text{Emily (PER)}, \text{P19 (place of death)}, \text{1987 (TIME)} \rangle\) violates \(\text{TIME} \notin \text{ran}(\text{P19})\). - Missing Inverse Relations: Under a local closed-world assumption over each document-level graph, if relation \(r\) is ontologically paired with \(r' \equiv \text{owl:inverseOf}(r)\), the presence of \(\langle e_s, r, e_o \rangle \in d\) entails the presence of \(\langle e_o, r', e_s \rangle \in d\). A pair is flagged as missing an inverse when the forward triple exists while the inverse triple is absent (for reflexive symmetric properties such as spouse or sibling, \(r = r'\)). - Asymmetric Relation Inconsistencies: Except for spouse (P26) and sibling (P3373) which are symmetric (owl:SymmetricProperty), all remaining relations in DocRED are asymmetric (owl:AsymmetricProperty). An asymmetric violation occurs if \(\langle e_s, r, e_o \rangle \in d \land \langle e_o, r, e_s \rangle \in d\) holds simultaneously for \(s \neq o\). - Relation Cardinality Violations: While functional properties (owl:FunctionalProperty) mathematically constrain maximum cardinality to \(k_r = 1\), empirical scrutiny reveals that ground truth texts often contain both full dates (e.g., "2 July 1931") and standalone years (e.g., "1931") as distinct entity mentions. Consequently, the threshold is relaxed to \(k_r = 2\) for 11 functional relations (e.g., date of death, inception). A violation is triggered when \(|O(e_s, r)| > k_r\).

2. Statistically Grounded Root-Cause Attribution: Uncovering the Entity Annotation Bottleneck

To rigorously explain the origin of invalid triples without undertaking the prohibitive manual verification of all 46,211 invalid triples in DocRED distant, the authors introduce a binomial proportion estimation protocol using Simple Random Sampling (SRS) with sample size \(n_S = 400\). Rather than relying on standard frequentist intervals, the analysis computes a \(1-\alpha\) Bayesian Highest Posterior Density (HPD) credible interval (\(\alpha = 0.05\)), which operates directly in probabilistic parameter space and avoids interpretational pitfalls in one-shot knowledge graph evaluation.

Each sampled triple is manually categorized into either an entity annotation error (the entity mention itself is mistyped, inducing an invalid relation) or a relation annotation error (the entity types are correct, but the relation was misaligned by distant heuristics). The evaluation reveals that \(69\% \pm 4.5\%\) of invalid triples originate from named entity typing errors, while only \(31\%\) stem from erroneous relation assignment. This empirical discovery confirms that named entity recognition represents a fundamental bottleneck for relation extraction: upstream typing errors propagate inexorably downstream, placing an upper bound on structural graph quality.

3. Model-Agnostic Structural Regularization Pipeline: Entity-Level Pruning and Inverse Closure Synthesis

Rather than undertaking manual revision of 58,619 problematic instances (46,211 invalid triples and 12,408 asymmetric conflicts), the authors institute a scalable, conservative pruning and augmentation strategy. For every invalid triple or asymmetric inconsistency, the pipeline discards the entity involved in the violation from the document graph, thereby automatically eliminating all associated valid and invalid triples anchored to that corrupted entity. This eliminates structural violations without requiring costly entity re-annotation, removing 208,665 triples due to typing invalidities, 31,178 triples due to asymmetric conflicts, and discarding 3 completely corrupted documents.

For missing inverse relations, the pipeline leverages the Wikidata inverse mapping table to deterministically generate and inject reciprocal triples \(\langle e_o, r', e_s \rangle\) into the document, contributing 72,609 structurally valid training instances. The resulting sanitized dataset contains 1,338,404 training triples (a net reduction of 167,234 triples compared to raw DocRED distant) across 101,870 documents. Because this procedure operates strictly as an offline data pre-processing step, it requires zero modifications to neural architectures, objective functions, or inference routines.

Loss & Training

The framework is completely model-agnostic. Downstream models retain their native objective formulations, hyper-parameters, and optimization schedules. The empirical evaluation examines two state-of-the-art Transformer-based architectures: - ATLOP: Employs BERT or RoBERTa encoders equipped with Localized Context Pooling and Adaptive Thresholding to dynamically resolve multi-label class imbalance across document-level entity pairs. - DREEAM: Extends ATLOP with an evidence-aware attention distillation framework under a teacher-student training paradigm, wherein the student network undergoes self-training over distantly supervised data guided by weak teacher signals before fine-tuning on manual splits.

To rigorously disprove the hypothesis that performance improvements stem merely from dataset subsampling (i.e., regularization from training on 167,234 fewer triples), the authors implement a random subsampling baseline. Specifically, 167,234 triples are randomly removed from DocRED distant across 10 independent random seeds, and the mean performance across runs is evaluated.

Key Experimental Results

Main Results

Evaluation on the ReDocRED test set confirms the efficacy of ontology-driven structural regularization. The table below compares ATLOP and DREEAM (backed by BERT and RoBERTa) trained on raw distant data (DocRED DS), the structurally sanitized version (Corrected), and the 10-seed random subsampling baseline (Random). In addition to standard Precision, Recall, and F1, the evaluation reports ignored metrics (IgnPrec, IgnF1) that exclude entity pairs encountered during training.

Architecture Training Data Setting Precision (%) IgnPrec (%) Recall (%) F1 (%) IgnF1 (%)
ATLOP (BERT) DocRED DS 75.03 58.77 29.45 42.30 39.24
Corrected (Ours) 76.65 (+2.16%) 61.65 (+4.90%) 30.70 (+4.24%) 43.84 (+3.64%) 40.99 (+4.46%)
Random Subsample 75.28 (+0.33%) 59.73 (+1.63%) 29.64 (+0.65%) 42.53 (+0.43%) 39.62 (+0.97%)
ATLOP (RoBERTa) DocRED DS 75.71 60.06 30.60 43.58 40.54
Corrected (Ours) 76.57 (+1.14%) 61.72 (+2.76%) 31.20 (+1.96%) 44.34 (+1.74%) 42.45 (+4.71%)
Random Subsample 75.78 (+0.09%) 60.82 (+1.27%) 30.49 (-0.36%) 43.48 (-0.23%) 40.61 (+0.17%)
DREEAM (BERT) DocRED DS 80.87 67.91 26.09 39.45 37.70
Corrected (Ours) 80.42 (-0.56%) 69.41 (+2.21%) 31.13 (+19.32%) 44.88 (+13.76%) 42.98 (+14.01%)
Random Subsample 79.74 (-1.40%) 67.46 (-0.66%) 26.83 (+2.84%) 40.13 (+1.72%) 38.37 (+1.78%)
DREEAM (RoBERTa) DocRED DS 81.70 69.64 28.61 42.38 40.56
Corrected (Ours) 81.43 (-0.33%) 70.83 (+1.71%) 31.13 (+8.81%) 45.04 (+6.28%) 43.53 (+7.32%)
Random Subsample 80.87 (-1.02%) 69.74 (+0.14%) 28.97 (+1.26%) 42.63 (+0.59%) 40.90 (+0.84%)

Ablation Study

To isolate the individual contribution of each structural constraint, the authors evaluate models trained on distant variants where only a single constraint was enforced, compared against the unified pipeline on the ReDocRED development set:

Model Backbone Sanitization Configuration Precision (%) IgnPrec (%) Recall (%) F1 (%) IgnF1 (%)
ATLOP-BERT DocRED DS Baseline 76.57 60.69 29.83 42.93 40.00
Filter Invalid Triples only (w/o invalid) 76.84 (+0.35%) 61.96 (+2.08%) 30.09 (+0.87%) 43.24 (+0.73%) 40.50 (+1.27%)
Inject Missing Inverses only (w/o inverse) 76.05 (-0.68%) 59.80 (-1.46%) 30.88 (+3.55%) 43.93 (+2.33%) 40.73 (+1.84%)
Remove Asymmetric Violations only (w/o asymmetric) 76.60 (+0.03%) 60.94 (+0.41%) 30.09 (+0.89%) 43.21 (+0.65%) 40.29 (+0.73%)
All Rules Combined 77.04 (+0.60%) 61.97 (+2.11%) 31.03 (+4.05%) 44.24 (+3.06%) 41.36 (+3.41%)
DREEAM-BERT DocRED DS Baseline 81.16 76.67 26.13 39.53 38.97
Filter Invalid Triples only (w/o invalid) 79.70 (-1.80%) 68.47 (-10.69%) 27.47 (+5.14%) 40.86 (+3.36%) 39.21 (+0.61%)
Inject Missing Inverses only (w/o inverse) 81.09 (-0.09%) 68.39 (-10.80%) 27.72 (+6.09%) 41.32 (+4.52%) 39.45 (+1.22%)
Remove Asymmetric Violations only (w/o asymmetric) 81.48 (+0.39%) 68.88 (-10.16%) 26.35 (+0.86%) 39.83 (+0.75%) 38.12 (-2.19%)
All Rules Combined 79.92 (-1.54%) 69.09 (-9.89%) 30.18 (+15.52%) 43.82 (+10.85%) 42.01 (+7.80%)

Crucially, structural evaluation of test set predictions reveals dramatic reductions in model logical incoherence: - Invalid triples in predictions drop by an average of 62.92% (e.g., DREEAM-RoBERTa invalid predictions decrease from 52 to 23, reducing the error rate from 0.85% to 0.34%); - Missing inverse relations plummet by 74% to 87% (ATLOP-RoBERTa missing inverse rate drops from 41.18% to 5.68%); - Asymmetric inconsistencies are essentially eliminated (ATLOP-RoBERTa achieves a 0.00% asymmetric contradiction rate in predictions).

Key Findings

  • Inverse relation completion acts as the primary engine for recall and F1 gains: Injecting reciprocal triples rectifies severe false-negative bias in distant supervision, boosting DREEAM-BERT test recall by +19.32% and driving a net F1 gain of +5.43 points (+13.76% relative improvement).
  • Invalid triple pruning drives precision and open-set generalization: Eliminating domain/range mismatches prevents encoders from hallucinating illegal relation types. Under the ignored setting (excluding training entity pairs), ATLOP exhibits substantial gains (+4.46% to +4.71% in IgnF1), proving that structural sanitization shields models against spurious entity-pair memorization.
  • Random pruning fails to replicate gains: The 10-seed random subsampling baseline produces negligible shifts and even degrades performance in several configurations (e.g., ATLOP-RoBERTa F1 declines by -0.23%), definitively refuting the hypothesis that smaller data volume explains the observed benefits.

Highlights & Insights

  • Establishing ontological well-formedness as a data-centric frontier: Moves beyond conventional loss re-weighting or uncertainty filtering by proving that formal OWL semantic axioms act as cost-effective, inductive structural regularizers for distant supervision.
  • Pinpointing the upstream entity recognition bottleneck: Leverages rigorous Bayesian HPD interval analysis to uncover that 69% of structural extraction errors originate from entity misclassification, formally establishing that relation extraction quality is inherently bounded by upstream entity typing accuracy.
  • Zero-overhead, plug-and-play practicality: Because sanitization occurs strictly as an offline dataset preparation phase, it is universally compatible with any DocRED-formatted model without introducing training overhead or inference latency.

Limitations & Future Work

  • Coarse entity-level pruning discards benign triples: To bypass the prohibitive manual cost of correcting 58k violations, the current pipeline discards the entire entity involved in an invalidity, inadvertently deleting legitimate triples associated with that entity; future iterations could leverage LLMs to perform targeted entity re-typing.
  • Residual structural noise in evaluation benchmarks: Even the curated ReDocRED test set contains ~0.8% invalid triples and ~9% missing inverses, meaning that structurally coherent model predictions may be unfairly penalized by flawed ground-truth annotations.
  • Evaluation restricted to discriminative encoders: The empirical analysis focuses on discriminative models (ATLOP/DREEAM); extending structural regularization principles to generative LLM-based relation extraction represents a promising future avenue.
  • vs UGDRE (Sun et al., 2023): UGDRE uses Monte Carlo dropout to gauge semantic uncertainty; however, audit results in Table 4 reveal that UGDRE-cleaned data still harbors 3.01% invalid triples and 53.50% missing inverses, failing to resolve structural incoherence. The proposed framework provides an orthogonal, structural layer of purification.
  • vs DOREMI (Menotti et al., 2026): DOREMI employs active learning for long-tail relation discovery but leaves substantial cardinality and inverse deficiencies unaddressed; combining DOREMI with ontological pre-filtering could yield mutually reinforcing benefits.
  • vs Sentence-Level Constraint RE (Liu et al., 2014; Liang et al., 2023): Earlier constraint-based approaches inject type priors into neural architectures via attention biases or constraint graphs at the sentence level; this work scales formal OWL reasoning to document-level graphs while maintaining complete model agnosticism.

Rating

  • Novelty: ⭐⭐⭐⭐☆ (Pioneering perspective establishing formal OWL ontology axioms to diagnose and eliminate structural noise in DocRE)
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Encompasses multiple splits, two leading model architectures, Bayesian HPD error attribution, and a 10-seed control baseline)
  • Writing Quality: ⭐⭐⭐⭐⭐ (Exceptionally well-structured, rigorous mathematical grounding, and lucid presentation of empirical findings)
  • Value: ⭐⭐⭐⭐⭐ (Delivers an open-source, model-agnostic, zero-cost data-centric pipeline with immediate utility for large-scale KG construction)