Skip to content

Ontology-Driven Structural Regularization for Document-Level Relation Extraction

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/mntlra/DocRE-StructuralConsistency
Area: Graph Learning
Keywords: Document-Level Relation Extraction, Ontology Regularization, Distant Supervision, Knowledge Graph Construction, Structural Consistency

TL;DR

Addressing pervasive and previously overlooked structural noise in distantly supervised Document-Level Relation Extraction (DocRE)—such as ontology constraint violations, missing inverse relations, and asymmetric contradictions—this paper formalizes structural well-formedness under OWL axioms and demonstrates that lightweight, model-agnostic data-centric structural regularization drastically eliminates logical contradictions in model predictions while consistently improving generalization performance.

Background & Motivation

Document-Level Relation Extraction (DocRE) aims to identify all semantic relationships holding between entity pairs across a multi-sentence document, serving as a foundational technology for automatically constructing Knowledge Graphs (KGs) that empower downstream information retrieval, question answering, and reasoning systems. Because exhaustive manual annotation across entire documents is prohibitively labor-intensive, the research community heavily relies on Distant Supervision (DS)—heuristically aligning document entity co-occurrences with structured triples from reference KGs (such as Wikidata)—to assemble massive corpora like DocRED distant (comprising over 100,000 documents). However, distant supervision hinges on the strong assumption that whenever two entities co-occur within a text, every document expresses that known KG relation; across multi-sentence paragraphs, this assumption induces severe label noise, resulting in rampant false positive supervision signals that prevent state-of-the-art models from leveraging distant data effectively.

To mitigate this supervision noise, existing denoising techniques follow a model-centric paradigm, deploying techniques such as Monte Carlo dropout to estimate pseudo-label uncertainty or active learning to reweight long-tail relations. Nevertheless, these approaches strictly evaluate isolated relational triples along the dimension of semantic correctness or prediction confidence, remaining fundamentally blind to the structural consistency of the extracted document graph. In reality, relations are not independent discrete categories; they are predicates subject to formal ontological constraints and domain/range specifications. When distantly supervised corpora harbor invalid entity types, unannotated inverse facts, or asymmetric bidirectional contradictions, models are exposed to structurally corrupted gradients during training, inevitably reproducing illogical and conflicting triples during inference.

This disconnect between isolated label classification and global knowledge graph structure represents a critical bottleneck in scaling distant supervision for DocRE. The core idea of this paper is to frame document-level relation extraction as the construction of a local knowledge graph governed by Web Ontology Language (OWL) axioms, formally quantifying four categories of structural violations (invalid triples, missing inverse relations, asymmetric contradictions, and cardinality conflicts) and demonstrating that a lightweight, model-agnostic preprocessing pipeline—which prunes conflicting entity nodes and automatically enforces inverse closures—acts as an effective structural regularizer that suppresses prediction contradictions and substantially boosts downstream generalization.

Method

Overall Architecture

The proposed methodology comprises two sequential phases: formal structural consistency diagnosis using OWL semantics, followed by a model-agnostic data cleaning and augmentation procedure. First, each document-level entity pair \((e_s, e_o)\) and relation \(r \in \mathcal{R}\) is formalized as an RDF triple \(\langle s, p, o \rangle\). Drawing on Wikidata property specifications, four OWL structural axioms are defined: property domain/range constraints, inverse property entailments, asymmetric property exclusions, and qualified cardinality upper bounds. After auditing the corpus and uncovering that named entity recognition errors dominate invalid triples, the framework applies a data-centric structural regularization step: entities involved in type violations or asymmetric contradictions are conservatively removed alongside their associated triples, while missing inverse and symmetric relations are deterministically added. The resulting structurally consistent dataset is directly fed into standard DocRE architectures (such as ATLOP and DREEAM) without altering their internal layers or loss formulations.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Distantly Supervised Dataset DocRED DS<br/>Wikipedia text, named entities & heuristically aligned triples"] --> B["1. Formalization of Four OWL Structural Axioms<br/>Domain/range, inverse relations, asymmetric exclusion, cardinality"]
    B --> C["2. Structural Noise Audit & Root-Cause Attribution<br/>Binomial proportion sampling with 95% HPD credible intervals"]
    C --> D["3. Model-Agnostic Structural Regularization Preprocessing<br/>Entity-level violation pruning + automatic inverse closure completion"]
    D --> E["Output: Structurally Consistent Training Corpus<br/>Seamlessly trains downstream DocRE models (ATLOP / DREEAM)"]

Key Designs

1. Formalization of Four OWL Structural Axioms: Mapping Relational Classification to Ontology Constraints

Standard DocRE models formulate extraction as multi-label binary cross-entropy across all entity pairs, which cannot penalize structurally incoherent joint predictions. To establish formal criteria for dataset well-formedness, the authors map each relation \(r \in \mathcal{R}\) to an owl:ObjectProperty and establish four distinct structural rules: - Invalid Triples: Based on Wikidata metadata fields (subject type constraint and value type constraint) combined with empirical distributions in ReDocRED, permissible domain \(\text{dom}(r)\) and range \(\text{ran}(r)\) sets are defined over entity types \(\mathcal{T} = \{\text{PER}, \text{ORG}, \text{LOC}, \text{TIME}, \text{NUM}, \text{MISC}\}\). A triple \(\langle e_s, r, e_o \rangle\) is marked structurally invalid if the representative type of the subject or object violates the ontology: \(t(e_s) \notin \text{dom}(r) \lor t(e_o) \notin \text{ran}(r)\). For instance, \(\langle\text{Emily(PER)}, \text{P19(place of death)}, \text{1987(TIME)}\rangle\) is invalid because \(\text{TIME} \notin \text{ran}(\text{P19})\). - Missing Inverse Relations: Adopting a local closed-world assumption within each document graph, if an ontology declares \(r' \equiv \text{owl:inverseOf } r\) (such as capital P36 and capital of P1376), the presence of \(\langle e_s, r, e_o \rangle\) logically entails \(\langle e_o, r', e_s \rangle\). Failure to observe the inverse triple indicates an incomplete supervision label. For symmetric properties (\(r = r'\), such as spouse P26 and sibling P3373), the relation must hold symmetrically in both directions. - Asymmetric Relation Inconsistencies: Except for P26 and P3373, all other relations in DocRED are treated as owl:AsymmetricProperty. If a document simultaneously contains both \(\langle e_s, r, e_o \rangle\) and \(\langle e_o, r, e_s \rangle\) for an asymmetric predicate (with \(s \neq o\)), the pair represents an irreconcilable logical contradiction. - Relations Cardinality Violations: Certain functional relations (e.g., date of death, date of birth) conceptually have an upper bound of \(k_r = 1\). However, empirical observation reveals that ground-truth annotations often retain both full dates (e.g., "2 July 1931") and isolated years ("1931") as distinct entities. To avoid over-constraining, maximum cardinality is relaxed to \(k_r = 2\) for 11 functional relations, flagging a violation only when the number of distinct objects exceeds the bound: \(|O(e_s, r)| > k_r\).

2. Structural Noise Audit & Root-Cause Attribution: Quantifying the NER Bottleneck

An exhaustive audit of DocRED DS exposes severe structural degradation: 46,211 invalid triples (3.07%), 85,065 missing inverse relations (40.26%), and 12,408 asymmetric contradictions (0.84%), far exceeding the error rates of manually curated ReDocRED splits. Because manually inspecting over 46,000 invalid triples is intractable, the authors implement an unbiased binomial proportion estimation protocol: drawing a Simple Random Sample of \(n_S = 400\) invalid triples, manually annotating whether each error is attributable to relation misannotation versus entity misannotation, and evaluating the population proportion via a Bayesian Highest Posterior Density (HPD) credible interval at \(\alpha = 0.05\):

\[\hat{\mu} = \frac{1}{n_S} \sum_{\langle e_s, r, e_o \rangle \in S} \mathbf{1}(\langle e_s, r, e_o \rangle)\]

The statistical investigation reveals that 69% (\(\pm 4.5\%\)) of invalid triples stem from named entity misannotations (incorrect entity type assignment), while only 31% originate from relation classifier misassignment. For example, in \(\langle\text{France(LOC)}, \text{P16(capital)}, \text{Paris(PER)}\rangle\), "Paris" is mistyped as a person rather than a location. This establishes that entity extraction acts as a fundamental quality ceiling for relation extraction: errors in entity typing propagate directly into downstream relational supervision.

3. Model-Agnostic Structural Regularization Preprocessing: Efficient Pruning and Closure Completion

To rectify structural noise across more than 58,000 conflicting instances without prohibitive manual re-annotation, the authors engineer a scalable and conservative data-cleaning pipeline: - Pruning Invalid and Asymmetric Triples: For every triple violating domain/range constraints or asymmetric exclusion, the offending entity is completely removed from the document's entity set. Removing the entity automatically purges all triples dependent on that entity within the document, eliminating 208,665 invalid triples and 31,178 asymmetric conflict triples while discarding only 3 completely emptied documents. - Automatic Inverse Closure Completion: For valid triples associated with an inverse or symmetric relation, the corresponding reciprocal triples are deterministically added to the training set, injecting 72,609 verified inverse triples. - The resulting structurally consistent distant dataset contains 1,338,404 instances (a net reduction of 167,234 triples across 101,870 documents). Crucially, because this intervention operates purely at the data-preprocessing level, it is completely decoupled from model architectures, loss functions, and inference procedures, guaranteeing plug-and-play compatibility with any DocRE baseline.

Key Experimental Results

Main Results

To evaluate whether data-level structural regularization improves downstream generalization and reduces ill-formed predictions, the authors train two leading DocRE models—ATLOP (localized context pooling with adaptive thresholding) and DREEAM (evidence-guided teacher-student distillation)—using BERT-base and RoBERTa-large backbones. Models are trained on the original DocRED distant dataset (DocRED DS) versus the structurally regularized dataset (Corrected), and evaluated on the ReDocRED test set. To prove that performance gains do not merely stem from dataset reduction, a control baseline is evaluated by randomly pruning the identical number of triples (167,234) across 10 different random seeds (Random).

Model Architecture Encoder Backbone Training Dataset Precision IgnPrec Recall F1 IgnF1
ATLOP BERT DocRED DS 0.7503 0.5877 0.2945 0.4230 0.3924
ATLOP BERT Corrected (Ours) 0.7665 (+2.16%) 0.6165 (+4.90%) 0.3070 (+4.24%) 0.4384 (+3.64%) 0.4099 (+4.46%)
ATLOP BERT Random Removal 0.7528 (+0.33%) 0.5973 (+1.63%) 0.2964 (+0.65%) 0.4253 (+0.43%) 0.3962 (+0.97%)
ATLOP RoBERTa DocRED DS 0.7571 0.6006 0.3060 0.4358 0.4054
ATLOP RoBERTa Corrected (Ours) 0.7657 (+1.14%) 0.6172 (+2.76%) 0.3120 (+1.96%) 0.4434 (+1.74%) 0.4245 (+4.71%)
ATLOP RoBERTa Random Removal 0.7578 (+0.09%) 0.6082 (+1.27%) 0.3049 (-0.36%) 0.4348 (-0.23%) 0.4061 (+0.17%)
DREEAM BERT DocRED DS 0.8087 0.6791 0.2609 0.3945 0.3770
DREEAM BERT Corrected (Ours) 0.8042 (-0.56%) 0.6941 (+2.21%) 0.3113 (+19.32%) 0.4488 (+13.76%) 0.4298 (+14.01%)
DREEAM BERT Random Removal 0.7974 (-1.40%) 0.6746 (-0.66%) 0.2683 (+2.84%) 0.4013 (+1.72%) 0.3837 (+1.78%)
DREEAM RoBERTa DocRED DS 0.8170 0.6964 0.2861 0.4238 0.4056
DREEAM RoBERTa Corrected (Ours) 0.8143 (-0.33%) 0.7083 (+1.71%) 0.3113 (+8.81%) 0.4504 (+6.28%) 0.4353 (+7.32%)
DREEAM RoBERTa Random Removal 0.8087 (-1.02%) 0.6974 (+0.14%) 0.2897 (+1.26%) 0.4263 (+0.59%) 0.4090 (+0.84%)

Note: IgnPrec and IgnF1 exclude entity pairs present during training, directly measuring genuine out-of-sample generalization.

Ablation Study

To isolate the individual contribution of each structural rule, ablation experiments on the ReDocRED development set evaluate the impact of correcting single constraint categories in isolation:

Model Architecture Backbone Cleaning Configuration Precision IgnPrec Recall F1 IgnF1
ATLOP BERT Original DocRED DS 0.7657 0.6069 0.2983 0.4293 0.4000
ATLOP BERT w/o invalid triples only 0.7684 (+0.35%) 0.6196 (+2.08%) 0.3009 (+0.87%) 0.4324 (+0.73%) 0.4050 (+1.27%)
ATLOP BERT w/o missing inverse only 0.7605 (-0.68%) 0.5980 (-1.46%) 0.3088 (+3.55%) 0.4393 (+2.33%) 0.4073 (+1.84%)
ATLOP BERT w/o asymmetric errors only 0.7660 (+0.03%) 0.6094 (+0.41%) 0.3009 (+0.89%) 0.4321 (+0.65%) 0.4029 (+0.73%)
ATLOP BERT All rules combined 0.7704 (+0.60%) 0.6197 (+2.11%) 0.3103 (+4.05%) 0.4424 (+3.06%) 0.4136 (+3.41%)
DREEAM BERT Original DocRED DS 0.8116 0.7667 0.2613 0.3953 0.3897
DREEAM BERT w/o invalid triples only 0.7970 (-1.80%) 0.6847 (-10.69%) 0.2747 (+5.14%) 0.4086 (+3.36%) 0.3921 (+0.61%)
DREEAM BERT w/o missing inverse only 0.8109 (-0.09%) 0.6839 (-10.80%) 0.2772 (+6.09%) 0.4132 (+4.52%) 0.3945 (+1.22%)
DREEAM BERT w/o asymmetric errors only 0.8148 (+0.39%) 0.6888 (-10.16%) 0.2635 (+0.86%) 0.3983 (+0.75%) 0.3812 (-2.19%)
DREEAM BERT All rules combined 0.7992 (-1.54%) 0.6909 (-9.89%) 0.3018 (+15.52%) 0.4382 (+10.85%) 0.4201 (+7.80%)

Furthermore, an audit of the structural well-formedness of test-set predictions reveals dramatic improvements: - Invalid Triples Reduced: The proportion of invalid triples in model predictions decreases by an average of -62.92% (e.g., dropping from 1.05% to 0.59% for ATLOP-RoBERTa, and from 0.85% to 0.34% for DREEAM-RoBERTa). - Missing Inverses Eradicated: The missing inverse rate plummets from 41%~45% down to 5%~10% (achieving an -87.44% relative decrease for ATLOP-RoBERTa). - Asymmetric Contradictions Eliminated: For ATLOP-RoBERTa, asymmetric violations in model predictions drop from 0.60% to 0.00%.

Key Findings

  • Structural Noise Directly Propagates to Inference: Missing inverse triples in training data (40.6%) propagate almost linearly into model predictions (41.2%~45.2%), and asymmetric errors (0.84%) replicate at 0.53%~0.79%, proving that deep DocRE models faithfully absorb structural pathologies from noisy supervision.
  • Complementary Roles of Individual Rules: Removing invalid triples acts primarily as a precision filter (preventing the model from associating incompatible entity types), whereas adding missing inverse triples drives substantial recall gains (+15.52% recall in DREEAM-BERT). Combining all constraints achieves the optimal balance.
  • Data Reduction Is Not the Driver: Randomly pruning 167,234 triples across 10 random seeds yields negligible performance shifts (<1% variance) and even degrades recall/F1 in several configurations, proving that observed gains stem exclusively from the elimination of structural contradictions.

Highlights & Insights

  • Pioneering a Structural Quality Dimension: Broadens the scope of DocRE data denoising beyond semantic truthfulness by grounding relational learning in formal OWL ontology constraints.
  • Principled Statistical Error Attribution: Leverages binomial proportion sampling with Bayesian HPD credible intervals to rigorously uncover that 69% of invalid relational triples are driven by named entity recognition errors rather than relation labeling failures.
  • Model-Agnostic and Architecture-Independent: Requires zero modifications to downstream neural architectures, objective functions, or inference routines, providing immediate out-of-the-box utility for large-scale KG extraction pipelines.

Limitations & Future Work

  • Aggressive Entity Removal Discards Valid Triples: Removing entire entity instances upon encountering an invalid triple conservatively eliminates over 200,000 potentially informative triples; future work could deploy targeted entity re-annotation or LLM-based type correction to retain more training data.
  • Residual Inconsistencies in Benchmark Splits: The ReDocRED test set itself retains minor structural noise (0.81% invalid triples, 9.27% missing inverses); models that correctly generate structurally valid triples may be penalized as false positives by unrevised ground-truth annotations.
  • Scope Limited to Transformer Backbones: The empirical validation focuses on discriminative Transformer architectures (ATLOP and DREEAM); expanding the analysis to graph neural networks and generative LLM-based relation extractors remains an open research avenue.
  • vs. UGDRE (Sun et al., 2023) & DOREMI (Menotti et al., 2026): While previous denoising frameworks focus on predictive uncertainty and active learning for long-tail classes, empirical tests reveal that their denoised datasets still contain 1.7%~3.0% invalid triples and amplify cardinality violations up to 1.43%; structural regularization provides a vital orthogonal axis of data cleansing.
  • vs. Logic Rule Integration (Ru et al., 2021; Zhang et al., 2024): Rather than learning latent first-order logic rules via Expectation-Maximization or adding complex secondary reasoning modules inside the network, this work enforces ontology constraints at the data preprocessing stage, achieving structural consistency with complete architectural independence.

Rating

  • Novelty: ⭐⭐⭐⭐☆ First systematic formalization of OWL structural consistency for distant DocRE denoising.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous sampling analysis, multiple backbones, ablation breakdown, and randomized downsampling controls.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical definitions, excellent structure, and transparent error analysis.
  • Value: ⭐⭐⭐⭐⭐ Highly actionable preprocessing methodology for constructing high-integrity knowledge graphs from unstructured text.