Ontology-Based Knowledge Graphs for Industrial Standards with Scope-Aware Logical Rule Structuring¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/Ontology-KG/ontology_based_kg_paper
Area: LLM Reasoning
Keywords: Industrial Standards, Knowledge Graph, Scope-Aware Reasoning, Logical Rule Structuring, Retrieval-Augmented Generation
TL;DR¶
Addressing the vulnerability of standard RAG systems to cross-clause rule collision in hierarchical, table-dense industrial specifications, this paper proposes an Ontology-based KG-RAG framework that formalizes document section hierarchies into explicit scope ontologies and couples them with condition–consequence rule subgraphs utilizing case and logic_group nodes. Evaluated on the expert-validated IndusSpec-QA benchmark across three real-world standards, the proposed approach significantly outperforms traditional text- and graph-based baselines, demonstrating a 120.2% relative F1 gain on complex table-based reasoning.
Background & Motivation¶
Industrial standard specifications (such as ASTM, API, ISO, and DIN regulations) serve as authoritative technical foundations across civil engineering, marine architecture, offshore construction, and precision manufacturing. These documents dictate stringent design rules, material selection criteria, and quality inspection protocols. Unlike open-domain narrative text, industrial standards exhibit an inherently hierarchical semantic architecture: root and major section headings establish foundational domain concepts, while nested subsections, conditional clauses, structured specification tables, and supplementary footnotes progressively refine, constrain, and scope those concepts. Consequently, identical terminology, material grades, or physical parameter names correspond to drastically different rules and tolerances depending strictly on their surrounding section context.
However, prevailing Large Language Model (LLM) and Retrieval-Augmented Generation (RAG) paradigms suffer from fundamental structural blind spots when processing such documents. Mainstream text-based RAG methods fragment documents into arbitrary linear chunks, discarding the hierarchical section tree that defines rule applicability boundaries and cross-clause prerequisites. Even frontier long-context LLMs, when provided with entire multi-page industrial standards directly, frequently succumb to "scope collision"—wrongly combining criteria from conflicting subsections. Furthermore, conventional triplet-based knowledge graph (KG) extraction techniques operate at sentence-level granularity, extracting flat (subject, predicate, object) facts. They fail to retain the multidimensional row/column condition intersections of technical tables and collapse compound rules governed by nested Boolean logic (AND/OR operators), leading to broken logical coherence and high error rates in automated compliance assessment.
The fundamental tension lies in the fact that engineering rules cannot be represented as isolated facts; their validity is strictly scoped by hierarchical document provenance while requiring atomic preservation of compound condition–consequence relationships. Core idea: formalize the hierarchical section numbering structure of industrial standards into an explicit scope ontology (Scope Path) and seamlessly integrate it with condition–consequence rule triplets—structured via dedicated case and Boolean logic_group nodes extracted from text and tables—enabling scope-constrained collaborative graph retrieval and reasoning.
Method¶
Overall Architecture¶
The framework encompasses three core construction stages alongside two tailored retrieval-reasoning engines. The first stage, Section-grounded Ontology Modeling, extracts the explicit numbering and heading patterns of the specification to construct a hierarchical concept ontology, establishing a scope path \(p_i = s_1 \to s_2 \to \dots \to s_k\) from the root section to each terminal content block and packaging content into scope-grounded knowledge units \(U_i = (p_i, x_i)\). The second stage, Rule-aware Triplet Extraction, uses an LLM guided by specialized prompts to convert conditional statements and specification tables into condition–consequence units anchored around case nodes, employing logic_group nodes to preserve Boolean operator hierarchies. The third stage, Ontology-based KG Construction, attaches extracted triplets to their corresponding section concepts via [mention] edges, followed by multimodal-embedding-based synonym normalization and multi-step graph pruning. Downstream inference is executed via two complementary paradigms: Ontology-based MindMap (structural ontology-constrained graph traversal) and Ontology-based KG-Retriever (coarse-to-fine multi-level retrieval with Guided Search).
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Industrial Standard Documents<br/>(Text / Tables / Footnotes)"] --> B["Section-as-Concept & Scope Path Modeling<br/>Hierarchical Tree: s1 → s2 → ... → sk"]
B --> C["Rule-aware Triplet & Logic Group Extraction<br/>Condition-Consequence via Case & Logic_group Nodes"]
C --> D["Multimodal KG Normalization & Pruning<br/>PPR + Spectral Clustering & Low-info Pruning"]
D --> E["Scope-Constrained Collaborative Retrieval<br/>Ontology-guided Traversal & Guided Search"]
E --> F["High-Fidelity Compliance QA<br/>IndusSpec-QA (Rule / Multi-hop / Table)"]
Key Designs¶
1. Section-as-Concept & Scope Path Modeling: Resolving cross-clause semantic collisions
To eliminate rule confusion caused by linear chunking that detaches technical clauses from their governing headings, this design formalizes every section heading as an ontological section concept. Following the explicit numbering system (e.g., Section 7 \(\to\) Subsection 7.4 \(\to\) Subsubsection 7.4.2), parent–child structural relationships are interconnected via directed [has_child] edges, yielding a canonical scope path:
$\(p_i = s_1 \to s_2 \to \dots \to s_k\)$
Each content unit \(x_i\) (prose paragraph, specification matrix, or footnote) is combined with its structural path into an indivisible knowledge unit \(U_i = (p_i, x_i)\). When fine-grained entity and rule triplets are instantiated, they are linked exclusively to the terminal section concept \(s_k\) via [mention] relations. Consequently, an identical term such as "Grade 50" appearing simultaneously under "5. Chemical Requirements" and "6. Mechanical Requirements" is physically partitioned into separate ontological subtrees, completely preventing property cross-contamination during retrieval.
2. Rule-aware Triplet & Logic Group Extraction: Preserving compound conditions and Boolean precedence
To resolve the breakdown of conventional triplet representations when handling compound conditional specifications and nested AND/OR requirements, this design models both prose rules and table cells as unified condition–consequence structures. Each distinct rule instance or non-empty table cell is assigned a dedicated rule node caseX. When a requirement incorporates mixed logical operators—such as \(A \land (B \lor C) \to D\)—the schema introduces a logic_groupX node that functions analogously to parentheses in formal Boolean algebra.
Inside caseX, conditions and consequences are connected via typed relations (has_condition_AND, has_condition_OR, has_consequence). Inside logic_groupX, only uniform operators are permitted, and the group itself connects to the parent caseX via the higher-precedence operator. For tabular matrices, row headers and column headers are systematically parsed into composite antecedent conditions, while the cell value is extracted as the consequent specification, cleanly converting complex 2D tabular constraints into logically sound graph subgraphs.
3. Multimodal KG Normalization & Pruning: Eliminating extraction redundancy and structural noise
To address surface-form variations introduced by LLM extraction and prune redundant graph loops, this design introduces a multimodal representation learning framework for canonical synonym discovery alongside conservative structural pruning. For every entity node \(a\), a fused embedding is generated: $\(e_{final}(a) = [e_{text}(a) \,;\, e_{ctx}(a) \,;\, e_{pos}(a)]\)$ where \(e_{text}(a)\) captures semantic similarity via Sentence-BERT (all-MiniLM-L6-v2), \(e_{ctx}(a)\) captures local contextual importance using Personalized PageRank (PPR), and \(e_{pos}(a)\) reflects global graph topological positioning via spectral embeddings of the graph Laplacian. For relation predicates \(r\), a selectional preference vector \(e_{prefer}(r)\) is additionally concatenated to model the probability distribution \(p(t \mid u, v)\) of linked head–tail entity types. Unsupervised density clustering via HDBSCAN identifies synonym clusters, electing the cluster medoid as the canonical predicate/concept. Subsequently, multi-stage pruning conservatively eliminates uninformative self-loops \(T_{self} = \{(u, r, v) \in T \mid u = v\}\), low-frequency reciprocal edges \(T_{inv}\), and duplicate extractions \(T_{dup}\), preserving graph density without discarding essential engineering constraints.
4. Scope-Constrained Collaborative Retrieval: Suppressing global graph drift and hallucination
To prevent graph traversal algorithms from straying into irrelevant document branches due to ubiquitous generic engineering terms, this design constrains evidence mining strictly within activated ontology scopes through two collaborative mechanisms:
- Ontology-based MindMap: Extracts query entities, anchors them to graph nodes, and restricts neighbor exploration and shortest-path mining (1–3 hops) strictly within the hierarchical subgraph defined by the relevant section concepts. Structural ontology relations ([has_child]) are filtered out from the final reasoning path, leaving a clean, coherent condition–consequence subgraph for LLM generation.
- Ontology-based KG-Retriever: Operates a two-tiered evidence collection strategy. At the Ontology KG-Level, query similarity against knowledge unit embeddings identifies Top-3 relevant section paths, extracting 35 candidate triplets. At the Global KG-Level, a Guided Search module computes direct semantic cosine similarity between the query and triplet textualizations, collecting 30 candidate triplets to capture cross-clause context. The combined 65 candidates undergo Maximal Marginal Relevance (MMR, \(\lambda=0.7\)) re-ranking to yield 30 diverse, high-precision triplets for response synthesis.
A Worked Example¶
Consider an inspection requirement from ASTM A578/A578M where Table S8.1 (under Supplementary Requirement S8: Ultrasonic Examination Using Flat Bottom Hole Calibration) and Table S9.1 (under Supplementary Requirement S9: Ultrasonic Examination of Electroslag Remelted ESR / Vacuum-Arc Remelted VAR Plates) both define calibration hole diameters based on plate thickness ranges.
- Query: "What is the required hole diameter for ultrasonic examination of plates with a thickness greater than 225 mm and up to 300 mm?" (Gold answer: 22 mm per Table S8.1).
- Knowledge Representation: Table S8.1 is placed under the scope path A578/A578M -> SUPPLEMENTARY REQUIREMENTS -> S8, creating a case node with condition 225 mm < plate_thickness <= 300 mm and consequence hole_diameter = 22 mm. Table S9.1 is anchored under ... -> S9 where a competing rule specifies hole_diameter = 10 mm.
- Failure in Traditional RAG: Conventional KG-based RAG (MindMap without ontology) matches entities plate_thickness and hole_diameter. Because both tables share nearly identical numerical bounds, the graph search commingles S8 and S9 candidates, erroneously selecting the S9 rule and predicting 10 mm.
- Resolution via Proposed Framework: Ontology-based KG-RAG identifies that the user query specifies general ultrasonic examination without mentioning ESR/VAR remelt processes, grounding the query scope firmly in Section S8. Traversing strictly within the S8 scope path, it retrieves the exact rule node and returns the verified standard value 22 mm.
Benchmark & Evaluation Protocol¶
To rigorously evaluate QA under industrial constraints, the authors constructed IndusSpec-QA, derived from three real-world industrial standard specifications: 1. ASTM A578/A578M: A concise, rule-centric standard for straight-beam ultrasonic testing; 2. API 2W: A challenging specification for offshore structural steel plates, saturated with multi-parameter property tables; 3. ASTM A6/A6M: A comprehensive, large-scale general requirements standard featuring complex multi-hop conditional clauses, extensive exception rules, and vast dimensional profiles. The benchmark categorizes questions into Rule (single-sentence/clause direct requirements), Multi-hop (inter-clause and multi-condition cross-referencing), and Table (interdependent row/column condition-to-cell value lookups). All items were validated by 8 industrial domain experts and researchers, complete with exact source page numbers and bounding box coordinates for provenance. Evaluation utilizes token-level F1 as the primary metric, supplemented by BERTScore, BLEU, and ROUGE.
Key Experimental Results¶
Main Results: Multi-Standard Benchmark Comparison¶
Experiments evaluated zero-shot generation (LLM-only), direct long-context input (LLM w/full doc), chunk-based text retrieval (Text-only RAG: BM25 / Dense Ada-002), document graph RAG (Document-to-Graph RAG: GraphRAG / LightRAG / HippoRAG2), flat rule graphs (KG-based RAG: MindMap / KG-Retriever), and the proposed Ontology-based KG-RAG. GPT-5-mini was employed as the primary backbone (weighted average results from Table 3):
| Knowledge Representation & Baseline | Method | A578/A578M (F1) | API_2W (F1) | A6/A6M (F1) | Average F1 |
|---|---|---|---|---|---|
| LLM-only | GPT-5-mini | 0.000 | 0.000 | 0.000 | 0.000 |
| LLM w/full doc | GPT-5-mini (Full Document Context) | 0.372 | 0.376 | 0.273 | 0.340 |
| Text-only RAG | BM25 + GPT-5-mini | 0.348 | 0.290 | 0.170 | 0.269 |
| Text-only RAG | Dense (Ada-002) + GPT-5-mini | 0.316 | 0.365 | 0.161 | 0.281 |
| Document-to-Graph | GraphRAG (Microsoft) | 0.410 | 0.416 | 0.252 | 0.359 |
| Document-to-Graph | LightRAG | 0.288 | 0.340 | 0.163 | 0.264 |
| Document-to-Graph | HippoRAG2 | 0.280 | 0.203 | 0.195 | 0.226 |
| KG-based RAG | MindMap (Flat Rule KG) | 0.601 | 0.413 | 0.339 | 0.451 |
| KG-based RAG | KG-Retriever (Flat Rule KG) | 0.706 | 0.435 | 0.271 | 0.471 |
| Ontology-based KG-RAG | Ontology-based MindMap (Ours) | 0.719 | 0.564 | 0.543 | 0.609 |
| Ontology-based KG-RAG | Ontology-based KG-Retriever (Ours) | 0.722 | 0.472 | 0.322 | 0.505 |
(Note: On A578/A578M, Ontology-based KG-Retriever achieved peak performance of 0.722 F1; on the highly complex API_2W and A6/A6M benchmarks, Ontology-based MindMap led with 0.564 and 0.543 F1 respectively, outperforming all competitive baselines by substantial margins).
Ablation Study: Structural Disentanglement of Ontology & KG¶
To isolate the distinct contributions of the section ontology and the rule-aware KG, systematic ablations were conducted on the API_2W dataset (Table 5):
| Configuration | F1 | BERTScore | BLEU | ROUGE | Relative Gain over Full Doc |
|---|---|---|---|---|---|
| LLM w/full doc (Baseline) | 0.447 | 0.883 | 0.218 | 0.510 | Baseline |
| + Ontology (Section Tree Only) | 0.421 | 0.879 | 0.206 | 0.482 | -5.8% (Lacks fine-grained facts) |
| + KG (Rule Triplets w/o Ontology) | 0.591 | 0.926 | 0.403 | 0.612 | +32.2% (Rule structuring benefit) |
| + Ontology + KG (Full Proposed Method) | 0.612 | 0.933 | 0.434 | 0.638 | +36.9% (Synergy of scope & rules) |
Furthermore, breakdown analysis across question types (Figure 4) revealed the performance leap when transitioning from flat KG-based RAG to the full Ontology-based KG-RAG: - Rule Questions: F1 increased from 0.601 to 0.668 (+11.1%); - Multi-hop Questions: F1 improved from 0.515 to 0.573 (+11.3%); - Table Questions: F1 escalated from 0.287 to 0.632 (an exceptional +120.2% increase).
Key Findings¶
- Long-context windows cannot compensate for missing hierarchical scoping: Feeding entire standards directly to GPT-5-mini yielded an average F1 of only 0.340 (and 0.273 on the large A6/A6M standard). Attention mechanisms frequently suffer from cross-clause distraction and fail to enforce strict subsection boundaries.
- Generic Graph RAG falters on exact engineering specifications: GraphRAG, LightRAG, and HippoRAG2 averaged an F1 of only 0.283. Because these methods rely on summary-driven community clusters and unconstrained entity graphs, they collapse tabular cell boundaries and omit exact numerical tolerances.
- Tabular reasoning is the primary beneficiary of scope-aware structuring: The 120.2% relative surge in Table question F1 highlights that specification matrices cannot be resolved via flat entity search. Fixed-traversal graph-swap ablations confirm that introducing ontology structure alone provides an immediate +0.168 F1 gain on tabular queries.
Highlights & Insights¶
- Reifying Boolean grouping parentheses as graph nodes: The introduction of
logic_groupnodes provides an elegant topological solution to complex nested conditions like \(A \land (B \lor C)\), preserving logical operator precedence within standard directed graph triples without requiring hypergraph schema overhauls. - Section hierarchies as hard topological scope boundaries: Rather than treating document headers as passive prefix metadata in text chunks, formalizing them into a backbone ontology enables graph traversals to enforce physical scope isolation, actively suppressing spurious cross-clause entity matching.
- Unified representation of prose rules and multidimensional tables: Systematically decomposing tabular row/column intersections into antecedent condition sets and mapping cell contents to consequent nodes harmonizes unstructured prose rules and structured tables under a unified, queryable schema.
Limitations & Future Work¶
- Dependence on structural formatting cues: The ontology extraction pipeline relies on explicit section numbering and title conventions (e.g., 1.1, 1.1.1). Its effectiveness may degrade when applied to loosely formatted, unstructured reports or specifications lacking distinct numbering hierarchies.
- Sensitivity to LLM extraction quality: While the extraction prompts are highly structured, the initial extraction of rules and
logic_groupnodes is powered by prompting an external LLM, leaving potential vulnerability to hallucinated predicates or malformed triples. - Future Directions: The authors highlight plans to distill the extraction and ontology construction capabilities into specialized, lightweight domain models, reducing dependence on proprietary LLM APIs and ensuring deterministic execution in safety-critical engineering deployments.
Related Work & Insights¶
- vs Document Ontologies (DoCO, DITA-RDF): Prior document ontologies focus primarily on structural markup vocabularies (paragraphs, lists, document components). In contrast, this work uses the document's conceptual section tree as an active semantic scope boundary directly governing rule applicability and retrieval paths.
- vs Document-to-Graph RAG (GraphRAG, LightRAG, HippoRAG2): While existing graph RAG frameworks excel at broad summarization and global theme clustering, they dilute precise numerical inequalities and tabular constraints into fuzzy text summaries. The proposed framework maintains strict atomic rule units with exact condition–consequence fidelity.
- vs Standard Triplet KGs (MindMap, KG-Retriever): Conventional open-IE or triplet KGs drop relational context across sections, inducing catastrophic scope collisions between identical material grades. By coupling the section ontology with rule-aware graph representations, the proposed architecture provides strict structural scope isolation.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ [Pioneering integration of hierarchical section ontologies with logic-grouped condition–consequence subgraphs for industrial standards]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Comprehensive benchmarking on the expert-validated IndusSpec-QA dataset across three real standards, supported by thorough ablations and qualitative retrieval traces]
- Writing Quality: ⭐⭐⭐⭐⭐ [Exceptionally rigorous formulation, precise terminology, cohesive motivation, and meticulous visual/empirical presentation]
- Value: ⭐⭐⭐⭐⭐ [Provides an essential, highly practical blueprint for deploying high-reliability RAG systems across technical auditing, engineering compliance, and legal domains]