Adaptive Knowledge Routing for Complex Reasoning in LLMs¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/Eunchan24/PEEL
Area: LLM Reasoning
Keywords: Knowledge Routing, Hierarchical Representations, Order Embeddings, Minimum Spanning Arborescence, Concept Ontology
TL;DR¶
This paper discovers that Matryoshka Representation Learning (MRL) retrieval embeddings inherently encode ontological granularity along their prefix dimensions, and proposes PEEL—a minimal 98K-parameter order-embedding probe that decodes high-fidelity taxonomic DAGs directly from concept vocabularies without needing document corpora or encoder retraining.
Background & Motivation¶
Domain taxonomies and hierarchical concept systems are fundamental backbones for structured knowledge systems, supporting everything from biomedical terminologies like MeSH to massive category hierarchies in Wikipedia and academic classification schemes. Structured taxonomic is-a relations provide the essential scaffold for knowledge-intensive inference and complex semantic routing. However, automated taxonomy construction remains prohibitively expensive: embedding-based flat clustering requires delicate per-corpus tuning and yields unstable hierarchies; specialized hyperbolic or box geometries demand task-specific, hierarchy-supervised training; and recent LLM-based pipelines require compute-heavy fine-tuning or complex multi-turn prompting that fails to generalize across domains while incurring massive token latency.
This creates a persistent structural tension: bottom-up statistical representations lack explicit asymmetric hierarchy constraints, while top-down generative LLMs are computationally heavy and struggle with global acyclic graph consistency. Concurrently, the structural probing literature has demonstrated that pre-trained language models encode rich linguistic trees, yet these probes have almost exclusively focused on local syntactic dependencies rather than global semantic ontologies. A foundational question has therefore remained unaddressed: do frozen text embeddings trained purely for dense retrieval already encode exploitable taxonomic hierarchy within their representation geometry?
The authors examine the mechanics of Matryoshka Representation Learning (MRL), wherein nested prefix structures enforce coarse semantic information to concentrate in early dimensions. Through rigorous shuffled-dimension controls (destroying ordering while preserving content) and reverse-dimension controls (tail dimensions), they confirm across three distinct encoder families (BERT, XLM-R, and decoder-only) that prefix length consistently tracks the coarse-to-fine granularity of gold taxonomies. Core idea: exploit the intrinsic coarse-to-fine granularity gradient in pre-trained MRL retrieval embeddings via a minimal 98K-parameter order-embedding probe with random prefix truncation, decoding global taxonomic DAGs via Chu-Liu/Edmonds minimum spanning arborescence directly from concept vocabularies without document access.
Method¶
Overall Architecture¶
PEEL (Peeling Embeddings to Extract Latent ontologies) operates on the principle that since pre-trained MRL embeddings already mix latent hierarchy with retrieval-specific features, a lightweight projection can cleanly isolate taxonomic containment into coordinate-wise dominance without retraining the underlying backbone. The pipeline comprises two decoupled phases: lightweight offline probe training and threshold-free online graph decoding. During training, frozen 768-dimensional MRL representations of gold parent-child pairs are mapped into a 128-dimensional non-negative orthant using a single linear layer with ReLU, optimized with an order-embedding loss. To ensure multi-scale robustness across resolutions, random dimensional truncation and zero-padding are applied at each training step. During inference, given only a target concept vocabulary, the trained probe produces coordinates to construct a pairwise violation matrix; the Chu-Liu/Edmonds algorithm then extracts a rooted minimum spanning arborescence (MSA), which is subsequently expanded into a polyhierarchical DAG by greedily admitting low-violation non-tree edges under an acyclicity constraint.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Target Concept Vocabulary<br/>Unordered concept terms"] --> B["Frozen MRL Encoder<br/>Multi-scale prefix embeddings e ∈ ℝ⁷⁶⁸"]
B --> C["Multi-Resolution Random Truncation<br/>d ∈ {64, ..., 768} zero-padded training"]
C --> D["Single-Layer Order-Embedding Probe<br/>768→128 ReLU projection into ℝ¹²⁸+"]
D --> E["Threshold-Free Tree Decoding<br/>Chu-Liu/Edmonds MSA algorithm"]
E --> F["Greedy Low-Violation DAG Expansion<br/>Add acyclic near-miss edges within budget"]
F --> G["Output Taxonomic DAG<br/>Global parent-child is-a hierarchy"]
Key Designs¶
1. Multi-Resolution Random Truncation: Resolution-Invariant Taxonomic Alignment Fixing a probe to a single truncation length ties its parameters to a specific feature distribution, preventing it from generalizing across diverse semantic grains. PEEL addresses this by dynamically sampling a prefix dimension \(d \in \{64, 128, 256, 384, 512, 768\}\) at each training step. The truncated \(d\)-dimensional prefix is zero-padded back to \(D=768\) before feeding into the probe. This stochastic truncation forces the probe to learn order-embedding relationships that hold robustly across all resolution scales simultaneously, allowing a single lightweight model to adapt flexibly to any downstream dataset granularity.
2. Order-Embedding Projection Probe: Geometric Encoding of Asymmetric Is-A Relations Symmetric distance metrics like cosine similarity fail to represent directional parent-child containment. PEEL adopts a single linear transformation followed by a ReLU activation \(f = \text{ReLU}(We + b)\) projecting into \(\mathbb{R}^{d_{out}}_+\) (\(d_{out}=128\), totaling only 98K trainable parameters). In this non-negative space, taxonomic hierarchy is modeled as coordinate-wise dominance: a parent node \(i\) coordinates must be smaller than or equal to its child node \(j\) across every dimension (\(f_k(i) \le f_k(j)\)). The degree of hierarchy violation between any pair \((i, j)\) is formulated as: $\(v(i, j) = \sum_{k=1}^{d_{out}} \max(0, f_k(i) - f_k(j))^2\)$ A zero violation strictly indicates perfect geometric containment of concept \(j\) under concept \(i\).
3. Chu-Liu/Edmonds Arborescence & Greedy DAG Expansion: Threshold-Free Topology Decoding Conventional clustering and containment-ratio thresholding are sensitive to tuning and frequently generate disconnected components or spurious cycles. PEEL models the pairwise violation scores as directed edge weights and executes the Chu-Liu/Edmonds (CLE) algorithm to extract the minimum spanning arborescence. This guarantees an optimal, cycle-free spanning tree of exactly \(N-1\) directed edges without requiring any threshold hyperparameter. To accommodate polyhierarchy (concepts with multiple parent classes), the method sorts all non-tree node pairs by their violation scores and greedily inserts the lowest-violation edges that preserve graph acyclicity, up to a target edge budget \(n_{expand}\).
Loss & Training¶
The probe is trained exclusively on source-domain gold parent-child pairs using the asymmetric Order-Embedding Loss: $\(\mathcal{L} = \mathcal{L}_+ + \mathcal{L}_-\)$ $\(\mathcal{L}_+ = \frac{1}{|\mathcal{P}|} \sum_{(p,c) \in \mathcal{P}} \sum_{k=1}^{d_{out}} \max(0, f_k(p) - f_k(c))^2\)$ $\(\mathcal{L}_- = \frac{1}{|\mathcal{N}|} \sum_{(p,n) \in \mathcal{N}} \max\left(0, m - \sum_{k=1}^{d_{out}} \max(0, f_k(p) - f_k(n))^2\right)\)$ where \(\mathcal{P}\) represents positive parent-child pairs and \(\mathcal{N}\) represents sampled negative pairs with a 10:1 negative-to-positive ratio. The negative sampling distribution is composed of 50% uniform random nodes, 30% sibling distractors (nodes sharing the same parent), and 20% reversed parent-child pairs. Training uses Adam with learning rate \(5 \times 10^{-4}\), batch size 512, and margin \(m=4.0\). Due to its tiny parameter footprint, training converges in 10 epochs on a single GPU in approximately 4 seconds.
Key Experimental Results¶
Main Results¶
Evaluated on the standardized OLLM benchmark suite, PEEL receives strictly the concept vocabulary (zero document access). Baselines include statistical extraction, Mistral-7B prompting/fine-tuning, and the full end-to-end OLLM pipeline (which consumes document titles and abstracts). The primary structure-sensitive metrics are Cont. F1 (threshold-free Hungarian edge matching scored by weaker endpoint similarity) and Graph F1 (Hungarian node matching over SG-Conv neighborhood embeddings).
| Dataset | Method | Input Modality | Cont. F1 ↑ | Graph F1 ↑ | Motif Dist. ↓ |
|---|---|---|---|---|---|
| wiki-ol (8,033 concepts) | Memorisation | Gold training edges | .314 | .419 | .063 |
| wiki-ol | Hearst Patterns | Corpus + concepts | .350 | .544 | .163 |
| wiki-ol | Zero-shot (Mistral-7B) | Title + abstract text | .455 | .639 | .341 |
| wiki-ol | Finetune (Mistral-7B) | Title + abstract text | .470 | .588 | .050 |
| wiki-ol | OLLM (cross-task ref.) | Title + abstract text | .500 | .644 | .080 |
| wiki-ol | PEEL (Ours) | Concepts only (no text) | .551 | .885 | .000 |
| arXiv-ol (61 categories) | OLLM (transfer, 2K arXiv SFT) | Title + abstract text | .357 | .633 | .097 |
| arXiv-ol | PEEL (Zero-shot transfer) | Concepts only (zero-shot) | .604 | .931 | .011 |
Ablation Study¶
Table A analyzes the impact of input prefix truncation dimension \(d\) on wiki-ol with fixed probe output dimension \(d_{out}=128\).
| Input Prefix Dimension \(d\) | Cont. F1 ↑ | Graph F1 ↑ | Observation / Trend |
|---|---|---|---|
| \(d = 64\) | .463 | .879 | Captures coarse high-level taxonomy, lacks fine distinctions |
| \(d = 128\) | .485 | .880 | Intermediate granularity separation begins emerging |
| \(d = 256\) | .519 | .879 | Subtree depth and branching factor expand smoothly |
| \(d = 384\) | .537 | .880 | Approaches optimal balance of semantics and hierarchy |
| \(d = 512\) (Default Best) | .551 | .885 | Optimal peak for continuous semantic resolution |
| \(d = 768\) (Full Vector) | .479 | .906 | Dense retrieval noise dilutes coordinate-wise containment |
| OLLM (Reference) | .500 | .644 | 7B LLM joint pipeline baseline |
Table B examines the sensitivity to probe output dimension \(d_{out}\) with fixed input prefix \(d=512\).
| Probe Output \(d_{out}\) | Trainable Params | Cont. F1 ↑ | Graph F1 ↑ | Finding |
|---|---|---|---|---|
| \(d_{out} = 32\) | 24K | .517 | .870 | Extremely tiny probe already dramatically outperforms 7B LLMs |
| \(d_{out} = 64\) | 49K | .544 | .878 | Smooth monotonic gain as coordinate capacity increases |
| \(d_{out} = 128\) (Ours) | 98K | .551 | .885 | Ideal sweet spot balancing capacity and minimal complexity |
Key Findings¶
- Prefix Length Controls Subtree Unfolding: Subtree extraction across prefix lengths reveals an intrinsic coarse-to-fine hierarchy: for the Mathematics subtree, \(d=64\) decodes only 1 child, \(d=256\) unfolds 19 children, and \(d=768\) unfolds 68 fine-grained concepts across 12 depth levels. The pairwise edge Jaccard similarity across dimensions remains below 0.18, proving that different prefix lengths encode qualitatively distinct, complementary granularities.
- Full Dimensionality Suffers from Retrieval Dilution: Performance on Cont. F1 peaks at \(d=512\) (.551) and drops to .479 at full dimension \(d=768\). This confirms that while the earliest dimensions concentrate pure hierarchical taxonomy, the trailing dimensions mix in passage-matching and keyword-retrieval features that dilute coordinate dominance.
- Exceptional Zero-Shot Cross-Domain Transfer: The probe trained exclusively on Wikipedia gold edges transfers directly to the academic arXiv-ol hierarchy without seeing any target-domain edges or document texts, attaining .931 Graph F1. This surpasses the OLLM model (.633 Graph F1) despite the latter fine-tuning Mistral-7B on 2,048 arXiv documents.
Highlights & Insights¶
- Groundbreaking Geometric Discovery: Systematically proves that standard MRL embeddings, trained solely for retrieval efficiency without any tree supervision, naturally organize semantic granularity along their dimension axis. Controlled shuffled and reverse permutations confirm that early dimensions specifically concentrate taxonomic generality.
- Unprecedented Compute Efficiency: Replaces heavy generative LLM pipelines (7B parameters, minute-long token generation) with a 98K-parameter single-layer probe that trains in 4 seconds on one GPU and performs non-parametric inference via combinatorial graph algorithms.
- Decoupled Modular Architecture: Separates taxonomy learning into front-end vocabulary discovery and back-end structure induction. Any text mining or entity extraction system can output raw terms, while PEEL decodes the global ontology topology with mathematical rigor.
Limitations & Future Work¶
- Source-to-Target Depth Asymmetry: Zero-shot transfer is currently demonstrated from a deeper source ontology (Wikipedia, depth 3) to a shallower target ontology (arXiv, depth 2). Whether a probe trained on shallow domains can successfully decode deeper, highly specialized hierarchies (e.g., complex legal codes or clinical MeSH subtrees) remains an open research question.
- Evaluation Restricted to Graph Topology: The benchmark focuses strictly on structural topology fidelity (Graph F1, Cont. F1, Motif Distance). Validating the downstream utility of these extracted DAGs within practical hierarchical RAG routing or complex multi-hop agent decision-making is left for future work.
- Fixed Edge Budget for Polyhierarchy: The greedy DAG expansion step relies on a pre-set edge budget \(n_{expand}\) to control density. Developing an adaptive stopping criterion based on curvature or global energy would further automate the pipeline.
Related Work & Insights¶
- vs HiTs (He et al., NeurIPS 2024): HiTs injects hierarchy by retraining the entire Transformer encoder (22M–109M parameters) in hyperbolic Poincaré space using custom contrastive losses. PEEL demonstrates that hierarchy is already present in standard Euclidean MRL embeddings, recovering higher-fidelity taxonomies using a frozen encoder and a 98K probe—reducing trainable parameters by three orders of magnitude.
- vs OLLM (Lo et al., NeurIPS 2024): OLLM couples concept discovery and taxonomy induction into an end-to-end LLM generative framework, requiring full document texts and fine-tuning. PEEL proves that once concept terms are available, structure induction can be solved with superior structural fidelity (Graph F1 .885 vs .644) without processing document text, highlighting the benefit of modular task decomposition.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ First study to uncover and operationalize the intrinsic alignment between MRL prefix geometry and taxonomic hierarchy.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive controls across 3 encoder families, ANOVA significance tests, multi-resolution sweeps, and zero-shot transfer.
- Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally concise, mathematically grounded, and rigorous structural analysis.
- Value: ⭐⭐⭐⭐⭐ Provides a fast, highly scalable paradigm for automated ontology construction, knowledge routing, and hierarchical retrieval.