TaxoAdapt: Aligning LLM-Based Multidimensional Taxonomy Construction to Evolving Research Corpora

Conference: ACL2025
arXiv: 2506.10737
Code: pkargupta/taxoadapt
Area: LLM/NLP
Keywords: Automatic Taxonomy Construction, LLM Alignment, Multidimensional Taxonomy, Hierarchical Text Classification, Scientific Literature Organization

TL;DR

Proposes the TaxoAdapt framework. By utilizing hierarchical classification-driven depth/breadth expansion and taxonomy-aware clustering, TaxoAdapt dynamically aligns LLM-generated multidimensional taxonomies with specific scientific corpora, outperforming state-of-the-art baselines by 26.51% in path granularity and 50.41% in sibling coherence.

Background & Motivation

Explosive Growth of Scientific Literature: In recent years, the volume of scientific literature has increased dramatically, and new research branches are constantly emerging (e.g., the rise of generative models), making the organization and retrieval of domain knowledge extremely challenging.

Limitations of Manually Curated Taxonomies: Handcrafting taxonomies by experts guarantees quality but is costly, time-consuming, and struggles to keep pace with rapidly evolving research fields.

Limitations of Prior Work in Corpus-Driven Methods: Traditional Automatic Taxonomy Construction (ATC) methods extract topics and relationships directly from text. However, they are bounded by the corpus vocabulary, lack broad background knowledge, and fail to leverage the capabilities of LLMs.

Blind Spots of LLM-Based Methods: Current LLM methods can generate general taxonomies, yet they rely excessively on general knowledge from their pre-training data and lack mechanisms to align with domain-specific corpora, failing to reflect research trends within a specific corpus.

Lack of Multidimensional Perspectives: A paper may simultaneously contribute to multiple dimensions (tasks, methods, datasets, evaluation metrics, etc.), but existing methods are restricted to constructing taxonomies along a single dimension, ignoring the multi-faceted nature of scientific literature.

Need for Dynamic Evolution: Research fields continuously evolve, with new subfields emerging and old ones fading (e.g., from the BERT era to the RLHF era). Taxonomies need to reflect these temporal shifts.

Method

Overall Architecture

TaxoAdapt is a multidimensional framework that dynamically aligns LLM-generated taxonomies with scientific corpora. Given a target topic \(t\), a set of dimensions \(D\) (Task, Methodology, Datasets, Evaluation Methods, Real-World Domains), and a scientific corpus \(P\), it outputs \(|D|\) dimension-specific taxonomies. The overall process consists of three stages:

1. Multi-Dimension Classification: Classifies the corpus via multi-label classification according to the dimensions a paper contributes to, dividing it into \(|D|\) potentially overlapping subsets \(P_d \subseteq P\).
2. Top-Down Construction: Identifies nodes that need expansion via hierarchical text classification, performing depth expansion and breadth expansion.
3. Taxonomy-Aware Clustering: Leverages LLMs' clustering capabilities to generate fine-grained, low-redundancy child nodes with consistent granularity for the nodes to be expanded.

Key Designs

Multi-Dimension Classification: Performs multi-label classification using an LLM to determine the contributing dimensions of a paper based on its title and abstract. The five dimensions are defined as follows: - Task: All papers are associated with at least one task by default. - Methodology: Papers that introduce or improve methods/approaches. - Datasets: Papers that introduce new datasets. - Evaluation Methods: Papers that evaluate model performance or propose new evaluation metrics. - Real-World Domains: Papers that address practical problems in specific domains.

Depth Expansion Signal: Triggered when the density of a leaf node \(n_{i,d}\) satisfies \(\rho(n_{i,d}) \geq \delta\), indicating that the topic is deeply explored in the corpus but the taxonomy is not granular enough, requiring a downward expansion into finer-grained subtopics.

Breadth Expansion Signal: Based on the unmapped density \(\tilde{\rho}(n_{i,d})\) of non-leaf nodes, representing the number of papers mapped to the parent node but not covered by any existing child node. When \(\tilde{\rho} > \delta\), it suggests that existing child nodes are insufficient to cover the research directions in the corpus, requiring the addition of new sibling nodes.

Subtopic Pseudo-Label Generation: For each paper mapped to a node undergoing expansion, the LLM is leveraged to generate dimension- and granularity-consistent pseudo-labels based on its title/abstract and context (such as the target node's dimension, level, and ancestor path).

Subtopic Clustering: Based on the pseudo-label list, the LLM's clustering capability is utilized in a dimension- and granularity-aware context to determine optimal subtopic clusters, generating a label and description for each cluster to be added as new child nodes in the taxonomy.

Loss & Training

• A hybrid model strategy is adopted to optimize costs: Llama-3.1-8B handles dimension classification, hierarchical classification signals, and subtopic pseudo-label generation; GPT-4o-mini handles initial taxonomy construction and subtopic clustering.
• The taxonomy is defined as a Directed Acyclic Graph (DAG), allowing a single node to have multiple parent nodes (e.g., "Scientific QA" can simultaneously belong to "QA" and "Scientific Reasoning").
• Iterative layer-by-layer processing: performing classification \(\rightarrow\) identifying expansion signals \(\rightarrow\) clustering expansion for each layer, until no nodes trigger expansion or the maximum depth is reached.

Key Experimental Results

Datasets

Conference	Papers	Topics
EMNLP 2022	828	NLP
EMNLP 2024	2954	NLP
ICRA 2020	1000	Robotics
ICLR 2024	2260	Deep Learning
Total	7042	-

Main Results (Averaged across all dataset dimensions, ×100)

Model	Path↑	Sib↑	Dim↑	Rel↑	Cover↑
Chain-of-Layers	47.5	55.5	95.0	81.1	50.9
With-Corpus LLM	65.2	31.7	89.5	79.7	39.4
TaxoCom	27.7	53.6	91.8	92.6	61.6
TaxoAdapt	82.4	83.5	99.4	85.3	55.5
- No Dim	89.1	81.3	99.6	82.5	64.8
- No Clustering	73.8	71.3	96.0	81.1	54.2

Standard Deviation Comparison

Model	Path	Sib	Dim	Rel	Cover
Chain-of-Layers	0.078	0.109	0.008	0.043	0.005
TaxoAdapt	0.027	0.021	0.007	0.043	0.015

Key Findings

1. Significant Improvement in Granularity Maintenance: TaxoAdapt's taxonomies exceed the best baselines by 26.51% in Path Granularity and 50.41% in Sibling Coherence, indicating more precise hierarchical relationships and more consistent granularity among sibling nodes.
2. Strong Robustness Across Dimensions: TaxoAdapt achieves the lowest standard deviation across all granularity metrics, demonstrating stable performance across different research dimensions without bias toward any specific one.
3. Corpus Evolution Tracking: In the EMNLP'22 \(\rightarrow\) EMNLP'24 case study, TaxoAdapt successfully captured trends such as the decline in the popularity of masked language modeling and the rise of instruction-based LMs, with the number of nodes increasing from 62 to 99.
4. Highly Competitive Pure Open-Source Models: A TaxoAdapt variant using solely Llama-3.1-8B still achieves performance comparable to or even exceeding GPT-based baselines.

Highlights & Insights

• Classification-signal-driven expansion is a highly ingenious design: using the aggregation density and unmapped density of papers on nodes as expansion signals avoids blind expansion and achieves "growth on demand."
• The multidimensional perspective is a key innovation: recognizing that a single paper can simultaneously contribute to multiple dimensions such as tasks, methods, and datasets, rather than being restricted purely to a single task dimension.
• Modeling the taxonomy as a DAG instead of a tree aligns better with the overlapping relationships among concepts in scientific literature.
• Leveraging LLM clustering capability instead of traditional clustering algorithms allows the clustering process to incorporate dimensional and granularity contexts, yielding child nodes with stronger semantic consistency.
• The hybrid open/closed-source model strategy effectively reduces costs while maintaining high performance.

Limitations & Future Work

1. Risk of Outdated LLM Knowledge: Dimension classification relies on the parametric knowledge of LLMs; when a new concept shares a name with an old one, classification errors may occur (e.g., confusing a new benchmark with a method of the same name).
2. Downstream Applications Pending Validation: The practical application of the generated taxonomies in retrieval augmentation and research assistants has not yet been fully explored.
3. Evaluation Heavily Reliant on LLM Scoring: Although supplemented by human evaluation, main metrics are judged by GPT-4o/4o-mini, potentially introducing some bias.
4. Manually Defined Dimensions: The selection and definition of the five dimensions rely on domain experts, which may require adjustments when generalizing to non-CS fields.
5. Scalability: The processing efficiency for large-scale corpora has not been discussed in detail; performing LLM-based classification document-by-document can be highly costly.

Related Work & Insights

vs Chain-of-Layer (Zeng et al., 2024)

Chain-of-Layer is a pure LLM method that constructs taxonomies layer by layer using only pre-trained knowledge, lacking corpus alignment. By introducing classification signals, TaxoAdapt enables corpus-driven expansion, leading by a wide margin in Path (+34.9) and Sib (+28.0) metrics, indicating that pure LLM knowledge is insufficient to capture research trends within a specific corpus.

vs TaxoCom (Lee et al., 2022)

TaxoCom is a purely corpus-driven method that clusters and extracts entities from text to complete the taxonomy. Although its Relevance and Coverage are high (due to selecting coarse-grained nodes), its Path Granularity is extremely low (27.7 vs 82.4), suggesting that the lack of LLM background knowledge leads to poor hierarchical relations. TaxoAdapt strikes a better balance by combining the strengths of both.

vs TaxoInstruct (Shen et al., 2024)

TaxoInstruct unifies three tasks: entity set expansion, taxonomy expansion, and seed-guided construction, but still relies on pre-defined entity sets. TaxoAdapt does not require user-provided entity sets but automatically discovers new entities through document-level reasoning, making it more applicable to rapidly evolving domains.

Rating

• Novelty: 8/10 — Proposes a multidimensional and corpus-aligned taxonomy construction framework for the first time, with an innovative idea of classification-signal-driven expansion.
• Experimental Thoroughness: 8/10 — Evaluated across 4 datasets × 5 dimensions, complemented by ablation studies, open-source model experiments, evolution case studies, and human evaluation.
• Writing Quality: 8/10 — Clear problem formulation, complete algorithmic pseudocode, and rich diagrams and tables.
• Value: 7/10 — Possesses practical value for scientific literature organization and knowledge management, though validations on downstream application scenarios are still insufficient.

Related Papers

• [ACL 2025] Awes, Laws, and Flaws From Today's LLM Research
• [ICML 2025] LLM Social Simulations Are a Promising Research Method
• [ACL 2025] Leveraging Large Language Models to Measure Gender Representation Bias in Gendered Language Corpora
• [ACL 2025] Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers
• [ACL 2025] Aligning Large Language Models with Implicit Preferences from User-Generated Content