A Manifold-Aware Topic Modeling Approach via Rank-Based Prototypes¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/thcastilho/maretopic
Area: LLM Reasoning
Keywords: topic modeling, manifold learning, rank-based prototypes, training-free architecture, maximal marginal relevance
TL;DR¶
MARETopic introduces a training-free, manifold-aware topic modeling framework that casts topic discovery as greedy prototype document selection over ordinal ranked lists, eliminating high-dimensional embedding distortions while achieving superior clustering purity and topic coherence without a single gradient step.
Background & Motivation¶
Discovering latent thematic structure across large-scale document collections is foundational for information retrieval, corpus summarization, automated categorization, and semantic reasoning in language systems. For decades, topic modeling has been largely bifurcated into two primary paradigms: generative probabilistic models (ranging from classical LDA and NMF to neural embedding models like ETM, ECRTM, and optimal transport formulations like FASTopic) that represent topics as continuous latent variables or global distributions over the vocabulary; and clustering-based pipelines (such as BERTopic and Top2Vec) that group document embeddings in dense semantic spaces and extract topic keywords from the resulting clusters. Despite substantial progress, both paradigms suffer from critical architectural deficiencies that limit their reliability and human interpretability.
The first fundamental limitation is the complete absence of an architectural, document-level semantic anchor for each topic. Human inspection remains the definitive gold standard for assessing topic model validity because automated coherence metrics frequently diverge from human interpretability judgments. Yet, neural models represent topics purely as ungrounded latent vectors or probability vectors over vocabulary tokens, leaving the fundamental question—"which concrete document in the corpus best exemplifies and defines this topic?"—without an architectural answer. Concurrently, while clustering models such as BERTopic attempt to present "representative documents" per topic, these are merely selected post hoc by scoring a random subsample against a c-TF-IDF vector rather than being the foundational definition of the topic itself. Furthermore, density-based clustering algorithms like HDBSCAN cannot take a target topic budget \(K\) as an explicit input, leading to uncontrollable topic counts, fragmentation, and sensitive reliance on dataset-specific hyperparameter heuristics.
The second core bottleneck stems from the geometric pathologies inherent in high-dimensional embedding spaces. Contextual representations produced by deep language models (e.g., Sentence-BERT) exhibit severe anisotropy, concentrating dense vectors within a narrow cone and causing unrelated documents to share deceptively high cosine similarities. Simultaneously, high-dimensional spaces suffer from hubness, whereby a small fraction of outlier points appear disproportionately often across nearest-neighbor lists, heavily distorting absolute metric distances. In contrast, ordinal ranked lists depend solely on relative neighborhood order: they are strictly invariant under any monotonic transformation of the underlying distance metric and naturally capture the local topology of the data manifold. Core idea: by replacing absolute high-dimensional metric distances with ordinal ranked lists over a manifold projection, topic discovery can be formulated as greedy exemplar selection of \(K\) real documents whose neighborhoods cover the corpus without redundancy, producing an entirely training-free topic model with rigorous document anchors and diverse vocabularies via inter-topic MMR.
Method¶
Overall Architecture¶
The MARETopic framework reframes topic modeling as selecting a set of \(K\) leader exemplar documents \(\mathcal{L} = \{\ell_1, \dots, \ell_K\} \subset \mathcal{D}\) whose local neighborhoods collectively cover the corpus manifold while minimizing mutual overlap. The pipeline operates in four streamlined phases: first, corpus documents are encoded using Sentence-BERT and projected onto a low-dimensional manifold via UMAP; second, for every document, an ordinal ranked list is indexed, sorting all other documents by increasing manifold distance to eliminate absolute metric distortions; third, a greedy prototype selection algorithm iteratively identifies \(K\) exemplar documents that balance topological centrality against neighborhood redundancy relative to previously chosen leaders; and finally, a document-topic affinity matrix is computed to establish cluster assignments, followed by class-based TF-IDF and an inter-topic Maximal Marginal Relevance (MMR) step that dynamically penalizes cross-cluster vocabulary overlap to generate coherent and distinct topic word lists.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Raw Document Corpus & Standard Preprocessing"] --> B["SBERT Encoding & UMAP Manifold Projection"]
B --> C["Manifold Ranking & Indexing<br/>Ordinal Neighborhood Construction"]
C --> D["Greedy Exemplar Selection<br/>Centrality Scoring & Redundancy Penalty"]
D --> E["Document-Topic Matrix Assignment<br/>JaccardMax or Diffusion Paths"]
E --> F["Inter-Topic MMR Vocabulary Diversification<br/>Dynamic Maximum Penalty Reranking"]
F --> G["Output K Anchored Exemplar Texts & Top Word Lists"]
Key Designs¶
1. Manifold Ranking & Indexing: High-Dimensional Distortions -> Ordinal Neighborhood Construction -> Metric-Agnostic Geometric Robustness
High-dimensional transformer representations suffer from severe anisotropy and hubness, which fundamentally degrade conventional Euclidean and cosine distance metrics. MARETopic overcomes this by projecting \(d_{\text{SBERT}}=384\) dimensional embeddings into a compact \(d_{\text{UMAP}}=5\) dimensional manifold, \(L_2\)-normalizing them, and querying an exact nearest-neighbor index to construct an ordinal ranked list \(\tau_i\) for each document \(d_i\):
$\(\tau_i(d_j) < \tau_i(d_m) \iff \rho(d_i, d_j) \le \rho(d_i, d_m)\)$
where \(\rho(\cdot, \cdot)\) denotes the Euclidean distance on the projected manifold, and \(\tau_i(d_j)\) represents the rank position of document \(d_j\) in the context of \(d_i\). The top-\(k\) neighborhood is defined as \(\mathcal{N}(d_i, k) = \{d_j \in \mathcal{D} \mid \tau_i(d_j) \le k\}\). According to ordinal embedding theory, ordinal comparisons preserve underlying topological structures while being completely invariant to monotonic distance transformations, effectively discarding metric distortions and grounding similarity in local manifold geometry.
2. Greedy Exemplar Selection: Abstract Latent Topics -> Centrality Scoring & Redundancy Penalty -> Exact K-Exemplar Anchors
To eliminate the disconnect between abstract latent topic distributions and concrete corpus documents, MARETopic selects exactly \(K\) actual documents as topic exemplars. Given the leaders \(\mathcal{L}_{t-1}\) selected up to step \(t-1\), the next exemplar \(\ell_t\) is chosen greedily by maximizing a ratio of local centrality to cumulative redundancy:
$\(\ell_t = \arg\max_{d \notin \mathcal{L}_{t-1}} \frac{\eta(d)}{1 + \sum_{\ell \in \mathcal{L}_{t-1}} s(\tau_d, \tau_\ell)}\)$
where \(\eta(d)\) measures the candidate's topological centrality and \(s(\tau_d, \tau_\ell)\) penalizes neighborhood overlap with previously chosen leaders. The framework provides two complementary realizations:
- MARETopicCorr (Correlation Variant): Estimates centrality via a Query Performance Prediction (QPP) metric called Reciprocal Density:
$\(\eta(d) = \frac{1}{k^4} \sum_{d_j \in \mathcal{N}(d, k)} \sum_{d_i \in \mathcal{N}(d, k)} f_r(d_j, d_i) w_r(d, d_j) w_r(d, d_i)\)$
with \(f_r(d_j, d_i) = |\mathcal{N}(d_j, k) \cap \{d_i\}|\) indicating mutual reciprocal agreement and \(w_r(d_j, d_i) = k + 1 - \tau_j(d_i)\) applying rank decay. Redundancy \(s(\tau_d, \tau_\ell)\) is quantified using JaccardMax, which scans the maximum Jaccard coefficient across all prefix depths: \(s(\tau_d, \tau_\ell) = \max_{l=1}^{k-1} \frac{|\mathcal{N}(d, l) \cap \mathcal{N}(\ell, l)|}{|\mathcal{N}(d, l) \cup \mathcal{N}(\ell, l)|}\).
- MARETopicDiff (Diffusion Variant): Constructs a sparse log-weighted affinity matrix \(W \in \mathbb{R}^{N \times N}\) from the ranked lists (where \(w_{ij} = 1 - \log_k \tau_i(d_j)\) for \(\tau_i(d_j) \le k\) and 0 otherwise), computing the 2-step diffusion matrix \(A = W^2\). Here, the diagonal entry \(\eta(d) = a_{dd}\) directly provides globally smoothed reciprocal affinity, while off-diagonal entries \(s(\tau_d, \tau_\ell) = a_{d\ell}\) represent diffusion-based redundancy penalties. This variant avoids prefix scanning and runs 1.7x to 1.9x faster than the correlation counterpart.
3. Inter-Topic MMR Vocabulary Diversification: Cross-Cluster Lexical Overlap -> Dynamic Maximum Penalty Reranking -> High Topic Diversity Preservation
In clustering-based topic models, clusters frequently share general domain-specific vocabulary, leading to low Topic Diversity (TD) compared to optimal-transport models regularized by Sinkhorn divergence. To eliminate this deficiency without damaging coherence, MARETopic introduces an inter-topic Maximal Marginal Relevance (MMR) reranking step. Once hard cluster assignments \(z_i = \arg\max_j \theta_{ij}\) are established from the affinity matrix \(\Theta\), topics are ordered by decreasing cluster size \(|c|\). For each topic \(c\), each word \(w\) is scored as:
$\(\text{score}(c, w) = (1 - \lambda) \cdot \text{c-TF-IDF}(c, w) - \lambda \cdot \max_{c' \prec c} \text{c-TF-IDF}(c', w)\)$
where \(\lambda \in [0, 1]\) controls diversity weight (default \(\lambda = 0.3\)), and \(c' \prec c\) iterates over all previously processed larger topics. By penalizing words that have already received high c-TF-IDF scores in dominant clusters, this post-processing mechanism suppresses generic high-frequency terms, boosting TD by roughly \(+0.2\) across benchmarks at minimal coherence cost (\(\le 0.02\) drop in \(C_v\)).
Benchmark & Evaluation Protocol¶
The empirical evaluation was conducted using the TopMost topic modeling framework on three benchmark datasets representing varied domains, document lengths, and label cardinalities: 1. 20 Newsgroups (20NG): 18,846 newsgroup posts across 20 fine-grained categories, with a standard vocabulary of 5,000 tokens. 2. New York Times (NYT): 9,172 full-length news articles categorized into 12 sections, with a vocabulary of 10,000 tokens. 3. Web of Science (WoS): 11,967 scientific abstracts across 7 broad disciplines, evaluated under an 80/20 stratified split with a 10,000-token vocabulary.
All models were evaluated under an identical budget of \(K = 50\) topics across 3 independent runs, extracting \(m = 15\) top words per topic. Baselines included classical models (LDA, NMF), neural generative and optimal transport architectures (ETM, ECRTM, FASTopic, each trained for 200 epochs), and clustering methods (BERTopic, Top2Vec). All embedding-based methods utilized the identical all-MiniLM-L6-v2 SBERT backbone. Evaluation metrics tracked topic coherence (\(C_v\), NPMI), vocabulary diversity (TD), and clustering alignment with ground truth (Purity, NMI).
Key Experimental Results¶
Main Results¶
The main benchmark comparisons across the three datasets are summarized in the table below (reporting mean \(\pm\) standard deviation over 3 runs; wall-clock runtime in seconds measured on an Apple M4 CPU):
| Dataset | Metric | LDA | NMF | ETM | ECRTM | FASTopic | BERTopic | Top2Vec | MARETopic-Diff | MARETopic-Corr |
|---|---|---|---|---|---|---|---|---|---|---|
| 20NG | \(C_v \uparrow\) | 0.5352 | 0.5573 | 0.4522 | 0.6116 | 0.5993 | 0.5942 | 0.6649 | 0.6708 | 0.6818 |
| NPMI \(\uparrow\) | 0.0472 | 0.0613 | -0.0171 | -0.0372 | -0.0689 | 0.0842 | 0.1032 | 0.0958 | 0.1027 | |
| TD \(\uparrow\) | 0.5196 | 0.5093 | 0.8471 | 0.8689 | 0.9849 | 0.7733 | 0.8662 | 0.8613 | 0.8787 | |
| Purity \(\uparrow\) | 0.4204 | 0.2153 | 0.3390 | 0.6114 | 0.5738 | 0.4493 | 0.5812 | 0.4144 | 0.6227 | |
| NMI \(\uparrow\) | 0.3763 | 0.1800 | 0.3032 | 0.5348 | 0.5202 | 0.3797 | 0.5265 | 0.4011 | 0.5419 | |
| Time (s) \(\downarrow\) | 8.6 | 5.5 | 114.3 | 184.5 | 128.7 | 37.1 | 52.6 | 49.6 | 92.6 | |
| NYT | \(C_v \uparrow\) | 0.4534 | 0.4919 | 0.4696 | 0.4457 | 0.5864 | 0.6189 | 0.6944 | 0.6570 | 0.6496 |
| NPMI \(\uparrow\) | 0.0002 | 0.0468 | 0.0109 | -0.0997 | -0.0587 | 0.1166 | 0.1295 | 0.1079 | 0.1060 | |
| TD \(\uparrow\) | 0.5769 | 0.4560 | 0.8427 | 0.9595 | 0.9996 | 0.7951 | 0.9071 | 0.8831 | 0.8702 | |
| Purity \(\uparrow\) | 0.5454 | 0.4612 | 0.5556 | 0.6529 | 0.6583 | 0.6184 | 0.6874 | 0.6046 | 0.7084 | |
| NMI \(\uparrow\) | 0.2661 | 0.1952 | 0.2815 | 0.3549 | 0.3670 | 0.3221 | 0.3890 | 0.3305 | 0.3975 | |
| Time (s) \(\downarrow\) | 7.2 | 4.5 | 148.2 | 223.7 | 123.9 | 36.9 | 44.2 | 31.6 | 53.6 | |
| WoS | \(C_v \uparrow\) | 0.4797 | 0.4869 | 0.4198 | 0.4655 | 0.6000 | 0.6920 | 0.7018 | 0.7198 | 0.7150 |
| NPMI \(\uparrow\) | -0.0287 | 0.0396 | -0.0045 | -0.1076 | -0.0308 | 0.1244 | 0.1344 | 0.1359 | 0.1447 | |
| TD \(\uparrow\) | 0.6636 | 0.4876 | 0.8938 | 0.9996 | 1.0000 | 0.7782 | 0.8796 | 0.8569 | 0.8547 | |
| Purity \(\uparrow\) | 0.6529 | 0.6086 | 0.6456 | 0.7733 | 0.7765 | 0.7934 | 0.8293 | 0.6082 | 0.8106 | |
| NMI \(\uparrow\) | 0.3875 | 0.2962 | 0.3853 | 0.4518 | 0.4565 | 0.4473 | 0.4901 | 0.3660 | 0.4664 | |
| Time (s) \(\downarrow\) | 6.5 | 5.2 | 170.7 | 253.7 | 137.2 | 35.0 | 49.6 | 37.7 | 65.2 |
Ablation Study¶
Ablation analysis on the inter-topic MMR diversity parameter \(\lambda \in \{0.0, 0.3, 0.5, 0.7\}\), mapping the trade-off between coherence \(C_v\) and vocabulary diversity TD (\(\lambda = 0.3\) is the default configuration):
| Method Variant | Parameter \(\lambda\) | 20NG \(C_v\) | 20NG TD | NYT \(C_v\) | NYT TD | WoS \(C_v\) | WoS TD |
|---|---|---|---|---|---|---|---|
| MARETopic-Corr | \(\lambda = 0.0\) | 0.6777 | 0.6627 | 0.6398 | 0.6845 | 0.7299 | 0.6938 |
| \(\lambda = 0.3\) (Default) | 0.6674 (-0.010) | 0.8813 (+0.219) | 0.6384 (-0.001) | 0.8729 (+0.188) | 0.7206 (-0.009) | 0.8600 (+0.166) | |
| \(\lambda = 0.5\) | 0.6539 (-0.014) | 0.9538 (+0.073) | 0.6349 (-0.004) | 0.9582 (+0.085) | 0.6850 (-0.036) | 0.9533 (+0.093) | |
| \(\lambda = 0.7\) | 0.6257 (-0.028) | 0.9925 (+0.039) | 0.6043 (-0.031) | 0.9893 (+0.031) | 0.6689 (-0.016) | 0.9902 (+0.037) | |
| MARETopic-Diff | \(\lambda = 0.0\) | 0.6854 | 0.6502 | 0.6516 | 0.7080 | 0.7287 | 0.6906 |
| \(\lambda = 0.3\) (Default) | 0.6676 (-0.018) | 0.8800 (+0.230) | 0.6677 (+0.016) | 0.8747 (+0.167) | 0.7080 (-0.021) | 0.8569 (+0.166) | |
| \(\lambda = 0.5\) | 0.6487 (-0.019) | 0.9564 (+0.076) | 0.6506 (-0.017) | 0.9569 (+0.082) | 0.6816 (-0.026) | 0.9582 (+0.101) | |
| \(\lambda = 0.7\) | 0.6322 (-0.017) | 0.9920 (+0.036) | 0.6322 (-0.018) | 0.9893 (+0.032) | 0.6577 (-0.024) | 0.9938 (+0.036) |
Key Findings¶
- Training-Free Clustering Superiority: Without requiring a single epoch of neural training, MARETopicCorr establishes the highest Purity and NMI on 20NG (0.6227 / 0.5419) and NYT (0.7084 / 0.3975), consistently outperforming neural topic models trained for 200 epochs (e.g., FASTopic: 0.5738 Purity on 20NG). This confirms that ordinal manifold structures discriminate fine-grained class boundaries better than globally fitted variational distributions.
- Paradigm Polarization in NPMI: While neural generative models (ETM, ECRTM, FASTopic) repeatedly produce negative NPMI scores across benchmarks—indicating that words assigned to their topics co-occur less frequently than expected by random chance—clustering-based models and MARETopic maintain robustly positive NPMI (0.0958 to 0.1447), with MARETopicCorr securing the top score on WoS (0.1447).
- Complementary Properties Across Variants: MARETopicCorr excels at granular document discrimination via prefix-level JaccardMax overlap, whereas MARETopicDiff leverages 2-step graph diffusion to achieve superior topological smoothing, leading \(C_v\) on WoS (0.7198) with nearly double the execution speed. When increasing the topic budget to \(K=100\), the Purity of the Diff variant surges (e.g., from 0.6082 to 0.8140 on WoS), proving its effectiveness under fine spatial resolutions.
- Model-Agnostic Vocabulary Diversification: The inter-topic MMR step successfully bridges the topic diversity gap inherent to clustering-based models. Setting \(\lambda = 0.3\) increases TD by \(+0.16\) to \(+0.23\) with negligible coherence degradation. Furthermore, when ported to BERTopic's native c-TF-IDF, inter-topic MMR elevates BERTopic's TD from ~0.78 to 0.92-0.94, demonstrating complete architectural modularity.
Highlights & Insights¶
- Grounding Topics in Concrete Exemplar Texts: Rather than defining topics as abstract latent vectors or ungrounded vocabulary distributions, MARETopic assigns an actual, inspectable corpus document to every topic, bridging the long-standing semantic gap in topic interpretability.
- Ordinal Ranking as a Shield Against High-Dimensional Geometric Pathologies: By replacing raw Euclidean or cosine metric distances with relative ranked lists over projected manifolds, the method completely sidesteps Sentence-BERT's anisotropy and hubness distortions.
- Extreme Efficiency via Zero-Gradient Pipeline: The entire model operates non-parametrically without neural optimization. Exact nearest-neighbor indexing takes less than 0.5 seconds (<1% of runtime), making the end-to-end pipeline substantially faster than 200-epoch neural alternatives.
Limitations & Future Work¶
- Corpus Scalability and Indexing Constraints: Experiments were conducted on corpora of moderate size (9K to 19K documents). While exact nearest-neighbor indexing overhead is minimal at this scale, scaling to hundreds of thousands of documents will necessitate approximate nearest neighbor (ANN) structures like HNSW.
- Degradation on Unstructured or Ultra-Simple Manifolds: On corpora characterized by highly separable, broad semantic categories (such as Web of Science), centroid-based clustering (Top2Vec) slightly outperforms prototype selection, indicating that exemplar selection provides the greatest relative advantage on complex, irregular, or overlapping manifolds.
- Dynamic and Hierarchical Generalization: The current framework is restricted to flat, static collections; future extensions should explore rank-based prototypes for hierarchical topic trees and temporal dynamic modeling.
Related Work & Insights¶
- vs BERTopic (Grootendorst, 2022): BERTopic relies on HDBSCAN density clustering, which cannot enforce an exact topic budget \(K\), and selects representative documents post hoc via random sampling. MARETopic architecturalizes exemplar documents as the primary definition of topics, enforces an exact \(K\)-budget, and eliminates high-dimensional metric distortions.
- vs FASTopic (Wu et al., 2024a): FASTopic utilizes dual optimal transport with Sinkhorn regularization to achieve near-perfect vocabulary diversity but incurs negative NPMI and requires 200 training epochs. MARETopic is entirely training-free, attains significantly higher Purity and NMI, and produces robustly positive NPMI coherence.
- vs RaDE & GRaCE (de Fernando et al., 2022; Almeida et al., 2025): While RaDE and GRaCE applied rank-based graph diffusion and QPP estimators to general representation learning, MARETopic bridges these principles into end-to-end NLP topic modeling with out-of-sample inference and inter-topic MMR diversification.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Pioneering introduction of manifold ordinal ranking and greedy exemplar selection to solve core topic modeling challenges.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation across 7 baselines, 3 datasets, 5 metrics, detailed ablations, parameter sensitivity curves, and multi-backbone validation.
- Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally rigorous mathematical formulation, lucid problem motivation, and thorough geometric analysis.
- Value: ⭐⭐⭐⭐⭐ Highly practical, interpretable, and reproducible framework for zero-training semantic discovery and document intelligence.