Disentangling Semantics and Popularity: Adversarial Debiasing and Asymmetric Alignment for LLM-Enhanced Long-Tail Sequential Recommendation¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Sequential Recommendation, Long-Tail Recommendation, LLM Semantic Priors, Adversarial Debiasing, Asymmetric Alignment
TL;DR¶
Addressing popularity-induced semantic distortion and the erosion of semantic priors in LLM-enhanced sequential recommendation, ADAA introduces a decoupled two-stage framework combining adversarial popularity debiasing with head-tail asymmetric anchoring, improving long-tail recommendation accuracy with zero additional online inference latency.
Background & Motivation¶
Sequential recommendation (SR) aims to capture dynamic user preference transitions by mining chronological interaction histories. Mainstream collaborative filtering backbones, such as self-attentive architectures (SASRec) or bidirectional masked transformers (BERT4Rec), rely fundamentally on capturing behavioural dependencies and co-occurrence transitions among discrete item identifiers (IDs). However, real-world user feedback inevitably follows an extreme long-tail Pareto distribution, where a small fraction of popular items accounts for the vast majority of user engagements, while the remaining ~80% of long-tail items have sparse historical interaction logs. Operating in closed collaborative loops, pure ID-based backbones cannot establish reliable transition trajectories for these sparse items, leading to severe representation degradation and compromised recommendation accuracy on the long tail.
To mitigate interaction sparsity, recent paradigms incorporate semantic embeddings extracted from item metadata by frozen Large Language Models (LLMs) to inject rich external open-world knowledge. However, empirical investigations reveal an acute "seesaw effect" in existing semantic adaptation methods: the accuracy enhancements achieved by standard semantic initialization are heavily skewed toward high-frequency head items, while performance improvements on sparse tail items remain negligible. This bottleneck stems from three fundamental failure modes: first, textual semantic embeddings and collaborative ID representations occupy fundamentally heterogeneous latent geometric spaces, where naive uniform projections induce information collapse and lose fine-grained distinctions; second, cross-space mapping rules trained over user interaction distributions are inevitably dominated by head-item frequencies, causing the projector to overfit popularity signals and project tail items into dense high-frequency clusters; third, during subsequent downstream sequential training, high-frequency gradient updates predominantly overwrite and erode the fragile semantic priors initialized for sparse items.
Overcoming these failure modes necessitates abandoning the conventional symmetric assumption that treats head and tail items identically during cross-space mapping and sequential adaptation. Because popular and sparse items exhibit fundamentally asymmetric needs regarding collaborative reliability and semantic dependence, their representation updates must be decoupled and guided by frequency-specific structural constraints. Core idea: decouple cross-space semantic projection from sequential training into an asymmetric adaptation pipeline, employing adversarial domain adaptation to purge popularity bias from projected semantic representations, while applying asymmetric anchoring during sequential fine-tuning to protect tail-item semantic priors from gradient erosion while permitting head items to fit dynamic collaborative transitions without introducing online serving overhead.
Method¶
Overall Architecture¶
The ADAA framework consists of two decoupled stages: an offline semantic projection phase with adversarial debiasing, and an asymmetric sequential fine-tuning phase with parameter-level injection. In the first phase, frozen collaborative representations of head items serve as structural anchors; a lightweight multi-layer perceptron projector is optimized using frequency-weighted contrastive alignment and a dual-metric retention loss, alongside a domain discriminator trained via a Gradient Reversal Layer (GRL) in a minimax game to eliminate popularity signals. In the second phase, the learned debiased semantic representations overwrite the item embedding matrix of the sequential backbone via parameter-level injection, followed by sequence fine-tuning regularized by frequency-partitioned asymmetric anchoring constraints that are discarded entirely prior to online inference.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Item Metadata & Pre-trained Collaborative Anchors<br/>$E_{llm}$ and frozen $E_{id}^H$"] --> B["Collaborative Anchored Semantic Alignment<br/>Frequency-weighted contrastive loss & dual-metric retention"]
B --> C["Adversarial Domain Adaptation for Popularity Debiasing<br/>GRL-driven minimax game with domain discriminator"]
C --> D["Parameter-Level Semantic Injection<br/>Direct non-intrusive backbone embedding overwriting"]
D --> E["Asymmetric Semantic Anchoring<br/>Head collaborative distillation & tail semantic regularization"]
E --> F["Standard Sequential Backbone Inference<br/>Pure forward propagation with zero serving overhead"]
Key Designs¶
1. Collaborative Anchored Semantic Alignment: Heterogeneous structural disparity and mapping collapse -> Head collaborative anchors with frequency-weighted contrastive alignment -> Robust cross-space topology preservation
Directly projecting frozen LLM textual embeddings \(e_{llm}^v \in \mathbb{R}^{d_{llm}}\) into the collaborative latent space \(\mathbb{R}^d\) using Mean Squared Error (MSE) forces absolute coordinate regression, which collapses diverse semantic features into dense collaborative clusters and erases intrinsic semantic variance. ADAA extracts the well-optimized collaborative ID embeddings of the top 20% head items \(\mathcal{V}_H\) from a pre-trained sequential backbone to construct frozen structural anchors \(E_{id}^H \in \mathbb{R}^{|\mathcal{V}_H| \times d}\), thereby isolating the mapping from unreliable tail representations. A two-layer MLP projector \(\mathcal{P}(\cdot)\) with GELU activations maps semantic embeddings into collaborative space. To prevent sparse anchors from destabilizing the cross-space geometry, ADAA weights the noise-contrastive estimation loss across batch \(\mathcal{B}\) with a continuous log-frequency scaling coefficient \(c_i = \omega \cdot \frac{\log(\text{count}(i) + 1)}{\log(\text{count}_{\max} + 1)} + b\):
$$
\mathcal{L}{\text{align}} = \frac{1}{\sum}} c_i} \sum_{i \in \mathcal{B}} c_i \cdot \left( -\log \frac{\exp(\text{sim}(\tilde{\mathcal{P}}(e_{llm}^i), \tilde{e{id}^i)/\tau)}{\sum \right)
$$
where }} \exp(\text{sim}(\tilde{\mathcal{P}}(e_{llm}^i), \tilde{e}_{id}^j)/\tau)\(\tilde{x} = x / \|x\|_2\) removes magnitude distortions, ensuring that cross-space projection is guided exclusively by reliable collaborative structures.
2. Adversarial Domain Adaptation for Popularity Debiasing: Projector overfitting to head-item interaction frequencies -> Minimax game via Gradient Reversal Layer -> Explicit decoupling of popularity and semantics
Because the alignment targets are dominated by high-frequency interactions of head items, an unconstrained projector \(\mathcal{P}(\cdot)\) inadvertently absorbs popularity-specific characteristics, shifting projected tail items toward high-frequency collaborative manifolds. ADAA formulates debiasing as a domain adaptation task, treating the head set \(\mathcal{V}_H\) as the source domain (\(d_i = 1\)) and the tail set \(\mathcal{V}_T\) as the target domain (\(d_i = 0\)). A binary domain discriminator \(\mathcal{D}(\cdot)\) is trained to identify the popularity origin of the projected feature \(z_i = \mathcal{P}(e_{llm}^i)\), competing against \(\mathcal{P}(\cdot)\) through a Gradient Reversal Layer (GRL):
$$
\mathcal{L}{\text{adv}} = -\mathbb{E}(z_i)) \right]
$$
To prevent representation collapse where the projector maps all inputs to an uninformative constant vector to fool the discriminator, a dual-metric retention loss }} \left[ d_i \log \mathcal{D}(z_i) + (1 - d_i) \log (1 - \mathcal{D\(\mathcal{L}_{\text{ret}} = \sum_{v_i \in \mathcal{V}_H} \left( \|\mathcal{P}(e_{llm}^i) - e_{id}^i\|_2^2 + \gamma \cdot (1 - \text{sim}(\mathcal{P}(e_{llm}^i), e_{id}^i)) \right)\) is imposed on head items, bounding coordinate distances while preserving directional alignment. Joint optimization under \(\mathcal{L}_{\text{proj}} = \mathcal{L}_{\text{align}} + \mathcal{L}_{\text{ret}} + \lambda \mathcal{L}_{\text{adv}}\) strips frequency attributes, yielding an uncorrupted semantic matrix \(Z^* = \mathcal{P}^*(E_{llm})\).
3. Asymmetric Semantic Anchoring: Erosion of injected tail priors by high-frequency gradients -> Frequency-differentiated reference targets and exponential weight decay -> Balancing sequential transitions and semantic integrity
Instead of inserting parameter-heavy cross-attention adapters or gating layers that degrade online serving latency, ADAA adopts a parameter-level injection strategy by directly overwriting the backbone's embedding layer with the debiased semantic representations \(Z^*\). However, subsequent sequence training with standard objectives (\(\mathcal{L}_{\text{seq}}\)) quickly overwrites the injected priors of sparse items due to dominant head-item gradient updates. ADAA introduces an asymmetric anchoring regularization \(\mathcal{L}_{\text{anchor}}\) that dynamically assigns frequency-stratified reference targets \(\hat{e}_i\):
$$
\hat{e}i = \begin{cases} e}^i, & \text{if } v_i \in \mathcal{VH \ z_i^*, & \text{if } v_i \in \mathcal{V}T \end{cases}, \quad \mathcal{L} = \sum}_i))
$$
where }} (1 - \text{sim}(h_i, \hat{e\(h_i\) represents the updated embedding during sequential training. For head items, anchoring to pre-trained collaborative vectors \(e_{id}^i\) serves as knowledge distillation of behavioural transitions; for tail items, anchoring to their debiased semantic representations \(z_i^*\) acts as an elastic constraint preventing representation drift. By applying an exponentially decaying weight \(\alpha(e) = \alpha_0 \cdot \eta^e\) (\(\eta \in (0, 1)\)), ADAA enforces structural preservation early in training before smoothly transitioning focus toward personalized sequence modeling.
Loss & Training¶
The optimization follows a strictly decoupled two-stage scheme: - Stage 1 (Offline Debiased Semantic Alignment): Keeping pre-trained collaborative anchors \(E_{id}^H\) and LLM embeddings \(E_{llm}\) frozen, the projector \(\mathcal{P}(\cdot)\) and discriminator \(\mathcal{D}(\cdot)\) are trained via the Adam optimizer under \(\mathcal{L}_{\text{train}} = \mathcal{L}_{\text{align}} + \mathcal{L}_{\text{ret}} + \lambda \mathcal{L}_{\text{adv}}\). Upon convergence, \(\mathcal{P}^*\) maps all item texts to obtain \(Z^*\). - Stage 2 (Asymmetric Sequential Fine-Tuning): The embedding table of the target backbone (e.g., SASRec or BERT4Rec) is initialized with \(Z^*\). The backbone parameters are updated using \(\mathcal{L}_{\text{tune}} = \mathcal{L}_{\text{seq}} + \alpha(e) \mathcal{L}_{\text{anchor}}\) across user interaction sequences. - Hyper-parameter Setup: Latent dimension \(d = 64\), batch size 128, learning rate 0.001, dropout 0.5. The adversarial weight \(\lambda\) is tuned over \(\{0.001, 0.01, 0.05, 0.1, 0.5\}\), and the initial anchoring coefficient \(\alpha_0\) over \(\{5, 10, 15, 20, 25\}\). All auxiliary networks are discarded post-training, guaranteeing identical inference complexity to the original backbone.
Key Experimental Results¶
Main Results¶
The models were evaluated across three benchmark datasets (Yelp, Amazon Beauty, and Amazon Fashion) using Top-10 ranking metrics (Hit Rate@10 and NDCG@10) under the standard sampled evaluation protocol (100 negative candidates per test instance).
| Dataset | Backbone & Model | Overall H@10 | Overall N@10 | Tail Item H@10 | Tail Item N@10 | Head Item H@10 | Head Item N@10 |
|---|---|---|---|---|---|---|---|
| Yelp | SASRec (Vanilla) | 0.5724 | 0.3413 | 0.0763 | 0.0318 | 0.7184 | 0.4324 |
| Yelp | SASRec + CITIES | 0.5828 | 0.3540 | 0.1532 | 0.0700 | 0.7093 | 0.4376 |
| Yelp | SASRec + MELT | 0.6257 | 0.3791 | 0.1015 | 0.0371 | 0.7801 | 0.4799 |
| Yelp | SASRec + LLMInit | 0.6415 | 0.3997 | 0.1760 | 0.0789 | 0.7785 | 0.4941 |
| Yelp | SASRec + SRA-CL | 0.6534 | 0.4169 | 0.1312 | 0.0564 | 0.8072 | 0.5231 |
| Yelp | SASRec + LLM-ESR | 0.6673 | 0.4208 | 0.1893 | 0.0845 | 0.8080 | 0.5199 |
| Yelp | SASRec + ADAA (Ours) | 0.6770 | 0.4279 | 0.1927 | 0.0840 | 0.8196 | 0.5297 |
| Yelp | BERT4Rec (Vanilla) | 0.5812 | 0.3544 | 0.0173 | 0.0076 | 0.7473 | 0.4565 |
| Yelp | BERT4Rec + LLM-ESR | 0.6623 | 0.4222 | 0.1227 | 0.0500 | 0.8212 | 0.5318 |
| Yelp | BERT4Rec + ADAA (Ours) | 0.6765 | 0.4409 | 0.2178 | 0.0934 | 0.8216 | 0.5433 |
| Beauty | SASRec (Vanilla) | 0.4388 | 0.3030 | 0.0870 | 0.0649 | 0.5227 | 0.3598 |
| Beauty | SASRec + LLM-ESR | 0.5672 | 0.3713 | 0.2257 | 0.1108 | 0.6486 | 0.4334 |
| Beauty | SASRec + ADAA (Ours) | 0.5770 | 0.4046 | 0.2698 | 0.1572 | 0.6503 | 0.4637 |
| Beauty | BERT4Rec (Vanilla) | 0.4601 | 0.2935 | 0.0125 | 0.0051 | 0.5656 | 0.3616 |
| Beauty | BERT4Rec + ADAA (Ours) | 0.5498 | 0.3821 | 0.1556 | 0.0826 | 0.6428 | 0.4544 |
| Fashion | SASRec (Vanilla) | 0.4956 | 0.4429 | 0.0454 | 0.0235 | 0.6748 | 0.6099 |
| Fashion | SASRec + LLM-ESR | 0.5619 | 0.4743 | 0.1095 | 0.0520 | 0.7420 | 0.6424 |
| Fashion | SASRec + ADAA (Ours) | 0.6096 | 0.5240 | 0.2254 | 0.1342 | 0.7605 | 0.6771 |
Under the rigorous full-catalog ranking evaluation where candidates are not sampled, ADAA achieves statistically significant improvements on Yelp with SASRec, lifting overall H@10/N@10 from 0.0254/0.0118 (vanilla) and 0.0482/0.0241 (LLM-ESR) to 0.0491/0.0248, and boosting tail-item H@10/N@10 from 0.0003/0.0001 (vanilla) to 0.0062/0.0029, confirming that performance gains are genuine and persistent across the entire candidate catalog.
Ablation Study¶
Ablation experiments conducted on Yelp with the SASRec backbone analyze the impact of each core component:
| Architecture Variant | Overall H@10 | Overall N@10 | Tail Item H@10 | Tail Item N@10 | Head Item H@10 | Head Item N@10 | Description & Effect |
|---|---|---|---|---|---|---|---|
| ADAA (Full Model) | 0.6770 | 0.4279 | 0.1927 | 0.0840 | 0.8196 | 0.5297 | Full model with adversarial debiasing and asymmetric anchoring |
w/o Adv (No Adversarial) |
0.6246 | 0.3795 | 0.1317 | 0.0519 | 0.7698 | 0.4759 | Removes GRL and discriminator; popularity bias corrupts projection |
w/o FT (No Fine-Tuning) |
0.5070 | 0.3152 | 0.2992 | 0.1433 | 0.5387 | 0.3364 | Evaluates debiased semantic embeddings directly without sequence training |
w/o Anc (No Anchoring) |
0.6485 | 0.4003 | 0.1163 | 0.0473 | 0.8053 | 0.5042 | Unconstrained sequence fine-tuning; tail priors are washed out |
Sym-Col (Symmetric ID) |
0.5892 | 0.3573 | 0.1096 | 0.0465 | 0.7305 | 0.4488 | Anchors all items to collaborative IDs; noisy sparse targets hurt tail |
Sym-Sem (Symmetric Semantic) |
0.5924 | 0.3561 | 0.1499 | 0.0655 | 0.7127 | 0.4317 | Anchors all items to semantic embeddings; restricts head learning |
Key Findings¶
- Adversarial debiasing is indispensable for sparse item recovery: Removing the adversarial objective (
w/o Adv) causes tail-item H@10 to plummet from 0.1927 to 0.1317 (a 31.7% drop). Independent post-hoc probing analysis confirms that while collaborative vectors yield a linear AUC of 0.815 in recovering item popularity, and the unconstrained projector retains an AUC of 0.718, ADAA decreases popularity prediction AUC to 0.535 (near random guess 0.5) and popularity regression \(R^2\) from 0.275 to 0.039, confirming effective popularity removal. - Sequence fine-tuning bridges the semantic-sequential gap: While zero-shot semantic inference (
w/o FT) achieves an inflated tail H@10 of 0.2992, its overall H@10 collapses to 0.5070 and head H@10 drops to 0.5387. Sequence training is fundamentally necessary to capture behavioral co-occurrence patterns, and ADAA successfully preserves tail semantics during this training phase. - Symmetric anchoring fails to resolve the seesaw dilemma:
Sym-Coldrags tail items toward noisy collaborative anchors (tail H@10 drops to 0.1096), whileSym-Semprevents head items from absorbing dense collaborative interactions (head H@10 drops to 0.7127). The head-collaborative / tail-semantic asymmetric partition is necessary to balance both groups. - Zero online latency overhead: In empirical serving benchmarks, SASRec requires 0.78M parameters and 2.39ms batch latency (BL); ADAA requires identical 0.78M parameters and 2.35ms BL with 54,591 QPS throughput. In contrast, LLM-ESR requires 20.31M parameters, increasing batch latency to 13.04ms (a 5.5x slowdown) and consuming 4x peak GPU memory.
Highlights & Insights¶
- Asymmetric Modeling Philosophy: Rather than forcing a homogeneous embedding space across all items, ADAA recognizes the disparate reliability of collaborative signals and establishes asymmetric head/tail preservation targets, resolving the long-standing long-tail seesaw effect.
- Clean Online Serving Profile: By separating cross-space adversarial alignment into an offline pre-processing step and parameter-level initialization, ADAA provides state-of-the-art long-tail accuracy without altering the backbone's architecture, caching, or runtime inference graph.
- Domain Adaptation Formulation of Popularity Bias: Casting interaction frequency disparity as a domain shift problem and solving it via minimax GRL optimization offers a principled mathematical blueprint for debiasing multi-modal recommender systems.
Limitations & Future Work¶
- Static Offline Alignment in Streaming Settings: The decoupled pipeline requires periodic retraining or incremental re-projection when new items continuously arrive in streaming production systems.
- Vulnerability to Sparse Textual Metadata: The framework relies on expressive item metadata (titles, categories, descriptions). For items with missing or extremely noisy text, the language model cannot provide distinguishable priors, weakening debiasing efficacy.
- Scale Validation Beyond Moderate Benchmarks: While validated across Yelp and Amazon sub-domains, testing the stability of adversarial convergence on industrial catalogs containing tens of millions of items and billions of interactions remains an open engineering challenge.
Related Work & Insights¶
- vs. CITIES / MELT (Collaborative Long-Tail Methods): CITIES and MELT attempt to transfer structural knowledge purely within the closed collaborative interaction matrix via co-occurrence modeling or curriculum learning. They cannot overcome fundamental information scarcity for cold items, whereas ADAA injects external LLM semantic priors.
- vs. LLM-ESR (Dual-Branch LLM Recommendation): LLM-ESR combines collaborative and text views via runtime cross-attention adapters and dual-tower distillation, introducing severe online memory and latency bottlenecks. ADAA attains equal or superior long-tail accuracy while operating at the latency and parameter scale of standard single-tower backbones.
- vs. RLMRec / SRA-CL (Contrastive Semantic Alignment): Standard contrastive methods apply symmetric alignment objectives across all items, leading to popularity bias during projection and subsequent semantic erosion during sequence training. ADAA explicitly eliminates popularity bias via adversarial GRL and preserves tail semantics via asymmetric anchoring.
Rating¶
- Novelty: ⭐⭐⭐⭐☆ Principled formulation of frequency debiasing via adversarial domain adaptation coupled with head-tail asymmetric anchoring.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Extensive validation covering sampled ranking, full-catalog ranking, fine-grained decile breakdown, and representation probing.
- Writing Quality: ⭐⭐⭐⭐⭐ Clear theoretical justification, precise mathematical formulations, and rigorous terminology.
- Value: ⭐⭐⭐⭐⭐ High industrial relevance due to zero online inference latency combined with significant tail-item accuracy gains.